Firmware Over-the-air Programming Techniques for IoT Networks - A Survey

The devices forming Internet of Things (IoT) networks need to be re-programmed over the air, so that new features are added, software bugs or security vulnerabilities are resolved, and their applications can be re-purposed. The limitations of IoT devices, such as installation in locations with limited physical access, resource-constrained nature, large scale, and high heterogeneity, should be taken into consideration for designing an efficient and reliable pipeline for over-the-air programming (OTAP). In this work, we present a survey of OTAP techniques, which can be applied to IoT networks. We highlight the main challenges and limitations of OTAP for IoT devices and analyze the essential steps of the firmware update process, along with different approaches and techniques that implement them. In addition, we discuss schemes that focus on securing the OTAP process. Finally, we present a collection of state-of-the-art open-source and commercial platforms that integrate secure and reliable OTAP.

A Survey of Flaky Tests

Tests that fail inconsistently, without changes to the code under test, are described as flaky . Flaky tests do not give a clear indication of the presence of software bugs and thus limit the reliability of the test suites that contain them. A recent survey of software developers found that 59% claimed to deal with flaky tests on a monthly, weekly, or daily basis. As well as being detrimental to developers, flaky tests have also been shown to limit the applicability of useful techniques in software testing research. In general, one can think of flaky tests as being a threat to the validity of any methodology that assumes the outcome of a test only depends on the source code it covers. In this article, we systematically survey the body of literature relevant to flaky test research, amounting to 76 papers. We split our analysis into four parts: addressing the causes of flaky tests, their costs and consequences, detection strategies, and approaches for their mitigation and repair. Our findings and their implications have consequences for how the software-testing community deals with test flakiness, pertinent to practitioners and of interest to those wanting to familiarize themselves with the research area.

Automated Classification of Unstructured Bilingual Software Bug Reports: An Industrial Case Study Research

Software bug report classification is a critical process to understand the nature, implications, and causes of software failures. Furthermore, classification enables a fast and appropriate reaction to software bugs. However, for large-scale projects, one must deal with a broad set of bugs from multiple types. In this context, manually classifying bugs becomes cumbersome and time-consuming. Although several studies have addressed automated bug classification using machine learning techniques, they have mainly focused on academic case studies, open-source software, and unilingual text input. This paper presents our automated bug classification approach applied and validated in an industrial case study. In contrast to earlier studies, our study is applied to a commercial software system based on unstructured bilingual bug reports written in English and Turkish. The presented approach adopts and integrates machine learning (ML), text mining, and natural language processing (NLP) techniques to support the classification of software bugs. The approach has been applied within an industrial case study. Compared to manual classification, our results show that bug classification can be automated and even performs better than manual bug classification. Our study shows that the presented approach and the corresponding tools effectively reduce the manual classification time and effort.

Programming and execution models for parallel bounded exhaustive testing

Bounded-exhaustive testing (BET), which exercises a program under test for all inputs up to some bounds, is an effective method for detecting software bugs. Systematic property-based testing is a BET approach where developers write test generation programs that describe properties of test inputs. Hybrid test generation programs offer the most expressive way to write desired properties by freely combining declarative filters and imperative generators. However, exploring hybrid test generation programs, to obtain test inputs, is both computationally demanding and challenging to parallelize. We present the first programming and execution models, dubbed Tempo, for parallel exploration of hybrid test generation programs. We describe two different strategies for mapping the computation to parallel hardware and implement them both for GPUs and CPUs. We evaluated Tempo by generating instances of various data structures commonly used for benchmarking in the BET domain. Additionally, we generated CUDA programs to stress test CUDA compilers, finding four bugs confirmed by the developers.

A Framework for Detecting System Performance Anomalies Using Tracing Data Analysis

Advances in technology and computing power have led to the emergence of complex and large-scale software architectures in recent years. However, they are prone to performance anomalies due to various reasons, including software bugs, hardware failures, and resource contentions. Performance metrics represent the average load on the system and do not help discover the cause of the problem if abnormal behavior occurs during software execution. Consequently, system experts have to examine a massive amount of low-level tracing data to determine the cause of a performance issue. In this work, we propose an anomaly detection framework that reduces troubleshooting time, besides guiding developers to discover performance problems by highlighting anomalous parts in trace data. Our framework works by collecting streams of system calls during the execution of a process using the Linux Trace Toolkit Next Generation(LTTng), sending them to a machine learning module that reveals anomalous subsequences of system calls based on their execution times and frequency. Extensive experiments on real datasets from two different applications (e.g., MySQL and Chrome), for varying scenarios in terms of available labeled data, demonstrate the effectiveness of our approach to distinguish normal sequences from abnormal ones.

Methodological approach to identifying software bugs in special software for systems of critical applications

The paper focuses on a methodological approach to identifying software in special software for systems of critical applications. The approach relies on the analysis of the subject area associated with the functioning of such systems. The term ‘software bugs’ is local and depends on the system into which they are embedded. In this regard, the methodological approach to identifying software bugs is illustrated by an automated system for preparing data for aircraft flights. By software bugs, we mean malicious software that can affect the algorithms for the functioning of the system, disrupting the normal mode of its operation and causing significant damage to the goals of the system. To find where software bugs are likely to be embedded, we specified actions which consist in understanding the features of assessing the quality of each of the main elements of the system and the essence of software bugs, with account for the features of the automated data preparation system; describing the system and its specifics; determining the most likely place for embedding software bugs and conditions for their initialization.

Application Threats to Exploit Northbound Interface Vulnerabilities in Software Defined Networks

Software Defined Networking (SDN) is an evolving technology that decouples the control functionality from the underlying hardware managed by the control plane. The application plane supports programmers to develop numerous applications (such as networking, management, security, etc.) that can even be executed from remote locations. Northbound interface (NBI) bridges the control and application planes to execute the third-party applications business logic. Due to the software bugs in applications and existing vulnerabilities such as illegal function calling, resource exhaustion, lack of trust, and so on, NBIs are susceptible to different attacks. Based on the extensive literature review, we have identified that the researchers and academia have mainly focused on the security of the control plane, data plane, and southbound interface (SBI). NBI, in comparison, has received far less attention. In this article, the security of the least explored, but a critical component of the SDN architecture, i.e., NBI, is analyzed. The article provides a brief overview of SDN, followed by a detailed discussion on the categories of NBI, vulnerabilities of NBI, and threats posed by malicious applications to NBI. Efforts of the researchers to counter malicious applications and NBI issues are then discussed in detail. The standardization efforts for the single acceptable NBI and security requirements of SDN by Open Networking Foundation (ONF) are also presented. The article concludes with the future research directions for the security of a single acceptable NBI.