Efficiency and Effectiveness of Web Application Vulnerability Detection Approaches: A Review

Most existing surveys and reviews on web application vulnerability detection (WAVD) approaches focus on comparing and summarizing the approaches’ technical details. Although some studies have analyzed the efficiency and effectiveness of specific methods, there is a lack of a comprehensive and systematic analysis of the efficiency and effectiveness of various WAVD approaches. We conducted a systematic literature review (SLR) of WAVD approaches and analyzed their efficiency and effectiveness. We identified 105 primary studies out of 775 WAVD articles published between January 2008 and June 2019. Our study identified 10 categories of artifacts analyzed by the WAVD approaches and 8 categories of WAVD meta-approaches for analyzing the artifacts. Our study’s results also summarized and compared the effectiveness and efficiency of different WAVD approaches on detecting specific categories of web application vulnerabilities and which web applications and test suites are used to evaluate the WAVD approaches. To our knowledge, this is the first SLR that focuses on summarizing the effectiveness and efficiencies of WAVD approaches. Our study results can help security engineers choose and compare WAVD tools and help researchers identify research gaps.

A Survey of Flaky Tests

Tests that fail inconsistently, without changes to the code under test, are described as flaky . Flaky tests do not give a clear indication of the presence of software bugs and thus limit the reliability of the test suites that contain them. A recent survey of software developers found that 59% claimed to deal with flaky tests on a monthly, weekly, or daily basis. As well as being detrimental to developers, flaky tests have also been shown to limit the applicability of useful techniques in software testing research. In general, one can think of flaky tests as being a threat to the validity of any methodology that assumes the outcome of a test only depends on the source code it covers. In this article, we systematically survey the body of literature relevant to flaky test research, amounting to 76 papers. We split our analysis into four parts: addressing the causes of flaky tests, their costs and consequences, detection strategies, and approaches for their mitigation and repair. Our findings and their implications have consequences for how the software-testing community deals with test flakiness, pertinent to practitioners and of interest to those wanting to familiarize themselves with the research area.

Learning how to search: generating effective test cases through adaptive fitness function selection

Search-based test generation is guided by feedback from one or more fitness functions—scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals—such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage—do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show limited improvements on the third when the number of generations of evolution is fixed. Additionally, for two of the three goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows strategic choices that efficiently produce more effective test suites, and examining these choices offers insight into how to attain our testing goals. We find that adaptive fitness function selection is a powerful technique to apply when an effective fitness function does not already exist for achieving a testing goal.

Hybrid Selection Based Multi/Many-Objective Evolutionary Algorithm

In the last decade, numerous Multi/Many-Objective Evolutionary Algorithms (MOEAs) have been proposed to handle Multi/Many-Objective Problems (MOPs) with challenges such as discontinuous Pareto Front (PF), degenerate PF, etc. MOEAs in the literature can be broadly divided into three categories based on the selection strategy employed such as dominance, decomposition, and indicator-based MOEAs. Each category of MOEAs have their advantages and disadvantages when solving MOPs with diverse characteristics. In this work, we propose a Hybrid Selection based MOEA, referred to as HS-MOEA, which is a simple yet effective hybridization of dominance, decomposition and indicator-based concepts. In other words, we propose a new environmental selection strategy where the Pareto-dominance, reference vectors and an indicator are combined to effectively balance the diversity and convergence properties of MOEA during the evolution. The superior performance of HS-MOEA compared to the state-of-the-art MOEAs is demonstrated through experimental simulations on DTLZ and WFG test suites with up to 10 objectives.

Adaptive Strategies Based on Differential Evolutionary Algorithm for Many-Objective Optimization

The decomposition-based algorithm, for example, multiobjective evolutionary algorithm based on decomposition (MOEA/D), has been proved effective and useful in a variety of multiobjective optimization problems (MOPs). On the basis of MOEA/D, the MOEA/D-DE replaces the simulated binary crossover (SBX) operator with differential evolution (DE) operator, which is used to enhance the diversity of the solutions more effectively. However, the amplification factor and the crossover probability are fixed in MOEA/D-DE, which would lead to a low convergence rate and be more likely to fall into local optimum. To overcome such a prematurity problem, this paper proposes three different adaptive operators in DE with crossover probability and amplification factors to adjust the parameter settings adaptively. We incorporate these three adaptive operators in MOEA/D-DE and MOEA/D-PaS to solve MOPs and many-objective optimization problems (MaOPs), respectively. This paper also designs a sensitive experiment for the changeable parameter η in the proposed adaptive operators to explore how η would affect the convergence of the proposed algorithms. These adaptive algorithms are tested on many benchmark problems, including ZDT, DTLZ, WFG, and MaF test suites. The experimental results illustrate that the three proposed adaptive algorithms have better performance on most benchmark problems.

A Mating Selection Based on Modified Strengthened Dominance Relation for NSGA-III

In multi/many-objective evolutionary algorithms (MOEAs), to alleviate the degraded convergence pressure of Pareto dominance with the increase in the number of objectives, numerous modified dominance relationships were proposed. Recently, the strengthened dominance relation (SDR) has been proposed, where the dominance area of a solution is determined by convergence degree and niche size (θ¯). Later, in controlled SDR (CSDR), θ¯ and an additional parameter (k) associated with the convergence degree are dynamically adjusted depending on the iteration count. Depending on the problem characteristics and the distribution of the current population, different situations require different values of k, rendering the linear reduction of k based on the generation count ineffective. This is because a particular value of k is expected to bias the dominance relationship towards a particular region on the Pareto front (PF). In addition, due to the same reason, using SDR or CSDR in the environmental selection cannot preserve the diversity of solutions required to cover the entire PF. Therefore, we propose an MOEA, referred to as NSGA-III*, where (1) a modified SDR (MSDR)-based mating selection with an adaptive ensemble of parameter k would prioritize parents from specific sections of the PF depending on k, and (2) the traditional weight vector and non-dominated sorting-based environmental selection of NSGA-III would protect the solutions corresponding to the entire PF. The performance of NSGA-III* is favourably compared with state-of-the-art MOEAs on DTLZ and WFG test suites with up to 10 objectives.

An enhanced version of Harris Hawks Optimization by dimension learning-based hunting for Breast Cancer Detection

Swarm intelligence techniques have a vast range of real world applications.Some applications are in the domain of medical data mining where, main attention is on structure models for the classification and expectation of numerous diseases. These biomedical applications have grabbed the interest of numerous researchers because these are most serious and prevalent causes of death among the human whole world out of which breast cancer is the most serious issue. Mammography is the initial screening assessment of breast cancer. In this study, an enhanced version of Harris Hawks Optimization (HHO) approach has been developed for biomedical databases, known as DLHO. This approach has been introduced by integrating the merits of dimension learning-based hunting (DLH) search strategy with HHO. The main objective of this study is to alleviate the lack of crowd diversity, premature convergence of the HHO and the imbalance amid the exploration and exploitation. DLH search strategy utilizes a dissimilar method to paradigm a neighborhood for each search member in which the neighboring information can be shared amid search agents. This strategy helps in maintaining the diversity and the balance amid global and local search. To evaluate the DLHO lot of experiments have been taken such as (i) the performance of optimizers have analysed by using 29-CEC -2017 test suites, (ii) to demonstrate the effectiveness of the DLHO it has been tested on different biomedical databases out of which we have used two different databases for Breast i.e. MIAS and second database has been taken from the University of California at Irvine (UCI) Machine Learning Repository.Also to test the robustness of the proposed method its been tested on two other databases of such as Balloon and Heart taken from the UCI Machine Learning Repository. All the results are in the favour of the proposed technique.

Applying test case prioritization to software microbenchmarks

Regression testing comprises techniques which are applied during software evolution to uncover faults effectively and efficiently. While regression testing is widely studied for functional tests, performance regression testing, e.g., with software microbenchmarks, is hardly investigated. Applying test case prioritization (TCP), a regression testing technique, to software microbenchmarks may help capturing large performance regressions sooner upon new versions. This may especially be beneficial for microbenchmark suites, because they take considerably longer to execute than unit test suites. However, it is unclear whether traditional unit testing TCP techniques work equally well for software microbenchmarks. In this paper, we empirically study coverage-based TCP techniques, employing total and additional greedy strategies, applied to software microbenchmarks along multiple parameterization dimensions, leading to 54 unique technique instantiations. We find that TCP techniques have a mean APFD-P (average percentage of fault-detection on performance) effectiveness between 0.54 and 0.71 and are able to capture the three largest performance changes after executing 29% to 66% of the whole microbenchmark suite. Our efficiency analysis reveals that the runtime overhead of TCP varies considerably depending on the exact parameterization. The most effective technique has an overhead of 11% of the total microbenchmark suite execution time, making TCP a viable option for performance regression testing. The results demonstrate that the total strategy is superior to the additional strategy. Finally, dynamic-coverage techniques should be favored over static-coverage techniques due to their acceptable analysis overhead; however, in settings where the time for prioritzation is limited, static-coverage techniques provide an attractive alternative.

Into the Noddyverse: A massive data store of 3D geological models for Machine Learning & inversion applications

Unlike some other well-known challenges such as facial recognition, where Machine Learning and Inversion algorithms are widely developed, the geosciences suffer from a lack of large, labelled datasets that can be used to validate or train robust Machine Learning and inversion schemes. Publicly available 3D geological models are far too restricted in both number and the range of geological scenarios to serve these purposes. With reference to inverting geophysical data this problem is further exacerbated as in most cases real geophysical observations result from unknown 3D geology, and synthetic test datasets are often not particularly geological, nor geologically diverse. To overcome these limitations, we have used the Noddy modelling platform to generate one million models, which represent the first publicly accessible massive training set for 3D geology and resulting gravity and magnetic datasets. This model suite can be used to train Machine Learning systems, and to provide comprehensive test suites for geophysical inversion. We describe the methodology for producing the model suite, and discuss the opportunities such a model suit affords, as well as its limitations, and how we can grow and access this resource.