Multi-objective Regression Test Suite Minimisation for Mockito

Author(s):  
Andrew J. Turner ◽  
David R. White ◽  
John H. Drake

Regression testing is one of the most critical testing activities among software product verification activities. Nevertheless, resources and time constraints could inhibit the execution of a full regression test suite, hence leaving us in confusion on what test cases to run to preserve the high quality of software products. Different techniques can be applied to prioritize test cases in resource-constrained environments, such as manual selection, automated selection, or hybrid approaches. Different Multi-Objective Evolutionary Algorithms (MOEAs) have been used in this domain to find an optimal solution to minimize the cost of executing a regression test suite while obtaining maximum fault detection coverage as if the entire test suite was executed. MOEAs achieve this by selecting set of test cases and determining the order of their execution. In this paper, three Multi Objective Evolutionary Algorithms, namely, NSGA-II, IBEA and MoCell are used to solve test case prioritization problems using the fault detection rate and branch coverage of each test case. The paper intends to find out what’s the most effective algorithm to be used in test cases prioritization problems, and which algorithm is the most efficient one, and finally we examined if changing the fitness function would impose a change in results. Our experiment revealed that NSGA-II is the most effective and efficient MOEA; moreover, we found that changing the fitness function caused a significant reduction in evolution time, although it did not affect the coverage metric.


PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0242708
Author(s):  
Ayesha Kiran ◽  
Wasi Haider Butt ◽  
Arslan Shaukat ◽  
Muhammad Umar Farooq ◽  
Urooj Fatima ◽  
...  

In the process of software development, regression testing is one of the major activities that is done after making modifications in the current system or whenever a software system evolves. But, the test suite size increases with the addition of new test cases and it becomes in-efficient because of the occurrence of redundant, broken, and obsolete test cases. For that reason, it results in additional time and budget to run all these test cases. Many researchers have proposed computational intelligence and conventional approaches for dealing with this problem and they have achieved an optimized test suite by selecting, minimizing or reducing, and prioritizing test cases. Currently, most of these optimization approaches are single objective and static in nature. But, it is mandatory to use multi-objective dynamic approaches for optimization due to the advancements in information technology and associated market challenges. Therefore, we have proposed three variants of self-tunable Adaptive Neuro-fuzzy Inference System i.e. TLBO-ANFIS, FA-ANFIS, and HS-ANFIS, for multi-objective regression test suites optimization. Two benchmark test suites are used for evaluating the proposed ANFIS variants. The performance of proposed ANFIS variants is measured using Standard Deviation and Root Mean Square Error. A comparison of experimental results is also done with six existing methods i.e. GA-ANFIS, PSO-ANFIS, MOGA, NSGA-II, MOPSO, and TOPSIS and it is concluded that the proposed method effectively reduces the size of regression test suite without a reduction in the fault detection rate.


2016 ◽  
Vol 25 (2) ◽  
pp. 123-146 ◽  
Author(s):  
Zeeshan Anwar ◽  
Ali Ahsan ◽  
Cagatay Catal

AbstractRegression testing is a type of testing activity, which ensures that source code changes do not affect the unmodified portions of the software adversely. This testing activity may be very expensive in, some cases, due to the required time to execute the test suite. In order to execute the regression tests in a cost-effective manner, the optimization of regression test suite is crucial. This optimization can be achieved by applying test suite reduction (TSR), regression test selection (RTS), or test case prioritization (TCP) techniques. In this paper, we designed and implemented an expert system for TSR problem by using neuro-fuzzy modeling-based approaches known as “adaptive neuro-fuzzy inference system with grid partitioning” (ANFIS-GP) and “adaptive neuro-fuzzy inference system with subtractive clustering” (ANFIS-SC). Two case studies were performed to validate the model and fuzzy logic, multi-objective genetic algorithms (MOGAs), non-dominated sorting genetic algorithm (NSGA-II) and multi-objective particle swarm optimization (MOPSO) algorithms were used for benchmarking. The performance of the models were evaluated in terms of reduction of test suite size, reduction in fault detection rate, reduction in test suite execution time, and reduction in requirement coverage. The experimental results showed that our ANFIS-based optimization system is very effective to optimize the regression test suite and provides better performance than the other approaches evaluated in this study. Size and execution time of the test suite is reduced up to 50%, whereas loss in fault detection rate is between 0% and 25%.


Author(s):  
Sarika Sharma ◽  
Deepak Kumar

Objective: From the literature review, it is evident that the concept of “regression testing” inherited in agile software testing originates from software maintenance practices. Therefore, the existing algorithms for regression testing revolve around the software maintenance principles rather than agile methodology. The objective of this paper is to evaluate the degree of fitness of the existing regression test-suite development algorithms for performing the regression testing in agile. Methods: This paper performs a systematic literature review for research work published from 2006 to 2018, which includes survey of the existing regression testing algorithms to identify and overcome the challenges associated with them while performing regression testing in agile. This research paper considers the four research questions into scope for analyzing the fitness of existing regression test-suite development algorithm for performing regression testing under agile methodology. Further, this paper attempts to propose approach for the development of the regression test-suite suitable for regression testing under agile methodology. Results: The current regression test-suite development algorithm were found unsuitable for performing the regression testing under agile methodology due to the newly identified four key challenges associated with them. Conclusion: The current regression test-suite development algorithms aligned with software maintenance principles rather than agile methodology. In addition, the newly proposed approach for regression test-suite development found to be easily adaptable by agile teams as it aligns with agile methodology principles. Finally, this paper recommends the adoption of agile principle through the newly proposed approach for developing regression test-suite for performing regression testing under agile methodology.


Algorithms ◽  
2019 ◽  
Vol 12 (11) ◽  
pp. 244 ◽  
Author(s):  
Chunling Ye ◽  
Zhengyan Mao ◽  
Mandan Liu

Inspired by the mechanism of generation and restriction among five elements in Chinese traditional culture, we present a novel Multi-Objective Five-Elements Cycle Optimization algorithm (MOFECO). During the optimization process of MOFECO, we use individuals to represent the elements. At each iteration, we first divide the population into several cycles, each of which contains several individuals. Secondly, for every individual in each cycle, we judge whether to update it according to the force exerted on it by other individuals in the cycle. In the case of an update, a local or global update is selected by a dynamically adjustable probability P s ; otherwise, the individual is retained. Next, we perform combined mutation operations on the updated individuals, so that a new population contains both the reserved and updated individuals for the selection operation. Finally, the fast non-dominated sorting method is adopted on the current population to obtain an optimal Pareto solution set. The parameters’ comparison of MOFECO is given by an experiment and also the performance of MOFECO is compared with three classic evolutionary algorithms Non-dominated Sorting Genetic Algorithm II (NSGA-II), Multi-Objective Particle Swarm Optimization algorithm (MOPSO), Pareto Envelope-based Selection Algorithm II (PESA-II) and two latest algorithms Knee point-driven Evolutionary Algorithm (KnEA) and Non-dominated Sorting and Local Search (NSLS) on solving test function sets Zitzler et al’s Test suite (ZDT), Deb et al’s Test suite (DTLZ), Walking Fish Group (WFG) and Many objective Function (MaF). The experimental results indicate that the proposed MOFECO can approach the true Pareto-optimal front with both better diversity and convergence compared to the five other algorithms.


Sign in / Sign up

Export Citation Format

Share Document