Comparison of Multi-Objective Evolutionary Algorithms to Prioritize Regression Test Cases

2021 ◽  
Vol 9 (12) ◽  
pp. 1445-1454
Keyword(s):  
Fault Detection ◽  
Evolutionary Algorithms ◽  
Fitness Function ◽  
Optimal Solution ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Nsga Ii ◽  
Multi Objective ◽  
Regression Test

Regression testing is one of the most critical testing activities among software product verification activities. Nevertheless, resources and time constraints could inhibit the execution of a full regression test suite, hence leaving us in confusion on what test cases to run to preserve the high quality of software products. Different techniques can be applied to prioritize test cases in resource-constrained environments, such as manual selection, automated selection, or hybrid approaches. Different Multi-Objective Evolutionary Algorithms (MOEAs) have been used in this domain to find an optimal solution to minimize the cost of executing a regression test suite while obtaining maximum fault detection coverage as if the entire test suite was executed. MOEAs achieve this by selecting set of test cases and determining the order of their execution. In this paper, three Multi Objective Evolutionary Algorithms, namely, NSGA-II, IBEA and MoCell are used to solve test case prioritization problems using the fault detection rate and branch coverage of each test case. The paper intends to find out what’s the most effective algorithm to be used in test cases prioritization problems, and which algorithm is the most efficient one, and finally we examined if changing the fitness function would impose a change in results. Our experiment revealed that NSGA-II is the most effective and efficient MOEA; moreover, we found that changing the fitness function caused a significant reduction in evolution time, although it did not affect the coverage metric.

scholarly journals Efficiency Enhancement in Regression Test Case Prioritization Technique

2019 ◽  
Vol 8 (12) ◽  
pp. 5445-5451
Keyword(s):  
Fault Detection ◽  
Software Quality ◽  
Detection Rate ◽  
Regression Testing ◽  
Test Case ◽  
Test Cases ◽  
Efficiency Enhancement ◽  
Average Percentage ◽  
Regression Test ◽  
Expensive Process

Regression testing is an important, but expensive, process that has a powerful impact on software quality. Unfortunately all the test cases, existing and newly added, cannot be re-executed due to insufficient resources. In this scenario, prioritization of test case helps in improving the efficacy of regression testing by arranging the test cases in such a way that the most beneficial (that has the potential to detect the more number of faults) are executed first. Previous work and existing prioritization techniques, though detects faults, but there is a need of improved techniques to enhance the process of regression testing by improving the fault detection rate. The new technique, proposed in this paper, gives improved result than the existing ones. The comparison of the effectiveness of the proposed approach is done with other prioritization and non-prioritization orderings. The result of the proposed approach shows higher average percentage of faults detected (APFD) values. Also, the performance is evaluated and it is observed that the capability of the proposed method outperforms other algorithms by enhancing the fault detection rate.

Multi- Objective Crow Search and Fruit Fly Optimization for Combinatorial Test Case Prioritization

2021 ◽  
Vol 9 (4) ◽  
pp. 0-0
Keyword(s):  
Performance Metrics ◽  
Search Algorithm ◽  
Fitness Function ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Optimal Test ◽  
Fruit Fly Optimization ◽  
Average Percentage ◽  
Multi Objective

This paper proposes a novel test case prioritization technique, namely Multi- Objective Crow Search and Fruitfly Optimization (MOCSFO) for test case prioritization. The proposed MOCSFO is designed by integrating Crow search algorithm (CSA) and Chaotic Fruitfly optimization algorithm (CFOA). The optimal test cases are selected based on newly modelled fitness function, which consist of two parameters, namely average percentage of combinatorial coverage (APCC) and Normalized average of the percentage of faults detected (NAPFD). The test case to be selected is decided using a searching criterion or fitness based on sequential weighed coverage size. Accordingly, the effective searching criterion is formulated to determine the optimal test cases based on the constraints. The experimentation of the proposed MOCSFO method is performed by considering the performance metrics, like NAPFD, and APCC. The proposed MOCSFO outperformed the existing methods with enhanced NAPFD of 0.7, and APCC of 0.837.

Functional Test Case Generator for Contiki Operating System and Cooja Simulator

2019 ◽  
Vol 10 (1) ◽  
pp. 1251-1257
Author(s):  
Abhinandan H Patil
Keyword(s):  
Operating System ◽  
Functional Test ◽  
Test Suite ◽  
Target System ◽  
Test Case ◽  
Test Cases ◽  
Code Coverage ◽  
Regression Test ◽  
Run Time ◽  
Configuration Systems

Evolving multi-parameter, multi-configuration systems require regression test suite that can be customized. This is in terms of run time. Run time can be customized by generating the combinations using combinatorial techniques. For systems like Contiki operating system, the test cases need to be executed in its simulator Cooja. Executing test cases in a simulator requires functional test cases to be generated from the combinatorial parameter combinations obtained. In this work we present a methodology to generate the functional test cases. We present Functional Test Case Generator for Contiki and Cooja (FTCGCC), which is a tool developed using our methodology. We demonstrate use of our tool by generating customizable regression test suite for Contiki and Cooja using code coverage as criteria. FTCGCC is developed for the test case generation when target System Under Test is IoT operating system Contiki and its simulator Cooja.

scholarly journals Multi-objective regression test suite optimization using three variants of adaptive neuro fuzzy inference system

PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0242708
Author(s):  
Ayesha Kiran ◽  
Wasi Haider Butt ◽  
Arslan Shaukat ◽  
Muhammad Umar Farooq ◽  
Urooj Fatima ◽  
...  
Keyword(s):  
Fuzzy Inference System ◽  
Fuzzy Inference ◽  
Current System ◽  
Test Suite ◽  
Test Cases ◽  
Inference System ◽  
Multi Objective ◽  
Neuro Fuzzy ◽  
Regression Test ◽  
Test Suites

In the process of software development, regression testing is one of the major activities that is done after making modifications in the current system or whenever a software system evolves. But, the test suite size increases with the addition of new test cases and it becomes in-efficient because of the occurrence of redundant, broken, and obsolete test cases. For that reason, it results in additional time and budget to run all these test cases. Many researchers have proposed computational intelligence and conventional approaches for dealing with this problem and they have achieved an optimized test suite by selecting, minimizing or reducing, and prioritizing test cases. Currently, most of these optimization approaches are single objective and static in nature. But, it is mandatory to use multi-objective dynamic approaches for optimization due to the advancements in information technology and associated market challenges. Therefore, we have proposed three variants of self-tunable Adaptive Neuro-fuzzy Inference System i.e. TLBO-ANFIS, FA-ANFIS, and HS-ANFIS, for multi-objective regression test suites optimization. Two benchmark test suites are used for evaluating the proposed ANFIS variants. The performance of proposed ANFIS variants is measured using Standard Deviation and Root Mean Square Error. A comparison of experimental results is also done with six existing methods i.e. GA-ANFIS, PSO-ANFIS, MOGA, NSGA-II, MOPSO, and TOPSIS and it is concluded that the proposed method effectively reduces the size of regression test suite without a reduction in the fault detection rate.

Hybrid Regression Testing Based on Path Pruning

2015 ◽  
Vol 5 (1) ◽  
pp. 35-55 ◽  
Author(s):  
Varun Gupta ◽  
Durg Singh Chauhan ◽  
Kamlesh Dutta
Keyword(s):  
Fault Detection ◽  
Regression Testing ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Testing Time ◽  
Ripple Effect ◽  
Web Based ◽  
Testing Approach ◽  
Globally Distributed

Regression testing has been studied by various researchers for developing and testing the quality of software. Regression testing aims at re-execution of evolved software code to ensure that no new errors had been introduced during the process of modification. Since re-execution of all test cases is not feasible, selecting manageable number of test cases to execute modified code with good fault detection rate is a problem. In past few years, various hybrid based regression testing approaches have been proposed and successfully employed for software testing, aiming at reduction in the number of test cases and higher fault detection capabilities. These techniques are based on sequence of selections, prioritizations and minimization of test suite. However, these techniques suffer from major drawbacks like improper consideration of control dependencies, neglection of unaffected fragments of code for testing purpose. Further, these techniques have been employed on hypothetical or simple programs with test suites of smaller size. Present paper proposes hybrid regression testing, a combination of test case selections, test case prioritizations and test suite minimization. The technique works at statement level and is based on finding the paths containing statements that affects or gets affected by the addition/deletion or modification (both control and data dependency) of variables in statements. The modification in the code may cause ripple effect thereby resulting into faulty execution of the code. The hybrid regression testing approach is aimed at detecting such faults with lesser number of test cases. Reduction in number of test cases is possible because of the decreased number of paths to be tested. A web based framework to automate and parallelize this testing technique to maximum extend, making it well suited for globally distributed environments is also proposed in the present paper. Framework when implemented as a tool can handle large pool of test cases and will make use of parallel MIMD architectures like multicore systems. Technique is applied on prototype live system and results are compared with recently proposed hybrid regression testing approach against parameters of interest. Obtained optimized results are indicators of effectiveness of approach in terms of reduction in effort, cost as well as testing time in general and increment delivery time in particular.

TEST AGENT SYSTEM

2002 ◽  
Vol 12 (03) ◽  
pp. 269-290 ◽  
Author(s):  
JEONGEUN CHOI ◽  
BYOUNGJU CHOI
Keyword(s):  
Fault Detection ◽  
Test Case ◽  
Test Cases ◽  
Testing Time ◽  
Agent System ◽  
Social Ability ◽  
Test Agent ◽  
Regression Test ◽  
Agent Characteristics ◽  
Active Assistance

In this paper, we propose a Test Agent System (TAS) that employs intelligent agent characteristics of autonomy, social ability, and intelligence to provide active assistance to the tester. It consists of the 'User Interface Agent', the 'Test Case Selection & Testing Agent', and the 'Regression Test Agent'. These agents carry out the tests autonomously by employing their intelligent rules in the object-oriented test processes. This system has 2 advantages. Firstly, since the tests are carried out autonomously, it minimizes tester interference and secondly, since redundant-free and consistent effective test cases are intellectually selected, the testing time is reduced while the fault detection effectiveness improves. In this paper, we show the cut-down rate on the testing time and improvement in the fault detection effectiveness through experiments.

scholarly journals A Slice-Based Change Impact Analysis for Regression Test Case Prioritization of Object-Oriented Programs

2016 ◽  
Vol 2016 ◽  
pp. 1-20 ◽  
Author(s):  
S. Panda ◽  
D. Munjal ◽  
D. P. Mohapatra
Keyword(s):  
Impact Analysis ◽  
Object Oriented ◽  
Test Suite ◽  
Optimal Order ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Regression Test ◽  
Static Approach ◽  
Change Impact

Test case prioritization focuses on finding a suitable order of execution of the test cases in a test suite to meet some performance goals like detecting faults early. It is likely that some test cases execute the program parts that are more prone to errors and will detect more errors if executed early during the testing process. Finding an optimal order of execution for the selected regression test cases saves time and cost of retesting. This paper presents a static approach to prioritizing the test cases by computing the affected component coupling (ACC) of the affected parts of object-oriented programs. We construct a graph named affected slice graph (ASG) to represent these affected program parts. We determine the fault-proneness of the nodes of ASG by computing their respective ACC values. We assign higher priority to those test cases that cover the nodes with higher ACC values. Our analysis with mutation faults shows that the test cases executing the fault-prone program parts have a higher chance to reveal faults earlier than other test cases in the test suite. The result obtained from seven case studies justifies that our approach is feasible and gives acceptable performance in comparison to some existing techniques.

Neuro-Fuzzy Modeling for Multi-Objective Test Suite Optimization

2016 ◽  
Vol 25 (2) ◽  
pp. 123-146 ◽  
Author(s):  
Zeeshan Anwar ◽  
Ali Ahsan ◽  
Cagatay Catal
Keyword(s):  
Fault Detection ◽  
Fuzzy Inference System ◽  
Execution Time ◽  
Detection Rate ◽  
Fuzzy Inference ◽  
Test Suite ◽  
Inference System ◽  
Multi Objective ◽  
Neuro Fuzzy ◽  
Regression Test

AbstractRegression testing is a type of testing activity, which ensures that source code changes do not affect the unmodified portions of the software adversely. This testing activity may be very expensive in, some cases, due to the required time to execute the test suite. In order to execute the regression tests in a cost-effective manner, the optimization of regression test suite is crucial. This optimization can be achieved by applying test suite reduction (TSR), regression test selection (RTS), or test case prioritization (TCP) techniques. In this paper, we designed and implemented an expert system for TSR problem by using neuro-fuzzy modeling-based approaches known as “adaptive neuro-fuzzy inference system with grid partitioning” (ANFIS-GP) and “adaptive neuro-fuzzy inference system with subtractive clustering” (ANFIS-SC). Two case studies were performed to validate the model and fuzzy logic, multi-objective genetic algorithms (MOGAs), non-dominated sorting genetic algorithm (NSGA-II) and multi-objective particle swarm optimization (MOPSO) algorithms were used for benchmarking. The performance of the models were evaluated in terms of reduction of test suite size, reduction in fault detection rate, reduction in test suite execution time, and reduction in requirement coverage. The experimental results showed that our ANFIS-based optimization system is very effective to optimize the regression test suite and provides better performance than the other approaches evaluated in this study. Size and execution time of the test suite is reduced up to 50%, whereas loss in fault detection rate is between 0% and 25%.

scholarly journals A Study of Regression Testing for Trade me Website

2021 ◽  
Author(s):  
Kenil Manishkumar Patel ◽  
Shahid Ali
Keyword(s):  
New Zealand ◽  
Fault Detection ◽  
Software Development ◽  
Critical Role ◽  
Regression Testing ◽  
Test Suite ◽  
Test Cases ◽  
Regression Test ◽  
Manual Testing ◽  
Research Analysis

Regression testing plays a critical role to verify the functionality testing of a product. Trade Me is New Zealand based website. It is one of the major websites in New Zealand dealing in buying and selling online. The aim of this research is to find out the functionalities of Trade Me website after injecting new features. Automation regression suite is used to execute test scripts which helped the company to save time and cost compared to manual testing. Automation regression test suite also helped to prioritize test cases are designed in such a way that it can maximize the fault detection. For research analysis scrum methodology is used to meet the ultimate desires of software development companies and to increase the client satisfaction.

A Multi-Objective Approach for Test Suite Reduction During Testing of Web Applications

2021 ◽  
Vol 12 (3) ◽  
pp. 81-122
Author(s):  
Munish Khanna ◽  
Naresh Chauhan ◽  
Dilip Kumar Sharma ◽  
Law Kumar Singh
Keyword(s):  
Genetic Algorithm ◽  
Web Applications ◽  
State Of The Art ◽  
Test Suite ◽  
Test Cases ◽  
Nsga Ii ◽  
Multi Objective ◽  
Reduction Problem ◽  
Test Suite Reduction ◽  
Experimental Findings

During the development and maintenance phases of evolving software, new test cases would be needed for the verification of the accuracy of the modifications as well as for new functionalities leading to an increase in the size of the test suite. Various related objectives are to be kept in mind while reducing the original test suite by removing redundancy and generating a practical representative set of the unique test cases, some of which may need to be maximized and the remaining ones minimized. This paper presents a multi-objective approach for the test suite reduction problem in which one objective is to be minimized and the remaining two maximized. In this study, experiments were performed on diverse versions of four web applications. Various state-of-the-art algorithms and their updated versions were compared with non-dominated sorting genetic algorithm-II (NSGA-II) for performance evaluation. Based on experimental findings, it was concluded that NSGA-II outperforms all other algorithms; moreover, the algorithm attempts to satisfy all the objectives without compromising coverage.

Sign in / Sign up

Export Citation Format

Share Document