Parallel Test Prioritization

2022 ◽  
Vol 31 (1) ◽  
pp. 1-50
Author(s):  
Jianyi Zhou ◽  
Junjie Chen ◽  
Dan Hao
Keyword(s):  
Extensive Study ◽  
Good Choice ◽  
Regression Testing ◽  
Sequential Test ◽  
Testing Time ◽  
Parallel Test ◽  
Parallel Testing ◽  
Test Execution ◽  
Coverage Criterion ◽  
Test Prioritization

Although regression testing is important to guarantee the software quality in software evolution, it suffers from the widely known cost problem. To address this problem, existing researchers made dedicated efforts on test prioritization, which optimizes the execution order of tests to detect faults earlier; while practitioners in industry leveraged more computing resources to save the time cost of regression testing. By combining these two orthogonal solutions, in this article, we define the problem of parallel test prioritization, which is to conduct test prioritization in the scenario of parallel test execution to reduce the cost of regression testing. Different from traditional sequential test prioritization, parallel test prioritization aims at generating a set of test sequences, each of which is allocated in an individual computing resource and executed in parallel. In particular, we propose eight parallel test prioritization techniques by adapting the existing four sequential test prioritization techniques, by including and excluding testing time in prioritization. To investigate the performance of the eight parallel test prioritization techniques, we conducted an extensive study on 54 open-source projects and a case study on 16 commercial projects from Baidu , a famous search service provider with 600M monthly active users. According to the two studies, parallel test prioritization does improve the efficiency of regression testing, and cost-aware additional parallel test prioritization technique significantly outperforms the other techniques, indicating that this technique is a good choice for practical parallel testing. Besides, we also investigated the influence of two external factors, the number of computing resources and time allowed for parallel testing, and find that more computing resources indeed improve the performance of parallel test prioritization. In addition, we investigated the influence of two more factors, test granularity and coverage criterion, and find that parallel test prioritization can still accelerate regression testing in parallel scenario. Moreover, we investigated the benefit of parallel test prioritization on the regression testing process of continuous integration, considering both the cumulative acceleration performance and the overhead of prioritization techniques, and the results demonstrate the superiority of parallel test prioritization.

A Microservice Regression Testing Selection Approach Based on Belief Propagation

2021 ◽  
Author(s):  
Lizhe Chen ◽  
Ji Wu ◽  
Haiyan Yang ◽  
Kui Zhang
Keyword(s):  
Directed Graph ◽  
Belief Propagation ◽  
Regression Testing ◽  
Change Propagation ◽  
Test Cases ◽  
Testing Time ◽  
Dependency Matrix ◽  
System Models ◽  
Selection Approach ◽  
Service Change

Abstract Regression testing is required in each iteration of microservice systems. Regression testing selection, which reduces testing costs by selecting a subset from the original test cases, is one of the main techniques to optimize regression testing. Existing techniques mainly rely on the information retrieved from artifacts such as code files and system models. For microservice systems with service autonomy, development method diversity and a large amount of services, such artifacts are too difficultly obtained and costly processed to apply those approaches. This paper presents a regression testing selection approach called MRTS-BP, which needs the API gateway layer logs instead of code files and system models as inputs. By parsing the API gateway layer logs, our approach establishes the service dependency matrix, which in further is transformed into a directed graph with the services as nodes. Then, to find out which test cases are affected by service changes, an algorithm based on belief propagation is presented to compute the quantitative results of service-change propagation from the directed graph. Finally, the relationships between original test cases and service-change propagation results are established to select test cases with three strategies. To evaluate the efficiency of MRTS-BP, the empirical study based on four microservice systems is presented. A typical technique RTS-CFG is compared with MRTS-CFG and four experiments are setup to investigate four research questions. The results show that MRTS-BP can not only reduce the number of test cases by half compared with the retest-all strategy while ensuring the safety, but also save at least 20% testing time costs more than that of RTS-CFG. MRTS-BP is more practical than the techniques relying on the artifacts when the latter cannot be implemented due to the artifacts are difficult to obtain and process.

A Synchronized Test Control Execution Model of Distributed Systems

2021 ◽  
pp. 1377-1395
Author(s):  
Salma Azzouzi ◽  
Sara Hsaini ◽  
My El Hassan Charaf
Keyword(s):  
Test Cases ◽  
Agent Based ◽  
Parallel Testing ◽  
Test Execution ◽  
Support Interaction ◽  
Distributed Testing ◽  
Implementation Under Test ◽  
Execution Model ◽  
Local Test ◽  
Control And Synchronization

Conformance testing may be seen as mean to execute an IUT (implementation under test) by carrying out test cases in order to observe whether the behavior of the IUT is conforming to its specifications. However, the development of distributed testing frameworks is more complex and the implementation of the parallel testing components (PTCs) should take into consideration the mechanisms and functions required to support interaction during PTC communication. In this article, the authors present another way to control the test execution of PTCs by introducing synchronization messages into the local test sequences. Then, they suggest an agent-based simulation to implement synchronized local test sequences and resolve the problem of control and synchronization.

scholarly journals Two-Dimensional Software Defect Models with Test Execution History

2012 ◽  
Vol 2012 ◽  
pp. 1-12 ◽  
Author(s):  
Tomotaka Ishii ◽  
Tadashi Dohi
Keyword(s):  
Software Testing ◽  
Time Scales ◽  
Software Reliability ◽  
Execution Time ◽  
Testing Time ◽  
Two Dimensional ◽  
Time Data ◽  
Test Execution ◽  
Software Defect ◽  
Software Reliability Models

In general, the software-testing time may be measured by two kinds of time scales: calendar time and test execution time. In this paper, we develop two-dimensional software reliability models with two-time measures and incorporate both of them to assess the software reliability with higher accuracy. Since the resulting software defect models are based on the familiar nonhomogeneous Poisson processes with two time scales, which are the natural extensions of one-dimensional software defect models, it is possible to treat the time data both simultaneously and effectively. We investigate the dependence of test-execution time as a testing effort on the software reliability assessment and validate quantitatively the software defect models with two-time scales. We also consider an optimization problem when to stop the software testing in terms of two-time measurements.

Image-Based Approach to Determining Regression Test Results of Dynamic Web Applications

2018 ◽  
Vol 28 (07) ◽  
pp. 1001-1025 ◽  
Author(s):  
Akihiro Hori ◽  
Shingo Takada ◽  
Toshiyuki Kurabayashi ◽  
Haruto Tanno
Keyword(s):  
Web Application ◽  
Web Applications ◽  
Regression Testing ◽  
Test Case ◽  
Test Cases ◽  
Test Results ◽  
Test Execution ◽  
Dynamic Web ◽  
Regression Test ◽  
Post Modification

Much work has been done on automating regression testing for applications. But most of them focus on test execution. Little work has been done on automatically determining if a test case passes or fails. This decision is often made by comparing the results of executing test cases on a base version of the application and post-modification version of the application. If the two results match, the test case passes, otherwise fails. However, to the best of our knowledge, there is no regression testing method for automatically deciding pass/fail of dynamic Web applications which use JavaScript or CSS. We propose a method that automatically decides if a dynamic Web application passes a regression test case. The basic idea is to obtain a screenshot each time the GUI of the Web application (i.e. Web page) changes its state, and then compare each corresponding screenshot to see if they match. The evaluation results showed that the accuracy rate of our approach is high and our approach can be considered as fast enough for practical use.

scholarly journals The Application Of Machine Learning In Test Case Prioritization - A Review

2020 ◽  
Vol 4 (1) ◽  
Author(s):  
Elinda Kajo Mece ◽  
Kleona Binjaku ◽  
Hakik Paci
Keyword(s):  
Machine Learning ◽  
Regression Testing ◽  
Machine Learning Techniques ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Testing Time ◽  
Engineering Software ◽  
Learning Techniques ◽  
Learning Machine

Regression testing is very important but also a very costly and time-consuming activity that ensures the developers that changes in the application will not bring new errors. Retest all, selection of test cases and prioritization of test cases (TCP)  approaches are used to enhance the efficiency and effectiveness in regression testing. While test case selection techniques decrease testing time and cost, it can exclude some critical test cases that can detect the faults. On the other hand, test case prioritization considers all test cases and execute them until resources are exhausted or all test cases are executed, while always focusing on the most important ones. Over the years, machine learning has found wide usage in solving different problems in software engineering. Software development and maintenance problems can be defined as learning problems and machine learning techniques have shown to be very effective in solving these problems. In the range of application of machine learning, machine learning techniques have also found usage in solving the test case prioritization problem. In this paper, we investigate the application of machine learning techniques in test case prioritization. We survey some of the most recent studies made in this field and provide information like techniques of machine learning used in TCP process, metrics used to measure the effectiveness of the proposed methods, data used to define the priority of test cases and some advantages or limitations of application of machine learning in TCP.

STUDI ALGORITMA babelRTS UNTUK PELAKSANAAN REGRESSION TEST SELECTION (RST)

2020 ◽  
Vol 3 (2) ◽  
Author(s):  
Ani - Rahmani
Keyword(s):  
Execution Time ◽  
Regression Testing ◽  
Test Case ◽  
Test Cases ◽  
Test Selection ◽  
Test Execution ◽  
Regression Test Selection ◽  
Regression Test ◽  
Long Time ◽  
Testing Techniques

Software testing (testing) is a crucial stage in software development. The success of the testing process will ensure the quality of the software. In the regression testing process, one issue is that not all test cases (retest all) in the test suite need to be executed. Retest all will consume massive resources, as well as a long time. Regression testing techniques seek to find ways to reduce test execution time. One of the regression testing techniques is test case selection, also known as regression test selection (RTS). This paper describes a study on babelRTS, an RTS algorithm, to see its effectiveness. Effectiveness is measured by comparing the execution time of the execution retest all and babelRTS. Experiments were carried out on five software under tests (SUT) that had some faults. Test cases are prepared by designing for each SUT. The results showed a reduction in time so that the effectiveness reached a maximum of 32%, and average of 23% .

Sign in / Sign up

Export Citation Format

Share Document