A Network-Based Approach for Reducing Test Suites While Maintaining Code Coverage

2019 ◽  
pp. 164-176 ◽  
Author(s):  
Misael Mongiovì ◽  
Andrea Fornaia ◽  
Emiliano Tramontana
Keyword(s):  
Code Coverage ◽  
Test Suites

scholarly journals REDUNET: reducing test suites by integrating set cover and network-based optimization

2020 ◽  
Vol 5 (1) ◽  
Author(s):  
Misael Mongiovì ◽  
Andrea Fornaia ◽  
Emiliano Tramontana
Keyword(s):  
Linear Programming ◽  
Integer Linear Programming ◽  
Automatic Generation ◽  
Control Flow ◽  
Test Suite ◽  
Set Cover ◽  
Software Systems ◽  
Code Coverage ◽  
Test Suite Reduction ◽  
Test Suites

Abstract The availability of effective test suites is critical for the development and maintenance of reliable software systems. To increase test effectiveness, software developers tend to employ larger and larger test suites. The recent availability of software tools for automatic test generation makes building large test suites affordable, therefore contributing to accelerating this trend. However, large test suites, though more effective, are resources and time consuming and therefore cannot be executed frequently. Reducing them without decreasing code coverage is a needed compromise between efficiency and effectiveness of the test, hence enabling a more regular check of the software under development. We propose a novel approach, namely REDUNET, to reduce a test suite while keeping the same code coverage. We integrate this approach in a complete framework for the automatic generation of efficient and effective test suites, which includes test suite generation, code coverage analysis, and test suite reduction. Our approach formulates the test suite reduction as a set cover problem and applies integer linear programming and a network-based optimisation, which takes advantage of the properties of the control flow graph. We find the optimal set of test cases that keeps the same code coverage in fractions of seconds on real software projects and test suites generated automatically by Randoop. The results on ten real software systems show that the proposed approach finds the optimal minimisation and achieves up to 90% reduction and more than 50% reduction on all systems under analysis. On the largest project our reduction algorithm performs more than three times faster than both integer linear programming alone and the state-of-the-art heuristic Harrold Gupta Soffa.

Investigating faults missed by test suites achieving high code coverage

2018 ◽  
Vol 144 ◽  
pp. 106-120 ◽  
Author(s):  
Amanda Schwartz ◽  
Daniel Puckett ◽  
Ying Meng ◽  
Gregory Gay
Keyword(s):  
Code Coverage ◽  
Test Suites

scholarly journals Test suites effectiveness evolution in open source systems: empirical study

2020 ◽  
Vol 19 (2) ◽  
pp. 992 ◽  
Author(s):  
Mohammed Akour ◽  
Mamdouh Alenezi
Keyword(s):  
Open Source ◽  
Test Suite ◽  
Software Systems ◽  
Test Size ◽  
Code Coverage ◽  
Java Programs ◽  
The Relationship ◽  
Open Source Systems ◽  
Test Suites ◽  
Real Test

<span>Test suite code coverage is usually used to indicate the capability of a test suite in detecting faults. Earlier research studies, which explored the relationship among test suite effectiveness and code coverage, have not addressed this relationship evolutionally. Moreover, some of these works were studied small or identical domain systems, which make the result generalization process unclear for other systems. Finally, some of these studies were conducted with automatically generated test suites, which might not present the real situation for studied systems, so the results cannot be generalized to real test suites. In this paper, the authors empirically explore three open-source software systems along with their 11 versions. These versions are evolved over time and might have more sources of code and test suites. This work objective is to study the correlation between test suite effectiveness, the size of the test suite, and coverage for three Java programs during their evolution. In this work, the code coverage, test suite LOC and mutation testing coverage are measured to assess the correlation between the effectiveness of fault detection, code coverage, and test suite size. Based on the result we cannot generalize the assumption that test size is always revealing a positive correlation with its effectiveness, but still weak to the high correlation between test effectiveness, test size, and coverage.</span><p> </p>

Automated Synthesis of Comprehensive Memory Model Litmus Test Suites

2017 ◽  
Vol 45 (1) ◽  
pp. 661-675
Author(s):  
Daniel Lustig ◽  
Andrew Wright ◽  
Alexandros Papakonstantinou ◽  
Olivier Giroux
Keyword(s):  
Memory Model ◽  
Automated Synthesis ◽  
Litmus Test ◽  
Test Suites

scholarly journals Fine-grained Code Coverage Measurement in Automated Black-box Android Testing

2020 ◽  
Vol 29 (4) ◽  
pp. 1-35
Author(s):  
Aleksandr Pilgun ◽  
Olga Gadyatskaya ◽  
Yury Zhauniarovich ◽  
Stanislav Dashevskyi ◽  
Artsiom Kushniarou ◽  
...  
Keyword(s):  
Black Box ◽  
Code Coverage ◽  
Fine Grained ◽  
Coverage Measurement

scholarly journals The Relation of Test-Related Factors to Software Quality: A Case Study on Apache Systems

2021 ◽  
Vol 26 (2) ◽  
Author(s):  
Fabiano Pecorelli ◽  
Fabio Palomba ◽  
Andrea De Lucia
Keyword(s):  
Software Quality ◽  
Compelling Evidence ◽  
Related Factors ◽  
Code Coverage ◽  
Production Code ◽  
Software Code ◽  
Class Test ◽  
Code Quality ◽  
Lack Of Knowledge

AbstractTesting represents a crucial activity to ensure software quality. Recent studies have shown that test-related factors (e.g., code coverage) can be reliable predictors of software code quality, as measured by post-release defects. While these studies provided initial compelling evidence on the relation between tests and post-release defects, they considered different test-related factors separately: as a consequence, there is still a lack of knowledge of whether these factors are still good predictors when considering all together. In this paper, we propose a comprehensive case study on how test-related factors relate to production code quality in Apache systems. We first investigated how the presence of tests relates to post-release defects; then, we analyzed the role played by the test-related factors previously shown as significantly related to post-release defects. The key findings of the study show that, when controlling for other metrics (e.g., size of the production class), test-related factors have a limited connection to post-release defects.

Sign in / Sign up

Export Citation Format

Share Document