scholarly journals Status Report on Software Testing: Test-Comp 2021

2021 ◽  
pp. 341-357 ◽  
Author(s):  
Dirk Beyer
Keyword(s):  
Software Testing ◽  
Test Generation ◽  
Software Test ◽  
Status Report ◽  
Generation Task ◽  
C Programs ◽  
Branch Coverage ◽  
Current State ◽  
Strong Focus ◽  
Automatic Software

AbstractThis report describes Test-Comp 2021, the 3rd edition of the Competition on Software Testing. The competition is a series of annual comparative evaluations of fully automatic software test generators for C programs. The competition has a strong focus on reproducibility of its results and its main goal is to provide an overview of the current state of the art in the area of automatic test-generation. The competition was based on 3 173 test-generation tasks for C programs. Each test-generation task consisted of a program and a test specification (error coverage, branch coverage). Test-Comp 2021 had 11 participating test generators from 6 countries.

scholarly journals Software Verification: 10th Comparative Evaluation (SV-COMP 2021)

2021 ◽  
pp. 401-422
Author(s):  
Dirk Beyer
Keyword(s):  
Comparative Evaluation ◽  
Software Verification ◽  
Verification Task ◽  
C Programs ◽  
Current State ◽  
Strong Focus ◽  
Java Programs ◽  
Automatic Software ◽  
Fully Automatic ◽  
Verification Systems

AbstractSV-COMP 2021 is the 10th edition of the Competition on Software Verification (SV-COMP), which is an annual comparative evaluation of fully automatic software verifiers for C and Java programs. The competition provides a snapshot of the current state of the art in the area, and has a strong focus on reproducibility of its results. The competition was based on 15 201 verification tasks for C programs and 473 verification tasks for Java programs. Each verification task consisted of a program and a property (reachability, memory safety, overflows, termination). SV-COMP 2021 had 30 participating verification systems from 27 teams from 11 countries.

scholarly journals Advances in Automatic Software Verification: SV-COMP 2020

2020 ◽  
pp. 347-367 ◽  
Author(s):  
Dirk Beyer
Keyword(s):  
Software Verification ◽  
State Of The Art ◽  
Verification Task ◽  
C Programs ◽  
Current State ◽  
Strong Focus ◽  
Java Programs ◽  
Automatic Software ◽  
Fully Automatic ◽  
Verification Systems

Abstract This report describes the 2020 Competition on Software Verification (SV-COMP), the 9$$^{\text {th}}$$ edition of a series of comparative evaluations of fully automatic software verifiers for C and Java programs. The competition provides a snapshot of the current state of the art in the area, and has a strong focus on replicability of its results. The competition was based on 11 052 verification tasks for C programs and 416 verification tasks for Java programs. Each verification task consisted of a program and a property (reachability, memory safety, overflows, termination). SV-COMP 2020 had 28 participating verification systems from 11 countries.

Concolic Test Generation and the Cloud

2015 ◽  
pp. 302-322
Author(s):  
Nikolai Kosmatov
Keyword(s):  
Software Testing ◽  
Test Generation ◽  
Symbolic Execution ◽  
Major Obstacle ◽  
Testing Methods ◽  
C Programs ◽  
Security Issues ◽  
Efficient Alternative ◽  
Cloud Environments ◽  
Testing Techniques

Software testing in the cloud can reduce the need for hardware and software resources and offer a flexible and efficient alternative to the traditional software testing process. A major obstacle to the wider use of testing in the cloud is related to security issues. This chapter focuses on test generation techniques that combine concrete and symbolic execution of the program under test. Their deployment in the cloud leads to complex technical and security issues that do not occur for other testing methods. This chapter describes recent online deployment of such a technique implemented by the PathCrawler test generation tool for C programs, where the author faced, studied, and solved many of these issues. Mixed concrete/symbolic testing techniques not only constitute a challenging target for deployment in the cloud, but they also provide a promising way to improve the reliability of cloud environments. The author argues that these techniques can be efficiently used to help to create trustworthy cloud environments.

scholarly journals First international competition on software testing

2021 ◽  
Author(s):  
Dirk Beyer
Keyword(s):  
Software Testing ◽  
Test Generation ◽  
Comparative Evaluation ◽  
State Of The Art ◽  
International Competition ◽  
Software Test ◽  
Test Specifications ◽  
Research Areas ◽  
Function Call ◽  
Generation Test

AbstractTool competitions are a special form of comparative evaluation, where each tool has a team of developers or supporters associated that makes sure the tool is properly configured to show its best possible performance. In several research areas, tool competitions have been a driving force for the development of mature tools that represent the state of the art in their field. This paper describes and reports the results of the 1$$^{\text {st}}$$ st International Competition on Software Testing (Test-Comp 2019), a comparative evaluation of automatic tools for software test generation. Test-Comp 2019 was presented as part of TOOLympics 2019, a satellite event of the conference TACAS. Nine test generators were evaluated on 2 356 test-generation tasks. There were two test specifications, one for generating a test that covers a particular function call and one for generating a test suite that tries to cover the branches of the program.

Knowledge Engineering Support for Intelligent Software Test Optimization

2011 ◽  
pp. 211-243
Author(s):  
D. Jeya Mala
Keyword(s):  
Software Testing ◽  
Intelligent Agents ◽  
Knowledge Engineering ◽  
Software Test ◽  
Test Sequence ◽  
Data Generation ◽  
Graph Searching ◽  
Branch Coverage ◽  
Development Life Cycle ◽  
Path Coverage

Software testing is one of the most important processes in Software Development Life Cycle (SDLC) to ensure quality of software product. A large number of approaches for test data generation and optimization in software testing have been reported with varying degrees of success. In recent years, the application of artificial intelligence (AI) techniques in software testing to achieve quality software is an emerging area of research. It combines the concepts of the two domains AI and Software Engineering and yields innovative mechanisms that have the advantages of both of them. This chapter demonstrates the development of a novel software test sequence optimization framework using intelligent agents based graph searching technique. To ensure the quality, a set of test adequacy criteria such as path coverage, state coverage, and branch coverage have been applied in the approach discussed in this chapter, while achieving optimization.

Automatic software test generation

1990 ◽  
Vol 32 (5) ◽  
pp. 337-346 ◽  
Author(s):  
M Camuffo ◽  
M Maiocchi ◽  
M Morselli
Keyword(s):  
Test Generation ◽  
Software Test ◽  
Automatic Software

Concolic Test Generation and the Cloud

2012 ◽  
pp. 231-251
Author(s):  
Nikolai Kosmatov
Keyword(s):  
Software Testing ◽  
Test Generation ◽  
Symbolic Execution ◽  
Major Obstacle ◽  
Testing Methods ◽  
C Programs ◽  
Security Issues ◽  
Efficient Alternative ◽  
Cloud Environments ◽  
Testing Techniques

Software testing in the cloud can reduce the need for hardware and software resources and offer a flexible and efficient alternative to the traditional software testing process. A major obstacle to the wider use of testing in the cloud is related to security issues. This chapter focuses on test generation techniques that combine concrete and symbolic execution of the program under test. Their deployment in the cloud leads to complex technical and security issues that do not occur for other testing methods. This chapter describes recent online deployment of such a technique implemented by the PathCrawler test generation tool for C programs, where the author faced, studied, and solved many of these issues. Mixed concrete/symbolic testing techniques not only constitute a challenging target for deployment in the cloud, but they also provide a promising way to improve the reliability of cloud environments. The author argues that these techniques can be efficiently used to help to create trustworthy cloud environments.

Constraint-Based Techniques for Software Testing

2010 ◽  
pp. 218-232 ◽  
Author(s):  
Nikolai Kosmatov
Keyword(s):  
Software Testing ◽  
Test Data ◽  
Test Generation ◽  
Automatic Testing ◽  
Constraint Solving ◽  
Future Research ◽  
C Programs ◽  
Black Box Testing ◽  
White Box Testing ◽  
Generate Test Data

In this chapter, the authors discuss some innovative applications of artificial intelligence techniques to software engineering, in particular, to automatic test generation. Automatic testing tools translate the program under test, or its model, and the test criterion, or the test objective, into constraints. Constraint solving allows then to find a solution of the constraint solving problem and to obtain test data. The authors focus on two particular applications: model-based testing as an example of black-box testing, and all-paths test generation for C programs as a white-box testing strategy. Each application is illustrated by a running example showing how constraint-based methods allow to automatically generate test data for each strategy. They also give an overview of the main difficulties of constraint-based software testing and outline some directions for future research.

Research Notes: Binary Test-Suites Using Covering Arrays

2018 ◽  
Vol 28 (09) ◽  
pp. 1321-1337 ◽  
Author(s):  
Jose Torres-Jimenez ◽  
Himer Avila-George ◽  
Ezra Federico Parra-González
Keyword(s):  
Software Testing ◽  
Test Generation ◽  
Software Systems ◽  
Combinatorial Testing ◽  
Covering Array ◽  
Covering Arrays ◽  
Almost All ◽  
Generation Problem ◽  
Test Suites

Software testing is an essential activity to ensure the quality of software systems. Combinatorial testing is a method that facilitates the software testing process; it is based on an empirical evidence where almost all faults in a software component are due to the interaction of very few parameters. The test generation problem for combinatorial testing can be represented as the construction of a matrix that has certain properties; typically this matrix is a covering array. Covering arrays have a small number of tests, in comparison with an exhaustive approach, and provide a level of interaction coverage among the parameters involved. This paper presents a repository that contains binary covering arrays involving many levels of interaction. Also, it discusses the importance of covering array repositories in the construction of better covering arrays. In most of the cases, the size of the covering arrays included in the repository reported here are the best upper bounds known, moreover, the files containing the matrices of these covering arrays are available to be downloaded. The final purpose of our Binary Covering Arrays Repository (BCAR) is to provide software testing practitioners the best-known binary test-suites.

scholarly journals Selection of Best Test Cases using Support Vector Machine for ARPT

2019 ◽  
Vol 8 (3) ◽  
pp. 4265-4271
Keyword(s):  
Support Vector Machine ◽  
Software Testing ◽  
Support Vector ◽  
Test Cases ◽  
Testing Strategy ◽  
Random Partition ◽  
Code Coverage ◽  
Time Consumption ◽  
Branch Coverage ◽  
Selection Of

Software testing is an essential activity in software industries for quality assurance; subsequently, it can be effectively removing defects before software deployment. Mostly good software testing strategy is to accomplish the fundamental testing objective while solving the trade-offs between effectiveness and efficiency testing issues. Adaptive and Random Partition software Testing (ARPT) approach was a combination of Adaptive Testing (AT) and Random Partition Approach (RPT) used to test software effectively. It has two variants they are ARPT-1 and ARPT-2. In ARPT-1, AT was used to select a certain number of test cases and then RPT was used to select a number of test cases before returning to AT. In ARPT-2, AT was used to select the first m test cases and then switch to RPT for the remaining tests. The computational complexity for random partitioning in ARPT was solved by cluster the test cases using a different clustering algorithm. The parameters of ARPT-1 and ARPT-2 needs to be estimated for different software, it leads to high computation overhead and time consumption. It was solved by Improvised BAT optimization algorithms and this approach is named as Optimized ARPT1 (OARPT1) and OARPT2. By using all test cases in OARPT will leads to high time consumption and computational overhead. In order to avoid this problem, OARPT1 with Support Vector Machine (OARPT1-SVM) and OARPT2- SVM are introduced in this paper. The SVM is used for selection of best test cases for OARPT-1 and OARPT-2 testing strategy. The SVM constructs hyper plane in a multi-dimensional space which is used to separate test cases which have high code and branch coverage and test cases which have low code and branch coverage. Thus, the SVM selects the best test cases for OARPT-1 and OARPT-2. The selected test cases are used in OARPT-1 and OARPT-2 to test software. In the experiment, three different software is used to prove the effectiveness of proposed OARPT1- SVM and OARPT2-SVM testing strategies in terms of time consumption, defect detection efficiency, branch coverage and code coverage.

Sign in / Sign up

Export Citation Format

Share Document