scholarly journals Diversity of graph models and graph generators in mutation testing

2019 ◽  
Vol 22 (1) ◽  
pp. 57-78 ◽  
Author(s):  
Oszkár Semeráth ◽  
Rebeka Farkas ◽  
Gábor Bergmann ◽  
Dániel Varró
Keyword(s):  
Mutation Testing ◽  
Distance Functions ◽  
Test Suite ◽  
Critical Systems ◽  
Graph Models ◽  
Bug Detection ◽  
Modeling Tools ◽  
Graph Constraints ◽  
Access Policies ◽  
Diversity Metrics

Abstract When custom modeling tools are used for designing complex safety-critical systems (e.g., critical cyber-physical systems), the tools themselves need to be validated by systematic testing to prevent tool-specific bugs reaching the system. Testing of such modeling tools relies upon an automatically generated set of models as a test suite. While many software testing practices recommend that this test suite should be diverse, model diversity has not been studied systematically for graph models. In the paper, we propose different diversity metrics for models by generalizing and exploiting neighborhood and predicate shapes as abstraction. We evaluate such shape-based diversity metrics using various distance functions in the context of mutation testing of graph constraints and access policies for two separate industrial DSLs. Furthermore, we evaluate the quality (i.e., bug detection capability) of different (random and consistent) model generation techniques for mutation testing purposes.

scholarly journals Diversifying Focused Testing for Unit Testing

2021 ◽  
Vol 30 (4) ◽  
pp. 1-24
Author(s):  
Héctor D. Menéndez ◽  
Gunel Jahangirova ◽  
Federica Sarro ◽  
Paolo Tonella ◽  
David Clark
Keyword(s):  
Search Algorithm ◽  
Symbolic Execution ◽  
Specific Area ◽  
Test Suite ◽  
Constraint Solving ◽  
Test Case ◽  
Unit Testing ◽  
Bug Detection ◽  
Mutation Score ◽  
The Given

Software changes constantly, because developers add new features or modifications. This directly affects the effectiveness of the test suite associated with that software, especially when these new modifications are in a specific area that no test case covers. This article tackles the problem of generating a high-quality test suite to cover repeatedly a given point in a program, with the ultimate goal of exposing faults possibly affecting the given program point. Both search-based software testing and constraint solving offer ready, but low-quality, solutions to this: Ideally, a maximally diverse covering test set is required, whereas search and constraint solving tend to generate test sets with biased distributions. Our approach, Diversified Focused Testing (DFT), uses a search strategy inspired by GödelTest. We artificially inject parameters into the code branching conditions and use a bi-objective search algorithm to find diverse inputs by perturbing the injected parameters, while keeping the path conditions still satisfiable. Our results demonstrate that our technique, DFT, is able to cover a desired point in the code at least 90% of the time. Moreover, adding diversity improves the bug detection and the mutation killing abilities of the test suites. We show that DFT achieves better results than focused testing, symbolic execution, and random testing by achieving from 3% to 70% improvement in mutation score and up to 100% improvement in fault detection across 105 software subjects.

Analysis of distance functions for similarity-based test suite reduction in the context of model-based testing

2014 ◽  
Vol 24 (2) ◽  
pp. 407-445 ◽  
Author(s):  
Ana Emília Victor Barbosa Coutinho ◽  
Emanuela Gadelha Cartaxo ◽  
Patrícia Duarte de Lima Machado
Keyword(s):  
Distance Functions ◽  
Test Suite ◽  
Model Based ◽  
Model Based Testing ◽  
Test Suite Reduction

scholarly journals A Model for Predicting Statement Mutation Scores

Mathematics ◽  
2019 ◽  
Vol 7 (9) ◽  
pp. 778
Author(s):  
Lili Tan ◽  
Yunzhan Gong ◽  
Yawen Wang
Keyword(s):  
Neural Network ◽  
Mutation Testing ◽  
Classification Model ◽  
Test Suite ◽  
Support Vector ◽  
Testing Methods ◽  
The Neural Network ◽  
Mutation Score ◽  
Powerful Approach ◽  
Hidden Layer

A test suite plays a key role in software testing. Mutation testing is a powerful approach to measure the fault-detection ability of a test suite. The mutation testing process requires a large number of mutants to be generated and executed. Hence, mutation testing is also computationally expensive. To solve this problem, predictive mutation testing builds a classification model to predict the test result of each mutant. However, the existing predictive mutation testing methods only can be used to estimate the overall mutation scores of object-oriented programs. To overcome the shortcomings of the existing methods, we propose a new method to directly predict the mutation score for each statement in process-oriented programs. Compared with the existing predictive mutation testing methods, our method uses more dynamic program execution features, which more adequately reflect dynamic dependency relationships among the statements and more accurately reflects information propagation during the execution of test cases. By comparing the prediction effects of logistic regression, artificial neural network, random forest, support vector machine, and symbolic regression, we finally decide to use a single hidden layer feedforward neural network as the predictive model to predict the statement mutation scores. In our two experiments, the mean absolute errors between the statement mutation scores predicted by the neural network and the real statement mutation scores both approximately reach 0.12.

scholarly journals Mutant Selecting According to the Nondominated Original Statements

2019 ◽  
Vol 2019 ◽  
pp. 1-11
Author(s):  
Gongjie Zhang ◽  
Chunli Xie ◽  
Yongquan Dong ◽  
Qiao Yu
Keyword(s):  
Mutation Testing ◽  
Experimental Results ◽  
New Method ◽  
Test Suite ◽  
Practical Application

Mutation testing is a technique for evaluating the quality of a test suite. However, the costly computation from a large number of mutants affects the practical application of mutation testing, and reducing the number of mutants is reasonably an efficient way for mutation testing. We propose a new method for reducing mutants by analyzing dominance between statements in the program under test. The proposed method only selects the mutants generated from the nondominated statements, and the mutants generated from the dominated statements are reduced. After applying the proposed method to nine programs, the experimental results show that our method reduces over 75% mutants and well maintains the mutation adequacy.

Sign in / Sign up

Export Citation Format

Share Document