scholarly journals Mutation testing with hyperproperties

2021 ◽  
Author(s):  
Andreas Fellner ◽  
Mitra Tabaei Befrouei ◽  
Georg Weissenbacher
Keyword(s):  
Model Checking ◽  
Model Transformation ◽  
Current Model ◽  
Original System ◽  
Mutation Testing ◽  
Test Case ◽  
Test Cases ◽  
Modeling Languages ◽  
Deterministic Models ◽  
Testing Tool

AbstractWe present a new method for model-based mutation-driven test case generation. Mutants are generated by making small syntactical modifications to the model or source code of the system under test. A test case kills a mutant if the behavior of the mutant deviates from the original system when running the test. In this work, we use hyperproperties—which allow to express relations between multiple executions—to formalize different notions of killing for both deterministic as well as non-deterministic models. The resulting hyperproperties are universal in the sense that they apply to arbitrary reactive models and mutants. Moreover, an off-the-shelf model checking tool for hyperproperties can be used to generate test cases. Furthermore, we propose solutions to overcome the limitations of current model checking tools via a model transformation and a bounded SMT encoding. We evaluate our approach on a number of models expressed in two different modeling languages by generating tests using a state-of-the-art mutation testing tool.

scholarly journals Coevolution of Second-order-mutant

2018 ◽  
Vol 8 (5) ◽  
pp. 3238
Author(s):  
Mohamad Syafri Tuloli ◽  
Benhard Sitohang ◽  
Bayu Hendradjaya
Keyword(s):  
Genetic Algorithm ◽  
Optimization Method ◽  
Mutation Testing ◽  
Second Order ◽  
Real Case ◽  
Test Case ◽  
Test Cases ◽  
Single Population ◽  
Testing Problem

<span>One of the obstacles that hinder the usage of mutation testing is its impracticality, two main contributors of this are a large number of mutants and a large number of test cases involves in the process. Researcher usually tries to address this problem by optimizing the mutants and the test case separately. In this research, we try to tackle both of optimizing mutant and optimizing test-case simultaneously using a coevolution optimization method. The coevolution optimization method is chosen for the mutation testing problem because the method works by optimizing multiple collections (population) of a solution. This research found that coevolution is better suited for multi-problem optimization than other single population methods (i.e. Genetic Algorithm), we also propose new indicator to determine the optimal coevolution cycle. The experiment is done to the artificial case, laboratory, and also a real case.</span>

The Framework for Testing of Web Services through Actions in Addition to Scripts

2014 ◽  
Vol 490-491 ◽  
pp. 1617-1623
Author(s):  
R. Deeptha ◽  
Rajeswari Mukesh
Keyword(s):  
Web Services ◽  
Web Service ◽  
Susceptibility Testing ◽  
Test Case Generation ◽  
Test Case ◽  
Test Cases ◽  
Testing Framework ◽  
Testing Tool ◽  
Current Scenario ◽  
Web Services Testing

.As Web Services draw modules within and across enterprises, dynamically and belligerently testing Web Services has become crucial. Comprehensive Functional, Concert, Interoperability and Susceptibility Testing form the Pillars of Web Services Testing. Only by adopting a comprehensive testing department, enterprises can safeguard that their Web Services is robust, scalable, interoperable, and secure. Overall functionality of web services would be informal towards test. But, only if we methodically trust the applications components (services) before we combine them to complete the application. In current scenario web service technology comprehends various testing apparatuses for manipulating and generating the test cases. But these tools and approaches were negotiating security and execution time and consume more resources. The existing methodologies will generate test cases for the low end web services and limited number of requests, due to these constraints we built new testing framework. In this paper we introduced the new basis with testing of actions, scripts and link for web services by the use of test cases. For this approach we used SOAP web services with SOA. The test case generation and testing reports will gives the accurate testing results and test cases. These test cases are generated using Java JUnit testing tool. We implemented our approach in a java based platform for efficient and secure manner.

scholarly journals A structural testing tool for MPI programs with loops

2019 ◽  
Author(s):  
Silvia Diaz Diaz ◽  
Paulo Lopes Souza ◽  
Simone Souza
Keyword(s):  
Software Tool ◽  
Parallel Programs ◽  
Structural Testing ◽  
Test Case ◽  
Test Cases ◽  
Test Model ◽  
Specific Test ◽  
Internal Structures ◽  
Testing Tool ◽  
Many Core

Há uma alta demanda por programas paralelos corretos, principalmente devido às arquiteturas paralelas atuais, como clusters e processadores multi/many cores. O teste estrutural permite identificar defeitos pela cobertura de estruturas internas de programas paralelos. O não determinismo em programas paralelos traz novos desafios ao teste estrutural. Ele requer ferramentas e modelos de teste especı́ficos, capazes de cobrir primitivas de comunicação e sincronização com comportamentos dinâmicos, tais como os presentes em loops. Este artigo propõe uma nova ferramenta de software para o teste estrutural, com o objetivo de auxiliar testadores na revelação de defeitos desconhecidos associados a comunicação e presentes em estruturas de repetição de programas paralelos em C/MPI. Baseando-se na cobertura obtida, testadores podem escolher casos de teste especı́ficos e avaliar o progresso da atividade de teste. A ferramenta de teste proposta é validada com a ingestão de defeitos no código de um programa, e com a análise do suporte dado pela ferramenta para a geração de elementos requeridos e seleção de casos de teste. A ferramenta proposta automatiza parte da atividade de teste, especificamente a geração de elementos requeridos e guia a execução dos testes, reduzindo o tempo para a aplicação da atividade de teste. Nossos resultados mostram que a ferramenta de teste é capaz de revelar defeitos desconhecidos em primitivas de comunicação presentes em iterações de loops. There is a growing demand for correct parallel programs, mainly due to nowadays parallel architectures, such as clusters and multi/many-core processors. Structural testing allows the identification of defects by covering internal structures of parallel programs. Nondeterminism in parallel programs brings new challenges to the structural testing. It requires specific test model and tools, capable to cover communication and synchronization primitives with dynamic behaviors, such as those present inside of loops. This paper proposes a novel software tool for the structural testing, aiming to help testers in revealing defects associated to communication present in repetition structures of C/MPI parallel programs. Based on the obtained coverage, testers can choose specific test cases and evaluate the progress of the testing activity. We validate the proposed testing software tool by injecting a defect in a program code, and analyzing the support for generation of required elements and selection of test cases. ValiMPI tool automates part of the test activity, specifically the generation of required elements to guide test case selection, reducing the application cost of the testing activity. Our results demonstrate that the testing tool is capable to reveal unknown defects from communication in different loop iterations.

scholarly journals Resource and dependency based test case generation for RESTful Web services

2021 ◽  
Vol 26 (4) ◽  
Author(s):  
Man Zhang ◽  
Bogdan Marculescu ◽  
Andrea Arcuri
Keyword(s):  
Web Services ◽  
Domain Knowledge ◽  
Automated System ◽  
Test Case ◽  
Test Cases ◽  
Evolutionary Search ◽  
Sampling Strategies ◽  
Mutation Operators ◽  
On Line ◽  
Restful Web Services

AbstractNowadays, RESTful web services are widely used for building enterprise applications. REST is not a protocol, but rather it defines a set of guidelines on how to design APIs to access and manipulate resources using HTTP over a network. In this paper, we propose an enhanced search-based method for automated system test generation for RESTful web services, by exploiting domain knowledge on the handling of HTTP resources. The proposed techniques use domain knowledge specific to RESTful web services and a set of effective templates to structure test actions (i.e., ordered sequences of HTTP calls) within an individual in the evolutionary search. The action templates are developed based on the semantics of HTTP methods and are used to manipulate the web services’ resources. In addition, we propose five novel sampling strategies with four sampling methods (i.e., resource-based sampling) for the test cases that can use one or more of these templates. The strategies are further supported with a set of new, specialized mutation operators (i.e., resource-based mutation) in the evolutionary search that take into account the use of these resources in the generated test cases. Moreover, we propose a novel dependency handling to detect possible dependencies among the resources in the tested applications. The resource-based sampling and mutations are then enhanced by exploiting the information of these detected dependencies. To evaluate our approach, we implemented it as an extension to the EvoMaster tool, and conducted an empirical study with two selected baselines on 7 open-source and 12 synthetic RESTful web services. Results show that our novel resource-based approach with dependency handling obtains a significant improvement in performance over the baselines, e.g., up to + 130.7% relative improvement (growing from + 27.9% to + 64.3%) on line coverage.

scholarly journals Performance of Enhanced Multiple-Searching Genetic Algorithm for Test Case Generation in Software Testing

Mathematics ◽  
2021 ◽  
Vol 9 (15) ◽  
pp. 1779
Author(s):  
Wanida Khamprapai ◽  
Cheng-Fa Tsai ◽  
Paohsi Wang ◽  
Chi-En Tsai
Keyword(s):  
Genetic Algorithm ◽  
Software Testing ◽  
Random Search ◽  
Automatic Generation ◽  
Superior Performance ◽  
Test Case Generation ◽  
Test Case ◽  
Test Cases ◽  
Network Systems ◽  
Routing Problem

Test case generation is an important process in software testing. However, manual generation of test cases is a time-consuming process. Automation can considerably reduce the time required to create adequate test cases for software testing. Genetic algorithms (GAs) are considered to be effective in this regard. The multiple-searching genetic algorithm (MSGA) uses a modified version of the GA to solve the multicast routing problem in network systems. MSGA can be improved to make it suitable for generating test cases. In this paper, a new algorithm called the enhanced multiple-searching genetic algorithm (EMSGA), which involves a few additional processes for selecting the best chromosomes in the GA process, is proposed. The performance of EMSGA was evaluated through comparison with seven different search-based techniques, including random search. All algorithms were implemented in EvoSuite, which is a tool for automatic generation of test cases. The experimental results showed that EMSGA increased the efficiency of testing when compared with conventional algorithms and could detect more faults. Because of its superior performance compared with that of existing algorithms, EMSGA can enable seamless automation of software testing, thereby facilitating the development of different software packages.

scholarly journals Automated Search-Based Robustness Testing for Autonomous Vehicle Software

2016 ◽  
Vol 2016 ◽  
pp. 1-15 ◽  
Author(s):  
Kevin M. Betts ◽  
Mikel D. Petty
Keyword(s):  
Monte Carlo ◽  
Performance Metrics ◽  
Initial Conditions ◽  
Autonomous Vehicle ◽  
Vehicle Control ◽  
Test Case ◽  
Test Cases ◽  
Control Software ◽  
Robustness Testing ◽  
Testing Techniques

Autonomous systems must successfully operate in complex time-varying spatial environments even when dealing with system faults that may occur during a mission. Consequently, evaluating the robustness, or ability to operate correctly under unexpected conditions, of autonomous vehicle control software is an increasingly important issue in software testing. New methods to automatically generate test cases for robustness testing of autonomous vehicle control software in closed-loop simulation are needed. Search-based testing techniques were used to automatically generate test cases, consisting of initial conditions and fault sequences, intended to challenge the control software more than test cases generated using current methods. Two different search-based testing methods, genetic algorithms and surrogate-based optimization, were used to generate test cases for a simulated unmanned aerial vehicle attempting to fly through an entryway. The effectiveness of the search-based methods in generating challenging test cases was compared to both a truth reference (full combinatorial testing) and the method most commonly used today (Monte Carlo testing). The search-based testing techniques demonstrated better performance than Monte Carlo testing for both of the test case generation performance metrics: (1) finding the single most challenging test case and (2) finding the set of fifty test cases with the highest mean degree of challenge.

scholarly journals PRIORITIZATION OF COMBINATORIAL TEST CASES BY INCREMENTAL INTERACTION COVERAGE

2013 ◽  
Vol 23 (10) ◽  
pp. 1427-1457 ◽  
Author(s):  
RUBING HUANG ◽  
XIAODONG XIE ◽  
DAVE TOWEY ◽  
TSONG YUEH CHEN ◽  
YANSHENG LU ◽  
...  
Keyword(s):  
Test Case ◽  
Test Cases ◽  
New Strategy ◽  
Testing Cost ◽  
Test Prioritization ◽  
Interaction Testing ◽  
Parameter Interactions ◽  
Combinatorial Interaction Testing ◽  
Alternative Assumption ◽  
Better Than

Combinatorial interaction testing is a well-recognized testing method, and has been widely applied in practice, often with the assumption that all test cases in a combinatorial test suite have the same fault detection capability. However, when testing resources are limited, an alternative assumption may be that some test cases are more likely to reveal failure, thus making the order of executing the test cases critical. To improve testing cost-effectiveness, prioritization of combinatorial test cases is employed. The most popular approach is based on interaction coverage, which prioritizes combinatorial test cases by repeatedly choosing an unexecuted test case that covers the largest number of uncovered parameter value combinations of a given strength (level of interaction among parameters). However, this approach suffers from some drawbacks. Based on previous observations that the majority of faults in practical systems can usually be triggered with parameter interactions of small strengths, we propose a new strategy of prioritizing combinatorial test cases by incrementally adjusting the strength values. Experimental results show that our method performs better than the random prioritization technique and the technique of prioritizing combinatorial test suites according to test case generation order, and has better performance than the interaction-coverage-based test prioritization technique in most cases.

MuSim: Mutation-based Fault Localization Using Test Case Proximity

2021 ◽  
Vol 31 (05) ◽  
pp. 725-744
Author(s):  
Arpita Dutta ◽  
Amit Jha ◽  
Rajib Mall
Keyword(s):  
Fault Localization ◽  
Test Case ◽  
Test Cases ◽  
Similarity Metrics ◽  
Localization Technique ◽  
Evaluation Metric

Fault localization techniques aim to localize faulty statements using the information gathered from both passed and failed test cases. We present a mutation-based fault localization technique called MuSim. MuSim identifies the faulty statement based on its computed proximity to different mutants. We study the performance of MuSim by using four different similarity metrics. To satisfactorily measure the effectiveness of our proposed approach, we present a new evaluation metric called Mut_Score. Based on this metric, on an average, MuSim is 33.21% more effective than existing fault localization techniques such as DStar, Tarantula, Crosstab, Ochiai.

Comparison of Supercritical CO2 Power Cycles to Steam Rankine Cycles in Coal-Fired Applications

2017 ◽  
Author(s):  
Jason D. Miller ◽  
David J. Buckmaster ◽  
Katherine Hart ◽  
Timothy J. Held ◽  
David Thimsen ◽  
...  
Keyword(s):  
Power Plants ◽  
Cooling Tower ◽  
Test Case ◽  
Test Cases ◽  
Power Cycles ◽  
Air Preheater ◽  
Power Cycle ◽  
Inlet Conditions ◽  
Cycle Efficiency ◽  
Steam Plant

Increasing the efficiency of coal-fired power plants is vital to reducing electricity costs and emissions. Power cycles employing supercritical carbon dioxide (sCO2) as the working fluid have the potential to increase power cycle efficiency by 3–5% points over state-of-the-art oxy-combustion steam-Rankine cycles operating under comparable conditions. To date, the majority of studies have focused on the integration and optimization of sCO2 power cycles in waste heat, solar, or nuclear applications. The goal of this study is to demonstrate the potential of sCO2 power cycles, and quantify the power cycle efficiency gains that can be achieved versus the state-of-the-art steam-Rankine cycles employed in oxy-fired coal power plants. Turbine inlet conditions were varied among the sCO2 test cases and compared with existing Department of Energy (DOE)/National Energy Technology6 Laboratory (NETL) steam base cases. Two separate sCO2 test cases were considered and the associated flow sheets developed. The turbine inlet conditions for this study were chosen to match conditions in a coal-fired ultra-supercritical steam plant (Tinlet = 593°C, Pinlet = 24.1 MPa) and an advanced ultra-supercritical steam plant (Tinlet = 730°C, Pinlet = 27.6 MPa). A plant size of 550 MWe, was selected to match available information on existing DOE/NETL bases cases. The effects of cycle architecture, combustion-air preheater temperature, and cooling source type were considered subject to comparable heat source and reference conditions taken from the steam Rankine reference cases. Combinations and variants of sCO2 power cycles — including cascade and recompression and variants with multiple reheat and compression steps — were considered with varying heat-rejection subsystems — air-cooled, direct cooling tower, and indirect-loop cooling tower. Where appropriate, combustion air preheater inlet temperature was also varied. Through use of a multivariate nonlinear optimization design process that considers both performance and economic impacts, curves of minimum cost versus efficiency were generated for each sCO2 test case and combination of architecture and operational choices. These curves indicate both peak theoretical efficiency and suggest practical limits based on incremental cost versus performance. For a given test case, results for individual architectural and operational options give insight to cost and performance improvements from step-changes in system complexity and design, allowing down selection of candidate architectures. Optimized designs for each test case were then selected based on practical efficiency limits within the remaining candidate architectures and compared to the relevant baseline steam plant. sCO2 cycle flowsheets are presented for each optimized design.

Sign in / Sign up

Export Citation Format

Share Document