Distributing test cases more evenly in adaptive random testing

Restricted Random Testing (RRT) is a new method of testing software that improves upon traditional Random Testing (RT) techniques. Research has indicated that failure patterns (portions of an input domain which, when executed, cause the program to fail or reveal an error) can influence the effectiveness of testing strategies. For certain types of failure patterns, it has been found that a widespread and even distribution of test cases in the input domain can be significantly more effective at detecting failure compared with ordinary RT. Testing methods based on RT, but which aim to achieve even and widespread distributions, have been called Adaptive Random Testing (ART) strategies. One implementation of ART is RRT. RRT uses exclusion zones around executed, but non-failure-causing, test cases to restrict the regions of the input domain from which subsequent test cases may be drawn. In this paper, we introduce the motivation behind RRT, explain the algorithm and detail some empirical analyses carried out to examine the effectiveness of the method. Two versions of RRT are presented: Ordinary RRT (ORRT) and Normalized RRT (NRRT). The two versions share the same fundamental algorithm, but differ in their treatment of non-homogeneous input domains. Investigations into the use of alternative exclusion shapes are outlined, and a simple technique for reducing the computational overheads of RRT, prompted by the alternative exclusion shape investigations, is also explained. The performance of RRT is compared with RT and another ART method based on maximized minimum test case separation (DART), showing excellent improvement over RT and a very favorable comparison with DART.

Download Full-text

Adaptive Random Testing with Combinatorial Input Domain

The Scientific World JOURNAL ◽

10.1155/2014/843248 ◽

2014 ◽

Vol 2014 ◽

pp. 1-16 ◽

Cited By ~ 2

Author(s):

Rubing Huang ◽

Jinfu Chen ◽

Yansheng Lu

Keyword(s):

Categorical Data ◽

Similarity Measures ◽

Test Case ◽

Test Cases ◽

Random Testing ◽

Detection Capability ◽

Adaptive Random Testing ◽

Random Manner ◽

Input Domain ◽

Better Than

Random testing (RT) is a fundamental testing technique to assess software reliability, by simply selecting test cases in a random manner from the whole input domain. As an enhancement of RT, adaptive random testing (ART) has better failure‐detection capability and has been widely applied in different scenarios, such as numerical programs, some object‐oriented programs, and mobile applications. However, not much work has been done on the effectiveness of ART for the programs with combinatorial input domain (i.e., the set of categorical data). To extend the ideas to the testing for combinatorial input domain, we have adopted different similarity measures that are widely used for categorical data in data mining and have proposed two similarity measures based on interaction coverage. Then, we propose a new version named ART‐CID as an extension of ART in combinatorial input domain, which selects an element from categorical data as the next test case such that it has the lowest similarity against already generated test cases. Experimental results show that ART‐CID generally performs better than RT, with respect to different evaluation metrics.

Download Full-text

A Proactive Approach to Test Case Selection — An Efficient Implementation of Adaptive Random Testing

International Journal of Software Engineering and Knowledge Engineering ◽

10.1142/s0218194020500308 ◽

2020 ◽

Vol 30 (08) ◽

pp. 1169-1198

Author(s):

Michael Omari ◽

Jinfu Chen ◽

Robert French-Baidoo ◽

Yunting Sun

Keyword(s):

Experimental Studies ◽

Test Case ◽

Test Cases ◽

Random Testing ◽

Major Drawback ◽

Computation Process ◽

Adaptive Random Testing ◽

New Strategy ◽

Valid Test ◽

Using Data

Fixed Sized Candidate Set (FSCS) is the first of a series of methods proposed to enhance the effectiveness of random testing (RT) referred to as Adaptive Random Testing methods or ARTs. Since its inception, test case generation overheads have been a major drawback to the success of ART. In FSCS, the bulk of this cost is embedded in distance computations between a set of randomly generated candidate test cases and previously executed but unsuccessful test cases. Consequently, FSCS is caught in a logical trap of probing the distances between every candidate and all executed test cases before the best candidate is determined. Using data mining, however, we discovered that about 50% of all valid test cases are encountered much earlier in the distance computations process but without any benefit of a hindsight, FSCS is unable to validate them; a wild goose chase. This paper then uses this information to propose a new strategy that predictively and proactively selects valid candidates anywhere during the distance computation process without vetting every candidate. Theoretical analysis, simulations and experimental studies conducted led to a similar conclusion: 25% of the distance computations are wasteful and can be discarded without any repercussion on effectiveness.

Download Full-text

An Enhanced Adaptive Random Testing by Dividing Dimensions Independently

Mathematical Problems in Engineering ◽

10.1155/2019/9381728 ◽

2019 ◽

Vol 2019 ◽

pp. 1-15

Author(s):

Zhibo Li ◽

Qingbao Li ◽

Lei Yu

Keyword(s):

Boundary Effect ◽

Real Life ◽

Failure Detection ◽

Test Case Generation ◽

Test Case ◽

Test Cases ◽

Random Testing ◽

Failure Rates ◽

Adaptive Random Testing ◽

Candidate Set

Random testing (RT) is widely applied in the area of software testing due to its advantages such as simplicity, unbiasedness, and easy implementation. Adaptive random testing (ART) enhances RT. It improves the effectiveness of RT by distributing test cases as evenly as possible. Fixed Size Candidate Set (FSCS) is one of the most well-known ART algorithms. Its high failure-detection effectiveness only shows at low failure rates in low-dimensional spaces. In order to solve this problem, the boundary effect of the test case distribution is analyzed, and the FSCS algorithm of a limited candidate set (LCS-FSCS) is proposed. By utilizing the information gathered from success test cases (no failure-causing test inputs), a tabu generation domain of candidate test case is produced. This tabu generation domain is eliminated from the current candidate test case generation domain. Finally, the number of test cases at the boundary is reduced by constraining the candidate test case generation domain. The boundary effect is effectively relieved, and the distribution of test cases is more even. The results of the simulation experiment show that the failure-detection effectiveness of LCS-FSCS is significantly improved in high-dimensional spaces. Meanwhile, the failure-detection effectiveness is also improved for high failure rates and the gap of failure-detection effectiveness between different failure rates is narrowed. The results of an experiment conducted on some real-life programs show that LCS-FSCS is less effective than FSCS only when the failure distribution is concentrated on the boundary. In general, the effectiveness of LCS-FSCS is higher than that of FSCS.

Download Full-text