Test Oracle Generation Based on BPNN by Using the Values of Variables at Different Breakpoints for Programs

Automatic test oracle generation is a bottleneck in realizing full automation of the entire software testing process. This study proposes a new method for automatically generating a test oracle for a new test input on the basis of several historical test cases by using a backpropagation neural network (BPNN) model. The new method is different from existing test oracle techniques. Specifically, our method has two steps. First, the values of variables are collected as training data when several historical test inputs are used to execute the program at different breakpoints. The test oracles (pass or fail) of these test cases are utilized to classify and label the training data. Second, a new test input is used to execute the program at different breakpoints, where the trained BPNN prediction model automatically generates its test oracle on the basis of the collected values of the variables involved. We conduct an experiment to validate our method. In the experiment, 113 faulty versions of seven types of programs are used as experimental objects. Results show that the average prediction accuracy rate of 74,651 test oracles is 95.8%. Although the failed test cases in the training data account for less than 5%, the overall average recall rate (prediction accuracy of test case execution failure) of all programs is 78.9%. Furthermore, the trained BPNN can reveal not only the impact of the values of variables but also the impact of the logical correspondence between variables in test oracle generation.

Path-Sensitive Oracle Data Selection via Static Analysis

A test oracle is a procedure that is used during testing to determine whether software behaves correctly or not. One of most important tasks for a test oracle is to choose oracle data (the set of variables monitored during testing) to observe. However, most literature on test oracles has focused either on formal specification generation or on automated test oracle construction, whereas little work exists for supporting oracle data selection. In this paper, we present a path-sensitive approach, PSODS (path-sensitive oracle data selection), to automatically select oracle data for use by expected value oracles. PSODS ranks paths according to the possibility that potential faults may exist in them, and the ranked paths help testers determine which oracle data should be considered first. To select oracle data for each path, we introduce quantity and quality analysis of oracle data, which use static analysis to estimate oracle data for their substitution capability and fault-detection capability. Quantity analysis can reduce the number of oracle data. Quality analysis can rank oracle data based on their fault-detection capability. By using quantity and quality analysis, PSODS reduces the cost of oracle construction and improves fault-detection efficiency and effectiveness. We have implemented our approach and applied it to a real-world project. The experimental results show that PSODS is efficient in helping testers construct test oracles. Moreover, the oracle datasets produced by our approach are more effective and efficient than output-only oracles at detecting faults.