Test Data Generation Approaches for Structural Testing and Automatic Programming Assessment: A Systematic Literature Review

This article discusses the application of an improved version of the firefly algorithm for the test suite optimization problem. Software test optimization refers to optimizing test data generation and selection for structural testing criteria for white box testing. This will subsequently reduce the two most costly activities performed during testing: time and cost. Recently, various search-based approaches proved very interesting results for the software test optimization problem. Also, due to no free lunch theorem, scientists are continuously searching for more efficient and convergent methods for the optimization problem. In this paper, firefly algorithm is modified in a way that local search ability is improved. Levy flights are incorporated into the firefly algorithm. This modified algorithm is applied to the software test optimization problem. This is the first application of Levy-based firefly algorithm for software test optimization. Results are shown and compared with some existing metaheuristic approaches.

Download Full-text

TEST DATA GENERATION FOR SOFTWARE TESTING BASED ON QUANTUM-INSPIRED GENETIC ALGORITHM

International Journal of Computational Intelligence and Applications ◽

10.1142/s1469026813500041 ◽

2013 ◽

Vol 12 (01) ◽

pp. 1350004 ◽

Cited By ~ 2

Author(s):

CHENGYING MAO ◽

XINXIN YU

Keyword(s):

Genetic Algorithm ◽

Software Testing ◽

Test Data ◽

Optimal Solution ◽

Search Space ◽

Structural Testing ◽

Test Data Generation ◽

Data Generation ◽

Quantum Bit

The quality of test data has an important impact on the effect of software testing, so test data generation has always been a key task for finding the potential faults in program code. In structural testing, the primary goal is to cover some kinds of structure elements with some specific inputs. Search-based test data generation provides a rational way to handle this difficult problem. In the past, some well-known meta-heuristic search algorithms have been successfully utilized to solve this issue. In this paper, we introduce a variant of genetic algorithm (GA), called quantum-inspired genetic algorithm (QIGA), to generate the test data with stronger coverage ability. In this new algorithm, the traditional binary bit is replaced by a quantum bit (Q-bit) to enlarge the search space so as to avoid falling into local optimal solution. On the other hand, some other strategies such as quantum rotation gate and catastrophe operation are also used to improve algorithm efficiency and quality of test data. In addition, experimental analysis on eight real-world programs is performed to validate the effectiveness of our method. The results show that QIGA-based method can generate test data with higher coverage in much smaller convergence generations than GA-based method. More importantly, our proposed method is more robust for algorithm parameter change.

Download Full-text

IMPROVING THE RELIABILITY AND VALIDITY OF TEST DATA ADEQUACY IN PROGRAMMING ASSESSMENTS

Jurnal Teknologi ◽

10.11113/jt.v77.6201 ◽

2015 ◽

Vol 77 (9) ◽

Author(s):

Rohaida Romli ◽

Shahida Sulaiman ◽

Kamal Zuhairi Zamli

Keyword(s):

Test Data ◽

Dynamic Testing ◽

Reliability And Validity ◽

Structural Testing ◽

Functional Testing ◽

Generation Process ◽

Test Data Generation ◽

Data Generation ◽

Data Adequacy ◽

Programming Courses

Automatic Programming Assessment (or APA) has recently become a notable method in assisting educators of programming courses to automatically assess and grade students’ programming exercises as its counterpart; the typical manual tasks are prone to errors and lead to inconsistency. Practically, this method also provides an alternative means of reducing the educators’ workload effectively. By default, test data generation process plays an important role to perform a dynamic testing on students’ programs. Dynamic testing involves the execution of a program against different inputs or test data and the comparison of the results with the expected output, which must conform to the program specifications. In the software testing field, there have been diverse automated methods for test data generation. Unfortunately, APA rarely adopts these methods. Limited studies have attempted to integrate APA and test data generation to include more useful features and to provide a precise and thorough quality program testing. Thus, we propose a framework of test data generation known as FaSt-Gen covering both the functional and structural testing of a program for APA. Functional testing is a testing that relies on specified functional requirements and focuses the output generated in response to the selected test data and execution, Meanwhile, structural testing looks at the specific program logic to verify how it works. Overall, FaSt-Gen contributes as a means to educators of programming courses to furnish an adequate set of test data to assess students’ programming solutions regardless of having the optimal expertise in the particular knowledge of test cases design. FaSt-Gen integrates the positive and negative testing criteria or so-called reliable and valid test adequacy criteria to derive desired test data and test set schema. As for the functional testing, the integration of specification-derived test and simplified boundary value analysis techniques covering both the criteria. Path coverage criterion guides the test data selection for structural testing. The findings from the conducted controlled experiment and comparative study evaluation show that FaSt-Gen improves the reliability and validity of test data adequacy in programming assessments.

Download Full-text

An Analysis on the Applicability of Meta-Heuristic Searching Techniques for Automated Test Data Generation in Automatic Programming Assessment

Baghdad Science Journal ◽

10.21123/bsj.2019.16.2(si).0515 ◽

2019 ◽

Vol 16 (2(SI)) ◽

pp. 0515 ◽

Cited By ~ 1

Author(s):

Musa Et al.

Keyword(s):

Test Data ◽

Comparative Evaluation ◽

Dynamic Testing ◽

Cuckoo Search ◽

Automatic Programming ◽

Generation Process ◽

Test Data Generation ◽

Data Generation ◽

Automated Test ◽

Automated Test Data Generation

Automatic Programming Assessment (APA) has been gaining lots of attention among researchers mainly to support automated grading and marking of students’ programming assignments or exercises systematically. APA is commonly identified as a method that can enhance accuracy, efficiency and consistency as well as providing instant feedback on students’ programming solutions. In achieving APA, test data generation process is very important so as to perform a dynamic testing on students’ assignment. In software testing field, many researches that focus on test data generation have demonstrated the successful of adoption of Meta-Heuristic Search Techniques (MHST) so as to enhance the procedure of deriving adequate test data for efficient testing. Nonetheless, thus far the researches on APA have not yet usefully exploited the techniques accordingly to include a better quality program testing coverage. Therefore, this study has conducted a comparative evaluation to identify any applicable MHST to support efficient Automated Test Data Generation (ATDG) in executing a dynamic-functional testing in APA. Several recent MHST are included in the comparative evaluation combining both the local and global search algorithms ranging from the year of 2000 until 2018. Result of this study suggests that the hybridization of Cuckoo Search with Tabu Search and lévy flight as one of promising MHST to be applied, as it’s outperforms other MHST with regards to number of iterations and range of inputs.

Download Full-text

Using Artificial Bee Colony Algorithm for Test Data Generation and Path Testing Coverage

Modern Applied Science ◽

10.5539/mas.v12n7p99 ◽

2018 ◽

Vol 12 (7) ◽

pp. 99

Author(s):

Faten Hamad

Keyword(s):

Software Testing ◽

Test Data ◽

Artificial Bee Colony Algorithm ◽

Artificial Bee Colony ◽

Structural Testing ◽

Test Data Generation ◽

Data Generation ◽

Bee Colony ◽

Development Lifecycle ◽

Path Testing

Software testing is a significant stage in software development lifecycle. There are different sorts of' structural software testing methodologies that may be generally utilized and moved forward through enhancing the traverse of all of the conceivable code software paths. The interest for automating data testing is growing; however, manual testing strategies utilization would be expensive and costly. Heuristic measure is being applied to; detect how better the result might be (solution fitness); result development mechanism; and suitableness criteria with stop search mechanism depending on wither a result is found or not. Testing experience could be exploited for finding a solution to the optimization problem by utilizing Meta heuristic procedures. The presented approach might have been tested for five programs to demonstrate the distinctive tests issues. This paper proposes an automatic test data generation approach that use artificial bee colony algorithm for software structural testing, particularly, path testing. This is brought on moving the centralization of data generation testing, as opposed to the automation of the whole testing operation. It executes artificial bee colony algorithm by creating testing data for the criteria of path coverage testing, and then applying the strategy to a group of test programs.

Download Full-text