scholarly journals SHFuzz: Selective Hybrid Fuzzing with Branch Scheduling Based on Binary Instrumentation

2020 ◽  
Vol 10 (16) ◽  
pp. 5449
Author(s):  
Xianya Mi ◽  
Baosheng Wang ◽  
Yong Tang ◽  
Pengfei Wang ◽  
Bo Yu
Keyword(s):  
Test Cases ◽  
Calculation Algorithm ◽  
Priority Score ◽  
Binary Instrumentation ◽  
Synchronization Mechanism ◽  
Code Coverage ◽  
Concolic Execution ◽  
Real World Applications ◽  
Bug Finding ◽  
Number Of Branches

Hybrid fuzzing is a popular software testing technique that combines random fuzzing with concolic execution. It is widely used in the security domain known for its ability to find deeply hidden vulnerabilities and reach high code coverage. Hybrid fuzzing is based on negating branches in the execution path of a specific input to generate new test cases. However, due to numerous inputs and related branches, it does not show the best of its effectiveness without input and branch selection methods. In this paper, we systematically analyze the branch scheduling problem in the internal attributes of hybrid fuzzing, focusing on the synchronization mechanism. To solve the problems, we propose the Selective Hybrid Fuzzing (SHF) approach with branch scheduling based on binary instrumentation. There are two major parts to the SHF approach: (1) we propose a critical branch selection algorithm to select critical branches by three metrics: hit accuracy, solvability, and complexity; (2) we propose a priority score calculation algorithm to select inputs by the number of critical branches. With the SHF approach, we choose only the branches that can be negated to generate new coverage, instead of repeatedly executing the same branches and generating duplicates of inputs. We implement a hybrid fuzzer called SHFuzz with our SHF approach and compare it with the state-of-the-art hybrid fuzzer QSYM. In the evaluation, SHFuzz outperforms QSYM in 20 real-world applications from the Google Fuzzer Test Suite and other program suites in a 12 h test. On average, SHFuzz achieves 8.40% more code coverage and 100 more unique crashes in each application. Our work also finds existing vulnerabilities 7.85× faster than QSYM. We also find new bugs by SHFuzz, which QSYM fails to find. Our evaluation shows that the selective hybrid fuzzing approach can reduce the number of branches executed in concolic execution, enhancing hybrid fuzzing on code coverage and bug finding capabilities.

scholarly journals Selection of Best Test Cases using Support Vector Machine for ARPT

2019 ◽  
Vol 8 (3) ◽  
pp. 4265-4271
Keyword(s):  
Support Vector Machine ◽  
Software Testing ◽  
Support Vector ◽  
Test Cases ◽  
Testing Strategy ◽  
Random Partition ◽  
Code Coverage ◽  
Time Consumption ◽  
Branch Coverage ◽  
Selection Of

Software testing is an essential activity in software industries for quality assurance; subsequently, it can be effectively removing defects before software deployment. Mostly good software testing strategy is to accomplish the fundamental testing objective while solving the trade-offs between effectiveness and efficiency testing issues. Adaptive and Random Partition software Testing (ARPT) approach was a combination of Adaptive Testing (AT) and Random Partition Approach (RPT) used to test software effectively. It has two variants they are ARPT-1 and ARPT-2. In ARPT-1, AT was used to select a certain number of test cases and then RPT was used to select a number of test cases before returning to AT. In ARPT-2, AT was used to select the first m test cases and then switch to RPT for the remaining tests. The computational complexity for random partitioning in ARPT was solved by cluster the test cases using a different clustering algorithm. The parameters of ARPT-1 and ARPT-2 needs to be estimated for different software, it leads to high computation overhead and time consumption. It was solved by Improvised BAT optimization algorithms and this approach is named as Optimized ARPT1 (OARPT1) and OARPT2. By using all test cases in OARPT will leads to high time consumption and computational overhead. In order to avoid this problem, OARPT1 with Support Vector Machine (OARPT1-SVM) and OARPT2- SVM are introduced in this paper. The SVM is used for selection of best test cases for OARPT-1 and OARPT-2 testing strategy. The SVM constructs hyper plane in a multi-dimensional space which is used to separate test cases which have high code and branch coverage and test cases which have low code and branch coverage. Thus, the SVM selects the best test cases for OARPT-1 and OARPT-2. The selected test cases are used in OARPT-1 and OARPT-2 to test software. In the experiment, three different software is used to prove the effectiveness of proposed OARPT1- SVM and OARPT2-SVM testing strategies in terms of time consumption, defect detection efficiency, branch coverage and code coverage.

scholarly journals FuSeBMC: A White-Box Fuzzer for Finding Security Vulnerabilities in C Programs (Competition Contribution)

2021 ◽  
pp. 363-367 ◽  
Author(s):  
Kaled M. Alshmrany ◽  
Rafael S. Menezes ◽  
Mikhail R. Gadelha ◽  
Lucas C. Cordeiro
Keyword(s):  
Model Checking ◽  
Symbolic Execution ◽  
Bounded Model Checking ◽  
Test Cases ◽  
Security Vulnerabilities ◽  
C Programs ◽  
Code Coverage

AbstractWe describe and evaluate a novel white-box fuzzer for C programs named , which combines fuzzing and symbolic execution, and applies Bounded Model Checking (BMC) to find security vulnerabilities in C programs. explores and analyzes C programs (1) to find execution paths that lead to property violations and (2) to incrementally inject labels to guide the fuzzer and the BMC engine to produce test-cases for code coverage. successfully participates in Test-Comp’21 and achieves first place in the category and second place in the category.

Morpheus Web Testing: A Tool for Generating Test Cases for Widget Based Web Applications

2021 ◽  
Author(s):  
Romulo de Almeida Neves ◽  
Willian Massami Watanabe ◽  
Rafael Oliveira
Keyword(s):  
User Interfaces ◽  
Web Application ◽  
Web Applications ◽  
Test Cases ◽  
Code Coverage ◽  
Web Testing ◽  
The Cost ◽  
Software Lifecycle ◽  
The Web

Context: Widgets are reusable User Interfaces (UIs) components frequently delivered in Web applications.In the web application, widgets implement different interaction scenarios, such as buttons, menus, and text input.Problem: Tests are performed manually, so the cost associated with preparing and executing test cases is high.Objective: Automate the process of generating functional test cases for web applications, using intermediate artifacts of the web development process that structure widgets in the web application. The goal of this process is to ensure the quality of the software, reduce overall software lifecycle time and the costs associated with tests.Method:We elaborated a test generation strategy and implemented this strategy in a tool, Morpheus Web Testing. Morpheus Web Testing extracts widget information from Java Server Faces artifacts to generate test cases for JSF web applications. We conducted a case study for comparing Morpheus Web Testing with a state of the art tool (CrawlJax).Results: The results indicate evidence that the approach Morpheus Web Testing managed to reach greater code coverage compared to a CrawlJax.Conclusion: The achieved coverage values represent evidence that the results obtained from the proposed approach contribute to the process of automated test software engineering in the industry.

Virtual Test Environment for Self-Optimizing Systems

2013 ◽  
Author(s):  
Jörg Stöcklein ◽  
Daniel Baldin ◽  
Wolfgang Müller ◽  
Tao Xie
Keyword(s):  
Operating System ◽  
Real Time ◽  
Virtual Prototype ◽  
Test Cases ◽  
Test Environment ◽  
Dead Code ◽  
Real Time Operating System ◽  
The Real ◽  
Code Coverage ◽  
Virtual Test

In our paper we present a virtual test environment for self-optimizing systems based on mutant based testing to validate user tasks of a real-time operating system. This allows the efficient validation of the code coverage of the test cases and therefore helps to detect errors in order to improving the reliability of the system software. Technically we are able to run and test the software on both systems. By writing application software and setting up the virtual test environment properly, we define our test cases. To validate the code coverage for our test cases, we use the approach of mutant based testing. By running this mutated code on our virtual prototype in the virtual test environment, we are able to efficiently validate the code coverage and are able to detect bugs in the application code or detect dead code that is not executed. Finding non-executing code leads to redefinition of our test cases by either changing the test environment or the application code in the case of dead code. We implemented the virtual test environment on top of the third party low cost VR system Unity 3D, which is frequently used in entertainment and education. We demonstrate our concepts by the example of our BeBot robot vehicles. The implementation is based on our self-optimizing real-time operating system ORCOS and we used the tool CERTITUDE(TM) for generating the mutations in our application code. Our BeBot virtual prototype in our virtual test environment implements the same low-level interface to the underlying hardware as the real BeBot. This allows a redirection of commands in ORCOS to either the real or the virtual BeBot in order to provide a VR based platform for early software development as well as ensures comparable conditions under both environments. Our example applies a virtual BeBot that drives through a labyrinth utilizing its IR sensors for navigation. The mutant based testing checks if all situations implemented by the software to navigate through the labyrinth are covered by our tests.

Test Case Prioritization Technique Based on Error Probability and Severity of UML Models

2018 ◽  
Vol 28 (06) ◽  
pp. 831-844 ◽  
Author(s):  
Tianning Zhang ◽  
Xingqi Wang ◽  
Dan Wei ◽  
Jinglong Fang
Keyword(s):  
Flight Control ◽  
Error Probability ◽  
Unified Modeling Language ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Unified Modeling ◽  
Code Coverage ◽  
Development Life Cycle ◽  
The Cost

Test case prioritization is one of the most useful activities in testing. Most existing test case prioritization techniques are based on code coverage, which requires access to source code. However, code-based testing comes late in the software development life cycle, when errors are detected, the cost of testing is very high. Therefore, in this paper, we provide a test case prioritization technique based on Unified Modeling Language (UML) model, built before coding, to detect errors as earlier as possible and reduce the cost of modification. The technique consists of the following main parts: (1) using C&K metrics to estimate the error probability of class; (2) using dependences, obtained from the model slicing, to estimate error severity; (3) generating test case priority from error probability and severity, then prioritizing the test case. With our technique, test engineers need the UML model only and the test cases can be prioritized automatically. To evaluate our technique, we applied our technique to unmanned aerial vehicles (UAV) flight control system and performed test case prioritization. The results show that the error can be detected effectively and stability can be increased significantly as compared to the current code-based techniques.

scholarly journals Survey on Various Testing Techniques and Strategies for Bug Finding

2013 ◽  
Vol 11 (1) ◽  
pp. 2150-2155
Author(s):  
Mohit Kumar ◽  
Geetika Gandhi ◽  
Sushil Garg
Keyword(s):  
Software Testing ◽  
Verification And Validation ◽  
Test Cases ◽  
Validation Process ◽  
Software Product ◽  
Bug Finding ◽  
Testing Techniques ◽  
The Right ◽  
Selection Of

Software testing is verification and validation process aimed for evaluating a program and ensures that it meets the required result. The main goal of software testing is to uncover the errors in software. So the main aim of test cases is to derive set of tests that have highest probability of finding bugs. There are many approaches to software testing, but effective testing of any software product is essentially a tough process. It is nearly impossible to find all the errors in the program. The major problem in testing is what would be the strategy that we should adopt for testing. Thus, the selection of right strategy at the right time will make the software testing efficient and effective. In this paper I have described software testing techniques which are classified by purpose.

scholarly journals K-Step Crossover Method based on Genetic Algorithm for Test Suite Prioritization in Regression Testing

2021 ◽  
Vol 27 (2) ◽  
pp. 170-189
Author(s):  
P. K. Gupta
Keyword(s):  
Genetic Algorithm ◽  
Regression Testing ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Average Percentage ◽  
Code Coverage ◽  
Test Case Selection ◽  
Case Selection

Software is an integration of numerous programming modules  (e.g., functions, procedures, legacy system, reusable components, etc.) tested and combined to build the entire module. However, some undesired faults may occur due to a change in modules while performing validation and verification. Retesting of entire software is a costly affair in terms of money and time. Therefore, to avoid retesting of entire software, regression testing is performed. In regression testing, an earlier created test suite is used to retest the software system's modified module. Regression Testing works in three manners; minimizing test cases, selecting test cases, and prioritizing test cases. In this paper, a two-phase algorithm has been proposed that considers test case selection and test case prioritization technique for performing regression testing on several modules ranging from a smaller line of codes to huge line codes of procedural language. A textual based differencing algorithm has been implemented for test case selection. Program statements modified between two modules are used for textual differencing and utilized to identify test cases that affect modified program statements. In the next step, test case prioritization is implemented by applying the Genetic Algorithm for code/condition coverage. Genetic operators: Crossover and Mutation have been applied over the initial population (i.e. test cases), taking code/condition coverage as fitness criterion to provide a prioritized test suite. Prioritization algorithm can be applied over both original and reduced test suite depending upon the test suite's size or the need for accuracy. In the obtained results, the efficiency of the prioritization algorithms has been analyzed by the Average Percentage of Code Coverage (APCC) and Average Percentage of Code Coverage with cost (APCCc). A comparison of the proposed approach is also done with the previously proposed methods and it is observed that APCC & APCCc values achieve higher percentage values faster in the case of the prioritized test suite in contrast to the non-prioritized test suite.

Test Case Reduction Using Data Mining Technique

2016 ◽  
Vol 4 (4) ◽  
pp. 56-70 ◽  
Author(s):  
Ahmad A. Saifan ◽  
Emad Alsukhni ◽  
Hanadi Alawneh ◽  
Ayat AL Sbaih
Keyword(s):  
Data Mining ◽  
Clustering Algorithm ◽  
Test Case ◽  
Test Cases ◽  
Data Mining Technique ◽  
Code Coverage ◽  
Test Case Selection ◽  
Data Mining Approach ◽  
Case Selection ◽  
Using Data

Software testing is a process of ratifying the functionality of software. It is a crucial area which consumes a great deal of time and cost. The time spent on testing is mainly concerned with testing large numbers of unreliable test cases. The authors' goal is to reduce the numbers and offer more reliable test cases, which can be achieved using certain selection techniques to choose a subset of existing test cases. The main goal of test case selection is to identify a subset of the test cases that are capable of satisfying the requirements as well as exposing most of the existing faults. The state of practice among test case selection heuristics is cyclomatic complexity and code coverage. The authors used clustering algorithm which is a data mining approach to reduce the number of test cases. Their approach was able to obtain 93 unique effective test cases out a total of 504.

Functional Test Case Generator for Contiki Operating System and Cooja Simulator

2019 ◽  
Vol 10 (1) ◽  
pp. 1251-1257
Author(s):  
Abhinandan H Patil
Keyword(s):  
Operating System ◽  
Functional Test ◽  
Test Suite ◽  
Target System ◽  
Test Case ◽  
Test Cases ◽  
Code Coverage ◽  
Regression Test ◽  
Run Time ◽  
Configuration Systems

Evolving multi-parameter, multi-configuration systems require regression test suite that can be customized. This is in terms of run time. Run time can be customized by generating the combinations using combinatorial techniques. For systems like Contiki operating system, the test cases need to be executed in its simulator Cooja. Executing test cases in a simulator requires functional test cases to be generated from the combinatorial parameter combinations obtained. In this work we present a methodology to generate the functional test cases. We present Functional Test Case Generator for Contiki and Cooja (FTCGCC), which is a tool developed using our methodology. We demonstrate use of our tool by generating customizable regression test suite for Contiki and Cooja using code coverage as criteria. FTCGCC is developed for the test case generation when target System Under Test is IoT operating system Contiki and its simulator Cooja.

Sign in / Sign up

Export Citation Format

Share Document