scholarly journals An Empirical Study of Software Metrics Diversity for Cross-Project Defect Prediction

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Yiwen Zhong ◽  
Kun Song ◽  
ShengKai Lv ◽  
Peng He
Keyword(s):  
Software Metrics ◽  
Historical Data ◽  
Prediction Performance ◽  
Defect Prediction ◽  
Improvement Rate ◽  
Modeling Techniques ◽  
Structural Metrics ◽  
The Impact ◽  
F Measure ◽  
Cross Project

Cross-project defect prediction (CPDP) is a mainstream method estimating the most defect-prone components of software with limited historical data. Several studies investigate how software metrics are used and how modeling techniques influence prediction performance. However, the software’s metrics diversity impact on the predictor remains unclear. Thus, this paper aims to assess the impact of various metric sets on CPDP and investigate the feasibility of CPDP with hybrid metrics. Based on four software metrics types, we investigate the impact of various metric sets on CPDP in terms of F-measure and statistical methods. Then, we validate the dominant performance of CPDP with hybrid metrics. Finally, we further verify the CPDP-OSS feasibility built with three types of metrics (orient-object, semantic, and structural metrics) and challenge them against two current models. The experimental results suggest that the impact of different metric sets on the performance of CPDP is significantly distinct, with semantic and structural metrics performing better. Additionally, trials indicate that it is helpful for CPDP to increase the software’s metrics diversity appropriately, as the CPDP-OSS improvement is up to 53.8%. Finally, compared with two baseline methods, TCA+ and TDSelector, the optimized CPDP model is viable in practice, and the improvement rate is up to 50.6% and 25.7%, respectively.

scholarly journals An Empirical Study on Software Defect Prediction Using CodeBERT Model

2021 ◽  
Vol 11 (11) ◽  
pp. 4793
Author(s):  
Cong Pan ◽  
Minyan Lu ◽  
Biao Xu
Keyword(s):  
Deep Learning ◽  
Software Engineering ◽  
Empirical Study ◽  
Empirical Studies ◽  
Language Model ◽  
Prediction Performance ◽  
Defect Prediction ◽  
Software Defect Prediction ◽  
Software Defect ◽  
Cross Project

Deep learning-based software defect prediction has been popular these days. Recently, the publishing of the CodeBERT model has made it possible to perform many software engineering tasks. We propose various CodeBERT models targeting software defect prediction, including CodeBERT-NT, CodeBERT-PS, CodeBERT-PK, and CodeBERT-PT. We perform empirical studies using such models in cross-version and cross-project software defect prediction to investigate if using a neural language model like CodeBERT could improve prediction performance. We also investigate the effects of different prediction patterns in software defect prediction using CodeBERT models. The empirical results are further discussed.

Estimation of Target Defect Prediction Coverage in Heterogeneous Cross Software Projects

2021 ◽  
Vol 12 (1) ◽  
pp. 73-93
Author(s):  
Rohit Vashisht ◽  
Syed Afzal Murtaza Rizvi
Keyword(s):  
Open Source Software ◽  
Software Metrics ◽  
Research Study ◽  
Defect Prediction ◽  
Gradient Boosting ◽  
Software Projects ◽  
Two Phase ◽  
New Strategy ◽  
Boosting Method ◽  
Cross Project

Heterogeneous cross-project defect prediction (HCPDP) is an evolving area under quality assurance domain which aims to predict defects in a target project that has restricted historical defect data as well as completely non-uniform software metrics from other projects using a model built on another source project. The article discusses a particular source project group's problem of defect prediction coverage (DPC) and also proposes a novel two phase model for addressing this issue in HCPDP. The study has evaluated DPC on 13 benchmarked datasets in three open source software projects. One hundred percent of DPC is achieved with higher defect prediction accuracy for two project group pairs. The issue of partial DPC is found in third prediction pairs and a new strategy is proposed in the research study to overcome this issue. Furthermore, this paper compares HCPDP modeling with reference to with-in project defect prediction (WPDP), both empirically and theoretically, and it is found that the performance of WPDP is highly comparable to HCPDP and gradient boosting method performs best among all three classifiers.

scholarly journals Local versus Global Models for Just-In-Time Software Defect Prediction

2019 ◽  
Vol 2019 ◽  
pp. 1-13 ◽  
Author(s):  
Xingguang Yang ◽  
Huiqun Yu ◽  
Guisheng Fan ◽  
Kai Shi ◽  
Liqiong Chen
Keyword(s):  
Cross Validation ◽  
Prediction Models ◽  
Prediction Performance ◽  
Defect Prediction ◽  
Just In Time ◽  
Software Defect Prediction ◽  
Local Models ◽  
Global Models ◽  
Software Defect ◽  
Cross Project

Just-in-time software defect prediction (JIT-SDP) is an active topic in software defect prediction, which aims to identify defect-inducing changes. Recently, some studies have found that the variability of defect data sets can affect the performance of defect predictors. By using local models, it can help improve the performance of prediction models. However, previous studies have focused on module-level defect prediction. Whether local models are still valid in the context of JIT-SDP is an important issue. To this end, we compare the performance of local and global models through a large-scale empirical study based on six open-source projects with 227417 changes. The experiment considers three evaluation scenarios of cross-validation, cross-project-validation, and timewise-cross-validation. To build local models, the experiment uses the k-medoids to divide the training set into several homogeneous regions. In addition, logistic regression and effort-aware linear regression (EALR) are used to build classification models and effort-aware prediction models, respectively. The empirical results show that local models perform worse than global models in the classification performance. However, local models have significantly better effort-aware prediction performance than global models in the cross-validation and cross-project-validation scenarios. Particularly, when the number of clusters k is set to 2, local models can obtain optimal effort-aware prediction performance. Therefore, local models are promising for effort-aware JIT-SDP.

scholarly journals complexFuzzy: A novel clustering method for selecting training instances of cross-project defect prediction

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Muhammed Maruf Ozturk
Keyword(s):  
Area Under The Curve ◽  
Prediction Performance ◽  
Training Data ◽  
Defect Prediction ◽  
Data Sets ◽  
Clustering Method ◽  
Testing Data ◽  
Proper Training ◽  
Comparison Algorithms ◽  
Cross Project

Over the last decade, researchers have investigated to what extent cross-project defect prediction (CPDP) shows advantages over traditional defect prediction settings. These works do not take training and testing data of defect prediction from the same project. Instead, dissimilar projects are employed. Selecting proper training data plays an important role in terms of the success of CPDP. In this study, a novel clustering method named complexFuzzy is presented for selecting training data of CPDP. The method is developed by determining membership values with the help of some metrics which can be considered as indicators of complexity. First, CPDP combinations are created on 29 different data sets. Subsequently, complexFuzzy is evaluated by considering cluster centers of data sets and comparing some performance measures including area under the curve (AUC) and F-measure. The method is superior to other five comparison algorithms in terms of the distance of cluster centers and prediction performance.

scholarly journals An Improved Method for Cross-Project Defect Prediction by Simplifying Training Data

2018 ◽  
Vol 2018 ◽  
pp. 1-18 ◽  
Author(s):  
Peng He ◽  
Yao He ◽  
Lvjun Yu ◽  
Bing Li
Keyword(s):  
Euclidean Distance ◽  
Historical Data ◽  
Training Data ◽  
Defect Prediction ◽  
Improved Method ◽  
Additional Experiment ◽  
Weighted Function ◽  
Public Repositories ◽  
Better Than ◽  
Cross Project

Cross-project defect prediction (CPDP) on projects with limited historical data has attracted much attention. To the best of our knowledge, however, the performance of existing approaches is usually poor, because of low quality cross-project training data. The objective of this study is to propose an improved method for CPDP by simplifying training data, labeled as TDSelector, which considers both the similarity and the number of defects that each training instance has (denoted by defects), and to demonstrate the effectiveness of the proposed method. Our work consists of three main steps. First, we constructed TDSelector in terms of a linear weighted function of instances’ similarity and defects. Second, the basic defect predictor used in our experiments was built by using the Logistic Regression classification algorithm. Third, we analyzed the impacts of different combinations of similarity and the normalization of defects on prediction performance and then compared with two existing methods. We evaluated our method on 14 projects collected from two public repositories. The results suggest that the proposed TDSelector method performs, on average, better than both baseline methods, and the AUC values are increased by up to 10.6% and 4.3%, respectively. That is, the inclusion of defects is indeed helpful to select high quality training instances for CPDP. On the other hand, the combination of Euclidean distance and linear normalization is the preferred way for TDSelector. An additional experiment also shows that selecting those instances with more bugs directly as training data can further improve the performance of the bug predictor trained by our method.

scholarly journals Software Defect Prediction Based on Elman Neural Network and Cuckoo Search Algorithm

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Kun Song ◽  
ShengKai Lv ◽  
Die Hu ◽  
Peng He
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Search Algorithm ◽  
Cuckoo Search ◽  
Defect Prediction ◽  
Main Task ◽  
Elman Neural Network ◽  
The Impact ◽  
F Measure

In software engineering, defect prediction is significantly important and challenging. The main task is to predict the defect proneness of the modules. It helps developers find bugs effectively and prioritize their testing efforts. At present, a lot of valuable researches have been done on this topic. However, few studies take into account the impact of time factors on the prediction results. Therefore, in this paper, we propose an improved Elman neural network model to enhance the adaptability of the defect prediction model to the time-varying characteristics. Specifically, we optimized the initial weights and thresholds of the Elman neural network by incorporating adaptive step size in the Cuckoo Search (CS) algorithm. We evaluated the proposed model on 7 projects collected from public PROMISE repositories. The results suggest that the contribution of the improved CS algorithm to Elman neural network model is prominent, and the prediction performance of our method is better than that of 5 baselines in terms of F-measure and Cliff’s Delta values. The F-measure values are generally increased with a maximum growth rate of 49.5% for the POI project.

Bug Severity Assessment in Cross Project Context and Identifying Training Candidates

2017 ◽  
Vol 16 (01) ◽  
pp. 1750005 ◽  
Author(s):  
V. B. Singh ◽  
Sanjay Misra ◽  
Meera Sharma
Keyword(s):  
Prediction Models ◽  
Historical Data ◽  
Support Vector ◽  
Reliable Prediction ◽  
Bug Reports ◽  
Severity Prediction ◽  
Data Problem ◽  
F Measure ◽  
Better Than ◽  
Cross Project

The automatic bug severity prediction will be useful in prioritising the development efforts, allocating resources and bug fixer. It needs historical data on which classifiers can be trained. In the absence of such historical data cross project prediction provides a good solution. In this paper, our objective is to automate the bug severity prediction by using a bug metric summary and to identify best training candidates in cross project context. The text mining technique has been used to extract the summary terms and trained the classifiers using these terms. About 63 training candidates have been designed by combining seven datasets of Eclipse projects to develop the severity prediction models. To deal with the imbalance bug data problem, we employed two approaches of ensemble by using two operators available in RapidMiner: Vote and Bagging. Results show that k-Nearest Neighbour (k-NN) performance is better than the Support Vector Machine (SVM) performance. Naive Bayes f-measure performance is poor, i.e. below 34.25%. In case of k-NN, developing training candidates by combining more than one training datasets helps in improving the performances (f-measure and accuracy). The two ensemble approaches have improved the f-measure performance up to 5% and 10% respectively for the severity levels having less number of bug reports in comparison of major severity level. We have further motivated the paper with a cross project bug severity prediction between Eclipse and Mozilla products. Results show that Mozilla products can be used to build reliable prediction models for Eclipse products and vice versa in case of SVM and k-NN classifiers.

Sign in / Sign up

Export Citation Format

Share Document