scholarly journals Quick remedy commits and their impact on mining software repositories

2021 ◽  
Vol 27 (1) ◽  
Author(s):  
Fengcai Wen ◽  
Csaba Nagy ◽  
Michele Lanza ◽  
Gabriele Bavota
Keyword(s):  
Software Maintenance ◽  
Noisy Data ◽  
Mining Software Repositories ◽  
Software Repositories ◽  
Bug Fixing ◽  
Manual Analysis ◽  
Data Points ◽  
Software Maintenance And Evolution ◽  
Code Changes ◽  
Different Parts

AbstractMost changes during software maintenance and evolution are not atomic changes, but rather the result of several related changes affecting different parts of the code. It may happen that developers omit needed changes, thus leaving a task partially unfinished, introducing technical debt or injecting bugs. We present a study investigating “quick remedy commits” performed by developers to implement changes omitted in previous commits. With quick remedy commits we refer to commits that (i) quickly follow a commit performed by the same developer, and (ii) aim at remedying issues introduced as the result of code changes omitted in the previous commit (e.g., fix references to code components that have been broken as a consequence of a rename refactoring) or simply improve the previously committed change (e.g., improve the name of a newly introduced variable). Through a manual analysis of 500 quick remedy commits, we define a taxonomy categorizing the types of changes that developers tend to omit. The taxonomy can (i) guide the development of tools aimed at detecting omitted changes and (ii) help researchers in identifying corner cases that must be properly handled. For example, one of the categories in our taxonomy groups the reverted commits, meaning changes that are undone in a subsequent commit. We show that not accounting for such commits when mining software repositories can undermine one’s findings. In particular, our results show that considering completely reverted commits when mining software repositories accounts, on average, for 0.07 and 0.27 noisy data points when dealing with two typical MSR data collection tasks (i.e., bug-fixing commits identification and refactoring operations mining, respectively).

Automatically Labelled Software Topic Model

2020 ◽  
Vol 11 (1) ◽  
pp. 57-78
Author(s):  
Youcef Bouziane ◽  
Mustapha Kamel Abdi ◽  
Salah Sadou
Keyword(s):  
Information Retrieval ◽  
Software Engineering ◽  
Topic Model ◽  
Mining Software Repositories ◽  
Valuable Data ◽  
Empirical Results ◽  
Software Repositories ◽  
Manual Analysis ◽  
Novel Approach ◽  
Support Software

Public software repositories (SR) maintain a massive amount of valuable data offering opportunities to support software engineering (SE) tasks. Researchers have applied information retrieval techniques in mining software repositories. Topic models are one of these techniques. However, this technique does not give an interpretation nor labels to the extracted topics and it requires manual analysis to identify them. Some approaches were proposed to automatically label the topics using tags in SR, but they do not consider the existence of spam-tags and they have difficulties to scale to large tag space. This article introduces a novel approach called automatically labelled software topic model (AL-STM) that labels the topics based on observed tags in SR. It mitigates the shortcomings of manual and automatic labelling of topics in SE. AL-STM is implemented using 22K GitHub projects and evaluated in a SE task (tag recommending) against the currently used techniques. The empirical results suggest that AL-STM is more robust in terms of MAP and nDCG, and more scalable to large tag space.

scholarly journals Augmenting Bug Localization with Part-of-Speech and Invocation

2017 ◽  
Vol 27 (06) ◽  
pp. 925-949 ◽  
Author(s):  
Yu Zhou ◽  
Yanxiang Tong ◽  
Taolue Chen ◽  
Jin Han
Keyword(s):  
Software Maintenance ◽  
Large Scale ◽  
Bug Localization ◽  
Bug Reports ◽  
Part Of Speech ◽  
Adaptive Technique ◽  
Bug Report ◽  
Software Maintenance And Evolution ◽  
Speech Features ◽  
Localization Approach

Bug localization represents one of the most expensive, as well as time-consuming, activities during software maintenance and evolution. To alleviate the workload of developers, numerous methods have been proposed to automate this process and narrow down the scope of reviewing buggy files. In this paper, we present a novel buggy source-file localization approach, using the information from both the bug reports and the source files. We leverage the part-of-speech features of bug reports and the invocation relationship among source files. We also integrate an adaptive technique to further optimize the performance of the approach. The adaptive technique discriminates Top 1 and Top N recommendations for a given bug report and consists of two modules. One module is to maximize the accuracy of the first recommended file, and the other one aims at improving the accuracy of the fixed defect file list. We evaluate our approach on six large-scale open source projects, i.e. ASpectJ, Eclipse, SWT, Zxing, Birt and Tomcat. Compared to the previous work, empirical results show that our approach can improve the overall prediction performance in all of these cases. Particularly, in terms of the Top 1 recommendation accuracy, our approach achieves an enhancement from 22.73% to 39.86% for ASpectJ, from 24.36% to 30.76% for Eclipse, from 31.63% to 46.94% for SWT, from 40% to 55% for ZXing, from 7.97% to 21.99% for Birt, and from 33.37% to 38.90% for Tomcat.

BIAS-VARIANCE CONTROL VIA HARD POINTS SHAVING

2004 ◽  
Vol 18 (05) ◽  
pp. 891-903 ◽  
Author(s):  
STEFANO MERLER ◽  
BRUNO CAPRILE ◽  
CESARE FURLANELLO
Keyword(s):  
Control Strategy ◽  
Noisy Data ◽  
Real Data ◽  
Training Data ◽  
Regularization Technique ◽  
Data Points ◽  
Classification Tasks ◽  
Bias Variance

In this paper, we propose a regularization technique for AdaBoost. The method implements a bias-variance control strategy in order to avoid overfitting in classification tasks on noisy data. The method is based on a notion of easy and hard training patterns as emerging from analysis of the dynamical evolutions of AdaBoost weights. The procedure consists in sorting the training data points by a hardness measure, and in progressively eliminating the hardest, stopping at an automatically selected threshold. Effectiveness of the method is tested and discussed on synthetic as well as real data.

scholarly journals GrimoireLab: A toolset for software development analytics

2021 ◽  
Vol 7 ◽  
pp. e601
Author(s):  
Santiago Dueñas ◽  
Valerio Cosentino ◽  
Jesus M. Gonzalez-Barahona ◽  
Alvaro del Castillo San Felix ◽  
Daniel Izquierdo-Cortazar ◽  
...  
Keyword(s):  
Software Development ◽  
Preliminary Analysis ◽  
Data Retrieval ◽  
Mining Software Repositories ◽  
Data Sources ◽  
Data Sets ◽  
Community Based ◽  
Academic Environments ◽  
Software Repositories ◽  
Main Components

Background After many years of research on software repositories, the knowledge for building mature, reusable tools that perform data retrieval, storage and basic analytics is readily available. However, there is still room to improvement in the area of reusable tools implementing this knowledge. Goal To produce a reusable toolset supporting the most common tasks when retrieving, curating and visualizing data from software repositories, allowing for the easy reproduction of data sets ready for more complex analytics, and sparing the researcher or the analyst of most of the tasks that can be automated. Method Use our experience in building tools in this domain to identify a collection of scenarios where a reusable toolset would be convenient, and the main components of such a toolset. Then build those components, and refine them incrementally using the feedback from their use in both commercial, community-based, and academic environments. Results GrimoireLab, an efficient toolset composed of five main components, supporting about 30 different kinds of data sources related to software development. It has been tested in many environments, for performing different kinds of studies, and providing different kinds of services. It features a common API for accessing the retrieved data, facilities for relating items from different data sources, semi-structured storage for easing later analysis and reproduction, and basic facilities for visualization, preliminary analysis and drill-down in the data. It is also modular, making it easy to support new kinds of data sources and analysis. Conclusions We present a mature toolset, widely tested in the field, that can help to improve the situation in the area of reusable tools for mining software repositories. We show some scenarios where it has already been used. We expect it will help to reduce the effort for doing studies or providing services in this area, leading to advances in reproducibility and comparison of results.

Sign in / Sign up

Export Citation Format

Share Document