Automatic detection of Long Method and God Class code smells through neural source code embeddings

<p>Code smells are structures in code that often have a negative impact on its quality. Manually detecting code smells is challenging and researchers proposed many automatic code smell detectors. Most of the studies propose detectors based on code metrics and heuristics. However, these studies have several limitations, including evaluating the detectors using small-scale case studies and an inconsistent experimental setting. Furthermore, heuristic-based detectors suffer from limitations that hinder their adoption in practice. Thus, researchers have recently started experimenting with machine learning (ML) based code smell detection. </p><p>This paper compares the performance of multiple ML-based code smell detection models against multiple traditionally employed metric-based heuristics for detection of God Class and Long Method code smells. We evaluate the effectiveness of different source code representations for machine learning: traditionally used code metrics and code embeddings (code2vec, code2seq, and CuBERT).<br></p><p>We perform our experiments on the large-scale, manually labeled MLCQ dataset. We consider the binary classification problem – we classify the code samples as smelly or non-smelly and use the F1-measure of the minority (smell) class as a measure of performance. In our experiments, the ML classifier trained using CuBERT source code embeddings achieved the best performance for both God Class (F-measure of 0.53) and Long Method detection (F-measure of 0.75). With the help of a domain expert, we perform the error analysis to discuss the advantages of the CuBERT approach.<br></p><p>This study is the first to evaluate the effectiveness of pre-trained neural source code embeddings for code smell detection to the best of our knowledge. A secondary contribution of our study is the systematic evaluation of the effectiveness of multiple heuristic-based approaches on the same large-scale, manually labeled MLCQ dataset.<br></p>

Automatic detection of Long Method and God Class code smells through neural source code embeddings

<p>Code smells are structures in code that often have a negative impact on its quality. Manually detecting code smells is challenging and researchers proposed many automatic code smell detectors. Most of the studies propose detectors based on code metrics and heuristics. However, these studies have several limitations, including evaluating the detectors using small-scale case studies and an inconsistent experimental setting. Furthermore, heuristic-based detectors suffer from limitations that hinder their adoption in practice. Thus, researchers have recently started experimenting with machine learning (ML) based code smell detection. </p><p>This paper compares the performance of multiple ML-based code smell detection models against multiple traditionally employed metric-based heuristics for detection of God Class and Long Method code smells. We evaluate the effectiveness of different source code representations for machine learning: traditionally used code metrics and code embeddings (code2vec, code2seq, and CuBERT).<br></p><p>We perform our experiments on the large-scale, manually labeled MLCQ dataset. We consider the binary classification problem – we classify the code samples as smelly or non-smelly and use the F1-measure of the minority (smell) class as a measure of performance. In our experiments, the ML classifier trained using CuBERT source code embeddings achieved the best performance for both God Class (F-measure of 0.53) and Long Method detection (F-measure of 0.75). With the help of a domain expert, we perform the error analysis to discuss the advantages of the CuBERT approach.<br></p><p>This study is the first to evaluate the effectiveness of pre-trained neural source code embeddings for code smell detection to the best of our knowledge. A secondary contribution of our study is the systematic evaluation of the effectiveness of multiple heuristic-based approaches on the same large-scale, manually labeled MLCQ dataset.<br></p>

Towards a systematic approach to manual annotation of code smells

<div>Code smells are structures in code that indicate the presence of maintainability issues. A significant problem with code smells is their ambiguity. They are challenging to define, and software engineers have a different understanding of what a code smell is and which code suffers from code smells.</div><div>A solution to this problem could be an AI digital assistant that understands code smells and can detect (and perhaps resolve) them. However, it is challenging to develop such an assistant as there are few usable datasets of code smells on which to train and evaluate it. Furthermore, the existing datasets suffer from issues that mostly arise from an unsystematic approach used for their construction.</div><div>Through this work, we address this issue by developing a procedure for the systematic manual annotation of code smells. We use this procedure to build a dataset of code smells. During this process, we refine the procedure and identify recommendations and pitfalls for its use. The primary contribution is the proposed annotation model and procedure and the annotators’ experience report. The dataset and supporting tool are secondary contributions of our study. Notably, our dataset includes open-source projects written in the C# programming language, while almost all manually annotated datasets contain projects written in Java.</div>

Revealing Developers’ Arguments on Validating the Incidence of Code Smells: A Focus Group Experience

Identifying code smells is considered a subjective task. Unfortunately, current automated detection tools cannot deal with such subjectivity, requiring human validation. Developers tend to follow different, albeit complementary, strategies when validating the identified smells. Intending to find out developers' arguments when validating the incidence of code smells, we conducted a focus group session with developers familiar with identifying code smells. We distributed them among two groups, in which they had to argue about the incidence of a code smell: either accepting or rejecting its presence. Based on their arguments, we compiled a set of general heuristics that developers follow when validating smells. We then used these heuristics for composing validation items. We understand that the set of validation items proposed may support developers in reflecting on the incidence of code smells. However, further studies are needed for reaching a more comprehensive and optimized set. The experience of this study reveals that conducting focus group sessions is helpful to emerge the tacit knowledge of developers when validating code smells.

Design and Implementation of a Web-Based Application for Code Smells Repository

Pitfalls in software development process can be prevented by learning from other people's mistakes. Software practitioners and researchers document lessons learned and the knowledge about best practices is spread over literature. Presence of code smells does not indicate that software won’t work, but it will reveal deeper problems and rising risk of failure in future. Software metrics are applied to detect code smells whereas refactoring can remove code smells, improve code quality and make it simpler and cleaner. Detection tools facilitate management of code smells. Knowledge about code smells and related concepts can assist the software maintenance process. Exploratory analysis of code smells carried out in this paper, covers collecting data about code smells, identifying related concepts, categorizing and organizing this knowledge into a code smell repository, which can be made available to software developers. A detailed literature survey is carried out to identify code smells and related concepts. An initial list of 22 code smells proposed in 1999 has grown over the years into 65 code smells. The relationship between code smells, software metrics, refactoring methods and detection tools available in literature is also documented. Templates are designed that capture knowledge about code smells and related concepts. A code smell repository is designed and implemented to maintain all the information gathered about code smells and related concepts and is made available to software practitioners. All the knowledge about code smells found in literature is collected, organized and made accessible.

Bad Smells of Gang of Four Design Patterns: A Decade Systematic Literature Review

Gang of Four (GoF) design patterns are widely approved solutions for recurring software design problems, and their benefits to software quality are extensively studied. However, the occurrence of bad smells in design patterns increases the crisis of degenerating design patterns’ structure and behavior. Their occurrences are detrimental to the benefits of design patterns and they influence software sustainability by increasing maintenance costs and energy consumption. Despite the destructive roles of bad smells in such designs, there are an absence of studies systematically reviewing bad smells of GoF design patterns. This study systematically reviews a 10-year state of the art sample, identifying 16 studies investigating this phenomenon. Following a thorough evaluation of the full contents, we observed that the occurrence of bad smells have been investigated in proportion to four granularity levels of analysis: Design level, category level, pattern level, and role level. We identified 28 bad smells, categorized under code smells and grime symptoms, and emphasized their relationship with GoF pattern types and categories. The utilization of design pattern bad smell detection approaches and datasets were also discussed. Consequently, we observed that the research phenomenon is growing intensively, with a prominent focus of studies analyzing code smell occurrences rather than grime occurrences, at various granularity levels. Finally, we uncovered research gaps and areas with significant potentials for future research.