scholarly journals Differential Testing of Static Analyzers

2020 ◽  
Author(s):  
Gabor Horvath ◽  
Reka Nikolett Kovacs ◽  
Peter Szecsi

Program faults, best known as bugs, are practically unavoidable in today's ever growing software systems. One increasingly popular way of eliminating them, besides tests, dynamic analysis, and fuzzing, is using static analysis based bug-finding tools. Such tools are capable of finding surprisingly sophisticated bugs automatically by inspecting the source code. Their analysis is usually both unsound and incomplete, but still very useful in practice, as they can find non-trivial problems in a reasonable time (e.g. within hours, for an industrial project) without human intervention Because the problems that static analyzers try to solve are hard, usually intractable, they use various approximations that need to be fine-tuned in order to grant a good user experience (i.e. as many interesting bugs with as few distracting false alarms as possible). For each newly introduced heuristic, this normally happens by performing differential testing of the analyzer on a lot of widely used open source software projects that are known to use related language constructs extensively. In practice, this process is ad hoc, error-prone, poorly reproducible and its results are hard to share. We present a set of tools that aim to support the work of static analyzer developers by making differential testing easier. Our framework includes tools for automatic test suite selection, automated differential experiments, coverage information of increased granularity, statistics collection, metric calculations, and visualizations, all resulting in a convenient, shareable HTML report.

2020 ◽  
Author(s):  
Willian N. Oizumi ◽  
Alessandro F. Garcia

Design problems affect most software projects and make their maintenance expensive and impeditive. Thus, the identification of potential design problems in the source code – which is very often the only available and upto-date artifact in a project – becomes essential in long-living software systems. This identification task is challenging as the reification of design problems in the source code tend to be scattered through several code elements. However, stateof-the-art techniques do not provide enough information to effectively help developers in this task. In this work, we address this challenge by proposing a new technique to support developers in revealing design problems. This technique synthesizes information about potential design problems, which are materialized in the implementation under the form of syntactic and semantic anomaly agglomerations. Our evaluation shows that the proposed synthesis technique helps to reveal more than 1200 design problems across 7 industry-strength systems, with a median precision of 71% and a median recall of 78%. The relevance of our work has been widely recognized by the software engineering community through 2 awards and 7 publications in international and national venues.


Author(s):  
D. Berry

Open source software (OSS) is computer software that has its underlying source code made available under a licence. This can allow developers and users to adapt and improve it (Raymond, 2001). Computer software can be broadly split into two development models: • Proprietary, or closed software, owned by a company or individual. Copies of the binary are made public; the source code is not usually made public. • Open-source software (OSS), where the source code is released with the binary. Users and developers can be licenced to use and modify the code, and to distribute any improvements they make. Both OSS and proprietary approaches allow companies to make a profit. Companies developing proprietary software make money by developing software and then selling licences to use the software. For example, Microsoft receives a payment for every copy of Windows sold with a personal computer. OSS companies make their money by providing services, such as advising clients on the GPL licence. The licencee can either charge a fee for this service or work free of charge. In practice, software companies often develop both types of software. OSS is developed by an ongoing, iterative process where people share the ideas expressed in the source code. The aim is that a large community of developers and users can contribute to the development of the code, check it for errors and bugs, and make the improved version available to others. Project management software is used to allow developers to keep track of the various versions. There are two main types of open-source licences (although there are many variants and subtypes developed by other companies): • Berkeley Software Distribution (BSD) Licence: This permits a licencee to “close” a version (by withholding the most recent modifications to the source code) and sell it as a proprietary product; • GNU General Public Licence (GNU, GPL, or GPL): Under this licence, licencees may not “close” versions. The licencee may modify, copy, and redistribute any derivative version, under the same GPL licence. The licencee can either charge a fee for this service or work free of charge. Free software first evolved during the 1970s but in the 1990s forked into two movements, namely free software and open source (Berry, 2004). Richard Stallman, an American software developer who believes that sharing source code and ideas is fundamental to freedom of speech, developed a free version of the widely used Unix operating system. The resulting GNU program was released under a specially created General Public Licence (GNU, GPL). This was designed to ensure that the source code would remain openly available to all. It was not intended to prevent commercial usage or distribution (Stallman, 2002). This approach was christened free software. In this context, free meant that anyone could modify the software. However, the term “free” was often misunderstood to mean no cost. Hence, during the 1990s, Eric Raymond and others proposed that open-source software was coined as a less contentious and more business-friendly term. This has become widely accepted within the software and business communities; however there are still arguments about the most appropriate term to use (Moody, 2002). The OSMs are usually organised into a network of individuals who work collaboratively on the Internet, developing major software projects that sometimes rival commercial software but are always committed to the production of quality alternatives to those produced by commercial companies (Raymond, 2001; Williams, 2002). Groups and individuals develop software to meet their own and others’ needs in a highly decentralised way, likened to a Bazaar (Raymond, 2001). These groups often make substantive value claims to support their projects and foster an ethic of community, collaboration, deliberation, and intellectual freedom. In addition, it is argued by Lessig (1999) that the FLOSS community can offer an inspiration in their commitment to transparency in their products and their ability to open up governmental regulation and control through free/libre and open source code.


2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Rongcun Wang ◽  
Rubing Huang ◽  
Binbin Qu

The object-oriented software systems frequently evolve to meet new change requirements. Understanding the characteristics of changes aids testers and system designers to improve the quality of softwares. Identifying important modules becomes a key issue in the process of evolution. In this context, a novel network-based approach is proposed to comprehensively investigate change distributions and the correlation between centrality measures and the scope of change propagation. First, software dependency networks are constructed at class level. And then, the number of times of cochanges among classes is minded from software repositories. According to the dependency relationships and the number of times of cochanges among classes, the scope of change propagation is calculated. Using Spearman rank correlation analyzes the correlation between centrality measures and the scope of change propagation. Three case studies on java open source software projects Findbugs, Hibernate, and Spring are conducted to research the characteristics of change propagation. Experimental results show that (i) change distribution is very uneven; (ii) PageRank, Degree, and CIRank are significantly correlated to the scope of change propagation. Particularly, CIRank shows higher correlation coefficient, which suggests it can be a more useful indicator for measuring the scope of change propagation of classes in object-oriented software system.


2021 ◽  
Vol 14 (3) ◽  
pp. 58-69
Author(s):  
Madanjit Singh ◽  
Munish Saini ◽  
Manevpreet Kaur

This paper has statically investigated the source code of open source software (OSS) projects to uncover the presence of vulnerabilities in the code. The conducted research emphasizes that the presence of vulnerabilities has adverse effects on the overall software quality. The authors found the increasing trends in the vulnerabilities as the lines of code (LOC) increases during the software evolution. This signifies the fact that the addition of new features or change requests into the OSS project may cause an increase in vulnerability. Further, the relation between software vulnerabilities and popularity is also examined. This research does not find the existence of any relationship among software vulnerabilities and popularity. This research will provide significant implications to the developers and project managers to better understand the present state of the software.


2011 ◽  
pp. 1171-1176
Author(s):  
David Berry

Open source software (OSS) is computer software that has its underlying source code made available under a licence. This can allow developers and users to adapt and improve it (Raymond, 2001). Computer software can be broadly split into two development models: • Proprietary, or closed software, owned by a company or individual. Copies of the binary are made public; the source code is not usually made public. • Open-source software (OSS), where the source code is released with the binary. Users and developers can be licenced to use and modify the code, and to distribute any improvements they make. Both OSS and proprietary approaches allow companies to make a profit. Companies developing proprietary software make money by developing software and then selling licences to use the software. For example, Microsoft receives a payment for every copy of Windows sold with a personal computer. OSS companies make their money by providing services, such as advising clients on the GPL licence. The licencee can either charge a fee for this service or work free of charge. In practice, software companies often develop both types of software. OSS is developed by an ongoing, iterative process where people share the ideas expressed in the source code. The aim is that a large community of developers and users can contribute to the development of the code, check it for errors and bugs, and make the improved version available to others. Project management software is used to allow developers to keep track of the various versions. There are two main types of open-source licences (although there are many variants and subtypes developed by other companies): • Berkeley Software Distribution (BSD) Licence: This permits a licencee to “close” a version (by withholding the most recent modifications to the source code) and sell it as a proprietary product; • GNU General Public Licence (GNU, GPL, or GPL): Under this licence, licencees may not “close” versions. The licencee may modify, copy, and redistribute any derivative version, under the same GPL licence. The licencee can either charge a fee for this service or work free of charge. Free software first evolved during the 1970s but in the 1990s forked into two movements, namely free software and open source (Berry, 2004). Richard Stallman, an American software developer who believes that sharing source code and ideas is fundamental to freedom of speech, developed a free version of the widely used Unix operating system. The resulting GNU program was released under a specially created General Public Licence (GNU, GPL). This was designed to ensure that the source code would remain openly available to all. It was not intended to prevent commercial usage or distribution (Stallman, 2002). This approach was christened free software. In this context, free meant that anyone could modify the software. However, the term “free” was often misunderstood to mean no cost. Hence, during the 1990s, Eric Raymond and others proposed that open-source software was coined as a less contentious and more business-friendly term. This has become widely accepted within the software and business communities; however there are still arguments about the most appropriate term to use (Moody, 2002). The OSMs are usually organised into a network of individuals who work collaboratively on the Internet, developing major software projects that sometimes rival commercial software but are always committed to the production of quality alternatives to those produced by commercial companies (Raymond, 2001; Williams, 2002). Groups and individuals develop software to meet their own and others’ needs in a highly decentralised way, likened to a Bazaar (Raymond, 2001). These groups often make substantive value claims to support their projects and foster an ethic of community, collaboration, deliberation, and intellectual freedom. In addition, it is argued by Lessig (1999) that the FLOSS community can offer an inspiration in their commitment to transparency in their products and their ability to open up governmental regulation and control through free/libre and open source code.


Author(s):  
Zhuoxuan Li ◽  
Warren Seering

Abstract The success of many open source software projects revealed the power of voluntary collaborative production of large/complex software systems. The research community is therefore curious about the viability of open source projects in other areas. In around 2000, open source practices started to take place in the commercial hardware realm, and so far, the phenomenon has not been fully explored. Using grounded theory, the authors studied 31 firms for an average of 2.3 years, discovering a 4-phase growth pattern of open source hardware firms, including starting the firm, identifying core competencies, business model improvement and business maturation. The firms behaviors in each stage are reported, as well as the evolution of community demography, behaviors and impact in different growth phases.


Information ◽  
2018 ◽  
Vol 9 (11) ◽  
pp. 273 ◽  
Author(s):  
Aloisio Cairo ◽  
Glauco Carneiro ◽  
Miguel Monteiro

Context: Code smells are associated to poor design and programming style, which often degrades code quality and hampers code comprehensibility and maintainability. Goal: identify published studies that provide evidence of the influence of code smells on the occurrence of software bugs. Method: We conducted a Systematic Literature Review (SLR) to reach the stated goal. Results: The SLR selected studies from July 2007 to September 2017, which analyzed the source code of open source software projects and several code smells. Based on evidence of 16 studies covered in this SLR, we conclude that 24 code smells are more influential in the occurrence of bugs relative to the remaining smells analyzed. In contrast, three studies reported that at least 6 code smells are less influential in such occurrences. Evidence from the selected studies also point out tools, techniques, and procedures that should be applied to analyze the influence of the smells. Conclusions: To the best of our knowledge, this is the first SLR to target this goal. This study provides an up-to-date and structured understanding of the influence of code smells on the occurrence of software bugs based on findings systematically collected from a list of relevant references in the latest decade.


2021 ◽  
Vol 30 (4) ◽  
pp. 1-56
Author(s):  
Chris Bogart ◽  
Christian Kästner ◽  
James Herbsleb ◽  
Ferdian Thung

Open source software projects often rely on package management systems that help projects discover, incorporate, and maintain dependencies on other packages, maintained by other people. Such systems save a great deal of effort over ad hoc ways of advertising, packaging, and transmitting useful libraries, but coordination among project teams is still needed when one package makes a breaking change affecting other packages. Ecosystems differ in their approaches to breaking changes, and there is no general theory to explain the relationships between features, behavioral norms, ecosystem outcomes, and motivating values. We address this through two empirical studies. In an interview case study, we contrast Eclipse, NPM, and CRAN, demonstrating that these different norms for coordination of breaking changes shift the costs of using and maintaining the software among stakeholders, appropriate to each ecosystem’s mission. In a second study, we combine a survey, repository mining, and document analysis to broaden and systematize these observations across 18 ecosystems. We find that all ecosystems share values such as stability and compatibility, but differ in other values. Ecosystems’ practices often support their espoused values, but in surprisingly diverse ways. The data provides counterevidence against easy generalizations about why ecosystem communities do what they do.


Author(s):  
Maria Ulan ◽  
Welf Löwe ◽  
Morgan Ericsson ◽  
Anna Wingkvist

AbstractA quality model is a conceptual decomposition of an abstract notion of quality into relevant, possibly conflicting characteristics and further into measurable metrics. For quality assessment and decision making, metrics values are aggregated to characteristics and ultimately to quality scores. Aggregation has often been problematic as quality models do not provide the semantics of aggregation. This makes it hard to formally reason about metrics, characteristics, and quality. We argue that aggregation needs to be interpretable and mathematically well defined in order to assess, to compare, and to improve quality. To address this challenge, we propose a probabilistic approach to aggregation and define quality scores based on joint distributions of absolute metrics values. To evaluate the proposed approach and its implementation under realistic conditions, we conduct empirical studies on bug prediction of ca. 5000 software classes, maintainability of ca. 15000 open-source software systems, and on the information quality of ca. 100000 real-world technical documents. We found that our approach is feasible, accurate, and scalable in performance.


Sign in / Sign up

Export Citation Format

Share Document