A practical guide to controlled experiments of software engineering tools with human participants

This is the record of a meeting held at the University of Aarhus in Denmark, 23-26 May, 1978. The meeting was a workshop concerned with Software Engineering. A mixed group of Computer Scientists met for 4 days with the intention of exchanging knowledge and the hope of becoming wiser. Herein is a small glint of their wisdom. The workshop was organised as a mixture of formal presentations and discussions. The organisers had given titles to the various sessions but these were only used as guidelines, and the discussions in particular ranged far and wide, sometimes heated, and sometimes rather undisciplined.

Download Full-text

Knowledge Area: Software Engineering Tools and Methods

The Road Map to Software Engineering ◽

10.1109/9780471746256.ch14 ◽

2011 ◽

Keyword(s):

Software Engineering ◽

Knowledge Area ◽

Tools And Methods ◽

Software Engineering Tools

Download Full-text

An Expert Judgment in Source Code Quality Research Domain—A Comparative Study between Professionals and Students

Applied Sciences ◽

10.3390/app10207088 ◽

2020 ◽

Vol 10 (20) ◽

pp. 7088

Author(s):

Luka Pavlič ◽

Marjan Heričko ◽

Tina Beranič

Keyword(s):

Experimental Data ◽

Software Engineering ◽

Research Evidence ◽

Source Code ◽

Scientific Research ◽

Controlled Experiments ◽

Quality Research ◽

Professional Responses ◽

Code Quality ◽

Research Domain

In scientific research, evidence is often based on empirical data. Scholars tend to rely on students as participants in experiments in order to validate their thesis. They are an obvious choice when it comes to scientific research: They are usually willing to participate and are often themselves pursuing an education in the experiment’s domain. The software engineering domain is no exception. However, readers, authors, and reviewers do sometimes question the validity of experimental data that is gathered in controlled experiments from students. This is why we will address this difficult-to-answer question: Are students a proper substitute for experienced professional engineers while performing experiments in a typical software engineering experiment. As we demonstrate in this paper, it is not a “yes or no” answer. In some aspects, students were not outperformed by professionals, but in others, students would not only give different answers compared to professionals, but their answers would also diverge. In this paper we will show and analyze the results of a controlled experiment in the source code quality domain in terms of comparing student and professional responses. We will show that authors have to be careful when employing students in experiments, especially when complex and advanced domains are addressed. However, they may be a proper substitution in cases, where non-advanced aspects are required.

Download Full-text