Is It Worth Using Gamification on Software Testing Education? An Extended Experience Report in the Context of Undergraduate Students

Context: Testing is fundamental in the software development process. Nevertheless, testing education faces the key challenge of ensuring that undergraduate students acquire knowledge and skills they need for their future careers by matching what is taught in the classroom to industry standards. In this context, gamification can be used as an alternative educational approach. It uses game elements in real-world contexts in order to increase people's motivation and engagement in tasks that require external stimuli, especially in educational contexts.Objective: Reporting on results of an experimental study designed to assess the impact of gamification on software testing education, as well as reporting on the experience of building a supporting gamified platform. Method: We performed a systematic literature mapping aiming at characterizing how gamification has been explored in the software testing context. In addition, some of the problems faced by testing education were identified through an ad-hoc search. Then, we developed a gamified approach and a platform that have been used to run five 4-hour functional testing classes with undergraduate students from four Brazilian institutions of higher education.Results: Overall, students who learned with the traditional approach felt more motivated than those who learned with the gamified approach, whereas the performance of both groups were similar. On the other hand, feedback questionnaires indicated that the gamified class was more attractive (in terms of attention) and funny. Moreover, we observed that building a gamified platform is complex and challenging, particularly for the definition of the game mechanics and dynamics.Conclusion: Even though the results in terms of students' performance was neutral, and the motivation of students did not increase due to the introduction of gamification, the experience of having used this alternative approach is considered positive, as it provided a more enjoyable and funny environment, both from the researcher's and students' point of view. Furthermore, with this experience we foresee we can do better in terms of gamified teaching in future work.

Reducing the Discard of MBT Test Cases

Model-Based Testing (MBT) is used for generating test suites from system models. However, as software evolves, its models tend to be updated, which may lead to obsolete test cases that are often discarded. Test case discard can be very costly since essential data, such as execution history, are lost. In this paper, we investigate the use of distance functions and machine learning to help to reduce the discard of MBT tests. First, we assess the problem of managing MBT suites in the context of agile industrial projects. Then, we propose two strategies to cope with this problem: (i) a pure distance function-based. An empirical study using industrial data and ten different distance functions showed that distance functions could be effective for identifying low impact edits that lead to test cases that can be updated with little effort. We also found the optimal configuration for each function. Moreover, we showed that, by using this strategy, one could reduce the discard of test cases by 9.53%; (ii) a strategy that combines machine learning with distance values. This strategy can classify the impact of edits in use case documents with accuracy above 80%; it was able to reduce the discard of test cases by 10.4% and to identify test cases that should, in fact, be discarded.

Software Operational Profile vs. Test Profile: Towards a Better Software Testing Strategy

Software Operational Profile (SOP) is a software specification based on how users use the software. This specification corresponds to a quantitative representation of software that identifies the most used software parts. As software reliability depends on the context in which users operate the software, the SOP is used in software reliability engineering. However, there are evidences of a misalignment between the software tested parts and SOP. Therefore, this paper investigates a possible misalignment between SOP and the tested software parts to obtain, based on experimental data, more evidence of this misalignment. We performed an exploratory study composed of four activities to verify: a) whether there are significant variations in how users operate the software; b) whether there is a misalignment between SOP and the tested software parts; c) if failures occur in untested SOP parts in case of misalignment; d) in case of misalignment between SOP and untested software parts, whether a test strategy based on the amplification of the existent test set with additional test data generated automatically, can contribute to reduce the misalignment. We collected data form four software while users were operating them. We analyzed this collected data in an attempt to reach the goals of this work. To evaluate the originality of this research, we performed a Literature Systematic Review (SLR) and presented its conclusions. The obtained results evidence that there are significant variations in how users operate the software and also that there is a misalignment between SOP and the tested software parts when we evaluated the four software mentioned above. There are also indications of the occurrence of failures in the untested SOP parts. Although the test strategy mentioned above has reduced the possible misalignment, the test strategy is not enough to avoid it, thus denoting the need of specifics test strategies using SOP as a test criterion. These results indicate that SOP becomes relevant not only to software reliability engineering but also to contribute to testing activities, regardless of the adopted strategy.

Editoral letter - special issue CIbSE2019

This issue of the JSERD contains seven extended and peer-reviewed papers from the XXII Ibero-American Conference on Software Engineering (CIbSE 2019), which was held in La Habana, Cuba, in April 2019.

Towards a new template for the specification of requirements in semi-structured natural language

Requirements engineering is a systematic and disciplined approach for the specification and management of software requirements; one of its objectives is to transform the requirements of the stakeholders into formal spec-ifications in order to analyze and implement a system. These requirements are usually expressed and articulated in natural language, this due to the universality and facility that natural language presents for communicating them. To facilitate the transformation processes and to improve the quality of the resulting requirements, several authors have proposed templates for writing requirements in structured natural language. However, these templates do not allow writing certain functional requirements, non-functional requirements and constraints, and they do not adapt correctly to certain types of systems such as self-adaptive, product line-based and embedded systems. This paper (i) presents evidence of the weaknesses of the template recommended by the IREB® (International Requirements Engineering Institute), and (ii) lays the foundations, through a new template, for facilitating the work of the re-quirements engineers and therefore improving the quality of the products specified with the new template. This new template was built and evaluated through two active research cycles. In each cycle we identified the problems specifying the requirements of the corresponding industrial case with the corresponding base-line template, pro-pose some improvements to address these problems and analyze the results of using the new template to specify the requirements of each case. Thus, the resulting template was able to correctly write all requirements of both industrial cases. Despite the promising results of this new template, it is still preliminary work regarding its cov-erage and the quality level of the requirements that can be written with it.

A Requirements engineering process for a quality model in Cuba

A high percentage of projects worldwide fail or are canceled due to incorrect requirements engineering. Incorporating good practices into this process provides the appropriate mechanism to understand and analyze what stakeholders want and need. It also allows to evaluate and negotiate a reasonable solution, specify, validate and manage the requirements as they are transformed into a functional system. The objective of this research is to elaborate a process of Requirements Engineering for the Quality Model for Software Development that contributes to raise the percentage of successful projects, in Cuban´s software development organizations, regarding the fulfillment of the agreed requirements. The solution proposal contains specific requirements and support elements (graphic and textual description of the process), divided by the three levels of maturity proposed by the Model. In order to reach the proposed goal, a bibliographic review was made on the requirements engineering discipline, as well as interviews and surveys to roles related to this activity in the software development organizations of Cuba. The solution was evaluated by experts in a focus group and put into practice, as a pilot, in three organizations, it was also measured the satisfaction of the users who used it using the Iadov technique.

Supporting a Hybrid Composition of Microservices. The EUCalipTool Platform

In order to provide complex and elaborated functionalities, Microservices may cooperate with each other either by following a centralized (orchestration) or decentralized (choreography) approach. It seems that the decentralized nature of microservices makes the choreography approach more appropriate to achieve such cooperation, where lighter solutions based on events and message queues are used. However, orchestration through the usage of a process model makes it is easier to analyze the flow of the composition when modifications are required. In order to benefit from the goodness of these two approaches, this paper presents a hybrid solution based on the choreography of business process pieces, that are obtained from a previously defined description of the complete microservice composition. To support this solution, the EUCalipTool platform is presented.

A revisited systematic literature mapping on the support of requirement patterns for the software development life cycle

In the past few years, the literature has shown that the practice of reuse through requirement patterns is an effective alternative to address specification quality issues, with the additional benefit of time savings. Due to the interactions between requirements engineering and other phases of the software development life cycle (SDLC), these benefits may extend to the entire development process. This paper describes a revisited systematic literature mapping (SLM) that identifies and analyzes research that demonstrates those benefits from the use of requirement patterns for software design, construction, testing, and maintenance. In this extended version, the SLM protocol includes automatic search over two additional sources of information and the application of the snowballing technique, resulting in ten primary studies for analysis and synthesis. In spite of this new version of the SLM protocol, results still point out a small number of studies on requirement patterns at the SDLC (excluding requirements engineering). Results indicate that there is yet an open field for research that demonstrates, through empirical evaluation and usage in practice, the pertinence of requirement patterns at software design, construction, testing, and maintenance.

On the contributions of non-technical stakeholders to describing UX requirements by applying proto-persona

Context: Requirements elicitation is a software development phase that should investigate both functional and user experience (UX) requirements. Proto-persona is a technique that encourages the attention on the needs of a group of users. Usually, its elaboration is conducted by software specialists, technical stakeholders. However, non-technical stakeholders usually know more about target users and frequently do not take part in proto-persona elaboration. Objective: This work has the goal of investigating the contribution of non-technical stakeholders in the specification of UX requirements by using the proto-persona technique. For this, we explored the construction of the proto-personas and the use of these to the prototyping of solutions. Method: We carried out an empirical study in two rounds from which we analyzed and compared the contribution that technical and non-technical stakeholders had on the specification of UX requirements. In the first, 8 non-technical and 5 technical stakeholders built proto-personas. Afterwards, 18 pairs of software developers created low fidelity prototypes by using the information of proto-personas.~For the two rounds, we conducted a qualitative analysis exploring which UX requirements were described and used. Results: Our results revealed that both stakeholders have written up details of UX requirements on the artifact, however, throughout different and complementary perspectives. We also could observe that proto-personas produced by both were used on the prototyping activity. Conclusion: Our paper contributed to demonstrate that non-technical stakeholders were able to contribute to the specification of UX requirements and that proto-persona can boost such activity.

The RoCS Framework to Support the Development of Autonomous Robots

With the expansion of autonomous robotics and its applications (e.g. medical, competition, military), the biggest hurdle in developing mobile robots lies in endowing them with the ability to interact with the environment and to make correct decisions so that their tasks can be executed successfully. However, as the complexity of robotic systems grows, the need to organize and modularize software for their correct functioning also becomes a challenge, making the development of software for controlling robots a complex and intricate task. In the robotics domain, there is a lack of reference software architectures and, although most robot architectures available in the literature facilitate the creation process with their modularity, existing solutions do not provide development guidance on reusing existing modules. Based on the well- known IBM Autonomic Computing reference architecture (known as MAPE-K), this work defines a refined architecture following the Robotics perspective. To explore the capabilities of the proposed refinement, we implemented the RoCS (Robotics and Cognitive Systems) framework for autonomous robots. We successfully tested the framework under simulated robotics scenarios that mimic typical robotics tasks and evidence the framework reuse capability. Finally, we understand the proposed framework needs further experimental evaluation, particularly, assessments on real-world scenarios.