Supporting End-User Understanding of Classification Errors: Visualization and Usability Issues

Classifiers are applied in many domains where classification errors have significant implications. However, end-users may not always understand the errors and their impact, as error visualizations are typically designed for experts and for improving classifiers. We discuss the specific needs of classifiers' end-users, and a simplified visualization designed to address them. We evaluate this design with users from three levels of expertise, and compare it with ROC curves and confusion matrices. We identify key difficulties with understanding the classification errors, and how visualizations addressed or aggravated them. The main issues concerned confusions of the actual and predicted classes (e.g., confusion of False Positives and False Negatives). The machine learning terminology, complexity of ROC curves, and symmetry of confusion matrices aggravated the confusions. The end-user-oriented visualization reduced the difficulties by using several visual features to clarify the actual and predicted classes, and more tangible metrics and representation. Our results contribute to supporting end-users' understanding of classification errors, and informed decisions when choosing or tuning classifiers.

A Framework for Monitoring Evolution and its Drivers in Training Simulators

Systems evolve with societal, business and technological changes. Because of these changes, socio-technical systems need to adapt to new situations that were unknown at the time of design. Good knowledge of system evolution can help with that adaption. Although the evolution of software and interactive systems has been broadly debated, little research has been conducted on the specific genre of systems and even less empirical research on the evolution of interactive software has been performed. We propose a three-dimensional framework which consists of what changes during the evolution of training simulators, what are the drivers for those changes and how the changes effect innovation and robustness of the training simulators. By reviewing the literature on training simulators, we argue for this framework. The contribution of the paper is a framework that can be used to carry out empirical studies on evolution of training simulators.

A Method for Formative Multi-Context Boundary Profiling

Permanent reachability via mobile communications technologies has become a ubiquitous phenomenon. The traditional boundaries between peoples’ different contexts in their lives become blurry and begin to dissolve, if they are not actively maintained. Such boundary management activities allow to individually determine which communication request are considered acceptable in a particular context. Existing research in this field has a used a fixed set of pre-specified contexts to examine boundary management activities and identify different boundary profiles. Based on results from context-aware computing and mental model research, we argue for an open-ended, individual set of contexts to be considered for boundary management. Consequently, we develop an open structure elaboration technique to allow for individual specification of contexts and the information necessary to create a boundary profile, as identified in related work. The method is validated in an exploratory study, which was designed to verify the hypothesis that boundary management should be based on individually specified contexts, and show the feasibility of the proposed method. The results indicatively confirm our assumptions and show that the method can be used to elicit the required information.

Entertainment with Public Displays and Personal Screens

Human-computer interaction (HCI) research has recently become more interested in studying practices. Looking beyond the novelty of technology, practice studies try to understand how technology becomes integrated into everyday life and how it shapes everyday practices in the longer time span. The contribution of this article is to demonstrate how ubiquitous computing practices develop. The article also sheds light on children’s and their families’ smart device practices in private and public settings. This paper responds to the recent call for practice studies in HCI and tries to understand technology-mediated practices of children and their families in their everyday lives. We first focused on children’s practices with a multipurpose public display through an ethnographic field study, and then broadened our focus to the children’s and families’ smart device practices through a diary study. We showed that children’s practices with a public display were surprisingly similar to their other information and communication technology (ICT) practices at home and elsewhere. In both settings, displays were used for entertainment and time-killing, as well as for babysitting and social interaction. This study indicates that technology-mediated practices do not spring up from the ground fully formed. There are several factors contributing to the practices’ emergence: the artefact itself and its affordances, the nature of the space, and the mind-set of the users. This finding has many implications for research and design, indicating that when developing technology, we should pay attention to a variety of factors that might be contributing to the emergence of practice around that technology – factors not yet fully explored by current research.

Know Thy Users by Interpretative Phenomenological Analysis

One approach to getting to know a user and understanding the user experience (UX) is phenomenology. Currently, there is a lack of clearly defined methods for phenomenological analysis of user experience in design projects. Interpretative Phenomenological Analysis (IPA) is an approach developed in psychology, and in this article, it is adapted to the case of a pro bono design project at a UX design agency supporting a disadvantaged group of people, newly arrived immigrants to Sweden. The design project involved research on how the immigrants experienced a service that introduced them to the job market. The adapted method, UX IPA, contributed to the pro bono project with a focus on both experience and meaning, which is important in design projects that relate to major events in users’ lives. The method was considered less appropriate in UX projects for specific products with highly instrumental use. The method can, in many cases, be too costly. However, costs can possibly be reduced by top-down approaches. In commercial UX projects, the method may be appropriate for the fuzzy front-end of design and innovation, but clients may be unimpressed by the small sample size. This can potentially be alleviated by mixed-methods approaches.

Exploring the Role of Artefacts to Coordinate Design Meetings

Design artefacts are vital to communicate design outcomes, both in remote and co-located settings. However, it is unclear how artefacts are used to mediate interactions between designers and stakeholders of the design process. The purpose of this paper is exploring how professional design teams use artefacts to guide and capture discussions involving multidisciplinary stakeholders while they work in a co-located setting. An earlier draft of this paper was paper published in the Proceedings of the European Conference on Cognitive Ergonomics (ECCE 2017). This work adds substantial clarification of the methodology followed, further details and photographs of the case studies, and an extended discussion about our findings and their relevance for designing interactive systems. We report the observations of six design meetings in three different projects, involving professional design teams that follow a user-centered design approach. Meetings with stakeholders are instrumental for design projects. However, design teams face the challenge of synthesizing large amounts of information, often in a limited time, and with minimal common ground between meeting attendees. We found that all the observed design meetings had a similar structure consisting of a series of particular phases, in which design activities were organized around artefacts. These artefacts were used as input to disseminate and gather feedback of previous design outcomes, or as output to collect and process a variety of perspectives. We discuss the challenges faced by design teams during design meetings, and propose three design directions for interactive systems to coordinate design meetings revolving around artefacts.