Modeling the Process by Which People Try to Explain Complex Things to Others

The process of explaining something to another person is more than offering a statement. Explaining means taking the perspective and knowledge of the Learner into account and determining whether the Learner is satisfied. While the nature of explanation—conceived of as a set of statements—has been explored philosophically and empirically, the process of explaining, as an activity, has received less attention. We conducted an archival study, looking at 73 cases of explaining. We were particularly interested in cases in which the explanations focused on the workings of complex systems or technologies. The results generated two models: local explaining to address why a device (such an intelligent system) acted in a surprising way, and global explaining about how a device works. The examination of the processes of explaining as it occurs in natural settings revealed a number of mistaken beliefs about how explaining happens, and what constitutes an explanation that encourages learning.

A Work-Centered Approach to System User-Evaluation

New systems are often based on optimistic assumptions of how they will improve human performance. In the cognitive engineering tradition, these assumed benefits are regarded as hypotheses that need to be tested. An important element of a system user evaluation is to determine whether the hypothesized benefits are realized. Evaluation may also uncover unsupported aspects of performance or unanticipated side-effects of introducing the new technology that need to be addressed. We present a work-centered approach to user evaluation intended to meet these objectives, focusing specifically on design of tailored user-feedback questionnaires ( work-centered questionnaires) that are intended to be diagnostic of how specific system elements do, or do not, support work. We summarize two recent evaluation studies we have conducted that illustrate our approach and the diagnostic power of work-centered questionnaires. We discuss how the goals and approach of a work-centered evaluation differ from more traditional approaches to usability evaluation that emphasize the use of standardized questionnaires and broad assessments of usability.

Rapid Interpretation of Temporal–Spatial Unmanned Aerial Vehicle (UAV) Operational Data – RITSUD: Aiding UAV Operators With Visualizations of Patterns-of-Life Activities

To accomplish their missions, the operators of military unmanned aerial vehicles (UAVs) aggregate and process information from multiple sources under time constraints, while experiencing partial sensory deprivation. We aimed to design, develop, and evaluate the need for designated information aids in the form of visualizations of spatial and temporal patterns-of-life for specific operational mission phases. In an evaluation conducted to assess the usefulness of the proposed visualizations, experienced UAV operators from two nations were presented with five scenarios in a narrative format. Each scenario contained one information-aid visualization for a specific mission phase. Participants had to interpret each mission phase and rate the information aid. Results showed that, in general, most operators favored the aids, but their appreciation of specific aids depended on the mission phase. There were also statistically significant differences between the nations. The results emphasize the potential for equipping UAV operators with information aids in the form of visualizations of spatial and temporal patterns-of-life. Moreover, the findings underline the need for more rigorous analyses of operators’ missions, as the need for specific visualizations was shown to vary with mission phase, and some visualizations were found to be more widely useful than others.

Use of Augmented Reality to Train Sensemaking in High-Stakes Medical Environments

We present a framework for using augmented reality (AR) to train sensemaking skills in combat medics and civilian emergency medical personnel. AR and other extended reality technologies create engaging training environments, but their effectiveness on training outcomes is not yet clear. One benefit of AR is that it can enhance simulation training with realism and context that naturalistic decision-making (NDM) models emphasize. We describe four key elements of sensemaking that leverage the strengths of AR: perceptual skills, assessment skills, mental models, and generating/evaluating hypotheses. We discuss how AR can be used to train each of these four elements, along with design implications. A focus on naturalistic tasks and environments while designing AR-based simulation training will likely lead to training that is not only engaging but also effective.

Ecological Design of an Augmentative and Alternative Communication Device Interface

Autism spectrum disorder, cerebral palsy, brain stem stroke, and neurological injury are examples of conditions that may limit vocal communication. Augmentative and alternative communication (AAC) systems can provide a communication pathway to users who experience such complex communication needs, facilitating their societal participation and supporting some ability to direct their own care. We adapted the cognitive work analysis (CWA) framework to a linguistic domain for insights into an AAC design that best supports users’ communication. First, we applied the work domain analysis (WDA) to a popular commercial AAC system, Proloquo2Go. Data were gathered from guided AAC system use, domain experts, and the syntactic rules of the English language. The WDA exposed unmet needs in the commercial system. We then applied worker competency analysis to consider different approaches to present information and support user actions. The design included graphic forms and process views, and their integration into viewports and the workspace. Our novel application of CWA uncovered new considerations in AAC interface design and presents a nascent area of investigation, namely, AAC displays that more effectively support users’ goals. Future investigation will evaluate the mental workload of this AAC interface compared to that of current commercially available systems.

Designing Affordance-Based Procedures Using Heuristic Cognitive Work Analysis: Application to Triage in an Emergency Department

This study describes the concept of affordance-based procedure and its implementation in a triage station in a hospital emergency department. Rather than seeking to increase operators’ adherence to procedures, an affordance-based procedure (1) aims to induce task steps using affordances that also (2) support degrees of freedom for action. The design of this procedure was guided by the application of an extended version of cognitive work analysis, named “heuristic cognitive work analysis.” This design process produced a new procedural document: a reception card. Ten months after its implementation, a qualitative evaluation with 10 triage nurses shows that the reception card is viewed as supporting coordination between the different nurses’ tasks and providing an external memory to cope with frequent interruptions during high patient inflow, even though the document is used for convenience and with unexpected and partial uses of its items. The document assessed also afforded emerging benefits, that is, acceleration of ambulance release, higher level of confidentiality, assistance for staff hand-overs. Finally, novice triage nurses are particularly sensitive to the benefits brought by this affordance-based procedure.

The Art of Effective Handoff Communication Among Medical and Surgery Residents

End-of-shift handoffs occur when physicians transfer care responsibilities from one shift to another. Typically viewed as a straightforward exchange of information, we argue that several contextually relevant factors shape the communication behaviors of outgoing and incoming residents during handoffs. Digital recordings and transcripts of resident handoffs in medicine and surgery were made at three VA Medical Centers. They were triangulated with cognitive task interviews that elicited residents’ reconstructions of their work practices. Analyses revealed clear distinctions among “day-to-night,” “night-to-day,” and “double handoffs” that involve transitions between day and night teams. Although residents preferred handing off in dedicated, quiet spaces, few (16%) occurred in such settings; 28% contained significant interruptions. The quality handoff artifacts (notes and forms) influenced interactions, especially in cases where multiple residents from different teams were involved, requiring incoming residents to adjust “on the fly.” This study demonstrated that there are multiple contextual factors that affect, and are affected by, handoff interactions. The findings suggest that handoffs are less like the delivery of a telegram (unidirectional) and more like complex adaptive systems (products of interactional co-construction). Teaching communication practices based on interaction complexity may reduce errors and adverse outcomes for hospitalized patients.

Team Situation Awareness and Conflict: A Study of Human–Machine Teaming

This article focuses on two fundamental human–human teamwork behaviors and seeks to understand them better in human–machine teams. Specifically, team situation awareness (TSA) and team conflict are examined in human–machine teams. There is a significant need to identify how TSA and team conflict occur during human–machine teaming, in addition to how they impact each other. In this work, we present an experiment aimed at understanding TSA and team conflict in the context of human–machine teaming in a remotely piloted aircraft system (RPAS). Three conditions were tested: (1) control: teams consisted of all humans; (2) synthetic: teams consisted of the pilot role being occupied by a computational agent based on ACT-R architecture that employed AI capabilities, with all other team roles being humans; and (3) experimenter: an experimenter playing the role of the pilot as a highly effective computational agent, with the other roles being humans. The results indicate that TSA improved over time in synthetic teams, improved and then stabilized over time in experimenter teams, and did not improve in control teams. In addition, results show that control teams had the most team conflict. Finally, in the control condition, team conflict negatively impacts TSA.

Exploring the Impact of Coordination in Human–Agent Teams

Teaming permits cognitively complex work to be rapidly executed by multiple entities. As artificial agents (AAs) participate in increasingly complex cognitive work, they hold the promise of moving beyond tools to becoming effective members of human–agent teams. Coordination has been identified as the critical process that enables effective teams and is required to achieve the vision of tightly coupled teams of humans and AAs. This paper characterizes coordination on the axes of types, content, and cost. This characterization is grounded in the human and AA literature and is evaluated to extract design implications for human–agent teams. These design implications are the mechanisms, moderators, and models employed within human–agent teams, which illuminate potential AA design improvements to support coordination.

Levels of What?Investigating Drivers’ Understanding of Different Levels of Automation in Vehicles

Extant levels of automation (LoAs) taxonomies describe variations in function allocations between the driver and the driving automation system (DAS) from a technical perspective. However, these taxonomies miss important human factors issues and when design decisions are based on them, the resulting interaction design leaves users confused. Therefore, the aim of this paper is to describe how users perceive different DASs by eliciting insights from an empirical driving study facilitating a Wizard-of-Oz approach, where 20 participants were interviewed after experiencing systems on two different LoAs under real driving conditions. The findings show that participants talked about the DAS by describing different relationships and dependencies between three different elements: the context (traffic conditions, road types), the vehicle (abilities, limitations, vehicle operations), and the driver (control, attentional demand, interaction with displays and controls, operation of vehicle), each with associated aspects that indicate what users identify as relevant when describing a vehicle with automated systems. Based on these findings, a conceptual model is proposed by which designers can differentiate LoAs from a human-centric perspective and that can aid in the development of design guidelines for driving automation.