Shared Gaze Visualizations in Collaborative Interactions: Past, Present and Future

The past decade has witnessed a growing interest for using dual eye tracking to understand and support remote collaboration, especially with studies that have established the benefits of displaying gaze information for small groups. While this line of work is promising, we lack a consistent framework that researchers can use to organize and categorize studies on the effect of shared gaze on social interactions. There exists a wide variety of terminology and methods for describing attentional alignment; researchers have used diverse techniques for designing gaze visualizations. The settings studied range from real-time peer collaboration to asynchronous viewing of eye-tracking video of an expert providing explanations. There has not been a conscious effort to synthesize and understand how these different approaches, techniques and applications impact the effectiveness of shared gaze visualizations (SGVs). In this paper, we summarize the related literature and the benefits of SGVs for collaboration, describe important terminology as well as appropriate measures for the dual eye-tracking space and discuss promising directions for future research. As eye-tracking technology becomes more ubiquitous, there is pressing need to develop a consistent approach to evaluation and design of SGVs. The present paper makes a first and significant step in this direction.

Model-based Engineering of Feedforward Usability Function for GUI Widgets

Feedback and feedforward are two fundamental mechanisms that support users’ activities while interacting with computing devices. While feedback can be easily solved by providing information to the users following the triggering of an action, feedforward is much more complex as it must provide information before an action is performed. For interactive applications where making a mistake has more impact than just reduced user comfort, correct feedforward is an essential step toward correctly informed, and thus safe, usage. Our approach, Fortunettes, is a generic mechanism providing a systematic way of designing feedforward addressing both action and presentation problems. Including a feedforward mechanism significantly increases the complexity of the interactive application hardening developers’ tasks to detect and correct defects. We build upon an existing formal notation based on Petri Nets for describing the behavior of interactive applications and present an approach that allows for adding correct and consistent feedforward.

AugIR Meets GestureCards: A Digital Sketching Environment for Gesture-Based Applications

To gain a common understanding of an application’s layouts, dialogs and interaction flows, development teams often sketch user interface (UI). Nowadays, they must also define multi-touch gestures, but tools for sketching UIs often lack support for custom gestures and typically just integrate a basic predefined gesture set, which might not suffice to specifically tailor the interaction to the desired use cases. Furthermore, sketching can be enhanced with digital means, but it remains unclear whether digital sketching is actually beneficial when designing gesture-based applications. We extended the AugIR, a digital sketching environment, with GestureCards, a hybrid gesture notation, to allow software engineers to define custom gestures when sketching UIs. We evaluated our approach in a user study contrasting digital and analog sketching of gesture-based UIs.

On the Benefits of Using MVC Pattern for Structuring Event-B Models of WIMP Interactive Applications

This paper presents a formal development approach for designing interactive applications using a correct-by-construction approach. In this work, we propose a refinement strategy using model-view-controller (MVC) to structure and design Event-B formal models of the interactive application. The proposed MVC-based refinement strategy facilitates the development of an abstract model and a series of refined models by introducing the possible modes, controller’s behaviour and visual components of the interactive application while preserving the required interaction-related safety properties. To demonstrate the effectiveness, scalability, reliability and feasibility of our approach, we use a small example (from automotive domain) and real-life industrial case studies (from aviation). The entire development is realized in Event-B and the associated Rodin tool is used to analyse and verify the correctness of the formalized model. Finally, the developed Event-B models are used to generate source code using EB2ALL tool for going from the specification to the implementation of the interactive application.

Using Task Models to Understand the Intersection of Numeracy Skills and Technical Competence With Medical Device Design

Task models are used in many different ways throughout the design and development of interactive systems. When the interactive systems are safety critical, task models can play an important role in ensuring system behaviours are consistent with user requirements, which may help to prevent errors. While task models can be used to describe a user’s goals and the steps required to achieve that goal, to understand where user errors may occur we also need to consider the users’ understanding of how to perform a task and how this relates to the system they are using. Our focus is on the use of medical devices such as syringe drivers and infusion pumps for intravenous medication, which forms a major part of hospital inpatient care throughout the world. While we might rely on software engineering and human factors techniques to ensure correctness of such devices, their use by medical personnel in practice includes other factors that are equally important. These include training medical personnel in the use of medical devices. Also numeracy education for medical staff to ensure that they are able to set up and perform the necessary calculations to convert prescribed medication into the appropriate values and measures for their delivery mechanisms. We have developed an approach that aims to bring together concepts of technology design (both functional correctness and usability concerns), numeracy and medication delivery competency. In order to do so we use task models as a common language that enables us to consider these different domains in a single way. We find that the ability to describe the two domains within a single process allows us to compare models of knowledge, tasks and use of devices, which can elicit potential mismatches and problems.

Balancing the formal and the informal in user-centred design

This paper explores the role of formal methods as part of the user-centred design of interactive systems. An iterative process is described, developing prototypes incrementally, proving user-centred requirements while at the same time evaluating the prototypes that are executable forms of the developed models using ‘traditional’ techniques for user evaluation. A formal analysis complements user evaluations. This approach enriches user-centred design that typically focuses understanding on context and producing sketch designs. These sketches are often non-functional (e.g. paper) prototypes. They provide a means of exploring candidate design possibilities using techniques such as cooperative evaluation. This paper describes a further step in the process using formal analysis techniques. The use of formal methods provides a systematic approach to checking plausibility and consistency during early design stages, while at the same time enabling the generation of executable prototypes. The technique is illustrated through an example based on a pill dispenser.

Managing User-Centered Design Activities in Distributed Agile Development

The integration of user-centered design (UCD) activities into agile information systems development has become more popular recently. Despite the fact that there are many ways the merging of UCD activities into agile development can be carried out, it has been widely recognized that coordinating design activities with development activities is one of the most common problems, especially in distributed environments where designers, developers and users are spread over several sites. The main approach to coordinate UCD activities with distributed agile development is the use of informal methods (e.g. communication through using video conference tools). In addition to the temporal, geographical and socio-cultural barriers associated with this type of methods, a major limitation is a lack of awareness of how UCD activities and development activities affect each other. Furthermore, some agile project management tools are integrated with design platforms but fail to provide the necessary coordination that helps team members understand how the design and development activities affect their daily work. This research aims to support the effective management of integrating UCD activities into distributed agile development by (i) identifying the key activity dependencies between UX design teams and development teams during distributed UCD/agile development and (ii) designing a computer-based system to provide coordination support through managing these activity dependencies. In order to achieve these objectives, two case studies are carried out. Our findings revealed 10 main dependencies between UCD design teams and development teams as shown by six types of activity. In addition, the participatory design approach shows that developing a computer-based system to manage seven of these selected dependencies is achievable.

The Effect of an Adaptive Simulated Inner Voice on User’s Eye-gaze Behaviour, Ownership Perception and Plausibility Judgement in Virtual Reality

Virtual cognitions (VCs) are a stream of simulated thoughts people hear while emerged in a virtual environment, e.g. by hearing a simulated inner voice presented as a voice over. They can enhance people’s self-efficacy and knowledge about, for example, social interactions as previous studies have shown. Ownership and plausibility of these VCs are regarded as important for their effect, and enhancing both might, therefore, be beneficial. A potential strategy for achieving this is the synchronization of the VCs with people’s eye fixation using eye-tracking technology embedded in a head-mounted display. Hence, this paper tests this idea in the context of a pre-therapy for spider and snake phobia to examine the ability to guide people’s eye fixation. An experiment with 24 participants was conducted using a within-subjects design. Each participant was exposed to two conditions: one where the VCs were adapted to eye gaze of the participant and the other where they were not adapted, i.e. the control condition. The findings of a Bayesian analysis suggest that credibly more ownership was reported and more eye-gaze shift behaviour was observed in the eye-gaze-adapted condition than in the control condition. Compared to the alternative of no or negative mediation, the findings also give some more credibility to the hypothesis that ownership, at least partly, positively mediates the effect eye-gaze-adapted VCs have on eye-gaze shift behaviour. Only weak support was found for plausibility as a mediator. These findings help improve insight into how VCs affect people.

Use of a Persona to Support the Interdisciplinary Design of an Assistive Technology for Meal Preparation in Traumatic Brain Injury

User-centered design (UCD) facilitates the creation of technologies that are specifically designed to answer users’ needs. This paper presents the first step of a UCD using a persona, a fictitious character representing the targeted population, which in this case is people having sustained a traumatic brain injury (TBI). The persona is used to better understand the possible interactions of a TBI population with a prototype of a technology that we wish to develop, namely the Cognitive Orthosis for coOKing (COOK). COOK is meant to be an assistive technology that will be designed to promote independence for cooking within a supported-living residence. More specifically, this paper presents the persona’s creation methodology based on the first four phases of the persona’s lifecycle. It also describes how the persona methodology served as a facilitator to initiate an interdisciplinary collaboration between a clinical team and a computer science team. Creation of personas relied on a clinical model (Disability Creation Process) that contextualized the needs of this population and an evaluation tool [Instrumental Activities of Daily Living (IADL) Profile] that presented a wide range of cognitive assistance needs found in this same population. This paper provides an in-depth description of some of the most frequent everyday difficulties experienced by individuals with TBI as well as the persona’s abilities, limitations and social participation during the realization of IADL, and an evaluation of the manifestations of these difficulties during IADL performance as represented through scenarios. The interdisciplinary team used the persona to complete a first description of the interactions of a persona with TBI with COOK. This work is an attempt at offering a communication tool, the persona, to facilitate interdisciplinary research among diverse disciplines who wish to learn to develop a common language, models and methodologies at the beginning of the design process.

Interactive Exploration of Large-Scale UI Datasets with Design Maps

Designers are increasingly using online resources for inspiration. How to best support design exploration without compromising creativity? We introduce and study Design Maps, a class of point-cloud visualizations that makes large user interface datasets explorable. Design Maps are computed using dimensionality reduction and clustering techniques, which we analyze thoroughly in this paper. We present concepts for integrating Design Maps into design tools, including interactive visualization, local neighborhood exploration and functionality to integrate existing solutions to the design at hand. These concepts were implemented in a wireframing tool for mobile apps, which was evaluated with actual designers performing realistic tasks. Overall, designers find Design Maps supporting their creativity (avg. CSI score of 74/100) and indicate that the maps producing consistent whitespacing within cloud points are the most informative ones.