“Services to Empower YOU” – Interfaces Multimodais Inclusivas numa Plataforma de Serviços para Pessoas com Deficiência ou Incapacidade

Services to Empower YOU” (SEU) responde a um desafio societal que visa fornecer a pessoas com deficiência ou incapacidade uma ferramenta agregadora e inclusiva através da qual possam requisitar com confiança serviços. Este artigo apresenta os resultados iniciais obtidos no projeto SEU que compreende o estudo da conceção e implementação de interfaces digitais adaptadas às necessidades específicas de cada utilizador, assim como o estudo de um modelo de negócio que permita aos prestadores de serviços fornecerem serviços para um público com características especiais. Partindo do levantamento das necessidades de pessoas com incapacidades e do estudo das normas de acessibilidade digital, especificou-se um Modelo de Adaptação para interfaces em função das características de cada tipo de utilizador e um Modelo de Serviços, que estende o modelo base de plataformas facilitadoras de serviços offline. Estes dois modelos foram aplicados no desenvolvimento do protótipo web apresentado.Palavras-chave: Plataforma de serviços web. Acessibilidade digital. Interfaces multimodais. Inclusão. Pessoas com deficiência ou incapacidade. "Services to Empower YOU" - Inclusive Multimodal Interfaces in a Service Platform for People with DisabilitiesAbstract: “Services to Empower YOU” (SEU) responds to a societal challenge that aims to provide people with disabilities with an aggregating and inclusive tool through which they can confidently request services that are appropriate to them. This article presents the initial results achieved in project SEU, which comprises the study of the design and implementation of digital interfaces adapted to the specific needs of each user, as well as the study of a business model that allows service providers to offer services for an audience with special characteristics. Starting from the survey of the needs of people with disabilities and the study of digital accessibility standards, two models were proposed: an Adaptation Model for designing interfaces according to the characteristics of each type of user; and a Services Model, which extends the base model of platforms facilitating offline services. These two models were instantiated in the initial web prototype presented.Keywords: Web services platform. Digital accessibility. Multimodal user-interfaces. Inclusion. People with disabilities.

Design and Development of a Prototype Heads-Up Display: Supporting Context-Aware, Semi-Automated, Hands-Free Medical Documentation

Military and civilian medical personnel across all echelons of medical care play a critical role in evaluating, caring for, and treating casualties. Accurate medical documentation is critical to effective, coordinated care and positive patient outcomes. We describe our prototype, Context-Aware Procedure Support Tools and User Interfaces for Rapid and Effective Workflows (CAPTURE). Leveraging human factors and usercentered design methods, and advanced artificial intelligence and computer vision capabilities, CAPTURE was designed to enable Tactical Combat Causality Care (TCCC) providers to more efficiently and effectively input critical medical information through hands-free interaction techniques and semiautomated data capture methods. We designed and prototyped a heads-up display that incorporates: multimodal interfaces, including augmented reality-based methods for input and information display to support visual image capture and heads-up interaction; post-care documentation support (e.g., artifacts to support post-care review and documentation); context-aware active and passive data capture methods, specifically natural language interpretation using systemic functional grammars; and computer vision technologies for semi-automated data capture capabilities. During the course of this project we encountered challenges towards effective design which fall into three main categories: (1) challenges related to designing novel multimodal interfaces; (2) technical challenges related to software and hardware development to meet design needs; and (3) challenges as a result of domain characteristics and operational constraints. We discuss how we addressed some of these challenges and provide additional considerations necessary for future research regarding next generation technology design for medical documentation in the field.

Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments

This paper presents a new architecture that integrates Internet of Things (IoT) devices, service robots, and users in a smart assistive environment. A new intuitive and multimodal interaction system supporting people with disabilities and bedbound patients is presented. This interaction system allows the user to control service robots and devices inside the room in five different ways: touch control, eye control, gesture control, voice control, and augmented reality control. The interaction system is comprised of an assistive robotic arm holding a tablet PC. The robotic arm can place the tablet PC in front of the user. A demonstration of the developed technology, a prototype of a smart room equipped with home automation devices, and the robotic assistive arm are presented. The results obtained from the use of the various interfaces and technologies are presented in the article. The results include user preference with regard to eye-base control (performing clicks, and using winks or gaze) and the use of mobile phones over augmented reality glasses, among others.

Investigating the Effect of Sensory Concurrency on Learning Haptic Spatiotemporal Signals

A new generation of multimodal interfaces and interactions is emerging. Drawing on the principles of Sensory Substitution and Augmentation Devices (SSADs), these new interfaces offer the potential for rich, immersive human-computer interactions, but are difficult to design well, and take time to master, creating significant barriers towards wider adoption. Following a review of the literature surrounding existing SSADs, their metrics for success and their growing influence on interface design in Human Computer Interaction, we present a medium term (4-day) study comparing the effectiveness of various combinations of visual and haptic feedback (sensory concurrencies) in preparing users to perform a virtual maze navigation task using haptic feedback alone. Participants navigated 12 mazes in each of 3 separate sessions under a specific combination of visual and haptic feedback, before performing the same task using the haptic feedback alone. Visual sensory deprivation was shown to be inferior to visual & haptic concurrency in enabling haptic signal comprehension, while a new hybridized condition combining reduced visual feedback with the haptic signal was shown to be superior. Potential explanations for the effectiveness of the hybrid mechanism are explored, and the scope and implications of its generalization to new sensory interfaces is presented.

MuMIA: Multimodal Interactions to Better Understand Art Contexts

Cultural heritage is a challenging domain of application for novel interactive technologies, where varying aspects in the way that cultural assets are delivered play a major role in enhancing the visitor experience, either onsite or online. Technology-supported natural human–computer interaction that is based on multimodalities is a key factor in enabling wider and enriched access to cultural heritage assets. In this paper, we present the design and evaluation of an interactive system that aims to support visitors towards a better understanding of art contexts through the use of a multimodal interface, based on visual and audio interactions. The results of the evaluation study shed light on the dimensions of evoking natural interactions within cultural heritage environments, using micro-narratives for self-exploration and understanding of cultural content, and the intersection between human–computer interaction and artificial intelligence within cultural heritage. We expect our findings to provide useful insights for practitioners and researchers of the broad human–computer interaction and cultural heritage communities on designing and evaluating multimodal interfaces to better support visitor experiences.