A Conceptual Framework for Interoperability of Mobile User Interfaces with Ambient Computing Environments

The use of mobile and hand-held devices is a desirable option for implementation of user interaction with remote services from a distance, whereby the user should be able to select the input device depending on personal preferences, capabilities and availability of interaction devices. Because of the heterogeneity of available devices and interaction styles, the interoperability needs particular attention by the developer. This paper describes the design of a general solution to enable mobile devices to have control on services at remote hosts. The applied approach enhances the idea of separating the user interface from the application logic, leading to the definition of virtual or logical input devices physically separated from the controlled services.

Pervasive Computing in the Supermarket

Shopping in the real world is becoming an increasingly interactive experience as stores integrate various technologies to support shoppers. Based on an empirical study of supermarket shoppers, the authors designed a mobile context-aware system called the Context-Aware Shopping Trolley (CAST). The purpose of CAST is to support shopping in supermarkets through context-awareness and acquiring user attention, thus, the authors’ interactive trolley guides and directs shoppers in the handling and finding of groceries. An empirical evaluation showed that shoppers using CAST behaved differently than shoppers using a traditional trolley. Specifically, shoppers using CAST exhibited a more uniform pattern of product collection and found products more easily while travelling a shorter distance. As such, the study finds that CAST supported the supermarket shopping activity.

MuseumScrabble

Mobile technology has created new possibilities for location-based playful learning experiences. This article describes the MuseumScrabble mobile game, aimed at children visiting a historical museum. The game requires that the players explore the museum and link abstract concepts with physical artefacts using a mobile device. The focus of this article is on the interaction design process and the subsequent observations made during field evaluation of the game. Design principles that guide the development of such a game are presented and concern playfulness, learning, social interaction, physical aspects of the game and flow between physical and digital space. This article explores how these design principles are reflected in the study and how problem-solving strategies and collaboration and competition patterns are developed by children in this multi-player educational game.

View and Share

Co-present viewing and sharing of images on mobile devices is a popular but very cumbersome activity. Firstly, it is difficult to show a picture to a group of friends due to the small mobile phone screen and secondly it is difficult to share media between multiple friends, e.g., when considering Bluetooth usage and technical limitations, limited input and repeated user interactions. This paper introduces the View & Share system which allows mobile phone users to spontaneously form a group and engage in the viewing and sharing of images. A member of the group has a personal projector (e.g., projector phone) which is used to view pictures collaboratively. View & Share supports sharing with a single user, multiple users or all users, allows members to borrow the projected display and provides a private viewing mode between co-located users. This paper reports on the View & Share system, its implementation and an explorative user study with 12 participants showing the advantages of our system and user feedback.

Framing the Context of Use for Mobile HCI

The need to better understand the role of context has emerged after the revolution of mobile computing, as such devices are used in heterogeneous circumstances. However, it is difficult to say what context of use in mobile human-computer interaction actually means. This study summarises past research in mobile contexts of use and not only provides a deeper understanding of the characteristics associated with it, but also indicates a path for future research. This article presents an extensive and systematic literature review of more than 100 papers published in five high-quality journals and one main conference in the field of HCI during the years 2000-2007. The authors’ results show that context of use is still explored as a relatively static phenomenon in mobile HCI. Its most commonly mentioned characteristics are linked to social, physical, and technical components, while transitions between the contexts were rarely listed. Based on this review, a descriptive model of context of use for mobile HCI (CoU-HMCI) summarising five components, their subcomponents and descriptive properties is presented. The model can help both practitioners and academics to identify broadly relevant contextual factors when designing, experimenting with, and evaluating, mobile contexts of use.

Designing for Children’s Mobile Storytelling

Mobile technologies offer novel opportunities for children to express themselves in-context, seamlessly, without disrupting the flow of their formal learning activities or informal play. Most contemporary mobile devices are equipped with multimedia support that can be used to create multimodal stories that represent the rich life narratives children experience, imagine, and want to share. The authors investigated these issues over a 9-month series of participatory design sessions in the Human-Computer Interaction Lab (HCIL) at the University of Maryland. In this article, the authors describe their work with children in designing mobile tools for story creation and collaboration. Throughout this work, they asked the following questions: What stories do children want to tell, and how do they want to convey them in a mobile context? The findings suggest the need for mobile technology-based applications that support children’s unique storytelling habits, particularly interruptability and multimodality.

Visual Demand Evaluation Methods for In-Vehicle Interfaces

The primary aim of the research presented in this paper is developing a method for assessing the visual demand (distraction) afforded by in-vehicle information systems (IVIS). In this respect, two alternative methods are considered within the research. The occlusion technique evaluates IVIS tasks in interrupted vision conditions, predicting likely visual demand. However, the technique necessitates performance-focused user trials utilising robust prototypes, and consequently has limitations as an economic evaluation method. In contrast, the Keystroke Level Model (KLM) has long been viewed as a reliable and valid means of modelling human performance and making task time predictions, therefore not requiring empirical trials or a working prototype. The research includes four empirical studies in which an extended KLM was developed and subsequently validated as a means of predicting measures relevant to the occlusion protocol. Future work will develop the method further to widen its scope, introduce new measures, and link the technique to existing design practices.

Ask and You Will Receive

Even though the effective usage of mobile devices has become a mandatory requirement in many professional and private areas, inexperienced users face especially great difficulties in acquiring computer skills. Based on the assumptions of constructivist learning theories, the effect of asking questions and repeated practice on PDA skill acquisition in adults (n = 36) was examined. Learners had the opportunity to ask questions and receive answers during the learning process. One learner group additionally received a manual with basic PDA-operating-principles; a control group received no instructional support at all. As dependent variables task effectiveness, efficiency, subjective ratings of perceived ease of use as well as number and content of questions were assessed. Findings showed that asking questions and repeated practice considerably enhanced PDA-performance in adult novice learners, but not perceived ease of use. Furthermore, the content-analysis of learner questions gave valuable insights into information needs, cognitive barriers and mental models of adult learners, which can contribute to the design of interfaces and computer-based tutors.

Cocktail

Today’s mobile devices are capable of creating and storing a large amount of multimedia data, but sharing such data with others remains challenging. Even when a user wishes to send a picture to a friend located next to the user, current wireless data transfer techniques usually demand several steps requiring inconvenient user involvement, such as manipulation of tiny buttons on a small touch screen. This paper presents ‘Cocktail’, a new gesture-based mobile interaction system that exploits gestures employed by bartenders for easy data sharing between co-located users. In our system, users can pour (transfer) multimedia data in their mobile devices to other friends’ devices in a manner akin to a bartender pouring a drink into a glass. Cocktail also provides an intuitive way of creating new content by mixing existing data with a shaking gesture. For example, users can make music videos with their favorite music and pictures in the mobile phone by selecting them and shaking the mobile phone. The authors have implemented a prototype of Cocktail with commercial smart phones and evaluated its usability via user studies.

Projector Phones

With the miniaturization of projection technology, the integration of tiny projection units into mobile devices is no longer fiction; therefore, such integrated projectors in mobile devices could make mobile projection ubiquitous. These phones will have the ability to project large-scale information onto any surfaces in the real world, and by doing so, the interaction space of the mobile device can be considerably expanded. In addition, physical objects in the environment can be augmented with additional information, which can support interaction concepts that are not even possible on modern desktop computers today. The authors believe that mobile camera-projector units can form a promising interface type for mobile Augmented Reality (AR) applications, thus, this paper identifies different application classes of such interfaces. In addition, different spatial setups of camera and projector units will have an effect on the possible applications and the interaction space with the focus on the augmentation of real word objects in the environment. This paper presents two examples of applications for mobile camera-projector units and different hardware prototypes that allow augmentation of real world objects.