A Modular Framework for Vision-Based Human Computer Interaction

This chapter introduces the VirtualBoard framework for building vision-based Perceptual User Interfaces (PUI). While most vision-based Human Computer Interaction applications developed over the last decade focus on the technological aspects related to image processing and computer vision, our main effort is towards ease and naturalness of use, integrability and compatibility with the existing systems and software, portability and efficiency. VirtualBoard is based on a modular architecture which allows the implementation of several classes of gestural and vision-based human-computer interaction approaches: it is extensible and portable and requires relatively few computational resources, thus also helping in reducing energy consumption and hardware costs. Particular attention is also devoted to robustness to environment conditions (such as illumination and noise level). We believe that current technologies can easily support vision-based PUIs and that PUIs are strongly needed by modern applications. With the exception of gaming industry, where vision-based PUIs are already being intensively studied and in some cases exploited, more effort is needed to merge the knowledge from HCI and computer vision communities to develop realistic and industrially appealing products. This work is intended as a stimulus in this direction.

A Modular Framework for Vision-Based Human Computer Interaction

This chapter introduces the VirtualBoard framework for building vision-based Perceptual User Interfaces (PUI). While most vision-based Human Computer Interaction applications developed over the last decade focus on the technological aspects related to image processing and computer vision, our main effort is towards ease and naturalness of use, integrability and compatibility with the existing systems and software, portability and efficiency. VirtualBoard is based on a modular architecture which allows the implementation of several classes of gestural and vision-based human-computer interaction approaches: it is extensible and portable and requires relatively few computational resources, thus also helping in reducing energy consumption and hardware costs. Particular attention is also devoted to robustness to environment conditions (such as illumination and noise level). We believe that current technologies can easily support vision-based PUIs and that PUIs are strongly needed by modern applications. With the exception of gaming industry, where vision-based PUIs are already being intensively studied and in some cases exploited, more effort is needed to merge the knowledge from HCI and computer vision communities to develop realistic and industrially appealing products. This work is intended as a stimulus in this direction.

Facial Gesture Recognition

From facial gestures, we can extract many kinds of messages in human communication: they represent visible speech signals and clarify whether our current focus of attention is important, funny or unpleasant for us. They are direct, naturally preeminent means for humans to communicate their emotions (Russell and Fernandez-Dols, 1997). Automatic analyzers of subtle facial changes, therefore, seem to have a natural place in various vision systems including automated tools for psychological research, lip reading, bimodal speech analysis, affective computing, face and visual-speech synthesis, and perceptual user interfaces.