In this paper, we discuss a novel approach to multimodal input design in Tangible User Interface (TUI). We present a prototype Bubble Journey, a game platform where users control the avatar in flash game by blowing a real handle. This computer game was combined multimodal input tool embedded sensor, which augment experience of user’s (children’s) five senses and body into game’s digital world with previous experience in daily life. Sensor embodied in multimodal input tool can convert data of sounds and movements produced by users (children) into digital signals to manipulate the virtual characters’ performance in the game.
A computer vision tangible user interface for mixed reality billiards
