Tangible user interface design for learners with different multiple intelligence

The creation of learning activities responsive to learners with different basic skills has been limited due to a classroom environment and applied technologies. The goals of this research were to develop Tang-MI, a game with a tangible user interface supporting primary school learners’ analytical skills based on the theory of multiple intelligences (MI), and to present design guidelines for a tangible user interface suitable for learners in different MI groups. In this research, the Tangible user interface for multiple intelligence (Tang-MI) was tested with thirty students initially evaluated for their multiple intelligences. The learners’ usage behavior was observed and recorded while the students performed the assigned tasks. The behavioral data were analyzed and grouped into behaviors occurring before performing the tasks, during the tasks, and after completing the tasks. Based on the learners’ usage behavior, the tangible user interface design guidelines for learners in different MI groups were proposed concerning physical equipment design, question design, interactive program design, audio design, and animated visual feedback design. These guidelines would help educators build learning games that respond to the learners’ intelligence styles and enhance students’ motivation to learn.

Learning and Tracking Ad Hoc Fiducial Markers in Spatial Augmented Reality

We describe a spatial augmented reality system with a tangible user interface used to control computer simulations of complex systems. In spatial augmented reality, the user’s physical space is augmented with projected imagery, blending real objects with projected information, and a tangible user interface enables users to manipulate physical objects as controllers for interactive visualizations. Our system learns ad hoc objects in the user’s environment as fiducial markers (i.e., objects that are visually recognized and tracked). When combined with simulation and visualization tools, these interfaces allow the user to control simulations or ensembles of simulations via physical objects using apt metaphors. While other research has leveraged the use of depth cameras, our system enables the use of standard cameras in readily available smartphones and webcams and has an implementation that runs completely in JavaScript in the web browser. We discuss the prerequisite object-recognition requirements for such tangible user interfaces and describe computer-vision and machine-learning algorithms meeting those requirements. We conclude by presenting example applications, which are also available online.

Perpetual Shift

Perpetual Shift, a responsive art installation, exists at the intersection of digital media, art and architecture, bearing a range of connections to the realm of ubiquitous computing and adding to evolving perspectives in the fields of installation art and interactive architecture by proposing a new mode of engaging with data through the physical built environment. The physical embodiment of the sculpture has the functional properties of a Tangible User Interface and employs an innovative approach to material application to generate multi-modal dynamic output (sight and sound). Though as a device Perpetual Shift is capable of being programmed to function as a form of data-physicalization, this artwork is a form of data-sculpture whereby location of the human body is interpreted as data which drives the actuation of the built environment.

Perpetual Shift

Perpetual Shift, a responsive art installation, exists at the intersection of digital media, art and architecture, bearing a range of connections to the realm of ubiquitous computing and adding to evolving perspectives in the fields of installation art and interactive architecture by proposing a new mode of engaging with data through the physical built environment. The physical embodiment of the sculpture has the functional properties of a Tangible User Interface and employs an innovative approach to material application to generate multi-modal dynamic output (sight and sound). Though as a device Perpetual Shift is capable of being programmed to function as a form of data-physicalization, this artwork is a form of data-sculpture whereby location of the human body is interpreted as data which drives the actuation of the built environment.