Tinmith-Metro: new outdoor techniques for creating city models with an augmented reality wearable computer

Wearable computing moves computation from the desktop to the user. We are forming a community of networked, wearable-computer users to explore, over a long period, the augmented realities that these systems can provide. By adapting its behavior to the user's changing environment, a body-worn computer can assist the user more intelligently, consistently, and continuously than a desktop system. A text-based augmented reality, the Remembrance Agent, is presented to illustrate this approach. Video cameras are used both to warp the visual input (mediated reality) and to sense the user's world for graphical overlay. With a camera, the computer could track the user's finger to act as the system's mouse; perform face recognition; and detect passive objects to overlay 2.5D and 3D graphics onto the real world. Additional apparatus such as audio systems, infrared beacons for sensing location, and biosensors for learning about the wearer's affect are described. With the use of input from these interface devices and sensors, a long-term goal of this project is to model the user's actions, anticipate his or her needs, and perform a seamless interaction between the virtual and physical environments.

Download Full-text

Augmented reality powers a cognitive assistant for the blind

eLife ◽

10.7554/elife.37841 ◽

2018 ◽

Vol 7 ◽

Cited By ~ 7

Author(s):

Yang Liu ◽

Noelle RB Stiles ◽

Markus Meister

Keyword(s):

Augmented Reality ◽

Assistive Devices ◽

Wearable Computer ◽

Cognitive Level ◽

Sensory Data ◽

Spatial Memories ◽

Intuitive User Interface ◽

Story Building ◽

Further Development ◽

The Brain

To restore vision for the blind, several prosthetic approaches have been explored that convey raw images to the brain. So far, these schemes all suffer from a lack of bandwidth. An alternate approach would restore vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer captures video and other data, extracts important scene knowledge, and conveys that to the user in compact form. Here, we implement an intuitive user interface for such a device using augmented reality: each object in the environment has a voice and communicates with the user on command. With minimal training, this system supports many aspects of visual cognition: obstacle avoidance, scene understanding, formation and recall of spatial memories, navigation. Blind subjects can traverse an unfamiliar multi-story building on their first attempt. To spur further development in this domain, we developed an open-source environment for standardized benchmarking of visual assistive devices.

Download Full-text