Mobile Recognition and Tracking of Objects in the Environment through Augmented Reality and 3D Audio Cues for People with Visual Impairments

Author(s):  
Oliver Beren Kaul ◽  
Kersten Behrens ◽  
Michael Rohs
Author(s):  
Terek Arce ◽  
Henry Fuchs ◽  
Kyla McMullen

Currently available augmented reality systems have a narrow field of view, giving users only a small window to look through to find holograms in the environment. The challenge for developers is to direct users’ attention to holograms outside this window. To alleviate this field of view constraint, most research has focused on hardware improvements to the head mounted display. However, incorporating 3D audio cues into programs could also aid users in this localization task. This paper investigates the effectiveness of 3D audio on hologram localization. A comparison of 3D audio, visual, and mixed-mode stimuli shows that users are able to localize holograms significantly faster under conditions that include 3D audio. To our knowledge, this is the first study to explore the use of 3D audio in localization tasks using augmented reality systems. The results provide a basis for the incorporation of 3D audio in augmented reality applications.


Author(s):  
Reinhold Behringer ◽  
Steven Chen ◽  
Venkataraman Sundareswaran ◽  
Kenneth Wang ◽  
Marius Vassiliou

2017 ◽  
Vol 111 (2) ◽  
pp. 148-164 ◽  
Author(s):  
Oana Bălan ◽  
Alin Moldoveanu ◽  
Florica Moldoveanu ◽  
Hunor Nagy ◽  
György Wersényi ◽  
...  

Introduction As the number of people with visual impairments (that is, those who are blind or have low vision) is continuously increasing, rehabilitation and engineering researchers have identified the need to design sensory-substitution devices that would offer assistance and guidance to these people for performing navigational tasks. Auditory and haptic cues have been shown to be an effective approach towards creating a rich spatial representation of the environment, so they are considered for inclusion in the development of assistive tools that would enable people with visual impairments to acquire knowledge of the surrounding space in a way close to the visually based perception of sighted individuals. However, achieving efficiency through a sensory substitution device requires extensive training for visually impaired users to learn how to process the artificial auditory cues and convert them into spatial information. Methods Considering all the potential advantages game-based learning can provide, we propose a new method for training sound localization and virtual navigational skills of visually impaired people in a 3D audio game with hierarchical levels of difficulty. The training procedure is focused on a multimodal (auditory and haptic) learning approach in which the subjects have been asked to listen to 3D sounds while simultaneously perceiving a series of vibrations on a haptic headband that corresponds to the direction of the sound source in space. Results The results we obtained in a sound-localization experiment with 10 visually impaired people showed that the proposed training strategy resulted in significant improvements in auditory performance and navigation skills of the subjects, thus ensuring behavioral gains in the spatial perception of the environment.


1999 ◽  
Vol 23 (6) ◽  
pp. 821-825 ◽  
Author(s):  
Reinhold Behringer ◽  
Steven Chen ◽  
Venkataraman Sundareswaran ◽  
Kenneth Wang ◽  
Marius Vassiliou

2021 ◽  
Vol 5 (ISS) ◽  
pp. 1-33
Author(s):  
Dr-Ing Lauren Thevin ◽  
Nicolas Rodier ◽  
Bernard Oriola ◽  
Martin HACHET ◽  
Christophe Jouffrais ◽  
...  

Board games allow us to share collective entertainment experiences. They entertain because of the interactions between players, physical manipulation of tokens and decision making. Unfortunately, most board games exclude people with visual impairments as they were not initially designed for players with special needs. Through a user-centered design process with an accessible game library and visually impaired players, we observed challenges and solutions in making existing board games accessible through handcrafted solutions (tactile stickers, braille labels, etc.). In a second step, we used Spatial Augmented Reality (SAR), to make existing board games inclusive by adding interactivity (GameARt). In a case study with an existing board game considered as not accessible (Jamaica), we designed an interactive SAR version with touch detection (JamaicAR). We evaluated this prototype in a user study with 5 groups of 3 players each, including sighted, low vision and blind players. All players, independent of visual status, were able to play the Augmented Reality game. Moreover, the game was rated positively by all players regarding attractiveness, play engrossment, enjoyment and social connectivity. Our work shows that Spatial Augmented Reality has the potential to make board games accessible to people with visual impairments when handcrafted adaptations fall short.


Sign in / Sign up

Export Citation Format

Share Document