Human temporal coordination of visual and auditory events in virtual reality

2012 ◽  
Vol 25 (0) ◽  
pp. 31
Author(s):  
Michiteru Kitazaki
Keyword(s):  
Virtual Reality ◽  
Temporal Order ◽  
Binocular Disparity ◽  
Color Change ◽  
Motion Parallax ◽  
Near Field ◽  
Auditory Stimuli ◽  
Visual Depth ◽  
Virtual Reality Environment ◽  
Depth Cues

Since the speed of sound is much slower than light, we sometimes hear a sound later than an accompanying light event (e.g., thunder and lightning at a far distance). However, Sugita and Suzuki (2003) reported that our brain coordinates a sound and its accompanying light to be perceived simultaneously within 20 m distance. Thus, the light accompanied with physically delayed sound is perceived simultaneously with the sound in near field. We aimed to test if this sound–light coordination occurs in a virtual-reality environment and investigate effects of binocular disparity and motion parallax. Six naive participants observed visual stimuli on a 120-inch screen in a darkroom and heard auditory stimuli from a headphone. A ball was presented in a textured corridor and its distance from the participant was varied from 3–20 m. The ball changed to be in red before or after a short (10 ms) white noise (time difference: −120, −60, −30, 0, +30, +60, +120 ms), and participants judged temporal order of the color-change and the sound. We varied visual depth cues (binocular disparity and motion parallax) in the virtual-reality environment, and measured the physical delay at which visual and auditory events were perceived simultaneously. In terms of the results, we did not find sound–light coordination without binocular disparity or motion parallax, but found it with both cues. These results suggest that binocular disparity and motion parallax are effective for sound–light coordination in virtual-reality environment, and richness of depth cues are important for the coordination.

scholarly journals Border-ownership-dependent tilt aftereffect for shape defined by binocular disparity and motion parallax

2018 ◽  
Author(s):  
Reuben Rideaux ◽  
William J Harrison
Keyword(s):  
Binocular Disparity ◽  
Motion Parallax ◽  
Neural Systems ◽  
Visual Object ◽  
Access To Information ◽  
Tilt Aftereffect ◽  
Critical Function ◽  
Depth Order ◽  
Depth Cues ◽  
Ground Segmentation

ABSTRACTDiscerning objects from their surrounds (i.e., figure-ground segmentation) in a way that guides adaptive behaviours is a fundamental task of the brain. Neurophysiological work has revealed a class of cells in the macaque visual cortex that may be ideally suited to support this neural computation: border-ownership cells (Zhou, Friedman, & von der Heydt, 2000). These orientation-tuned cells appear to respond conditionally to the borders of objects. A behavioural correlate supporting the existence of these cells in humans was demonstrated using two-dimensional luminance defined objects (von der Heydt, Macuda, & Qiu, 2005). However, objects in our natural visual environments are often signalled by complex cues, such as motion and depth order. Thus, for border-ownership systems to effectively support figure-ground segmentation and object depth ordering, they must have access to information from multiple depth cues with strict depth order selectivity. Here we measure in humans (of both sexes) border-ownership-dependent tilt aftereffects after adapting to figures defined by either motion parallax or binocular disparity. We find that both depth cues produce a tilt aftereffect that is selective for figure-ground depth order. Further, we find the effects of adaptation are transferable between cues, suggesting that these systems may combine depth cues to reduce uncertainty (Bülthoff & Mallot, 1988). These results suggest that border-ownership mechanisms have strict depth order selectivity and access to multiple depth cues that are jointly encoded, providing compelling psychophysical support for their role in figure-ground segmentation in natural visual environments.SIGNIFICANCE STATEMENTSegmenting a visual object from its surrounds is a critical function that may be supported by “border-ownership” neural systems that conditionally respond to object borders. Psychophysical work indicates these systems are sensitive to objects defined by luminance contrast. To effectively support figure-ground segmentation, however, neural systems supporting border-ownership must have access to information from multiple depth cues and depth order selectivity. We measured border-ownership-dependent tilt aftereffects to figures defined by either motion parallax or binocular disparity and found aftereffects for both depth cues. These effects were transferable between cues, but selective for figure-ground depth order. Our results suggest that the neural systems supporting figure-ground segmentation have strict depth order selectivity and access to multiple depth cues that are jointly encoded.

scholarly journals A neural mechanism for detecting object motion during self-motion

2021 ◽  
Author(s):  
HyungGoo Kim ◽  
Dora Angelaki ◽  
Gregory DeAngelis
Keyword(s):  
Visual System ◽  
Moving Objects ◽  
Binocular Disparity ◽  
Motion Parallax ◽  
Neural Mechanism ◽  
Object Motion ◽  
Depth Cues ◽  
Behavioral Studies ◽  
Observer Motion ◽  
Self Motion

Detecting objects that move in a scene is a fundamental computation performed by the visual system. This computation is greatly complicated by observer motion, which causes most objects to move across the retinal image. How the visual system detects scene-relative object motion during self-motion is poorly understood. Human behavioral studies suggest that the visual system may identify local conflicts between motion parallax and binocular disparity cues to depth, and may use these signals to detect moving objects. We describe a novel mechanism for performing this computation based on neurons in macaque area MT with incongruent depth tuning for binocular disparity and motion parallax cues. Neurons with incongruent tuning respond selectively to scene-relative object motion and their responses are predictive of perceptual decisions when animals are trained to detect a moving object during selfmotion. This finding establishes a novel functional role for neurons with incongruent tuning for multiple depth cues.

Does the Size-Arrival Effect Occur With an Active Collision-Avoidance Task in an Immersive 3D Virtual Reality Environment?

2021 ◽  
pp. 001872082110310
Author(s):  
Patricia R. DeLucia ◽  
Adam M. Braly ◽  
Bria R. Savoy
Keyword(s):  
Virtual Reality ◽  
Collision Avoidance ◽  
Binocular Disparity ◽  
Avoidance Task ◽  
Small Object ◽  
Warning Systems ◽  
Virtual Reality Environment ◽  
Large Object ◽  
Left Turn ◽  
3D Environments

Objective Determine whether the size-arrival effect (SAE) occurs with immersive, 3D visual experiences and active collision-avoidance responses. Background When a small near object and a large far object approach the observer at the same speeds, the large object appears to arrive before the small object, known as the size-arrival effect (SAE), which may contribute to crashes between motorcycles and cars. Prior studies of the SAE were limited because they used two dimensional displays and asked participants to make passive judgments. Method Participants viewed approaching objects using a virtual reality (VR) headset. In an active task, participants ducked before the object hit them. In a passive prediction-motion (PM) judgment, the approaching object disappeared, and participants pressed a button when they thought the object would hit them. In a passive relative TTC judgment, participants reported which of two approaching objects would reach them first. Results The SAE occurred with the PM and relative TTC tasks but not with the ducking task. The SAE can occur in immersive 3D environments but is limited by the nature of the task and display. Application Certain traffic situations may be more prone to the SAE and have higher risk for collisions. For example, in left-turn scenarios (e.g., see Levulis, 2018), drivers make passive judgments when oncoming vehicles are far and optical expansion is slow, and binocular disparity putatively is ineffective. Collision-avoidance warning systems may be needed more in such scenarios than when vehicles are near and drivers’ judgments of TTC may be more accurate (DeLucia, 2008).

The myth of visual depth cues V: Motion parallax

2021 ◽  
Author(s):  
Lydia Maniatis
Keyword(s):  
Depth Perception ◽  
Motion Parallax ◽  
Convincing Evidence ◽  
The Other ◽  
Visual Depth ◽  
Independent Factor ◽  
Depth Cues ◽  
Depth Cue

Motion parallax is conventionally described as a “depth cue.” Rogers & Graham (1979) are credited with providing fairly convincing evidence for this view. Here, I argue that, just as in the case of the other so-called “depth cues,” the claim that “motion parallax” constitutes an independent factor supporting shape and depth perception is circular. Authors offering apparent demonstrations of this cue fail to properly distinguish between proximal and distal stimulus and overlook the fundamental confound of figural organization.

scholarly journals Border ownership-dependent tilt aftereffect for shape defined by binocular disparity and motion parallax

2019 ◽  
Vol 121 (5) ◽  
pp. 1917-1923 ◽  
Author(s):  
Reuben Rideaux ◽  
William J. Harrison
Keyword(s):  
Binocular Disparity ◽  
Motion Parallax ◽  
Neural Systems ◽  
Tilt Aftereffect ◽  
Critical Function ◽  
Depth Order ◽  
Depth Cues ◽  
Ground Segmentation ◽  
Object Depth ◽  
The Brain

Discerning objects from their surrounds (i.e., figure-ground segmentation) in a way that guides adaptive behaviors is a fundamental task of the brain. Neurophysiological work has revealed a class of cells in the macaque visual cortex that may be ideally suited to support this neural computation: border ownership cells (Zhou H, Friedman HS, von der Heydt R. J Neurosci 20: 6594–6611, 2000). These orientation-tuned cells appear to respond conditionally to the borders of objects. A behavioral correlate supporting the existence of these cells in humans was demonstrated with two-dimensional luminance-defined objects (von der Heydt R, Macuda T, Qiu FT. J Opt Soc Am A Opt Image Sci Vis 22: 2222–2229, 2005). However, objects in our natural visual environments are often signaled by complex cues, such as motion and binocular disparity. Thus for border ownership systems to effectively support figure-ground segmentation and object depth ordering, they must have access to information from multiple depth cues with strict depth order selectivity. Here we measured in humans (of both sexes) border ownership-dependent tilt aftereffects after adaptation to figures defined by either motion parallax or binocular disparity. We find that both depth cues produce a tilt aftereffect that is selective for figure-ground depth order. Furthermore, we find that the effects of adaptation are transferable between cues, suggesting that these systems may combine depth cues to reduce uncertainty (Bülthoff HH, Mallot HA. J Opt Soc Am A 5: 1749–1758, 1988). These results suggest that border ownership mechanisms have strict depth order selectivity and access to multiple depth cues that are jointly encoded, providing compelling psychophysical support for their role in figure-ground segmentation in natural visual environments. NEW & NOTEWORTHY Figure-ground segmentation is a critical function that may be supported by “border ownership” neural systems that conditionally respond to object borders. We measured border ownership-dependent tilt aftereffects to figures defined by motion parallax or binocular disparity and found aftereffects for both cues. These effects were transferable between cues but selective for figure-ground depth order, suggesting that the neural systems supporting figure-ground segmentation have strict depth order selectivity and access to multiple depth cues that are jointly encoded.

scholarly journals Effects of Partial Binocular Disparity on Behavior in a Virtual Reality Environment

2020 ◽  
Vol 56 (Supplement) ◽  
pp. 1D1-02-1D1-02
Author(s):  
Junnosuke SOMA ◽  
Yutaro HIRAO ◽  
Takashi KAWAI
Keyword(s):  
Virtual Reality ◽  
Binocular Disparity ◽  
Virtual Reality Environment

Top-Down Processing and Visual Reorientation Illusions in a Virtual Reality Environment

2004 ◽  
Vol 63 (3) ◽  
pp. 143-149 ◽  
Author(s):  
Fred W. Mast ◽  
Charles M. Oman
Keyword(s):  
Virtual Reality ◽  
Visual Processing ◽  
Line Length ◽  
Perceptual Cues ◽  
Virtual Reality Environment ◽  
Top Down ◽  
Test Conditions ◽  
The Subject ◽  
Processing Mechanisms

The role of top-down processing on the horizontal-vertical line length illusion was examined by means of an ambiguous room with dual visual verticals. In one of the test conditions, the subjects were cued to one of the two verticals and were instructed to cognitively reassign the apparent vertical to the cued orientation. When they have mentally adjusted their perception, two lines in a plus sign configuration appeared and the subjects had to evaluate which line was longer. The results showed that the line length appeared longer when it was aligned with the direction of the vertical currently perceived by the subject. This study provides a demonstration that top-down processing influences lower level visual processing mechanisms. In another test condition, the subjects had all perceptual cues available and the influence was even stronger.

Sign in / Sign up

Export Citation Format

Share Document