Scaffolding depth cues and perceptual learning in VR to train stereovision: a proof of concept pilot study

Stereopsis is a valuable feature of human visual perception, which may be impaired or absent in amblyopia and/or strabismus but can be improved through perceptual learning (PL) and videogames. The development of consumer virtual reality (VR) may provide a useful tool for improving stereovision. We report a proof of concept study, especially useful for strabismic patients and/or those with reduced or null stereoacuity. Our novel VR PL strategy is based on a principled approach which included aligning and balancing the perceptual input to the two eyes, dichoptic tasks, exposure to large disparities, scaffolding depth cues and perception for action. We recruited ten adults with normal vision and ten with binocular impairments. Participants played two novel PL games (DartBoard and Halloween) using a VR-HMD. Each game consisted of three depth cue scaffolding conditions, starting with non-binocular and binocular cues to depth and ending with only binocular disparity. All stereo-anomalous participants improved in the game and most (9/10) showed transfer to clinical and psychophysical stereoacuity tests (mean stereoacuity changed from 569 to 296 arc seconds, P < 0.0001). Stereo-normal participants also showed in-game improvement, which transferred to psychophysical tests (mean stereoacuity changed from 23 to a ceiling value of 20 arc seconds, P = 0.001). We conclude that a VR PL approach based on depth cue scaffolding may provide a useful method for improving stereoacuity, and the in-game performance metrics may provide useful insights into principles for effective treatment of stereo anomalies.This study was registered as a clinical trial on 04/05/2010 with the identifier NCT01115283 at ClinicalTrials.gov.

The myth of visual depth cues V: Motion parallax

Motion parallax is conventionally described as a “depth cue.” Rogers &amp; Graham (1979) are credited with providing fairly convincing evidence for this view. Here, I argue that, just as in the case of the other so-called “depth cues,” the claim that “motion parallax” constitutes an independent factor supporting shape and depth perception is circular. Authors offering apparent demonstrations of this cue fail to properly distinguish between proximal and distal stimulus and overlook the fundamental confound of figural organization.

Explicit and implicit depth-cue integration: evidence of systematic biases with real objects

In a previous series of experiments using virtual stimuli, we found evidence that 3D shape estimation agrees to a superadditivity rule of depth-cue combination. According to this rule, adding depth cues leads to greater perceived depth magnitudes and, in principle, to depth overestimation. The mechanism underlying the superadditivity effect can be fully accounted for by a normative theory of cue integration, through the adaptation of a model of cue integration termed the Intrinsic Constraint (IC) model. As for its nature, it remains unclear whether superadditivity is a byproduct of the artificial nature of virtual environments, causing explicit reasoning to infiltrate behavior and inflate the depth judgments when a scene is richer in depth cues, or the genuine output of the process of depth-cue integration. In the present study, we addressed this question by testing whether the IC model’s prediction of superadditivity generalizes beyond VR environments to real world situations. We asked participants to judge the perceived 3D shape of cardboard prisms through a matching task. To assay the potential influence of explicit control over those perceptual estimates, we also asked participants to reach and hold the same objects with their fingertips and we analyzed the in-flight grip size during the reaching. Using physical objects ensured that all visual information was fully consistent with the stimuli’s 3D structure without computer-generated artifacts. We designed a novel technique to carefully control binocular and monocular 3D cues independently from one another, allowing to add or remove depth information from the scene seamlessly. Even with real objects, participants exhibited a clear superadditivity effect in both explicit and implicit tasks. Furthermore, the magnitude of this effect was accurately predicted by the IC model. These results confirm that superadditivity is an inherent feature of depth estimation.

The myth of visual depth cues IV: Cue integration

“Visual depth cues” are conventionally invoked to explain the perception of a 3D world. They are also said to be “combined” or “integrated” for even greater effectiveness. The logical and empirical problems (Maniatis 2021a-c) that apply to the various depth cues individually and the depth cue concept generally apply to “cue integration” as well. Evidence in favor of the view is ad hoc, “models” fundamentally incomplete and contradictions never resolved.

The myth of visual depth cues IV: Shape-from-shading

The popular idea that “shading” is a shape and depth “cue” is the result of a failure to appreciate that neither shading as a physical fact nor shading as a perceptual fact can serve to explain the process leading to visual experience, because the description “shading” does not apply to the proximal stimulation, where this process begins. Both perceived shape and perceived illumination are products of figural constraints.

The myth of visual depth cues II: Linear perspective

“Linear perspective,” used to refer to converging lines in an image, is conventionally deemed a “visual depth cue.” Perceptual evidence is not consistent with this view.

­Scaffolding Depth Cues and Perceptual Learning in VR to Train Stereovision: A Proof of Concept Pilot Study

Stereopsis is a valuable feature of human visual perception, which may be impaired or absent in amblyopia and/or strabismus but can be improved through perceptual learning (PL) and videogames. The development of consumer virtual reality (VR) may provide a useful tool for improving stereovision. We report a proof of concept study, especially useful for strabismic patients and/or those with reduced or null stereoacuity. Our novel VR PL strategy is based on a principled approach which included aligning and balancing the perceptual input to the two eyes, dichoptic tasks, exposure to large disparities, scaffolding depth cues and perception for action. We recruited ten adults with normal vision and ten with binocular impairments. Participants played two novel PL games (DartBoard and Halloween) using a VR-HMD. Each game consisted of three depth cue scaffolding conditions, starting with non-binocular and binocular cues to depth and ending with only binocular disparity. All stereo-anomalous participants improved in the game and most (9/10) showed transfer to clinical and psychophysical stereoacuity tests (mean stereoacuity changed from 569 to 296 arc seconds, p<0.0001). Stereo-normal participants also showed in-game improvement, which transferred to psychophysical tests (mean stereoacuity changed from 23 to a ceiling value of 20 arc seconds, p=0.001). We conclude that a VR PL approach based on depth cue scaffolding may provide a useful method for improving stereoacuity, and the in-game performance metrics may provide useful insights into principles for effective treatment of stereo anomalies.

Unimpaired perception of relative depth from perspective cues in strabismus

Strabismus is a relatively common ophthalmological condition where the coordination of eye muscles to binocularly fixate a single point in space is impaired. This leads to deficits in vision and particularly in three-dimensional (3D) space perception. The exact nature of the deficits in 3D perception is poorly understood as much of understanding has relied on anecdotal reports or conjecture. Here, we investigated, for the first time, the perception of relative depth comparing strabismic and typically developed binocular observers. Specifically, we assessed the susceptibility to the depth cue of perspective convergence as well as the capacity to use this cue to make accurate judgements of relative depth. Susceptibility was measured by examining a 3D bias in making two-dimensional (2D) interval equidistance judgements and accuracy was measured by examining 3D interval equidistance judgements. We tested both monocular and binocular viewing of images of perspective scenes under two different psychophysical methods: two-alternative forced-choice (2AFC) and the method of adjustment. The biasing effect of perspective information on the 2D judgements (3D cue susceptibility) was highly significant and comparable for both subject groups in both the psychophysical tasks (all p s < 0.001) with no statistically significant difference found between the two groups. Both groups showed an underestimation in the 3D task with no significant difference between the group's judgements in the 2AFC task, but a small statistically significant difference (ratio difference of approx. 10%, p = 0.016) in the method of adjustment task. A small but significant effect of viewing condition (monocular versus binocular) was revealed only in the non-strabismic group (ratio difference of approx. 6%, p = 0.002). Our results show that both the automatic susceptibility to, and accuracy in the use of, the perspective convergence cue in strabismus is largely comparable to that found in typically developed binocular vision, and have implications on the nature of the encoding of depth in the human visual system.