TheVeiled Virginillustrates visual segmentation of shape by cause

Three-dimensional (3D) shape perception is one of the most important functions of vision. It is crucial for many tasks, from object recognition to tool use, and yet how the brain represents shape remains poorly understood. Most theories focus on purely geometrical computations (e.g., estimating depths, curvatures, symmetries). Here, however, we find that shape perception also involves sophisticated inferences that parse shapes into features with distinct causal origins. Inspired by marble sculptures such as Strazza’sThe Veiled Virgin(1850), which vividly depict figures swathed in cloth, we created composite shapes by wrapping unfamiliar forms in textile, so that the observable surface relief was the result of complex interactions between the underlying object and overlying fabric. Making sense of such structures requires segmenting the shape based on their causes, to distinguish whether lumps and ridges are due to the shrouded object or to the ripples and folds of the overlying cloth. Three-dimensional scans of the objects with and without the textile provided ground-truth measures of the true physical surface reliefs, against which observers’ judgments could be compared. In a virtual painting task, participants indicated which surface ridges appeared to be caused by the hidden object and which were due to the drapery. In another experiment, participants indicated the perceived depth profile of both surface layers. Their responses reveal that they can robustly distinguish features belonging to the textile from those due to the underlying object. Together, these findings reveal the operation of visual shape-segmentation processes that parse shapes based on their causal origin.

Deconstructing the relief inversion effect: Contributors of the problem and its solutions

<p><strong>Abstract.</strong> <i>Terrain reversal</i> (also known as <i>relief inversion</i>) effect is a common and well-known illusion encountered in shaded relief maps and satellite imagery where the main depth cue is shading/shadows (Imhof, 1967; Bernabe-Poveda, Callejo, &amp; Ballari, 2005; Saraf, Das, Agarwal, &amp; Sundaram, 1996; Biland &amp; Çöltekin, 2016; Çöltekin, Rautenbach, Coetzee, &amp; Mokwena, 2018). This illusion interferes with our perception of <i>shape from shading</i> (e.g., see Kleffner &amp; Ramachandran, 1992; Prados &amp; Faugeras, 2006). If the light shines from below, the shadows are then above, and this conflicts with the human mind’s ‘unconscious statistics’, that is, our minds assume that the light is more or less always above. This cognitive phenomenon is termed <i>light from above prior</i> (Kleffner &amp; Ramachandran, 1992). When the prior is violated, we see three-dimensional (3D) shapes ambiguously, or inverted; such that a valley looks like a ridge in a terrain representation and vice versa (Bernabé-Poveda, Sánchez-Ortega, &amp; Çöltekin, 2011; Bernabé-Poveda &amp; Çöltekin, 2014). In a recent study, we demonstrated that adding texture and color as opposed to the shading alone (as in the shaded relief maps) affect 3D shape identification performance (Çöltekin &amp; Biland, 2018). More precisely, when texture is present, success rates are higher in correctly identifying valleys and ridges, as well as other 3D spatial relationships. This is possibly a result of interpreting the spatial relationships between terrain features because people can recognize them more easily (e.g., a river is easier to identify on a photo than on a shaded relief map). Somewhat surprisingly, we also observed that people are better with 3D shape identification with grayscale images than with the color images; which we interpreted as the result of more pronounced contrast in grayscale images (Çöltekin &amp; Biland, 2018). Because we see that presence of other visual cues do interfere with the relief inversion effect, in this study, we explore other additional factors (depth cues, labels, terrain types, task types, expertise levels and spatial abilities) that may contribute to, or alleviate, the relief inversion effect. Understanding how other factors contribute to the strength of the illusion can help develop better-informed use of the displays that contain this illusion and lead to better solutions. In this vein, in a series of experiments, we examined effects of stereoscopic viewing vs. monoscopic viewing, terrain types (highly rugged vs. subtly rugged), task types (3D shape identification vs. land cover identification; simple vs. complex), expertise levels, and spatial abilities. In a second experiment, we examined how well various solutions function for correcting the illusion in various combinations of variables (labels, motion, stereo). Figure 1 shows an example of the shape identification tasks used in the first study.</p><p>We are developing full papers reporting the effects of each tested factor in detail, which would be beyond the scope of this short paper. In this short paper, we focus on the effects of stereo on the strength of the relief inversion experience (does showing the terrain in stereo make the relief inversion effect stronger or weaker?), and on the solutions for satellite images (if we combine a ‘solution’ with stereo viewing, do shape perception and land cover identification success improve?). The second question, especially the mention of land cover identification, is related to the fact that a common solution to terrain reversal effect is to overlay a semi-transparent shaded relief map (SRM overlay) on top of the image that has the perceptual problem (Bernabé-Poveda, Sánchez-Ortega, &amp; Çöltekin, 2011; Saraf, Das, Agarwal, &amp; Sundaram, 1996b).</p><p> Our initial findings suggest that stereo viewing does help against the issues in 3D shape perception; although it does not entirely remove it: 3D shape perception accuracy is &amp;sim;15% with original images, &amp;sim;32% with the stereo. When stereoscopic viewing is combined with an SRM overlay solution, it improves the 3D shape perception: 3D shape perception accuracy with the SRM overlay solution alone is &amp;sim;40%, with added stereo &amp;sim;68%, but impairs the land cover identification (accuracy is &amp;sim;78% with the SRM overlay, and drops to &amp;sim;44% when stereo is added). These findings are based on two controlled experiments with 33 and 35 participants respectively, and the differences are statistically significant based on analysis of variance (<i>p</i>&amp;thinsp;&amp;lt;&amp;thinsp;0.05). We believe the impairment of the land cover identification is linked to the stereoscopic viewing method, as the tests were conducted with anaglyph stereo, where color perception is strongly affected. These observations, taken together, provide us the initial clues that providing the viewers with an additional depth cue (in this case stereopsis) is indeed helpful; but also suggest that the success of the solution depends on how it is implemented, and the nature of the task; for example, if color is important for the task or not.</p><p> We believe our findings are of key importance in understanding the relief inversion effect, and its future solutions, and will guide cartographers towards a more nuanced comprehension and work practices.</p></p>

Computational model for human 3D shape perception from a single specular image

In natural conditions the human visual system can estimate the 3D shape of specular objects even from a single image. Although previous studies suggested that the orientation field plays a key role for 3D shape perception from specular reflections, its computational plausibility and possible mechanisms have not been investigated. In this study, to complement the orientation field information, we first add prior knowledge that objects are illuminated from above and utilize the vertical polarity of the intensity gradient. Then we construct an algorithm that incorporates these two image cues to estimate 3D shapes from a single specular image. We evaluated the algorithm with glossy and mirrored surfaces and found that 3D shapes can be recovered with a high correlation coefficient of around 0.8 with true surface shapes. Moreover, under a specific condition, the algorithm’s errors resembled those made by human observers. These findings show that the combination of the orientation field and the vertical polarity of the intensity gradient is computationally sufficient and probably reproduces essential representations used in human shape perception from specular reflections.

Binocular stereo acuity affects monocular three-dimensional shape perception in patients with strabismus

Background/aimsTo evaluate the perception of three-dimensional (3D) shape in patients with strabismus and the contributions of stereopsis and monocular cues to this perception.MethodsTwenty-one patients with strabismus with and 20 without stereo acuity as well as 25 age-matched normal volunteers performed two tasks: (1) identifying the closest vertices of 3D shapes from monocular shading (3D-SfS), texture (3D-SfT) or motion cues (3D-SfM) and from binocular disparity (3D-SfD), (2) discriminating 1D elementary features of these cues.ResultsDiscrimination of the elementary features of luminance, texture and motion did not differ across groups. When the distances between reported and actual closest vertices were resolved into sagittal and frontoparallel plane components, sagittal components in 3D-SfS and frontoparallel components in 3D-SfT indicated larger errors in patients with strabismus without stereo acuity than in normal subjects. These patients could not discriminate one-dimensional elementary features of binocular disparity. Patients with strabismus with stereo acuity performed worse for both components of 3D-SfD and frontoparallel components of 3D-SfT compared with normal subjects. No differences were observed in the perception of 3D-SfM across groups. A comparison between normal subjects and patients with strabismus with normal stereopsis revealed no deficit in 3D shape perception from any cue.ConclusionsBinocular stereopsis is essential for fine perception of 3D shape, even when 3D shape is defined by monocular static cues. Interaction between these cues may occur in ventral occipitotemporal regions, where 3D-SfS, 3D-SfT and 3D-SfD are processed in the same or neighbouring cortical regions. Our findings demonstrate the perceptual benefit of binocular stereopsis in patients with strabismus.