Mental geometry of 3D size and shape perception

Judging poses, sizes and shapes of objects accurately is necessary for organisms and machines to operate successfully in the world. Retinal images of 3D objects are mapped by the rules of projective geometry, and preserve the invariants of that geometry. Since Plato, it has been debated whether geometry is innate to the human brain, and Poincare and Einstein thought it worth examining whether formal geometry arises from experience with the world. We examine if humans have learned to exploit projective geometry to estimate sizes and shapes of objects in 3D scenes.Numerous studies have examined size invariance as a function of physical distance, which changes scale on the retina, but surprisingly, possible constancy or inconstancy of relative size seems not to have been investigated for object pose, which changes retinal image size differently along different axes. We show systematic underestimation of length for extents pointing towards or away from the observer, both for static objects and dynamically rotating objects. Observers do correct for projected shortening according to the optimal back-transform, obtained by inverting the projection function, but the correction is inadequate by a multiplicative factor. The clue is provided by the greater underestimation for longer objects, and the observation that they appear more slanted towards the observer. Adding a multiplicative factor for perceived slant in the back-transform model provides good fits to the corrections used by observers. We quantify the slant illusion with relative slant measurements, and use a dynamic demonstration to show the power of the slant illusion.In biological and mechanical objects, distortions of shape are manifold, and changes in aspect ratio and relative limb sizes are functionally important. Our model shows that observers try to retain invariance of these aspects of shape to 3D rotation by correcting retinal image distortions due to perspective projection, but the corrections can fall short. We discuss how these results imply that humans have internalized particular aspects of projective geometry through evolution or learning, and how assuming that images are preserving the continuity, collinearity, and convergence invariances of projective geometry, supplements the Generic Viewpoint assumption, and simply explains other illusions, such as Ames’ Chair.