Using Immersive Virtual Reality to Examine How Visual and Tactile Cues Drive the Material-Weight Illusion

The material-weight illusion (MWI) demonstrates how our past experience with material and weight can create expectations that influence the perceived heaviness of an object. Here we used mixed-reality to place touch and vision in conflict, to investigate whether the modality through which materials are presented to a lifter could influence the top-down perceptual processes driving the MWI. University students lifted equally-weighted polystyrene, cork and granite cubes whilst viewing computer-generated images of the cubes in virtual reality (VR). This allowed the visual and tactile material cues to be altered, whilst all other object properties were kept constant. Representation of the objects’ material in VR was manipulated to create four sensory conditions: visual-tactile matched, visual-tactile mismatched, visual differences only and tactile differences only. A robust MWI was induced across all sensory conditions, whereby the polystyrene object felt heavier than the granite object. The strength of the MWI differed across conditions, with tactile material cues having a stronger influence on perceived heaviness than visual material cues. We discuss how these results suggest a mechanism whereby multisensory integration directly impacts how top-down processes shape perception.

“Impossible” Somatosensation and the (Ir)rationality of Perception

Impossible figures represent the world in ways it cannot be. From the work of M. C. Escher to any popular perception textbook, such experiences show how some principles of mental processing can be so entrenched and inflexible as to produce absurd and even incoherent outcomes that could not occur in reality. Surprisingly, however, such impossible experiences are mostly limited to visual perception; are there “impossible figures” for other sensory modalities? Here, we import a known magic trick into the laboratory to report and investigate an impossible somatosensory experience—one that can be physically felt. We show that, even under full-cue conditions with objects that can be freely inspected, subjects can be made to experience a single object alone as feeling heavier than a group of objects that includes the single object as a member—an impossible and phenomenologically striking experience of weight. Moreover, we suggest that this phenomenon—a special case of the size-weight illusion—reflects a kind of “anti-Bayesian” perceptual updating that amplifies a challenge to rational models of perception and cognition. Impossibility can not only be seen, but also felt—and in ways that matter for accounts of (ir)rational mental processing.

The size-weight illusion is unimpaired in individuals with a history of congenital visual deprivation

Visual deprivation in childhood can lead to lifelong impairments in multisensory processing. Here, the Size-Weight Illusion (SWI) was used to test whether visuo-haptic integration recovers after early visual deprivation. Normally sighted individuals perceive larger objects to be lighter than smaller objects of the same weight. In Experiment 1, individuals treated for dense bilateral congenital cataracts (who had no patterned visual experience at birth), individuals treated for developmental cataracts (who had patterned visual experience at birth, but were visually impaired), congenitally blind individuals and normally sighted individuals had to rate the weight of manually explored cubes that differed in size (Small, Medium, Large) across two possible weights (350 g, 700 g). In Experiment 2, individuals treated for dense bilateral congenital cataracts were compared to sighted individuals in a similar task using a string set-up, which removed haptic size cues. In both experiments, indistinguishable SWI effects were observed across all groups. These results provide evidence that early aberrant vision does not interfere with the development of the SWI, and suggest a recovery of the integration of size and weight cues provided by the visual and haptic modality.

Fooling the size–weight illusion—Using augmented reality to eliminate the effect of size on perceptions of heaviness and sensorimotor prediction

Augmented reality, whereby computer-generated images are overlaid onto the physical environment, is becoming significant part of the world of education and training. Little is known, however, about how these external images are treated by the sensorimotor system of the user – are they fully integrated into the external environmental cues, or largely ignored by low-level perceptual and motor processes? Here, we examined this question in the context of the size–weight illusion (SWI). Thirty-two participants repeatedly lifted and reported the heaviness of two cubes of unequal volume but equal mass in alternation. Half of the participants saw semi-transparent equally sized holographic cubes superimposed onto the physical cubes through a head-mounted display. Fingertip force rates were measured prior to lift-off to determine how the holograms influenced sensorimotor prediction, while verbal reports of heaviness after each lift indicated how the holographic size cues influenced the SWI. As expected, participants who lifted without augmented visual cues lifted the large object at a higher rate of force than the small object on early lifts and experienced a robust SWI across all trials. In contrast, participants who lifted the (apparently equal-sized) augmented cubes used similar force rates for each object. Furthermore, they experienced no SWI during the first lifts of the objects, with a SWI developing over repeated trials. These results indicate that holographic cues initially dominate physical cues and cognitive knowledge, but are dismissed when conflicting with cues from other senses.

The Size-Weight Illusion is unimpaired in individuals with a history of congenital visual deprivation

Visual deprivation in childhood can lead to lifelong impairments in visual and multisensory processing. Here, the Size-Weight-Illusion was used to test whether visuo-haptic integration recovers after sight restoration. In Experiment 1, congenital (CC: 7 (3F), 8–35 years) and developmental cataract reversal individuals (DC: 9 (2F), 8–37 years), as well as congenitally blind (CB: 2 (1F), 33 and 44 years) and normally sighted individuals (SC: 10 (7F), 19-36 years) perceived larger objects as lighter than smaller objects of the same weight. In Experiment 2, CC (6 (1F), 17–44.7 years) and SC (7 (5F), 21-29 years) individuals performed identically when tested without haptic size cues. Together, this suggested that early visual experience is not necessary to perceive the Size-Weight-Illusion.

Investigating the Role of Haptic and Visual Perception in the Material-Weight Illusion

Weight illusions provide a compelling demonstration that prior experience affects perception. Here we investigated how the expectation-inducing modality affected the Material-Weight Illusion (MWI), where dense-looking objects feel lighter than less dense-looking objects. Participants lifted equally-weighted polystyrene, cork, and granite cubes whilst viewing computer-generated images of the cubes in virtual reality (VR). The representation of the object in VR was manipulated to create four illusion-inducing sensory conditions: visual differences only, haptic differences only, congruent visual-haptic differences, and incongruent visual-haptic material differences. Although an MWI was induced in all conditions, whereby the polystyrene object was reported to feel heavier than the granite object, the strength of the MWI differed across conditions, with haptic material cues having a stronger influence on perceived heaviness than visual material cues. These results are consistent with optimal integration theories of multi-modal perception, highlighting that perception reflects individual cues’ reliability and relevance in specific contexts.