scholarly journals Microsaccadic Eye Movements but not Pupillary Dilation Response Characterizes the Crossmodal Freezing Effect

2020 ◽  
Vol 1 (1) ◽  
Author(s):  
Lihan Chen ◽  
Hsin-I Liao
Keyword(s):  
Temporal Dynamics ◽  
Pupil Dilation ◽  
Temporal Scale ◽  
Audiovisual Integration ◽  
Visual Localization ◽  
Pupillary Dilation ◽  
Working Mechanism ◽  
Spatial Orienting ◽  
Group Motion ◽  
Audiovisual Stimuli

Abstract In typical spatial orienting tasks, the perception of crossmodal (e.g., audiovisual) stimuli evokes greater pupil dilation and microsaccade inhibition than unisensory stimuli (e.g., visual). The characteristic pupil dilation and microsaccade inhibition has been observed in response to “salient” events/stimuli. Although the “saliency” account is appealing in the spatial domain, whether this occurs in the temporal context remains largely unknown. Here, in a brief temporal scale (within 1 s) and with the working mechanism of involuntary temporal attention, we investigated how eye metric characteristics reflect the temporal dynamics of perceptual organization, with and without multisensory integration. We adopted the crossmodal freezing paradigm using the classical Ternus apparent motion. Results showed that synchronous beeps biased the perceptual report for group motion and triggered the prolonged sound-induced oculomotor inhibition (OMI), whereas the sound-induced OMI was not obvious in a crossmodal task-free scenario (visual localization without audiovisual integration). A general pupil dilation response was observed in the presence of sounds in both visual Ternus motion categorization and visual localization tasks. This study provides the first empirical account of crossmodal integration by capturing microsaccades within a brief temporal scale; OMI but not pupillary dilation response characterizes task-specific audiovisual integration (shown by the crossmodal freezing effect).

scholarly journals Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming

PLoS ONE ◽  
2021 ◽  
Vol 16 (7) ◽  
pp. e0253130
Author(s):  
Nina Heins ◽  
Jennifer Pomp ◽  
Daniel S. Kluger ◽  
Stefan Vinbrüx ◽  
Ima Trempler ◽  
...  
Keyword(s):  
Sound Production ◽  
Audiovisual Integration ◽  
Visual Signals ◽  
Whole Body ◽  
Auditory Signal ◽  
Natural Action ◽  
Simultaneity Judgment ◽  
Audiovisual Stimuli ◽  
High Event ◽  
Audiovisual Asynchrony

Auditory and visual percepts are integrated even when they are not perfectly temporally aligned with each other, especially when the visual signal precedes the auditory signal. This window of temporal integration for asynchronous audiovisual stimuli is relatively well examined in the case of speech, while other natural action-induced sounds have been widely neglected. Here, we studied the detection of audiovisual asynchrony in three different whole-body actions with natural action-induced sounds–hurdling, tap dancing and drumming. In Study 1, we examined whether audiovisual asynchrony detection, assessed by a simultaneity judgment task, differs as a function of sound production intentionality. Based on previous findings, we expected that auditory and visual signals should be integrated over a wider temporal window for actions creating sounds intentionally (tap dancing), compared to actions creating sounds incidentally (hurdling). While percentages of perceived synchrony differed in the expected way, we identified two further factors, namely high event density and low rhythmicity, to induce higher synchrony ratings as well. Therefore, we systematically varied event density and rhythmicity in Study 2, this time using drumming stimuli to exert full control over these variables, and the same simultaneity judgment tasks. Results suggest that high event density leads to a bias to integrate rather than segregate auditory and visual signals, even at relatively large asynchronies. Rhythmicity had a similar, albeit weaker effect, when event density was low. Our findings demonstrate that shorter asynchronies and visual-first asynchronies lead to higher synchrony ratings of whole-body action, pointing to clear parallels with audiovisual integration in speech perception. Overconfidence in the naturally expected, that is, synchrony of sound and sight, was stronger for intentional (vs. incidental) sound production and for movements with high (vs. low) rhythmicity, presumably because both encourage predictive processes. In contrast, high event density appears to increase synchronicity judgments simply because it makes the detection of audiovisual asynchrony more difficult. More studies using real-life audiovisual stimuli with varying event densities and rhythmicities are needed to fully uncover the general mechanisms of audiovisual integration.

The pupillary response to light in the turtle

1995 ◽  
Vol 12 (6) ◽  
pp. 1127-1133 ◽  
Author(s):  
A. M. Granda ◽  
J. R. Dearworth ◽  
C. A. Kittila ◽  
W. D. Boyd
Keyword(s):  
Time Constant ◽  
White Light ◽  
Light Adaptation ◽  
Pupillary Response ◽  
Pupil Dilation ◽  
Pupillary Dilation ◽  
Time Constants ◽  
Visual Threshold ◽  
Color Sensitivity ◽  
Pseudemys Scripta

AbstractWhen intense adapting lights are turned off, the pupil of the turtle, Pseudemys scripta elegans, enlarges. The recovery functions for pupillary dilation have different time constants that are defined by red- and green-sensitive cones and rods as they are affected by prior light adaptation and time in the dark. Pupillary area related to dilation responds over at least a three- to four-fold range. Following white-light adaptation, the course of pupil dilation in the dark shows a three-legged curve of differing time constants. With spectral-light adaptations, the contributions of separate classes of photoreceptors can be isolated. Red- and green-sensitive cones contribute shorter time constants of 3.31 and 3.65 min to prior white-light adaptation—4.81 and 4.18 min to prior spectral-light adaptations. Rods contribute a much longer time constant of 6.69 min to prior white-light adaptation—7.60 min to prior spectral-light adaptation. The ratios are in keeping with the flash sensitivities of photoreceptors in this same animal, as well as with psychophysical visual threshold mechanisms of color sensitivity.

Bayesian Inference Explains Perception of Unity and Ventriloquism Aftereffect: Identification of Common Sources of Audiovisual Stimuli

2007 ◽  
Vol 19 (12) ◽  
pp. 3335-3355 ◽  
Author(s):  
Yoshiyuki Sato ◽  
Taro Toyoizumi ◽  
Kazuyuki Aihara
Keyword(s):  
Bayesian Inference ◽  
Computational Model ◽  
Audiovisual Integration ◽  
Experimental Results ◽  
Auditory Stimuli ◽  
Maximum A Posteriori ◽  
A Posteriori ◽  
Spatial Frame ◽  
Ventriloquism Aftereffect ◽  
Audiovisual Stimuli

We study a computational model of audiovisual integration by setting a Bayesian observer that localizes visual and auditory stimuli without presuming the binding of audiovisual information. The observer adopts the maximum a posteriori approach to estimate the physically delivered position or timing of presented stimuli, simultaneously judging whether they are from the same source or not. Several experimental results on the perception of spatial unity and the ventriloquism effect can be explained comprehensively if the subjects in the experiments are regarded as Bayesian observers who try to accurately locate the stimulus. Moreover, by adaptively changing the inner representation of the Bayesian observer in terms of experience, we show that our model reproduces perceived spatial frame shifts due to the audiovisual adaptation known as the ventriloquism aftereffect.

scholarly journals Speech and non-speech measures of audiovisual integration are not correlated

2021 ◽  
Author(s):  
Jonathan Wilbiks ◽  
Julia Feld Strand ◽  
Violet Aurora Brown
Keyword(s):  
Individual Differences ◽  
Convergent Validity ◽  
Visual Stimuli ◽  
Audiovisual Integration ◽  
Mcgurk Effect ◽  
Large Individual ◽  
Speech Stimuli ◽  
Multiple Tasks ◽  
Auditory Signals ◽  
Audiovisual Stimuli

Many natural events generate both visual and auditory signals, and humans are remarkably adept at integrating information from those sources. However, individuals appear to differ markedly in their ability or propensity to combine what they hear with what they see. Individual differences in audiovisual integration have been established using a range of materials including speech stimuli (seeing and hearing a talker) and simpler audiovisual stimuli (seeing flashes of light combined with tones). Although there are multiple tasks in the literature that are referred to as “measures of audiovisual integration,” the tasks themselves differ widely with respect to both the type of stimuli used (speech versus non-speech) and the nature of the tasks themselves (e.g., some tasks use conflicting auditory and visual stimuli whereas others use congruent stimuli). It is not clear whether these varied tasks are actually measuring the same underlying construct: audiovisual integration. This study tested the convergent validity of four commonly-used measures of audiovisual integration, two of which use speech stimuli (susceptibility to the McGurk effect and a measure of audiovisual benefit), and two of which use non-speech stimuli (the sound-induced flash illusion and audiovisual integration capacity). We replicated previous work showing large individual differences in each measure, but found no significant correlations between any of the measures. These results suggest that tasks that are commonly referred to as measures of audiovisual integration may not be tapping into the same underlying construct.

scholarly journals Inter-and intra-individual coupling between pupillary, electrophysiological, and behavioral responses in a visual oddball task

2020 ◽  
Author(s):  
Sara LoTemplio ◽  
Jack Silcox ◽  
Brennan Payne ◽  
Kara D. Federmeier
Keyword(s):  
Pupil Diameter ◽  
Strong Relationship ◽  
Individual Variability ◽  
Pupil Dilation ◽  
Pupillary Dilation ◽  
Strong Correlations ◽  
Control Center ◽  
The Relationship ◽  
Visual Oddball ◽  
Phasic Activation

Although the P3b component of the event-related brain potential is one of the most widely-studied components, its underlying generators are not currently well understood. Recent theories have suggested that the P3b is triggered by phasic activation of the locus-coeruleus norepinephrine (LC-NE) system, an important control center implicated in facilitating optimal task-relevant behavior. Previous research has reported strong correlations between pupil dilation and LC activity, suggesting that pupil diameter is a useful indicator for ongoing LC-NE activity. Given the strong relationship between LC activity and pupil dilation, if the P3b is driven by phasic LC activity, there should be a robust trial-to-trial relationship with the phasic pupillary dilation response (PDR). However, previous work examining relationships between concurrently recorded pupillary and P3b responses has not supported this. One possibility is that the relationship between the measures might be carried primarily by either inter-individual (i.e., between-participant) or intra-individual (i.e., within-participant) contributions to coupling, and prior work has not systematically delineated these relationships. Doing so in the current study, we do not find evidence for either inter-individual or intra-individual relationships between the PDR and P3b responses. However, baseline pupil dilation did predict the P3b. Interestingly, both the PDR and P3b independently predicted inter-individual and intra-individual variability in decision response time. Implications for the LC-P3b hypothesis are discussed.

scholarly journals Sustained Auditory Attentional Load Decreases Audiovisual Integration in Older and Younger Adults

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Yanna Ren ◽  
Yawei Hou ◽  
Jiayu Huang ◽  
Fanghong Li ◽  
Tao Wang ◽  
...  
Keyword(s):  
Older Adults ◽  
Visual Information ◽  
Race Model ◽  
Audiovisual Integration ◽  
Auditory Presentation ◽  
Younger Adults ◽  
Attentional Load ◽  
Audiovisual Stimuli ◽  
Effects Of Aging ◽  
Load Conditions

The modulation of attentional load on the perception of auditory and visual information has been widely reported; however, whether attentional load alters audiovisual integration (AVI) has seldom been investigated. Here, to explore the effect of sustained auditory attentional load on AVI and the effects of aging, nineteen older and 20 younger adults performed an AV discrimination task with a rapid serial auditory presentation task competing for attentional resources. The results showed that responses to audiovisual stimuli were significantly faster than those to auditory and visual stimuli ( AV > V ≥ A , all p < 0.001 ), and the younger adults were significantly faster than the older adults under all attentional load conditions (all p < 0.001 ). The analysis of the race model showed that AVI was decreased and delayed with the addition of auditory sustained attention ( no _ load > load _ 1 > load _ 2 > load _ 3 > load _ 4 ) for both older and younger adults. In addition, AVI was lower and more delayed in older adults than in younger adults in all attentional load conditions. These results suggested that auditory sustained attentional load decreased AVI and that AVI was reduced in older adults.

Different classes of audiovisual correspondences are processed at distinct levels of the cortical hierarchy

2012 ◽  
Vol 25 (0) ◽  
pp. 69
Author(s):  
Uta Noppeney ◽  
Ruth Adam ◽  
Sepideh Sadaghiani ◽  
Joost X. Maier ◽  
HweeLing Lee ◽  
...  
Keyword(s):  
Phonological Processing ◽  
Audiovisual Integration ◽  
Neural Systems ◽  
Common Source ◽  
Motion Direction ◽  
Sensory Inputs ◽  
Cortical Hierarchy ◽  
Audiovisual Stimuli ◽  
Linguistic Labels ◽  
The Brain

The brain should integrate sensory inputs only when they emanate from a common source and segregate those from different sources. Sensory correspondences are important cues informing the brain whether two sensory inputs are generated by a common event and should hence be integrated. Most prominently, sensory inputs should co-occur in time and space. More complex audiovisual stimuli may also be congruent in terms of semantics (e.g., objects and source sounds) or phonology (e.g., spoken and written words; linked via common linguistic labels). Surprisingly, metaphoric relations (e.g., pitch and height) have also been shown to influence audiovisual integration. The neural mechanisms that mediate these metaphoric congruency effects are only poorly understood. They may be mediated via (i) natural multisensory binding, (ii) common linguistic labels or (iii) semantics. In this talk, we will present a series of studies that investigate whether these different types of audiovisual correspondences are processed by distinct neural systems. Further, we investigate how those systems are employed by metaphoric audiovisual correspondences. Our results demonstrate that different classes of audiovisual correspondences influence multisensory integration at distinct levels of the cortical hierarchy. Spatiotemporal incongruency is detected already at the primary cortical level. Natural (e.g., motion direction) and phonological incongruency influences MSI in areas involved in motion or phonological processing. Critically, metaphoric interactions emerge in neural systems that are shared with natural and semantic incongruency. This activation pattern may reflect the ambivalent nature of metaphoric audiovisual interactions relying on both natural and semantic correspondences.

scholarly journals Crossmodal associations modulate multisensory spatial integration

2020 ◽  
Vol 82 (7) ◽  
pp. 3490-3506
Author(s):  
Jonathan Tong ◽  
Lux Li ◽  
Patrick Bruns ◽  
Brigitte Röder
Keyword(s):  
Multisensory Integration ◽  
Visual Stimulus ◽  
Multisensory Processing ◽  
Visual Stimuli ◽  
Audiovisual Integration ◽  
Spatial Integration ◽  
Association Learning ◽  
Sensory Evidence ◽  
Audiovisual Stimuli ◽  
Crossmodal Associations

Abstract According to the Bayesian framework of multisensory integration, audiovisual stimuli associated with a stronger prior belief that they share a common cause (i.e., causal prior) are predicted to result in a greater degree of perceptual binding and therefore greater audiovisual integration. In the present psychophysical study, we systematically manipulated the causal prior while keeping sensory evidence constant. We paired auditory and visual stimuli during an association phase to be spatiotemporally either congruent or incongruent, with the goal of driving the causal prior in opposite directions for different audiovisual pairs. Following this association phase, every pairwise combination of the auditory and visual stimuli was tested in a typical ventriloquism-effect (VE) paradigm. The size of the VE (i.e., the shift of auditory localization towards the spatially discrepant visual stimulus) indicated the degree of multisensory integration. Results showed that exposure to an audiovisual pairing as spatiotemporally congruent compared to incongruent resulted in a larger subsequent VE (Experiment 1). This effect was further confirmed in a second VE paradigm, where the congruent and the incongruent visual stimuli flanked the auditory stimulus, and a VE in the direction of the congruent visual stimulus was shown (Experiment 2). Since the unisensory reliabilities for the auditory or visual components did not change after the association phase, the observed effects are likely due to changes in multisensory binding by association learning. As suggested by Bayesian theories of multisensory processing, our findings support the existence of crossmodal causal priors that are flexibly shaped by experience in a changing world.

Temporal disparity effects on audiovisual integration in low vision individuals

2012 ◽  
Vol 25 (0) ◽  
pp. 175
Author(s):  
Stefano Targher ◽  
Valeria Occelli ◽  
Massimiliano Zampini
Keyword(s):  
Visual Stimulus ◽  
Visual Detection ◽  
Low Vision ◽  
Audiovisual Integration ◽  
Enhancement Effect ◽  
Temporal Aspects ◽  
Detection Enhancement ◽  
Audiovisual Stimuli ◽  
High Level ◽  
Temporal Disparity

Our recent findings have shown that sounds improve visual detection in low vision individuals when the audiovisual pairs are presented simultaneously. The present study purports to investigate possible temporal aspects of the audiovisual enhancement effect that we have previously reported. Low vision participants were asked to detect the presence of a visual stimulus (yes/no task) either presented in isolation or together with an auditory stimulus at different SOAs. In the first experiment, when the sound was always leading the visual stimuli, there was a significant visual detection enhancement even when the visual stimulus was temporally delayed by 400 ms. However, the visual detection improvement was reduced in the second experiment when the sound could randomly lead or lag the visual stimulus. A significant enhancement was found only when the audiovisual stimuli were synchronized. Taken together, the results of the present study seem to suggest that high-level associations between modalities might modulate audiovisual interactions in low vision individuals.

Sign in / Sign up

Export Citation Format

Share Document