Audio-Visual Causality and Stimulus Reliability Affect Audio-Visual Synchrony Perception

People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and speech stimuli. Moreover, participants were more tolerant of AVOA of speech stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of “synchronous” responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception.

Rapid Temporal Recalibration to Audiovisual Asynchrony Occurs Across the Difference in Neural Processing Speed Based on Spatial Frequency

Audiovisual integration relies on temporal synchrony between visual and auditory stimuli. The brain rapidly adapts to audiovisual asynchronous events by shifting the timing of subjective synchrony in the direction of the leading modality of the most recent event, a process called rapid temporal recalibration. This phenomenon is the flexible function of audiovisual synchrony perception. Previous studies found that neural processing speed based on spatial frequency (SF) affects the timing of subjective synchrony. This study examined the effects of SF on the rapid temporal recalibration process by discriminating whether the presentation of the visual and auditory stimuli was simultaneous. I compared the magnitudes of the recalibration effect between low and high SF visual stimuli using two techniques. First, I randomly presented each SF accompanied by a tone during one session, then in a second experiment, only a single SF was paired with the tone throughout the one session. The results indicated that rapid recalibration occurred regardless of difference in presented SF between preceding and test trials. The recalibration magnitude did not significantly differ between the SF conditions. These findings confirm that intersensory temporal process is important to produce rapid recalibration and suggest that rapid recalibration can be induced by the simultaneity judgment criterion changes attributed to the low-level temporal information of audiovisual events.

Rhythm judgments reveal a frequency asymmetry in the perception and neural coding of sound synchrony

In modern Western music, melody is commonly conveyed by pitch changes in the highest-register voice, whereas meter or rhythm is often carried by instruments with lower pitches. An intriguing and recently suggested possibility is that the custom of assigning rhythmic functions to lower-pitch instruments may have emerged because of fundamental properties of the auditory system that result in superior time encoding for low pitches. Here we compare rhythm and synchrony perception between low- and high-frequency tones, using both behavioral and EEG techniques. Both methods were consistent in showing no superiority in time encoding for low over high frequencies. However, listeners were consistently more sensitive to timing differences between two nearly synchronous tones when the high-frequency tone followed the low-frequency tone than vice versa. The results demonstrate no superiority of low frequencies in timing judgments but reveal a robust asymmetry in the perception and neural coding of synchrony that reflects greater tolerance for delays of low- relative to high-frequency sounds than vice versa. We propose that this asymmetry exists to compensate for inherent and variable time delays in cochlear processing, as well as the acoustical properties of sound sources in the natural environment, thereby providing veridical perceptual experiences of simultaneity.