scholarly journals Cross-modal phonetic encoding facilitates the McGurk illusion and phonemic restoration

2018 ◽  
Vol 120 (6) ◽  
pp. 2988-3000 ◽  
Author(s):  
Noelle T. Abbott ◽  
Antoine J. Shahin

In spoken language, audiovisual (AV) perception occurs when the visual modality influences encoding of acoustic features (e.g., phonetic representations) at the auditory cortex. We examined how visual speech (mouth movements) transforms phonetic representations, indexed by changes to the N1 auditory evoked potential (AEP). EEG was acquired while human subjects watched and listened to videos of a speaker uttering consonant vowel (CV) syllables, /ba/ and /wa/, presented in auditory-only or AV congruent or incongruent contexts or in a context in which the consonants were replaced by white noise (noise replaced). Subjects reported whether they heard “ba” or “wa.” We hypothesized that the auditory N1 amplitude during illusory perception (caused by incongruent AV input, as in the McGurk illusion, or white noise-replaced consonants in CV utterances) should shift to reflect the auditory N1 characteristics of the phonemes conveyed visually (by mouth movements) as opposed to acoustically. Indeed, the N1 AEP became larger and occurred earlier when listeners experienced illusory “ba” (video /ba/, audio /wa/, heard as “ba”) and vice versa when they experienced illusory “wa” (video /wa/, audio /ba/, heard as “wa”), mirroring the N1 AEP characteristics for /ba/ and /wa/ observed in natural acoustic situations (e.g., auditory-only setting). This visually mediated N1 behavior was also observed for noise-replaced CVs. Taken together, the findings suggest that information relayed by the visual modality modifies phonetic representations at the auditory cortex and that similar neural mechanisms support the McGurk illusion and visually mediated phonemic restoration. NEW & NOTEWORTHY Using a variant of the McGurk illusion experimental design (using the syllables /ba/ and /wa/), we demonstrate that lipreading influences phonetic encoding at the auditory cortex. We show that the N1 auditory evoked potential morphology shifts to resemble the N1 morphology of the syllable conveyed visually. We also show similar N1 shifts when the consonants are replaced by white noise, suggesting that the McGurk illusion and the visually mediated phonemic restoration rely on common mechanisms.

Author(s):  
Felipe A. Araujo ◽  
Eduardo B. Jacobi ◽  
Juliana Avila-Souza ◽  
Jose F. Rodrigues ◽  
Renan C. Moioli ◽  
...  

2004 ◽  
Vol 359 (1-2) ◽  
pp. 29-32 ◽  
Author(s):  
Eiko Hayashi ◽  
Yoshiyuki Kuroiwa ◽  
Shu Omoto ◽  
Toshiaki Kamitani ◽  
Mei Li ◽  
...  

1980 ◽  
Vol 51 (2) ◽  
pp. 643-646 ◽  
Author(s):  
Carl P. Browman ◽  
Helen T. Sullivan

Averaged auditory evoked potential waveforms were recorded from the vertex of human subjects 3, 9, and 15 hr. after awakening. Responses to pure tones of 60 and 80 dB SL were obtained at each session. Amplitude measures of N1–P2 and P2–N2 components were largest in the morning followed by a gradual attenuation across sessions. Waveform amplitude during the late evening ranged from 85% to 91% of the morning amplitude.


1993 ◽  
Vol 69 (6) ◽  
pp. 1837-1849 ◽  
Author(s):  
D. S. Barth ◽  
J. Kithas ◽  
S. Di

1. Two 8 x 8-channel microelectrode arrays were used to map epicortical field potentials from a 3.5 x 3.5-mm2 area in homologous regions of right and left parietotemporal cortex of four rats. Potentials were evoked with bilaterally presented click stimuli and with bilateral tactile stimulation of the 25 major vibrissae. The spatial distribution of temporal components of the somatosensory evoked potential (SEP) and auditory evoked potential (AEP) complex were compared directly with cytochrome oxidase-stained sections of the recorded region. 2. Epicortical responses in both hemispheres to bilateral vibrissal stimuli consisted of a biphasic sharp wave (P1a-N1) constrained to the vibrissa/barrel granular region of primary somatosensory cortex (SmI). A slightly later sharp positive wave (P1b) was localized to secondary somatosensory cortex (SmII) and to perigranular cortex medial to the vibrissa/barrel field. The SEP complex ended with a biphasic slow wave (P2-N2). The P2 was centered on SmI and spread to dysgranular lateral cortex, caudal to but excluding SmII. The N2 was centered on SmII and spread to dysgranular cortex caudal to but excluding SmI. 3. The anatomic organization of the AEP in many ways approximated that of the SEP in the same animals. The timing and morphology of the AEP were nearly identical to the SEP. The AEP consisted of a P1a-N1 sharp wave constrained to the estimated region of primary auditory cortex (AI) in the lateral parietotemporal region, a later P1b localized to secondary auditory cortex (AII), and subsequent slow waves (P2 and N2) that were centered on AI and AII, respectively, and spread to dysgranular regions overlapping the distributions of the P2 and N2 of the SEP complex. 4. These data suggest that the basic neural generators for the SEP and AEP in parietotemporal cortex are quite similar, and provide evidence for the functional anatomy of each temporal component of the sensory evoked potential complex. It is concluded that the early fast waves of the SEP and AEP are modality specific and may represent the parallel activation of primary and secondary sensory cortex through established parallel afferent projections from lateral and medial thalamic nuclei. The later slow waves of the SEP and AEP appear to selectively involve primary and secondary sensory cortex but are more widely distributed, possibly reflecting a less modality-specific level of information processing in dysgranular cortex.


2021 ◽  
Vol 31 (Supplement_2) ◽  
Author(s):  
Cláudia Reis ◽  
Margarida Teixeira

Abstract Background The objective of this study was to verify wether it was possible to observe greater plasticity of the auditory cortex and greater benefits in terms of auditory processing, better discrimination, attention and identification of rare stimuli, in musicians, verified through the performance of Long Latency Auditory Evoked Potential, P300, with and without competitive noise, in musicians compared to non-musician. Methods 20 individuals were divided into two groups: 8 in the musicians, and 12 in the control group. The P300 values were compared between the two groups and then between the results of the P300 with and without competitive noise, in both groups. Results When comparing the results without competitive noise, it appears that the average amplitude was higher in the group of musicians compared to the control group, in both ears. Latency was lower in the control group, only in the right ear. With competitive noise, in both groups, the average amplitude is lower, compared to the results of the P300 without competitive noise, both in the right ear and in the left ear, and this effect is more considerable in the group of musicians. Regarding latency, theaverage of the P300 with competitive noise, in both ears, with a greater increase in latency values, in the group of musicians. Conclusion Musicians show a greater cortical inhibition effect compared to non-musicians, demonstrating that the musician’s central auditory system shows greater activation, which can result in better performance in functions such as attention and discrimination, due to training by musical practice.


Sign in / Sign up

Export Citation Format

Share Document