Relations of Audio and Visual Speech Signals in a Physical Feature Space: Implications for the Hearing-impaired

1996 ◽  
pp. 445-460 ◽  
Author(s):  
Hans-Heinrich Bothe
Keyword(s):  
Feature Space ◽  
Hearing Impaired ◽  
Visual Speech ◽  
Physical Feature ◽  
Speech Signals

The Speech Spectrographic Display

1981 ◽  
Vol 46 (4) ◽  
pp. 379-387 ◽  
Author(s):  
Jean E. Maki ◽  
John M. Conklin ◽  
Marianne Streff Gustafson ◽  
Brenda K. Humphrey-Whitehead
Keyword(s):  
Speech Production ◽  
Hearing Impaired ◽  
Visual Speech ◽  
Visual Features ◽  
Visual Patterns ◽  
Training Aids ◽  
Interpretation Errors ◽  
Speech Training

If visual speech training aids are to be used effectively, it is important to assess whether hearing-impaired speakers can accurately interpret visual patterns and arrive at correct conclusions concerning the accuracy of speech production. In this investigation with the Speech Spectrographic Display (SSD), a pattern interpretation task was given to 10 hearing-impaired adults. Subjects viewed selected SSD patterns from hearing-impaired speakers, evaluated the accuracy of speech production, and identified the SSD visual features that were used in the evaluation. In general, results showed that subjects could use SSD patterns to evaluate speech production. For those pattern interpretation errors that occurred most were related either to phonetic/orthographic confusions or to misconceptions concerning production of speech.

scholarly journals Comparing heterogeneous visual gestures for measuring the diversity of visual speech signals

2018 ◽  
Vol 52 ◽  
pp. 165-190 ◽  
Author(s):  
Helen L. Bear ◽  
Richard Harvey
Keyword(s):  
Visual Speech ◽  
Speech Signals

scholarly journals Visual speech is processed differently in auditory and visual cortex: evidence from MEG and partial coherence analysis

2021 ◽  
Author(s):  
Mate Aller ◽  
Heidi Solberg Okland ◽  
Lucy J MacGregor ◽  
Helen Blank ◽  
Matthew H. Davis
Keyword(s):  
Visual Cortex ◽  
Speech Perception ◽  
Phase Locking ◽  
Visual Speech ◽  
Visual Signals ◽  
Partial Coherence ◽  
Speech Signals ◽  
Coherence Analysis ◽  
Modal Interactions ◽  
Auditory Signals

Speech perception in noisy environments is enhanced by seeing facial movements of communication partners. However, the neural mechanisms by which audio and visual speech are combined are not fully understood. We explore MEG phase locking to auditory and visual signals in MEG recordings from 14 human participants (6 female) that reported words from single spoken sentences. We manipulated the acoustic clarity and visual speech signals such that critical speech information is present in auditory, visual or both modalities. MEG coherence analysis revealed that both auditory and visual speech envelopes (auditory amplitude modulations and lip aperture changes) were phase-locked to 2-6Hz brain responses in auditory and visual cortex, consistent with entrainment to syllable-rate components. Partial coherence analysis was used to separate neural responses to correlated audio-visual signals and showed non-zero phase locking to auditory envelope in occipital cortex during audio-visual (AV) speech. Furthermore, phase-locking to auditory signals in visual cortex was enhanced for AV speech compared to audio-only (AO) speech that was matched for intelligibility. Conversely, auditory regions of the superior temporal gyrus (STG) did not show above-chance partial coherence with visual speech signals during AV conditions, but did show partial coherence in VO conditions. Hence, visual speech enabled stronger phase locking to auditory signals in visual areas, whereas phase-locking of visual speech in auditory regions only occurred during silent lip-reading. Differences in these cross-modal interactions between auditory and visual speech signals are interpreted in line with cross-modal predictive mechanisms during speech perception.

Sign in / Sign up

Export Citation Format

Share Document