Intermodal timing relations and audio‐visual speech recognition by normal‐hearing adults

1985 ◽  
Vol 77 (2) ◽  
pp. 678-685 ◽  
Author(s):  
Matthew McGrath ◽  
Quentin Summerfield
Keyword(s):  
Speech Recognition ◽  
Normal Hearing ◽  
Visual Speech ◽  
Visual Speech Recognition

scholarly journals Multisensory Integration-Attention Trade-Off in Cochlear-Implanted Deaf Individuals

2021 ◽  
Vol 15 ◽  
Author(s):  
Luuk P. H. van de Rijt ◽  
A. John van Opstal ◽  
Marc M. van Wanrooij
Keyword(s):  
Speech Recognition ◽  
Visual Cues ◽  
Normal Hearing ◽  
Situational Factors ◽  
Visual Speech ◽  
Noisy Environments ◽  
Trade Off ◽  
Visual Speech Recognition ◽  
Attention Tasks ◽  
Audiovisual Speech Recognition

The cochlear implant (CI) allows profoundly deaf individuals to partially recover hearing. Still, due to the coarse acoustic information provided by the implant, CI users have considerable difficulties in recognizing speech, especially in noisy environments. CI users therefore rely heavily on visual cues to augment speech recognition, more so than normal-hearing individuals. However, it is unknown how attention to one (focused) or both (divided) modalities plays a role in multisensory speech recognition. Here we show that unisensory speech listening and reading were negatively impacted in divided-attention tasks for CI users—but not for normal-hearing individuals. Our psychophysical experiments revealed that, as expected, listening thresholds were consistently better for the normal-hearing, while lipreading thresholds were largely similar for the two groups. Moreover, audiovisual speech recognition for normal-hearing individuals could be described well by probabilistic summation of auditory and visual speech recognition, while CI users were better integrators than expected from statistical facilitation alone. Our results suggest that this benefit in integration comes at a cost. Unisensory speech recognition is degraded for CI users when attention needs to be divided across modalities. We conjecture that CI users exhibit an integration-attention trade-off. They focus solely on a single modality during focused-attention tasks, but need to divide their limited attentional resources in situations with uncertainty about the upcoming stimulus modality. We argue that in order to determine the benefit of a CI for speech recognition, situational factors need to be discounted by presenting speech in realistic or complex audiovisual environments.

Asynchrony modeling for audio-visual speech recognition

2002 ◽  
Author(s):  
Guillaume Gravier ◽  
Gerasimos Potamianos ◽  
Chalapathy Neti
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Visual Speech Recognition

scholarly journals Comparison between different feature extraction techniques for audio-visual speech recognition

2007 ◽  
Vol 1 (1) ◽  
pp. 7-20 ◽  
Author(s):  
Alin G. Chiţu ◽  
Leon J. M. Rothkrantz ◽  
Pascal Wiggers ◽  
Jacek C. Wojdel
Keyword(s):  
Feature Extraction ◽  
Speech Recognition ◽  
Visual Speech ◽  
Extraction Techniques ◽  
Visual Speech Recognition

European and American Audio-Visual Speech Recognition, Using SVM in Portuguese Language

2008 ◽  
Author(s):  
Adriano de Andrade Bresolin ◽  
Diamantino Rui da Silva da Silva Freitas ◽  
Adriao Duarte Doria Neto ◽  
Pablo Javier Alsina
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Visual Speech Recognition
Sign in / Sign up

Export Citation Format

Share Document