Multimodal recognition of interrupted speech: Benefit from text and visual speech cues

2018 ◽  
Vol 144 (3) ◽  
pp. 1800-1800
Author(s):  
Rachel E. Miller ◽  
Courtney Strickland ◽  
Daniel Fogerty
Keyword(s):  
Visual Speech ◽  
Interrupted Speech ◽  
Speech Cues

Auditory and Auditory-Visual Perception of Clear and Conversational Speech

1997 ◽  
Vol 40 (2) ◽  
pp. 432-443 ◽  
Author(s):  
Karen S. Helfer
Keyword(s):  
Visual Cues ◽  
Presentation Mode ◽  
Visual Presentation ◽  
Visual Speech ◽  
Conversational Speech ◽  
Clear Speech ◽  
Nature Of Information ◽  
Speech Cues ◽  
Visual Speech Information ◽  
Speech Information

Research has shown that speaking in a deliberately clear manner can improve the accuracy of auditory speech recognition. Allowing listeners access to visual speech cues also enhances speech understanding. Whether the nature of information provided by speaking clearly and by using visual speech cues is redundant has not been determined. This study examined how speaking mode (clear vs. conversational) and presentation mode (auditory vs. auditory-visual) influenced the perception of words within nonsense sentences. In Experiment 1, 30 young listeners with normal hearing responded to videotaped stimuli presented audiovisually in the presence of background noise at one of three signal-to-noise ratios. In Experiment 2, 9 participants returned for an additional assessment using auditory-only presentation. Results of these experiments showed significant effects of speaking mode (clear speech was easier to understand than was conversational speech) and presentation mode (auditoryvisual presentation led to better performance than did auditory-only presentation). The benefit of clear speech was greater for words occurring in the middle of sentences than for words at either the beginning or end of sentences for both auditory-only and auditory-visual presentation, whereas the greatest benefit from supplying visual cues was for words at the end of sentences spoken both clearly and conversationally. The total benefit from speaking clearly and supplying visual cues was equal to the sum of each of these effects. Overall, the results suggest that speaking clearly and providing visual speech information provide complementary (rather than redundant) information.

Spectral Distribution of Prosodic Information

1996 ◽  
Vol 39 (2) ◽  
pp. 228-238 ◽  
Author(s):  
Ken W. Grant ◽  
Brian E. Walden
Keyword(s):  
High Frequency ◽  
Low Frequency ◽  
Visual Speech ◽  
Prosodic Features ◽  
Temporal Properties ◽  
Prosodic Cues ◽  
Visual Speech Recognition ◽  
Speech Cues ◽  
Boundary Location ◽  
Speech Spectrum

Prosodic speech cues for rhythm, stress, and intonation are related primarily to variations in intensity, duration, and fundamental frequency. Because these cues make use of temporal properties of the speech waveform they are likely to be represented broadly across the speech spectrum. In order to determine the relative importance of different frequency regions for the recognition of Prosodic cues, identification of four Prosodic features, syllable number, syllabic stress, sentence intonation, and phrase boundary location, was evaluated under six filter conditions spanning the range from 200–6100 Hz. Each filter condition had equal articulation index (Al) weights, Al ½ 0.10; p(C) isolated words ≈ 0.40. Results obtained with normally hearing subjects showed that there was an interaction between filter condition and the identification of specific Prosodic features. For example, information from high-frequency regions of speech was particularly useful in the identification of syllable number and stress, whereas information from low-frequency regions was helpful in identifying intonation patterns. In spite of these spectral differences, overall listeners performed remarkably well in identifying Prosodic patterns, although individual differences were apparent. For some subjects, equivalent levels of performance across the six filter conditions were achieved. These results are discussed in relation to auditory and auditory-visual speech recognition.

The role of visual speech cues in reducing energetic and informational masking

2005 ◽  
Vol 117 (2) ◽  
pp. 842-849 ◽  
Author(s):  
Karen S. Helfer ◽  
Richard L. Freyman
Keyword(s):  
Visual Speech ◽  
Informational Masking ◽  
Speech Cues

Speech identification in noise: Contribution of temporal, spectral, and visual speech cues

2009 ◽  
Vol 126 (6) ◽  
pp. 3246-3257 ◽  
Author(s):  
Jeesun Kim ◽  
Chris Davis ◽  
Christopher Groot
Keyword(s):  
Visual Speech ◽  
Speech Cues ◽  
Speech Identification ◽  
Noise Contribution

Short and sweet, or long and complex? Perceiving temporal synchrony in audiovisual events

2012 ◽  
Vol 25 (0) ◽  
pp. 96
Author(s):  
Ragnhild Eg ◽  
Dawn Behne
Keyword(s):  
Visual Speech ◽  
Temporal Synchrony ◽  
Synchrony Perception ◽  
Speech Cues ◽  
Complex Events ◽  
Lag Times ◽  
Natural Dynamics

Perceived synchrony varies depending on the audiovisual event. Typically, asynchrony is tolerated at greater lead- and lag-times for speech and music than for action events. The tolerance for asynchrony in speech has been attributed to the unity assumption, which proposes a bonding of auditory and visual speech cues through associations in several dimensions. However, the variations in synchrony perception for different audiovisual events may simply be related to their complexity; where speech and music fluctuate naturally, actions involve isolated events and anticipated moments of impact. The current study measured perception of synchrony for long (13 s) and short (1 s) variants of three types of stimuli: (1) action, represented by a game of chess, (2) music, played by a drummer and (3) speech, presented by an anchorwoman in a newscast. The long variants allowed events to play out with their natural dynamics, whereas short variants offered controlled and predictable single actions or events, selected from the longer segments. Results show that among the long stimuli, lead asynchrony was detected sooner for speech than for chess. This contrasts both with previous research and our own predictions, although it may be related to characteristics of the selected chess scene. Interestingly, tolerance to asynchrony was generally greater for short, than for long, stimuli, especially for speech. These findings suggest that the dynamics of complex events cannot account for previously observed differences in synchrony perception between speech and action events.

Sign in / Sign up

Export Citation Format

Share Document