Prosodic Cues in the Interpretation of Echo Questions in Chinese Spoken Dialogues

This paper presents the results of three perceptual experiments investigating the role of auditory and visual channels for the identification of statements and echo questions in Brazilian Portuguese. Ten Brazilian speakers (five male) were video-recorded (frontal view of the face) while they produced a sentence (“ Como você sabe”), either as a statement (meaning “ As you know.”) or as an echo question (meaning “ As you know?”). Experiments were set up including the two different intonation contours. Stimuli were presented in conditions with clear and degraded audio as well as congruent and incongruent information from both channels. Results show that Brazilian listeners were able to distinguish statements and questions prosodically and visually, with auditory cues being dominant over visual ones. In noisy conditions, the visual channel improved the interpretation of prosodic cues robustly, while it degraded them in conditions where the visual information was incongruent with the auditory information. This study shows that auditory and visual information are integrated during speech perception, also when applied to prosodic patterns.

Download Full-text

Investigating the Multimodal Nature of Human Communication

Journal of Psychophysiology ◽

10.1027/0269-8803.23.2.63 ◽

2009 ◽

Vol 23 (2) ◽

pp. 63-76 ◽

Cited By ~ 41

Author(s):

Silke Paulmann ◽

Sarah Jessen ◽

Sonja A. Kotz

Keyword(s):

Visual Information ◽

Empirical Studies ◽

Physical Property ◽

Event Related Potentials ◽

Accurate Information ◽

Human Communication ◽

Channel Condition ◽

Prosodic Cues ◽

Early Processing ◽

Related Potentials

The multimodal nature of human communication has been well established. Yet few empirical studies have systematically examined the widely held belief that this form of perception is facilitated in comparison to unimodal or bimodal perception. In the current experiment we first explored the processing of unimodally presented facial expressions. Furthermore, auditory (prosodic and/or lexical-semantic) information was presented together with the visual information to investigate the processing of bimodal (facial and prosodic cues) and multimodal (facial, lexic, and prosodic cues) human communication. Participants engaged in an identity identification task, while event-related potentials (ERPs) were being recorded to examine early processing mechanisms as reflected in the P200 and N300 component. While the former component has repeatedly been linked to physical property stimulus processing, the latter has been linked to more evaluative “meaning-related” processing. A direct relationship between P200 and N300 amplitude and the number of information channels present was found. The multimodal-channel condition elicited the smallest amplitude in the P200 and N300 components, followed by an increased amplitude in each component for the bimodal-channel condition. The largest amplitude was observed for the unimodal condition. These data suggest that multimodal information induces clear facilitation in comparison to unimodal or bimodal information. The advantage of multimodal perception as reflected in the P200 and N300 components may thus reflect one of the mechanisms allowing for fast and accurate information processing in human communication.

Download Full-text

Vocalic, Lexical and Prosodic Cues for the INTERSPEECH 2018 Self-Assessed Affect Challenge

10.21437/interspeech.2018-1331 ◽

2018 ◽

Cited By ~ 3

Author(s):

Claude Montacié ◽

Marie-José Caraty

Keyword(s):

Prosodic Cues

Download Full-text

A Study of Lexical and Prosodic Cues to Segmentation in a Hindi-English Code-switched Discourse

10.21437/interspeech.2018-1600 ◽

2018 ◽

Author(s):

Preeti Rao ◽

Mugdha Pandya ◽

Kamini Sabu ◽

Kanhaiya Kumar ◽

Nandini Bondale

Keyword(s):

Prosodic Cues

Download Full-text

Relations between prominence and articulatory-prosodic cues in emotional speech

10.21437/speechprosody.2016-183 ◽

2016 ◽

Cited By ~ 1

Author(s):

Jangwon Kim ◽

Anil Ramakrishna ◽

Sungbok Lee ◽

Shrikanth Narayanan

Keyword(s):

Emotional Speech ◽

Prosodic Cues

Download Full-text

Recognising regional varieties of Danish

Linguistics Vanguard ◽

10.1515/lingvan-2019-0020 ◽

2020 ◽

Vol 6 (s1) ◽

Author(s):

John Tøndering ◽

Nicolai Pharao

Keyword(s):

Online Survey ◽

Stress Group ◽

Reliable Identification ◽

Prosodic Cues ◽

Regional Background ◽

Group Pattern

AbstractRegional varieties of Danish are distinguished mainly by prosodic cues, in particular the tonal stress group pattern. We test the hypothesis that prosody is a strong cue for identifying the regional background of speakers. This hypothesis was studied using an online survey in which listeners were asked to identify the origin of speakers based on examples from four different regional varieties of Danish. Listeners identified speaker origin on the basis of both natural, monotonised and delexicalised speech samples. The results show that although most listeners are able to identify the four regional varieties correctly, it is more reasonable to say that Danes categorise the regional varieties into broader categories of Western and Eastern Danish. Prosody clearly plays an important role in the identification of the varieties, but reliable identification is difficult based on prosody alone.

Download Full-text

USING PROSODY TO PREDICT UPCOMING REFERENTS IN THE L1 AND THE L2

Studies in Second Language Acquisition ◽

10.1017/s0272263120000509 ◽

2020 ◽

pp. 1-28

Author(s):

Anouschka Foltz

Keyword(s):

Language Learners ◽

Second Language Learners ◽

Exposure Period ◽

Predictive Processing ◽

Bilingual Speakers ◽

Prosodic Cues ◽

Resource Deficit ◽

L1 And L2 ◽

Pitch Accents

Abstract While monolingual speakers can use contrastive pitch accents to predict upcoming referents, bilingual speakers do not always use this cue predictively in their L2. The current study examines the role of recent exposure for predictive processing in native German (L1) second language learners of English (L2). In Experiment 1, participants followed instructions to click on two successive objects, for example, Click on the red carrot/duck. Click on the green/GREEN carrot (where CAPS indicate a contrastive L + H* accent). Participants predicted a repeated noun following a L + H* accent in the L1, but not in the L2, where processing was delayed. Experiment 2 shows that after an exposure period with highly consistent prosodic cues, bilinguals engaged in predictive processing in both their L1 and L2. However, inconsistent prosodic cues showed different effects on bilinguals’ L1 and L2 predictive processing. The results are discussed in terms of exposure-based and resource-deficit models of processing.

Download Full-text

Zebra finches are sensitive to prosodic features of human speech

Proceedings of The Royal Society B Biological Sciences ◽

10.1098/rspb.2014.0480 ◽

2014 ◽

Vol 281 (1787) ◽

pp. 20140480 ◽

Cited By ~ 24

Author(s):

Michelle J. Spierings ◽

Carel ten Cate

Keyword(s):

Speech Perception ◽

Emotional State ◽

The Other ◽

Zebra Finches ◽

Prosodic Features ◽

Human Speech ◽

Prosodic Cues ◽

Paralinguistic Information ◽

Final Syllable ◽

Human Specific

Variation in pitch, amplitude and rhythm adds crucial paralinguistic information to human speech. Such prosodic cues can reveal information about the meaning or emphasis of a sentence or the emotional state of the speaker. To examine the hypothesis that sensitivity to prosodic cues is language independent and not human specific, we tested prosody perception in a controlled experiment with zebra finches. Using a go/no-go procedure, subjects were trained to discriminate between speech syllables arranged in XYXY patterns with prosodic stress on the first syllable and XXYY patterns with prosodic stress on the final syllable. To systematically determine the salience of the various prosodic cues (pitch, duration and amplitude) to the zebra finches, they were subjected to five tests with different combinations of these cues. The zebra finches generalized the prosodic pattern to sequences that consisted of new syllables and used prosodic features over structural ones to discriminate between stimuli. This strong sensitivity to the prosodic pattern was maintained when only a single prosodic cue was available. The change in pitch was treated as more salient than changes in the other prosodic features. These results show that zebra finches are sensitive to the same prosodic cues known to affect human speech perception.

Download Full-text