scholarly journals Integrating prosody in anticipatory language processing: how listeners adapt to unconventional prosodic cues

Author(s):  
Chie Nakamura ◽  
Jesse A. Harris ◽  
Sun-Ah Jun
PLoS ONE ◽  
2021 ◽  
Vol 16 (5) ◽  
pp. e0250969
Author(s):  
Tirza Biron ◽  
Daniel Baum ◽  
Dominik Freche ◽  
Nadav Matalon ◽  
Netanel Ehrmann ◽  
...  

Automatic speech recognition (ASR) and natural language processing (NLP) are expected to benefit from an effective, simple, and reliable method to automatically parse conversational speech. The ability to parse conversational speech depends crucially on the ability to identify boundaries between prosodic phrases. This is done naturally by the human ear, yet has proved surprisingly difficult to achieve reliably and simply in an automatic manner. Efforts to date have focused on detecting phrase boundaries using a variety of linguistic and acoustic cues. We propose a method which does not require model training and utilizes two prosodic cues that are based on ASR output. Boundaries are identified using discontinuities in speech rate (pre-boundary lengthening and phrase-initial acceleration) and silent pauses. The resulting phrases preserve syntactic validity, exhibit pitch reset, and compare well with manual tagging of prosodic boundaries. Collectively, our findings support the notion of prosodic phrases that represent coherent patterns across textual and acoustic parameters.


2021 ◽  
Author(s):  
◽  
Mengzhu Yan

<p>It is well established that focus plays an important role in facilitating language processing, i.e., focused words are recognised faster and remembered better. In addition, more recent research shows that alternatives to a word (e.g., sailor as an alternative to captain) are more activated when listeners hear the word with contrastive prominence (e.g., ‘The captain put on the raincoat) (bold indicates contrastive prominence). The mechanism behind these processing advantages is focus. Focus has two broad conceptions in relation to its effect on language processing: focus as updating the common ground and focus as indicating alternatives. Considerable psycholinguistic evidence has been obtained for processing advantages consistent with the first conception, and this evidence comes from studies across a reasonably wide range of languages. But the evidence for the second conception only comes from a handful of closely related languages (i.e., English, Dutch and German). Further, it has largely been confined to contrastive accenting as a marker of focus. Therefore, it is not clear if other types of focus marking (e.g., clefts) have similar processing effects. It is also not known if all this is true in Mandarin, as there is very little research in these areas in Mandarin. Mandarin uses pitch expansion to mark contrastive prominence, rather than the pitch accenting found in Germanic languages. Therefore, the investigation of Mandarin expands our knowledge of these speech processing effects to a different language and language family. It also expands our knowledge of the relative roles of prosody and syntax in marking focus and in speech processing in Mandarin, and in general.  This thesis tested how different types of focus marking affect the perception of focus and two aspects of language processing related to focus: the encoding and activation of discourse information (focused words and focus alternatives). The aim was to see whether there is a link between the relative importance of prosodic and syntactic focus marking in Mandarin and their effectiveness in these aspects of language processing. For focus perception, contrastive prominence and clefting have been claimed to mark focus in Mandarin, but it has not been well tested whether listeners perceive them as focus marking. For the first aspect of processing, it is not yet clear what cues listeners use to encode focused information beyond prominence when processing a discourse. For the second aspect, there has been rapidly growing interest in the role of alternatives in language processing, but little is known regarding the effect of clefting. In addition, it is not clear whether the prosodic and syntactic cues are equally effective, and again little research has been devoted to Mandarin. Therefore, the following experiments were conducted to look at these cues in Mandarin.  Experiment 1, a norming study, was conducted to help select stimuli for the following Experiments 2, 3, 4A and 4B. Experiment 2 investigated the relative weights of prosodic and syntactic focus cues in a question-answer appropriateness rating task. The findings show that in canonical word order sentences, the focus was perceived to be on the word that was marked by contrastive prominence. In clefts where the prominence and syntactic cues were on the same word, that word was perceived as being in focus. However, in ‘mismatch’ cases, e.g., 是[船长]F 穿上的[雨衣]F ‘It was the [captain]F who put on the [raincoat]F’ (F indicates focus), the focus was perceived to be on raincoat, the word that had contrastive prominence. In other words, participants weighted prosodic cues more highly. This suggests that prosodic prominence is a stronger focus cue than syntax in Mandarin.  Experiment 3 looked at the role of prosodic and syntactic cues in listeners’ encoding of discourse information in a speeded ‘false alternative’ rejection task. This experiment shows that false alternatives to a word in a sentence (e.g., sailor to captain in ‘The captain put on the raincoat’) were more easily rejected if captain was marked with prosodic cues than with syntactic cues. This experiment shows congruent results to those of Experiment 2, in that prosodic cues were more effective than syntactic cues in encoding discourse information. It seems that a more important marker of focus provides more effective encoding of discourse information.  Experiments 4A and 4B investigated the role of prosodic and syntactic focus cues in the activation of discourse information in Mandarin, using the cross-modal lexical priming paradigm. Both studies consistently show that prosodic focus marking, but not syntactic focus marking, facilitates the activation of identical targets (e.g., captain after hearing ‘The captain put on the raincoat’). Similarly, prosodic focus marking, but not syntactic focus marking, primes alternatives (e.g., sailor). But focus marking does not prime noncontrastive associates (e.g., deck). These findings, together with previous findings on focus particles (e.g., only), suggest that alternative priming is particularly related to contrastive prominence, at least in languages looked at to date. The relative priming effects of prosodic and syntactic focus cues in Experiments 4A and 4B are in line with their relative weights in Experiments 2 and 3.   This thesis presents a crucial link between the relative weights of prosodic and syntactic cues in marking focus, their degrees of effectiveness in encoding discourse information and their ability to activate discourse information in Mandarin. This research contributes significantly to our cross-linguistic understanding of prosodic and syntactic focus in speech processing, showing the processing advantages of focus may be common across languages, but what cues trigger the effects differ by language.</p>


2019 ◽  
Vol 36 (4) ◽  
pp. 503-527 ◽  
Author(s):  
Aleuna Lee ◽  
Michelle Perdomo ◽  
Edith Kaan

Prosody signals important aspects of meaning, and hence, is crucial for language comprehension and learning, yet remains under-investigated in second-language (L2) processing. The present electrophysiology study investigates the use of prosody to cue information structure, in particular, the use of contrastive pitch accent (L+H*) to define the set of elements that are contrasted. For instance, in We ate Angela’s cake, but saved BENjamin’s cake, the pitch accent on Benjamin’s is a cue that two cakes are contrasted; BENjamin’s ice cream is not plausible in this context. Native English speakers showed a large negativity on the target noun ( cake) when the preceding possessive was inappropriately accented. Event-related brain potential (ERP) results from Mandarin-Chinese L2 learners of English suggest they did not use contrastive pitch accent to cue the contrast set in the way native English speakers did, even though Mandarin is similar to English in the use of prosodic cues to express contrast. Our results are in line with previous studies suggesting that L2 speakers have difficulty integrating information across domains and building information structure, especially in demanding task situations like in the present study.


2021 ◽  
Author(s):  
◽  
Mengzhu Yan

<p>It is well established that focus plays an important role in facilitating language processing, i.e., focused words are recognised faster and remembered better. In addition, more recent research shows that alternatives to a word (e.g., sailor as an alternative to captain) are more activated when listeners hear the word with contrastive prominence (e.g., ‘The captain put on the raincoat) (bold indicates contrastive prominence). The mechanism behind these processing advantages is focus. Focus has two broad conceptions in relation to its effect on language processing: focus as updating the common ground and focus as indicating alternatives. Considerable psycholinguistic evidence has been obtained for processing advantages consistent with the first conception, and this evidence comes from studies across a reasonably wide range of languages. But the evidence for the second conception only comes from a handful of closely related languages (i.e., English, Dutch and German). Further, it has largely been confined to contrastive accenting as a marker of focus. Therefore, it is not clear if other types of focus marking (e.g., clefts) have similar processing effects. It is also not known if all this is true in Mandarin, as there is very little research in these areas in Mandarin. Mandarin uses pitch expansion to mark contrastive prominence, rather than the pitch accenting found in Germanic languages. Therefore, the investigation of Mandarin expands our knowledge of these speech processing effects to a different language and language family. It also expands our knowledge of the relative roles of prosody and syntax in marking focus and in speech processing in Mandarin, and in general.  This thesis tested how different types of focus marking affect the perception of focus and two aspects of language processing related to focus: the encoding and activation of discourse information (focused words and focus alternatives). The aim was to see whether there is a link between the relative importance of prosodic and syntactic focus marking in Mandarin and their effectiveness in these aspects of language processing. For focus perception, contrastive prominence and clefting have been claimed to mark focus in Mandarin, but it has not been well tested whether listeners perceive them as focus marking. For the first aspect of processing, it is not yet clear what cues listeners use to encode focused information beyond prominence when processing a discourse. For the second aspect, there has been rapidly growing interest in the role of alternatives in language processing, but little is known regarding the effect of clefting. In addition, it is not clear whether the prosodic and syntactic cues are equally effective, and again little research has been devoted to Mandarin. Therefore, the following experiments were conducted to look at these cues in Mandarin.  Experiment 1, a norming study, was conducted to help select stimuli for the following Experiments 2, 3, 4A and 4B. Experiment 2 investigated the relative weights of prosodic and syntactic focus cues in a question-answer appropriateness rating task. The findings show that in canonical word order sentences, the focus was perceived to be on the word that was marked by contrastive prominence. In clefts where the prominence and syntactic cues were on the same word, that word was perceived as being in focus. However, in ‘mismatch’ cases, e.g., 是[船长]F 穿上的[雨衣]F ‘It was the [captain]F who put on the [raincoat]F’ (F indicates focus), the focus was perceived to be on raincoat, the word that had contrastive prominence. In other words, participants weighted prosodic cues more highly. This suggests that prosodic prominence is a stronger focus cue than syntax in Mandarin.  Experiment 3 looked at the role of prosodic and syntactic cues in listeners’ encoding of discourse information in a speeded ‘false alternative’ rejection task. This experiment shows that false alternatives to a word in a sentence (e.g., sailor to captain in ‘The captain put on the raincoat’) were more easily rejected if captain was marked with prosodic cues than with syntactic cues. This experiment shows congruent results to those of Experiment 2, in that prosodic cues were more effective than syntactic cues in encoding discourse information. It seems that a more important marker of focus provides more effective encoding of discourse information.  Experiments 4A and 4B investigated the role of prosodic and syntactic focus cues in the activation of discourse information in Mandarin, using the cross-modal lexical priming paradigm. Both studies consistently show that prosodic focus marking, but not syntactic focus marking, facilitates the activation of identical targets (e.g., captain after hearing ‘The captain put on the raincoat’). Similarly, prosodic focus marking, but not syntactic focus marking, primes alternatives (e.g., sailor). But focus marking does not prime noncontrastive associates (e.g., deck). These findings, together with previous findings on focus particles (e.g., only), suggest that alternative priming is particularly related to contrastive prominence, at least in languages looked at to date. The relative priming effects of prosodic and syntactic focus cues in Experiments 4A and 4B are in line with their relative weights in Experiments 2 and 3.   This thesis presents a crucial link between the relative weights of prosodic and syntactic cues in marking focus, their degrees of effectiveness in encoding discourse information and their ability to activate discourse information in Mandarin. This research contributes significantly to our cross-linguistic understanding of prosodic and syntactic focus in speech processing, showing the processing advantages of focus may be common across languages, but what cues trigger the effects differ by language.</p>


Author(s):  
Martin Meyer ◽  
Matthias Keller ◽  
Nathalie Giroud

Prosody is a unique feature of spoken language and can be conceived as an umbrella term that covers several modulations of acoustic speech signals which mark paralinguistic information—namely stress, sentence accent, sentence mode, and phrasing—which are highly correlated with the semantic, syntactic, phonological organization and emotional tone of spoken and signed languages all over the world. Neuropsychological and neuroimaging research has grossly attributed prosodic functions to the right hemisphere (RH) of the human brain. The RH has been historically considered the site of emotions and this view is still the foundation of the widely held belief that prosody is primarily emotional in origin and nature. This chapter means to replace this ancient view by an innovative, parameter-based approach that considers prosody as an acoustic modulator that primarily serves as a cardinal structuring device to chunk suprasegmental units in spoken language. The chapter offers evidence from recent research that shows how the human brain exploits prosodic cues in spoken language during language acquisition, processing of foreign languages, and language processing under aversive listening conditions (e.g. age-related hearing loss). More importantly, it sketches how the human brain processes suprasegmental information during prelexical stages of language comprehension. In support of this view lies in the stunning fit between functional lateralization of computing prosodic cues and neuroanatomical asymmetry of auditory-related cortex (PARC). Finally, the chapter describes the essential role that prosodic modulations play in spoken language during early infancy and older age.


2016 ◽  
Vol 39 ◽  
Author(s):  
Giosuè Baggio ◽  
Carmelo M. Vicario

AbstractWe agree with Christiansen & Chater (C&C) that language processing and acquisition are tightly constrained by the limits of sensory and memory systems. However, the human brain supports a range of cognitive functions that mitigate the effects of information processing bottlenecks. The language system is partly organised around these moderating factors, not just around restrictions on storage and computation.


Author(s):  
Jennifer M. Roche ◽  
Arkady Zgonnikov ◽  
Laura M. Morett

Purpose The purpose of the current study was to evaluate the social and cognitive underpinnings of miscommunication during an interactive listening task. Method An eye and computer mouse–tracking visual-world paradigm was used to investigate how a listener's cognitive effort (local and global) and decision-making processes were affected by a speaker's use of ambiguity that led to a miscommunication. Results Experiments 1 and 2 found that an environmental cue that made a miscommunication more or less salient impacted listener language processing effort (eye-tracking). Experiment 2 also indicated that listeners may develop different processing heuristics dependent upon the speaker's use of ambiguity that led to a miscommunication, exerting a significant impact on cognition and decision making. We also found that perspective-taking effort and decision-making complexity metrics (computer mouse tracking) predict language processing effort, indicating that instances of miscommunication produced cognitive consequences of indecision, thinking, and cognitive pull. Conclusion Together, these results indicate that listeners behave both reciprocally and adaptively when miscommunications occur, but the way they respond is largely dependent upon the type of ambiguity and how often it is produced by the speaker.


Sign in / Sign up

Export Citation Format

Share Document