Cortical tracking of voice pitch in the presence of multiple speakers depends on selective attention

Voice pitch carries linguistic as well as non-linguistic information. Previous studies have described cortical tracking of voice pitch in clean speech, with responses reflecting both pitch strength and pitch value. However, pitch is also a powerful cue for auditory stream segregation, especially when competing streams have pitch differing in fundamental frequency, as is the case when multiple speakers talk simultaneously. We therefore investigated how cortical speech pitch tracking is affected in the presence of a second, task-irrelevant speaker. We analyzed human magnetoencephalography (MEG) responses to continuous narrative speech, presented either as a single talker in a quiet background, or as a two-talker mixture of a male and a female speaker. In clean speech, voice pitch was associated with a right-dominant response, peaking at a latency of around 100 ms, consistent with previous EEG and ECoG results. The response tracked both the presence of pitch as well as the relative value of the speaker’s fundamental frequency. In the two-talker mixture, pitch of the attended speaker was tracked bilaterally, regardless of whether or not there was simultaneously present pitch in the speech of the irrelevant speaker. Pitch tracking for the irrelevant speaker was reduced: only the right hemisphere still significantly tracked pitch of the unattended speaker, and only during intervals in which no pitch was present in the attended talker’s speech. Taken together, these results suggest that pitch-based segregation of multiple speakers, at least as measured by macroscopic cortical tracking, is not entirely automatic but strongly dependent on selective attention.

Cognitive constraints on advance planning of sentence intonation

Pitch peaks tend to be higher at the beginning of longer than shorter sentences (e.g., ‘A farmer is pulling donkeys’ vs ‘A farmer is pulling a donkey and goat’), whereas pitch valleys at the ends of sentences are rather constant for a given speaker. These data seem to imply that speakers avoid dropping their voice pitch too low by planning the height of sentence-initial pitch peaks prior to speaking. However, the length effect on sentence-initial pitch peaks appears to vary across different types of sentences, speakers and languages. Therefore, the notion that speakers plan sentence intonation in advance due to the limitations in low voice pitch leaves part of the data unexplained. Consequently, this study suggests a complementary cognitive account of length-dependent pitch scaling. In particular, it proposes that the sentence-initial pitch raise in long sentences is related to high demands on mental resources during the early stages of sentence planning. To tap into the cognitive underpinnings of planning sentence intonation, this study adopts the methodology of recording eye movements during a picture description task, as the eye movements are the established approximation of the real-time planning processes. Measures of voice pitch (Fundamental Frequency) and incrementality (eye movements) are used to examine the relationship between (verbal) working memory (WM), incrementality of sentence planning and the height of sentence-initial pitch peaks.

Rethinking the frequency code: a meta-analytic review of the role of acoustic body size in communicative phenomena

The widely cited frequency code hypothesis attempts to explain a diverse range of communicative phenomena through the acoustic projection of body size. The set of phenomena includes size sound symbolism (using /i/ to signal smallness in words such as teeny ), intonational phonology (using rising contours to signal questions) and the indexing of social relations via vocal modulation, such as lowering one's voice pitch to signal dominance. Among other things, the frequency code is commonly interpreted to suggest that polite speech should be universally signalled via high pitch owing to the association of high pitch with small size and submissiveness. We present a cross-cultural meta-analysis of polite speech of 101 speakers from seven different languages. While we find evidence for cross-cultural variation, voice pitch is on average lower when speakers speak politely, contrary to what the frequency code predicts. We interpret our findings in the light of the fact that pitch has a multiplicity of possible communicative meanings. Cultural and contextual variation determines which specific meanings become manifest in a specific interactional context. We use the evidence from our meta-analysis to propose an updated view of the frequency code hypothesis that is based on the existence of many-to-many mappings between speech acoustics and communicative interpretations. This article is part of the theme issue ‘Voice modulation: from origin and mechanism to social impact (Part I)’.

Human voice pitch measures are robust across a variety of speech recordings: methodological and theoretical implications

Fundamental frequency ( f o ), perceived as voice pitch, is the most sexually dimorphic, perceptually salient and intensively studied voice parameter in human nonverbal communication. Thousands of studies have linked human f o to biological and social speaker traits and life outcomes, from reproductive to economic. Critically, researchers have used myriad speech stimuli to measure f o and infer its functional relevance, from individual vowels to longer bouts of spontaneous speech. Here, we acoustically analysed f o in nearly 1000 affectively neutral speech utterances (vowels, words, counting, greetings, read paragraphs and free spontaneous speech) produced by the same 154 men and women, aged 18–67, with two aims: first, to test the methodological validity of comparing f o measures from diverse speech stimuli, and second, to test the prediction that the vast inter-individual differences in habitual f o found between same-sex adults are preserved across speech types. Indeed, despite differences in linguistic content, duration, scripted or spontan­­eous production and within-individual variability, we show that 42–81% of inter-individual differences in f o can be explained between any two speech types. Beyond methodological implications, together with recent evidence that inter-individual differences in f o are remarkably stable across the lifespan and generalize to emotional speech and nonverbal vocalizations, our results further substantiate voice pitch as a robust and reliable biomarker in human communication.

Harsh is large: nonlinear vocal phenomena lower voice pitch and exaggerate body size

A lion's roar, a dog's bark, an angry yell in a pub brawl: what do these vocalizations have in common? They all sound harsh due to nonlinear vocal phenomena (NLP)—deviations from regular voice production, hypothesized to lower perceived voice pitch and thereby exaggerate the apparent body size of the vocalizer. To test this yet uncorroborated hypothesis, we synthesized human nonverbal vocalizations, such as roars, groans and screams, with and without NLP (amplitude modulation, subharmonics and chaos). We then measured their effects on nearly 700 listeners' perceptions of three psychoacoustic (pitch, timbre, roughness) and three ecological (body size, formidability, aggression) characteristics. In an explicit rating task, all NLP lowered perceived voice pitch, increased voice darkness and roughness, and caused vocalizers to sound larger, more formidable and more aggressive. Key results were replicated in an implicit associations test, suggesting that the ‘harsh is large’ bias will arise in ecologically relevant confrontational contexts that involve a rapid, and largely implicit, evaluation of the opponent's size. In sum, nonlinearities in human vocalizations can flexibly communicate both formidability and intention to attack, suggesting they are not a mere byproduct of loud vocalizing, but rather an informative acoustic signal well suited for intimidating potential opponents.