Information flow across the cortical timescales hierarchy during narrative comprehension

When listening to spoken narratives, we must integrate information over multiple, concurrent timescales, building up from words to phrases to sentences to a coherent narrative. Recent evidence suggests that the brain relies on a chain of hierarchically organized areas with increasing temporal receptive windows to process naturalistic narratives. In this study, we use inter-subject functional connectivity to reveal a stimulus-driven information flow along the cortical hierarchy. Using cross-correlation analysis to estimate the time lags between six functional networks, we found a fixed temporal sequence of information flow, starting in early auditory areas, followed language areas, the attention network, and lastly the default mode network. This gradient is consistent across eight distinct stories but absent in resting-state and scrambled story data, indicating that the lag gradient reflects the construction of narrative features. Finally, we simulate a variety of narrative integration models and demonstrate that nested narrative structure along with the gradual accumulation of information within the boundaries of linguistic events at each level of the processing hierarchy is sufficient to reproduce the lag gradient. Taken together, this study provides a computational framework for how information flows along the cortical hierarchy during narrative comprehension.

Speech Perception in Noise Predicts Oral Narrative Comprehension in Children With Developmental Language Disorder

We examined the relative contribution of auditory processing abilities (tone perception and speech perception in noise) after controlling for short-term memory capacity and vocabulary, to narrative language comprehension in children with developmental language disorder. Two hundred and sixteen children with developmental language disorder, ages 6 to 9 years (Mean = 7; 6), were administered multiple measures. The dependent variable was children's score on the narrative comprehension scale of the Test of Narrative Language. Predictors were auditory processing abilities, phonological short-term memory capacity, and language (vocabulary) factors, with age, speech perception in quiet, and non-verbal IQ as covariates. Results showed that narrative comprehension was positively correlated with the majority of the predictors. Regression analysis suggested that speech perception in noise contributed uniquely to narrative comprehension in children with developmental language disorder, over and above all other predictors; however, tone perception tasks failed to explain unique variance. The relative importance of speech perception in noise over tone-perception measures for language comprehension reinforces the need for the assessment and management of listening in noise deficits and makes a compelling case for the functional implications of complex listening situations for children with developmental language disorder.

The “Narratives” fMRI dataset for evaluating models of naturalistic language comprehension

The “Narratives” collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

Perceiving and remembering speech depend on multifractal nonlinearity in movements producing and exploring speech

Speech perception and memory for speech require active engagement. Gestural theories have emphasized mainly the effect of speaker's movements on speech perception. They fail to address the effects of listener movement, focusing on communication as a boundary condition constraining movement among interlocutors. The present work attempts to break new ground by using multifractal geometry of physical movement as a common currency for supporting both sides of the speaker–listener dyads. Participants self-paced their listening to a narrative, after which they completed a test of memory querying their narrative comprehension and their ability to recognize words from the story. The multifractal evidence of nonlinear interactions across timescales predicted the fluency of speech perception. Self-pacing movements that enabled listeners to control the presentation of speech sounds constituted a rich exploratory process. The multifractal nonlinearity of this exploration supported several aspects of memory for the perceived spoken language. These findings extend the role of multifractal geometry in the speaker's movements to the narrative case of speech perception. In addition to posing novel basic research questions, these findings make a compelling case for calibrating multifractal structure in text-to-speech synthesizers for better perception and memory of speech.

Direct and Indirect Pathways of Variation in Length of Exposure to the Majority Language, Cognitive and Language Skills in Preschoolers’ Listening Narrative Comprehension

Listening narrative comprehension, according to the theoretical framework of the multicomponent model for comprehension, involves numerous skills that interact dynamically between each other and have the potential to give rise to individual differences in comprehension. The purpose of the current work was to define a comprehensive and complete multicomponent model of listening narrative comprehension in preschool age. We investigated how variation in Length of Exposure to majority Language (i.e., how long children have been exposed to the Italian language), lower-order cognitive (WM, inhibitory control, attention shifting), language skills (receptive vocabulary, syntactic knowledge, rapid naming), and higher-order cognitive skills (inferences, TOM, knowledge of story-structure) are related to listening narrative comprehension in Italian of 111 preschool children (Mage = 61 months; SD = 6.8) growing in a monolingual or multilingual context. Structural equation modeling results showed that the model explained 60% variance in listening narrative comprehension in Italian of children aged four to six and predicted the outcome both through direct and mediated paths, coherently with the multicomponent model of comprehension.

Conceptual combination in the LATL with and without syntactic composition

The relationship between syntactic, semantic, and conceptual processes in language comprehension is a central question to the neurobiology of language. Several studies have suggested that conceptual combination in particular can be localized to the left anterior temporal lobe (LATL), while syntactic processes are more often associated with the posterior temporal lobe or inferior frontal gyrus. However, LATL activity can also correlate with syntactic computations, particularly in narrative comprehension. Here we investigated the degree to which LATL conceptual combination is dependent on syntax, specifically asking whether rapid (∼200 ms) magnetoencephalography effects of conceptual combination in the LATL can occur in the absence of licit syntactic phrase closure and in the absence of a semantically plausible output for the composition. We find that such effects do occur: LATL effects of conceptual combination were observed even when there was no syntactic phrase closure or plausible meaning. But syntactic closure did have an additive effect such that LATL signals were the highest for expressions that composed both conceptually and syntactically. Our findings conform to an account in which LATL conceptual composition is influenced by local syntactic composition but is also able to operate without it.

EXPRESS: Lending a Hand to Storytelling: Gesture’s Effects on Narrative Comprehension Moderated by Task Difficulty and Cognitive Ability

Compelling evidence suggests observing iconic gestures benefits learning. While emerging evidence suggests typical iconic gestures benefit comprehension to a greater extent than atypical iconic gestures, it is unclear precisely when and for whom these gestures will be most helpful. The current study investigated factors that may moderate when and for whom gesture benefits narrative comprehension most, including the type of gesture, task difficulty, and individual differences in cognitive ability. Participants were shown a video narrative in which they observed either typical gestures (commonly produced gestures, highly semantically related to accompanying speech), atypical gestures (gestures that are seldom produced), or no gestures. The video narrative was either viewed with interference (background noise to increase task difficulty) or no interference (no background noise). To determine whether the effects of gesture observation and externally imposed task difficulty on narrative comprehension further depend on an individual’s cognitive abilities, participants completed four measures of cognitive abilities (immediate and delayed non-verbal memory, attention, and intellectual ability). Observing typical gestures significantly benefited narrative comprehension compared to atypical and no gestures combined, which did not differ significantly. Participants with below average and average levels of delayed non-verbal memory benefited more from typical gestures than atypical or no gestures compared to those with an above average level of delayed non-verbal memory. However, this interaction was only significant when the task was difficult (i.e., with interference) but not when the task was simple (i.e., no interference). This finding suggests that the type of iconic gesture observed may impact gesture’s beneficial effect on narrative comprehension, and that such gestures may be more beneficial in difficult tasks, but only for certain individuals.