Developmental Trajectories of Letter and Speech Sound Integration During Reading Acquisition

Reading acquisition in alphabetic languages starts with learning the associations between speech sounds and letters. This learning process is related to crucial developmental changes of brain regions that serve visual, auditory, multisensory integration, and higher cognitive processes. Here, we studied the development of audiovisual processing and integration of letter-speech sound pairs with an audiovisual target detection functional MRI paradigm. Using a longitudinal approach, we tested children with varying reading outcomes before the start of reading acquisition (T1, 6.5 yo), in first grade (T2, 7.5 yo), and in second grade (T3, 8.5 yo). Early audiovisual integration effects were characterized by higher activation for incongruent than congruent letter-speech sound pairs in the inferior frontal gyrus and ventral occipitotemporal cortex. Audiovisual processing in the left superior temporal gyrus significantly increased from the prereading (T1) to early reading stages (T2, T3). Region of interest analyses revealed that activation in left superior temporal gyrus (STG), inferior frontal gyrus and ventral occipitotemporal cortex increased in children with typical reading fluency skills, while poor readers did not show the same development in these regions. The incongruency effect bilaterally in parts of the STG and insular cortex at T1 was significantly associated with reading fluency skills at T3. These findings provide new insights into the development of the brain circuitry involved in audiovisual processing of letters, the building blocks of words, and reveal early markers of audiovisual integration that may be predictive of reading outcomes.

Intraoperative localization and preservation of reading in ventral occipitotemporal cortex

Resective surgery in language-dominant ventral occipitotemporal cortex (vOTC) carries the risk of causing impairment to reading. As it is not on the lateral surface, it is not easily accessible for intraoperative mapping and extensive stimulation mapping can be time consuming. Here we assess the feasibility of using task-based electrocorticography (ECoG) recordings intraoperatively to help guide stimulation mapping of reading in vOTC. In 11 patients undergoing extraoperative, intracranial seizure mapping we recorded induced broadband gamma activation (70 - 150 Hz) during a visual category localizer. Word-responsive cortex localized in this manner showed a high sensitivity (72%) to stimulation-induced reading deficits, and the confluence of ECoG and stimulation positive sites appears to demarcate the visual word form area. In two additional patients, with pathologies necessitating resections in language-dominant vOTC, task-based functional mapping was performed intraoperatively using subdural ECoG, alongside direct cortical stimulation. Cortical areas critical for reading were mapped and successfully preserved, while enabling pathological tissue to be completely removed. Data collection is possible in <3 minutes and initial intraoperative data analysis takes <3 minutes, allowing for rapid assessment of broad areas of cortex. Eloquent cortex in ventral visual cortex can be rapidly mapped intraoperatively using ECoG. This method acts to guide high-probability targets for stimulation, with limited patient participation, and can be used to avoid iatrogenic dyslexia following surgery.

Representation of motion concepts in occipitotemporal cortex: fMRI activation, decoding and connectivity analyses

Embodied theories of semantic cognition predict that brain regions involved in motion perception are engaged when people comprehend motion concepts expressed in language. Left lateral occipitotemporal cortex (LOTC) is implicated in both motion perception and motion concept processing but prior studies have produced mixed findings regarding which parts of this region are engaged by motion language. We scanned participants performing semantic judgements about sentences describing motion events and static events. We performed univariate analyses, multivariate pattern analyses (MVPA) and psychophysiological interaction (PPI) analyses to investigate the effect of motion on activity and connectivity in different parts of LOTC. In multivariate analyses that decoded whether a sentence described motion or not, the whole of LOTC showed above-chance level performance, with performance exceeding that of other brain regions. Univariate ROI analyses found that the middle part of LOTC was more active for motion events than static ones. Finally, PPI analyses found that when processing motion events, the middle and posterior parts of LOTC, overlapping with motion perception regions, increased their connectivity with cognitive control regions. Taken together, these results indicate that the whole of the LOTC responds differently to motion vs. static event descriptions, and that these effects are most pronounced in more posterior sites. These findings are consistent with embodiment accounts of semantic processing, and suggest that understanding verbal descriptions of motion engages areas of the occipitotemporal cortex involved in perceiving motion.

Action and object representation in the ventral "what" stream

The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition relies on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are restricted to medial areas of the ventral stream. We argue that the ventral stream bifurcates into lateral and medial pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and medial areas.

Predicting identity-preserving object transformations across the human ventral visual stream

In everyday life, we have no trouble recognizing and categorizing objects as they change in position, size, and orientation in our visual fields. This phenomenon is known as object invariance. Previous fMRI research suggests that higher-level object processing regions in the human lateral occipital cortex may link object responses from different affine states (i.e. size and viewpoint) through a general linear mapping function with the learned mapping capable of predicting responses of novel objects. In this study, we extended this approach to examine the mapping for both Euclidean (e.g. position and size) and non-Euclidean (e.g. image statistics and spatial frequency) transformations across the human ventral visual processing hierarchy, including areas V1, V2, V3, V4, ventral occipitotemporal cortex (VOT), and lateral occipitotemporal cortex (LOT). The predicted pattern generated from a linear mapping could capture a significant amount, but not all, of the variance of the true pattern across the ventral visual pathway. The derived linear mapping functions were not entirely category independent as performance was better for the categories included in the training. Moreover, prediction performance was not consistently better in higher than lower visual regions, nor were there notable differences between Euclidean and non-Euclidean transformations. Together, these findings demonstrate a near-orthogonal representation of object identity and non-identity features throughout the human ventral visual processing pathway, with the non-identity features largely untangled from the identity features early in the visual processing.Significance StatementPresently we still do not fully understand how object identity and non-identity (e.g. position, size) information are simultaneously represented in the primate ventral visual system to form invariant representations. Previous work suggests that the human lateral occipital cortex may be linking different affine states of object representations through general linear mapping functions. Here we show that across the entire human ventral processing pathway, we could link object responses in different states of non-identity transformations through linear mapping functions for both Euclidean and non-Euclidean transformations. These mapping functions are not identity-independent, suggesting that object identity and non-identity features are represented in a near, rather than a completely, orthogonal manner.