scholarly journals Sensory-modality independent activation of the brain network for language

2019 ◽  
Author(s):  
Sophie Arana ◽  
André Marquand ◽  
Annika Hultén ◽  
Peter Hagoort ◽  
Jan-Mathijs Schoffelen
Keyword(s):  
Language Processing ◽  
Sentence Processing ◽  
Human Subjects ◽  
Brain Activity ◽  
Activity Patterns ◽  
Sensory Modality ◽  
Brain Network ◽  
Large Network ◽  
Individual Subject ◽  
The Brain

AbstractThe meaning of a sentence can be understood, whether presented in written or spoken form. Therefore it is highly probable that brain processes supporting language comprehension are at least partly independent of sensory modality. To identify where and when in the brain language processing is independent of sensory modality, we directly compared neuromagnetic brain signals of 200 human subjects (102 males) either reading or listening to sentences. We used multiset canonical correlation analysis to align individual subject data in a way that boosts those aspects of the signal that are common to all, allowing us to capture word-by-word signal variations, consistent across subjects and at a fine temporal scale. Quantifying this consistency in activation across both reading and listening tasks revealed a mostly left hemispheric cortical network. Areas showing consistent activity patterns include not only areas previously implicated in higher-level language processing, such as left prefrontal, superior & middle temporal areas and anterior temporal lobe, but also parts of the control-network as well as subcentral and more posterior temporal-parietal areas. Activity in this supramodal sentence processing network starts in temporal areas and rapidly spreads to the other regions involved. The findings do not only indicate the involvement of a large network of brain areas in supramodal language processing, but also indicate that the linguistic information contained in the unfolding sentences modulates brain activity in a word-specific manner across subjects.

scholarly journals Characterization of deep neural network features by decodability from human brain activity

2018 ◽  
Author(s):  
Tomoyasu Horikawa ◽  
Shuntaro C. Aoki ◽  
Mitsuaki Tsukamoto ◽  
Yukiyasu Kamitani
Keyword(s):  
Human Brain ◽  
Deep Neural Networks ◽  
Human Subjects ◽  
Brain Activity ◽  
Activity Patterns ◽  
Human Vision ◽  
Highly Correlated ◽  
Feature Values ◽  
The Brain

AbstractAchievements of near human-level performances in object recognition by deep neural networks (DNNs) have triggered a flood of comparative studies between the brain and DNNs. Using a DNN as a proxy for hierarchical visual representations, our recent study found that human brain activity patterns measured by functional magnetic resonance imaging (fMRI) can be decoded (translated) into DNN feature values given the same inputs. However, not all DNN features are equally decoded, indicating a gap between the DNN and human vision. Here, we present a dataset derived through the DNN feature decoding analyses including fMRI signals of five human subjects during image viewing, decoded feature values of DNNs (AlexNet and VGG19), and decoding accuracies of individual DNN features with their rankings. The decoding accuracies of individual features were highly correlated between subjects, suggesting the systematic differences between the brain and DNNs. We hope the present dataset will contribute to reveal the gap between the brain and DNNs and provide an opportunity to make use of the decoded features for further applications.

scholarly journals Decoding Brain Activity Associated with Literal and Metaphoric Sentence Comprehension Using Distributional Semantic Models

2020 ◽  
Vol 8 ◽  
pp. 231-246
Author(s):  
Vesna G. Djokic ◽  
Jean Maillard ◽  
Luana Bulat ◽  
Ekaterina Shutova
Keyword(s):  
Language Processing ◽  
Sentence Comprehension ◽  
Brain Activity ◽  
Activity Patterns ◽  
Word Embeddings ◽  
Semantic Models ◽  
Distributional Semantic Models ◽  
Metaphor Processing ◽  
Meaning Representation ◽  
The Brain

Recent years have seen a growing interest within the natural language processing (NLP) community in evaluating the ability of semantic models to capture human meaning representation in the brain. Existing research has mainly focused on applying semantic models to decode brain activity patterns associated with the meaning of individual words, and, more recently, this approach has been extended to sentences and larger text fragments. Our work is the first to investigate metaphor processing in the brain in this context. We evaluate a range of semantic models (word embeddings, compositional, and visual models) in their ability to decode brain activity associated with reading of both literal and metaphoric sentences. Our results suggest that compositional models and word embeddings are able to capture differences in the processing of literal and metaphoric sentences, providing support for the idea that the literal meaning is not fully accessible during familiar metaphor comprehension.

scholarly journals Harmonic Brain Modes: A Unifying Framework for Linking Space and Time in Brain Dynamics

2017 ◽  
Vol 24 (3) ◽  
pp. 277-293 ◽  
Author(s):  
Selen Atasoy ◽  
Gustavo Deco ◽  
Morten L. Kringelbach ◽  
Joel Pearson
Keyword(s):  
Neural Activity ◽  
Brain Activity ◽  
Activity Patterns ◽  
Structural Connectivity ◽  
Building Blocks ◽  
Mental States ◽  
Brain Dynamics ◽  
Space And Time ◽  
Wide Range ◽  
The Brain

A fundamental characteristic of spontaneous brain activity is coherent oscillations covering a wide range of frequencies. Interestingly, these temporal oscillations are highly correlated among spatially distributed cortical areas forming structured correlation patterns known as the resting state networks, although the brain is never truly at “rest.” Here, we introduce the concept of harmonic brain modes—fundamental building blocks of complex spatiotemporal patterns of neural activity. We define these elementary harmonic brain modes as harmonic modes of structural connectivity; that is, connectome harmonics, yielding fully synchronous neural activity patterns with different frequency oscillations emerging on and constrained by the particular structure of the brain. Hence, this particular definition implicitly links the hitherto poorly understood dimensions of space and time in brain dynamics and its underlying anatomy. Further we show how harmonic brain modes can explain the relationship between neurophysiological, temporal, and network-level changes in the brain across different mental states ( wakefulness, sleep, anesthesia, psychedelic). Notably, when decoded as activation of connectome harmonics, spatial and temporal characteristics of neural activity naturally emerge from the interplay between excitation and inhibition and this critical relation fits the spatial, temporal, and neurophysiological changes associated with different mental states. Thus, the introduced framework of harmonic brain modes not only establishes a relation between the spatial structure of correlation patterns and temporal oscillations (linking space and time in brain dynamics), but also enables a new dimension of tools for understanding fundamental principles underlying brain dynamics in different states of consciousness.

scholarly journals Neurophysiological dynamics of phrase-structure building during sentence processing

2017 ◽  
Vol 114 (18) ◽  
pp. E3669-E3678 ◽  
Author(s):  
Matthew J. Nelson ◽  
Imen El Karoui ◽  
Kristof Giber ◽  
Xiaofang Yang ◽  
Laurent Cohen ◽  
...  
Keyword(s):  
Sentence Processing ◽  
Model Comparison ◽  
Formal Model ◽  
Syntactic Structure ◽  
Brain Activity ◽  
Transition Probability ◽  
Middle Temporal Gyrus ◽  
Similar Amount ◽  
Gamma Power ◽  
The Brain

Although sentences unfold sequentially, one word at a time, most linguistic theories propose that their underlying syntactic structure involves a tree of nested phrases rather than a linear sequence of words. Whether and how the brain builds such structures, however, remains largely unknown. Here, we used human intracranial recordings and visual word-by-word presentation of sentences and word lists to investigate how left-hemispheric brain activity varies during the formation of phrase structures. In a broad set of language-related areas, comprising multiple superior temporal and inferior frontal sites, high-gamma power increased with each successive word in a sentence but decreased suddenly whenever words could be merged into a phrase. Regression analyses showed that each additional word or multiword phrase contributed a similar amount of additional brain activity, providing evidence for a merge operation that applies equally to linguistic objects of arbitrary complexity. More superficial models of language, based solely on sequential transition probability over lexical and syntactic categories, only captured activity in the posterior middle temporal gyrus. Formal model comparison indicated that the model of multiword phrase construction provided a better fit than probability-based models at most sites in superior temporal and inferior frontal cortices. Activity in those regions was consistent with a neural implementation of a bottom-up or left-corner parser of the incoming language stream. Our results provide initial intracranial evidence for the neurophysiological reality of the merge operation postulated by linguists and suggest that the brain compresses syntactically well-formed sequences of words into a hierarchy of nested phrases.

scholarly journals Transient brain networks underlying interpersonal strategies during synchronized action

2020 ◽  
Author(s):  
Ole Adrian Heggli ◽  
Ivana Konvalinka ◽  
Joana Cabral ◽  
Elvira Brattico ◽  
Morten L Kringelbach ◽  
...  
Keyword(s):  
Computational Models ◽  
Brain Activity ◽  
Brain Regions ◽  
Brain Network ◽  
Human Interaction ◽  
Action Perception ◽  
Network Information ◽  
Mutual Adaptation ◽  
Underlying Mechanisms ◽  
The Brain

Abstract Interpersonal coordination is a core part of human interaction, and its underlying mechanisms have been extensively studied using social paradigms such as joint finger-tapping. Here, individual and dyadic differences have been found to yield a range of dyadic synchronization strategies, such as mutual adaptation, leading–leading, and leading–following behaviour, but the brain mechanisms that underlie these strategies remain poorly understood. To identify individual brain mechanisms underlying emergence of these minimal social interaction strategies, we contrasted EEG-recorded brain activity in two groups of musicians exhibiting the mutual adaptation and leading–leading strategies. We found that the individuals coordinating via mutual adaptation exhibited a more frequent occurrence of phase-locked activity within a transient action–perception-related brain network in the alpha range, as compared to the leading–leading group. Furthermore, we identified parietal and temporal brain regions that changed significantly in the directionality of their within-network information flow. Our results suggest that the stronger weight on extrinsic coupling observed in computational models of mutual adaptation as compared to leading–leading might be facilitated by a higher degree of action–perception network coupling in the brain.

scholarly journals Association Between Trait Empathy and Resting Brain Activity in Women With Primary Dysmenorrhea During the Pain and Pain-Free Phases

2020 ◽  
Vol 11 ◽  
Author(s):  
Wanghuan Dun ◽  
Tongtong Fan ◽  
Qiming Wang ◽  
Ke Wang ◽  
Jing Yang ◽  
...  
Keyword(s):  
Brain Activity ◽  
Inferior Frontal Gyrus ◽  
Brain Network ◽  
Primary Dysmenorrhea ◽  
Pain Experience ◽  
Current State ◽  
The Neural Network ◽  
The Right ◽  
The Brain

Empathy refers to the ability to understand someone else's emotions and fluctuates with the current state in healthy individuals. However, little is known about the neural network of empathy in clinical populations at different pain states. The current study aimed to examine the effects of long-term pain on empathy-related networks and whether empathy varied at different pain states by studying primary dysmenorrhea (PDM) patients. Multivariate partial least squares was employed in 46 PDM women and 46 healthy controls (HC) during periovulatory, luteal, and menstruation phases. We identified neural networks associated with different aspects of empathy in both groups. Part of the obtained empathy-related network in PDM exhibited a similar activity compared with HC, including the right anterior insula and other regions, whereas others have an opposite activity in PDM, including the inferior frontal gyrus and right inferior parietal lobule. These results indicated an abnormal regulation to empathy in PDM. Furthermore, there was no difference in empathy association patterns in PDM between the pain and pain-free states. This study suggested that long-term pain experience may lead to an abnormal function of the brain network for empathy processing that did not vary with the pain or pain-free state across the menstrual cycle.

An ERP Study in English Relative Clause Processing by Chinese-English Bilinguals

2015 ◽  
Vol 38 (1) ◽  
Author(s):  
Huili Wang ◽  
Liwen Ma ◽  
Youyou Wang ◽  
Melissa Troyer ◽  
Qiang Li
Keyword(s):  
Language Processing ◽  
Sentence Processing ◽  
Relative Clause ◽  
Relative Clauses ◽  
Native Speakers ◽  
Brain Activity ◽  
Working Memory Capacity ◽  
Large Body ◽  
Language Background ◽  
Study Support

AbstractThe processing of relative clauses receives much concern from linguists. The finding that object relatives are easier to process than subject relatives in Chinese challenges the notion that subject relative clauses are preferred universally. A large body of literature provides theories related to sentence processing mechanisms for native speakers but leaves one area relatively untouched: how bilinguals process sentences. This study is designed to examine the case where the individuals with a Chinese L1 language background process subject-extracted subject relative clauses (SS) and subject-extracted object relative clauses (SO) by using eventrelated potentials (ERPs) to probe into the real-time language processing and presents a direct manifestation of brain activity. The findings from this study support the subject relative clause preference due to the strong influence of English relative clause markedness and bilinguals’ relative lower working memory capacity.

scholarly journals The network properties of the brain at the time of normal birth support the acquisition of language processing

2018 ◽  
Author(s):  
Piergiorgio Salvan ◽  
Tomoki Arichi ◽  
Diego Vidaurre ◽  
J Donald Tournier ◽  
Shona Falconer ◽  
...  
Keyword(s):  
Language Acquisition ◽  
Language Processing ◽  
Developmental Trajectories ◽  
Brain Structures ◽  
Brain Network ◽  
Language Abilities ◽  
Normal Birth ◽  
Neural Substrate ◽  
Network Properties ◽  
The Brain

AbstractLanguage acquisition appears to rely at least in part on recruiting pre-existing brain structures. We hypothesized that the neural substrate for language can be characterized by distinct, non-trivial network properties of the brain, that modulate language acquisition early in development. We tested whether these brain network properties present at the normal age of birth predicted later language abilities, and whether these were robust against perturbation by studying infants exposed to the extreme environmental stress of preterm birth.We found that brain network controllability and integration predicted respectively phonological, ‘bottom-up’ and syntactical, ‘top-down’ language skills at 20 months, and that syntactical but not phonological functions were modulated by premature extrauterine life. These data show that the neural substrate for language acquisition is a network property present at term corrected age. These distinct developmental trajectories may be relevant to the emergence of social interaction after birth.

scholarly journals Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

2020 ◽  
Author(s):  
Sreejan Kumar ◽  
Cameron T. Ellis ◽  
Thomas O’Connell ◽  
Marvin M Chun ◽  
Nicholas B. Turk-Browne
Keyword(s):  
Human Brain ◽  
Language Processing ◽  
Brain Function ◽  
Computational Models ◽  
Brain Activity ◽  
Functional Space ◽  
Semantic Features ◽  
Brain Functions ◽  
Auditory Features ◽  
The Brain

AbstractThe extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model are more widely distributed across the brain than previously acknowledged. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.

Event-related brain potential (ERP) studies of sentence processing

2007 ◽  
pp. 384-406 ◽  
Author(s):  
Marta Kutas ◽  
Kara D. Federmeier
Keyword(s):  
Human Brain ◽  
Language Processing ◽  
Sentence Processing ◽  
Context Effects ◽  
Brain Potentials ◽  
Ion Flow ◽  
Invasive Techniques ◽  
Brain Processing ◽  
The Brain ◽  
Processing Language

The intact human brain is the only known system that can interpret and respond to various visual and acoustic patterns. Therefore, unlike researchers of other cognitive phenomena, (neuro)psycholinguists cannot avail themselves of invasive techniques in non-human animals to uncover the responsible mechanisms in the large parts of the (human) brain that have been implicated in language processing. Engagement of these different anatomical areas does, however, generate distinct patterns of biological activity (such as ion flow across neural membranes) that can be recorded inside and outside the heads of humans as they quickly, often seamlessly, and without much conscious reflection on the computations and linguistic regularities involved, understand spoken, written, or signed sentences. This article summarizes studies of event-related brain potentials and sentence processing. It discusses electrophysiology, language and the brain, processing language meaning, context effects in meaning processing, non-literal language processing, processing language form, parsing, slow potentials and the closure positive shift, and plasticity and learning.

Sign in / Sign up

Export Citation Format

Share Document