scholarly journals Characterization of deep neural network features by decodability from human brain activity

2018 ◽  
Author(s):  
Tomoyasu Horikawa ◽  
Shuntaro C. Aoki ◽  
Mitsuaki Tsukamoto ◽  
Yukiyasu Kamitani
Keyword(s):  
Human Brain ◽  
Deep Neural Networks ◽  
Human Subjects ◽  
Brain Activity ◽  
Activity Patterns ◽  
Human Vision ◽  
Highly Correlated ◽  
Feature Values ◽  
The Brain

AbstractAchievements of near human-level performances in object recognition by deep neural networks (DNNs) have triggered a flood of comparative studies between the brain and DNNs. Using a DNN as a proxy for hierarchical visual representations, our recent study found that human brain activity patterns measured by functional magnetic resonance imaging (fMRI) can be decoded (translated) into DNN feature values given the same inputs. However, not all DNN features are equally decoded, indicating a gap between the DNN and human vision. Here, we present a dataset derived through the DNN feature decoding analyses including fMRI signals of five human subjects during image viewing, decoded feature values of DNNs (AlexNet and VGG19), and decoding accuracies of individual DNN features with their rankings. The decoding accuracies of individual features were highly correlated between subjects, suggesting the systematic differences between the brain and DNNs. We hope the present dataset will contribute to reveal the gap between the brain and DNNs and provide an opportunity to make use of the decoded features for further applications.

scholarly journals Sensory-modality independent activation of the brain network for language

2019 ◽  
Author(s):  
Sophie Arana ◽  
André Marquand ◽  
Annika Hultén ◽  
Peter Hagoort ◽  
Jan-Mathijs Schoffelen
Keyword(s):  
Language Processing ◽  
Sentence Processing ◽  
Human Subjects ◽  
Brain Activity ◽  
Activity Patterns ◽  
Sensory Modality ◽  
Brain Network ◽  
Large Network ◽  
Individual Subject ◽  
The Brain

AbstractThe meaning of a sentence can be understood, whether presented in written or spoken form. Therefore it is highly probable that brain processes supporting language comprehension are at least partly independent of sensory modality. To identify where and when in the brain language processing is independent of sensory modality, we directly compared neuromagnetic brain signals of 200 human subjects (102 males) either reading or listening to sentences. We used multiset canonical correlation analysis to align individual subject data in a way that boosts those aspects of the signal that are common to all, allowing us to capture word-by-word signal variations, consistent across subjects and at a fine temporal scale. Quantifying this consistency in activation across both reading and listening tasks revealed a mostly left hemispheric cortical network. Areas showing consistent activity patterns include not only areas previously implicated in higher-level language processing, such as left prefrontal, superior & middle temporal areas and anterior temporal lobe, but also parts of the control-network as well as subcentral and more posterior temporal-parietal areas. Activity in this supramodal sentence processing network starts in temporal areas and rapidly spreads to the other regions involved. The findings do not only indicate the involvement of a large network of brain areas in supramodal language processing, but also indicate that the linguistic information contained in the unfolding sentences modulates brain activity in a word-specific manner across subjects.

scholarly journals Harmonic Brain Modes: A Unifying Framework for Linking Space and Time in Brain Dynamics

2017 ◽  
Vol 24 (3) ◽  
pp. 277-293 ◽  
Author(s):  
Selen Atasoy ◽  
Gustavo Deco ◽  
Morten L. Kringelbach ◽  
Joel Pearson
Keyword(s):  
Neural Activity ◽  
Brain Activity ◽  
Activity Patterns ◽  
Structural Connectivity ◽  
Building Blocks ◽  
Mental States ◽  
Brain Dynamics ◽  
Space And Time ◽  
Wide Range ◽  
The Brain

A fundamental characteristic of spontaneous brain activity is coherent oscillations covering a wide range of frequencies. Interestingly, these temporal oscillations are highly correlated among spatially distributed cortical areas forming structured correlation patterns known as the resting state networks, although the brain is never truly at “rest.” Here, we introduce the concept of harmonic brain modes—fundamental building blocks of complex spatiotemporal patterns of neural activity. We define these elementary harmonic brain modes as harmonic modes of structural connectivity; that is, connectome harmonics, yielding fully synchronous neural activity patterns with different frequency oscillations emerging on and constrained by the particular structure of the brain. Hence, this particular definition implicitly links the hitherto poorly understood dimensions of space and time in brain dynamics and its underlying anatomy. Further we show how harmonic brain modes can explain the relationship between neurophysiological, temporal, and network-level changes in the brain across different mental states ( wakefulness, sleep, anesthesia, psychedelic). Notably, when decoded as activation of connectome harmonics, spatial and temporal characteristics of neural activity naturally emerge from the interplay between excitation and inhibition and this critical relation fits the spatial, temporal, and neurophysiological changes associated with different mental states. Thus, the introduced framework of harmonic brain modes not only establishes a relation between the spatial structure of correlation patterns and temporal oscillations (linking space and time in brain dynamics), but also enables a new dimension of tools for understanding fundamental principles underlying brain dynamics in different states of consciousness.

scholarly journals Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

2020 ◽  
Author(s):  
Soma Nonaka ◽  
Kei Majima ◽  
Shuntaro C. Aoki ◽  
Yukiyasu Kamitani
Keyword(s):  
Neural Networks ◽  
Image Recognition ◽  
High Performance ◽  
Deep Neural Networks ◽  
Recognition Performance ◽  
Brain Activity ◽  
Spatial Integration ◽  
Hierarchical Processing ◽  
Visual Areas ◽  
The Brain

SummaryAchievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pretrained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.

scholarly journals Estimation of the Directions of Alpha Rhythm Elementary Sources Using the Method of Human Brain Functional Tomography Based On the Magnetic Encephalography Data

2018 ◽  
Vol 13 (2) ◽  
pp. 426-436 ◽  
Author(s):  
M.N. Ustinin ◽  
S.D. Rykunov ◽  
A.I. Boyko ◽  
O.A. Maslova ◽  
K.D. Walton ◽  
...  
Keyword(s):  
Inverse Problem ◽  
Time Series ◽  
Human Brain ◽  
Alpha Rhythm ◽  
Brain Activity ◽  
Problem Solution ◽  
New York University ◽  
Magnetic Encephalography ◽  
The Brain ◽  
Calculation Grid

New method for the magnetic encephalography data analysis was proposed. The method transforms multichannel time series into the spatial structure of the human brain activity. In this paper we further develop this method to determine the dominant direction of the electrical sources of brain activity at each node of the calculation grid. We have considered the experimental data, obtained with three 275-channel magnetic encephalographs in New York University, McGill University and Montreal University. The human alpha rhythm phenomenon was selected as a model object. Magnetic encephalograms of the brain spontaneous activity were registered for 5-7 minutes in magnetically shielded room. Detailed multichannel spectra were obtained by the Fourier transform of the whole time series. For all spectral components, the inverse problem was solved in elementary current dipole model and the functional structure of the brain activity was calculated in the frequency band 8-12 Hz. In order to estimate the local activity direction, at the each node of calculation grid the vector of the inverse problem solution was selected, having the maximal spectral power. So, the 3D-map of the brain activity vector field was produced – the directional functional tomogram. Such maps were generated for 15 subjects and some common patterns were revealed in the directions of the alpha rhythm elementary sources. The proposed method can be used to study the local properties of the brain activity in any spectral band and in any brain compartment.

scholarly journals The ventral visual pathway represents animal appearance over animacy, unlike human behavior and deep neural networks

2017 ◽  
Author(s):  
Stefania Bracci ◽  
Ioannis Kalfas ◽  
Hans Op de Beeck
Keyword(s):  
Neural Networks ◽  
Human Brain ◽  
Deep Neural Networks ◽  
Activity Patterns ◽  
Object Identification ◽  
Visual Appearance ◽  
Learning History ◽  
Fmri Study ◽  
Object Appearance ◽  
Biased Processing

AbstractRecent studies showed agreement between how the human brain and neural networks represent objects, suggesting that we might start to understand the underlying computations. However, we know that the human brain is prone to biases at many perceptual and cognitive levels, often shaped by learning history and evolutionary constraints. Here we explore one such bias, namely the bias to perceive animacy, and used the performance of neural networks as a benchmark. We performed an fMRI study that dissociated object appearance (how an object looks like) from object category (animate or inanimate) by constructing a stimulus set that includes animate objects (e.g., a cow), typical inanimate objects (e.g., a mug), and, crucially, inanimate objects that look like the animate objects (e.g., a cow-mug). Behavioral judgments and deep neural networks categorized images mainly by animacy, setting all objects (lookalike and inanimate) apart from the animate ones. In contrast, activity patterns in ventral occipitotemporal cortex (VTC) were strongly biased towards object appearance: animals and lookalikes were similarly represented and separated from the inanimate objects. Furthermore, this bias interfered with proper object identification, such as failing to signal that a cow-mug is a mug. The bias in VTC to represent a lookalike as animate was even present when participants performed a task requiring them to report the lookalikes as inanimate. In conclusion, VTC representations, in contrast to neural networks, fail to veridically represent objects when visual appearance is dissociated from animacy, probably due to a biased processing of visual features typical of animate objects.

scholarly journals Sleeping While Awake: The Intrusion of Neural Activity Associated with Sleep Onset in the Awake Human Brain

2020 ◽  
Author(s):  
Stephanie Hawes ◽  
Carrie R. H. Innes ◽  
Nicholas Parsons ◽  
Sean P.A. Drummond ◽  
Karen Caeyensberghs ◽  
...  
Keyword(s):  
Human Brain ◽  
Basal Forebrain ◽  
Brain Activity ◽  
Cognitive Task ◽  
Sleep Onset ◽  
Theoretical Modelling ◽  
Related Activity ◽  
Independent Dataset ◽  
Time Graph ◽  
The Brain

AbstractSleep can intrude into the awake human brain when sleep deprived or fatigued, even while performing cognitive tasks. However, how the brain activity associated with sleep onset can co-exist with the activity associated with cognition in the awake humans remains unexplored. Here, we used simultaneous fMRI and EEG to generate fMRI activity maps associated with EEG theta (4-7 Hz) activity associated with sleep onset. We implemented a method to track these fMRI activity maps in individuals performing a cognitive task after well-rested and sleep-deprived nights. We found frequent intrusions of the fMRI maps associated with sleep-onset in the task-related fMRI data. These sleep events elicited a pattern of transient fMRI activity, which was spatially distinct from the task-related activity in the frontal and parietal areas of the brain. They were concomitant with reduced arousal as indicated by decreased pupil size and increased response time. Graph theoretical modelling showed that the activity associated with sleep onset emerges from the basal forebrain and spreads anterior-posteriorly via the brain’s structural connectome. We replicated the key findings in an independent dataset, which suggests that the approach can be reliably used in understanding the neuro-behavioural consequences of sleep and circadian disturbances in humans.

Is Artistic Composition in Abstract Art Detected Automatically?

2021 ◽  
pp. 102-106
Author(s):  
Claudia Menzel ◽  
Gyula Kovács ◽  
Gregor U. Hayn-Leichsenring ◽  
Christoph Redies
Keyword(s):  
Human Brain ◽  
Significant Role ◽  
Mismatch Negativity ◽  
Brain Activity ◽  
Perceptual Processing ◽  
Abstract Art ◽  
Electrical Brain Activity ◽  
Image Set ◽  
The Brain

Most artists who create abstract paintings place the pictorial elements not at random, but arrange them intentionally in a specific artistic composition. This arrangement results in a pattern of image properties that differs from image versions in which the same pictorial elements are randomly shuffled. In the article under discussion, the original abstract paintings of the author’s image set were rated as more ordered and harmonious but less interesting than their shuffled counterparts. The authors tested whether the human brain distinguishes between these original and shuffled images by recording electrical brain activity in a particular paradigm that evokes a so-called visual mismatch negativity. The results revealed that the brain detects the differences between the two types of images fast and automatically. These findings are in line with models that postulate a significant role of early (low-level) perceptual processing of formal image properties in aesthetic evaluations.

“Grammar box” in the brain

2003 ◽  
Vol 26 (6) ◽  
pp. 672-673
Author(s):  
Valéria Csépe
Keyword(s):  
Human Brain ◽  
Brain Activity ◽  
Neural Correlates ◽  
Modular Architecture ◽  
Activity Data ◽  
The Brain ◽  
Semantic Domains

Brain activity data prove the existence of qualitatively different structures in the brain. However, the question is whether the human brain acts as linguists assume in their models. The modular architecture of grammar that has been claimed by many linguists raises some empirical questions. One of the main questions is whether the threefold abstract partition of language (into syntactic, phonological, and semantic domains) has distinct neural correlates.

Is the human brain only responsive?

1995 ◽  
Vol 18 (2) ◽  
pp. 365-366
Author(s):  
Rumyana Kristeva-Feige ◽  
Bernd Feige
Keyword(s):  
Human Brain ◽  
Spontaneous Activity ◽  
Cognitive Functions ◽  
Evoked Potential ◽  
Imaging Techniques ◽  
Brain Activity ◽  
External Events ◽  
Potential Methods ◽  
The Brain

AbstractPosner & Raichle's (1994) book is a fascinating and readable account of the studies the authors have conducted on the localization of cognitive functions in the brain mainly using PET and EEC evoked potential methods. Our criticism concerns the underrepresentation of some imaging techniques (magnetoencephalography) and some forms of brain activity (spontaneous activity). Furthermore, the book leaves the reader with the impression that the brain only responds to external events.

scholarly journals Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

2020 ◽  
Author(s):  
Sreejan Kumar ◽  
Cameron T. Ellis ◽  
Thomas O’Connell ◽  
Marvin M Chun ◽  
Nicholas B. Turk-Browne
Keyword(s):  
Human Brain ◽  
Language Processing ◽  
Brain Function ◽  
Computational Models ◽  
Brain Activity ◽  
Functional Space ◽  
Semantic Features ◽  
Brain Functions ◽  
Auditory Features ◽  
The Brain

AbstractThe extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model are more widely distributed across the brain than previously acknowledged. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.

Sign in / Sign up

Export Citation Format

Share Document