scholarly journals Neural dynamics of real-world object vision that guide behaviour

2017 ◽  
Author(s):  
Radoslaw M. Cichy ◽  
Nikolaus Kriegeskorte ◽  
Kamila M. Jozwik ◽  
Jasper J.F. van den Bosch ◽  
Ian Charest
Keyword(s):  
Human Brain ◽  
Real World ◽  
Brain Activity ◽  
Stimulus Onset ◽  
Visual Features ◽  
Object Representations ◽  
Neuronal Dynamics ◽  
Visual Stream ◽  
Semantic Categories ◽  
Ventral Visual Stream

1AbstractVision involves complex neuronal dynamics that link the sensory stream to behaviour. To capture the richness and complexity of the visual world and the behaviour it entails, we used an ecologically valid task with a rich set of real-world object images. We investigated how human brain activity, resolved in space with functional MRI and in time with magnetoencephalography, links the sensory stream to behavioural responses. We found that behaviour-related brain activity emerged rapidly in the ventral visual pathway within 200ms of stimulus onset. The link between stimuli, brain activity, and behaviour could not be accounted for by either category membership or visual features (as provided by an artificial deep neural network model). Our results identify behaviourally-relevant brain activity during object vision, and suggest that object representations guiding behaviour are complex and can neither be explained by visual features or semantic categories alone. Our findings support the view that visual representations in the ventral visual stream need to be understood in terms of their relevance to behaviour, and highlight the importance of complex behavioural assessment for human brain mapping.

scholarly journals Animacy and real world size shape object representations in the human medial temporal lobes

2018 ◽  
Author(s):  
Anna Blumenthal ◽  
Bobby Stojanoski ◽  
Chris Martin ◽  
Rhodri Cusack ◽  
Stefan Köhler
Keyword(s):  
Recognition Memory ◽  
Real World ◽  
Medial Temporal Lobe ◽  
Memory Task ◽  
Functional Differentiation ◽  
Object Representations ◽  
Neural Basis ◽  
Visual Stream ◽  
Ventral Visual Stream ◽  
Domain Level

ABSTRACTIdentifying what an object is, and whether an object has been encountered before, is a crucial aspect of human behavior. Despite this importance, we do not have a complete understanding of the neural basis of these abilities. Investigations into the neural organization of human object representations have revealed category specific organization in the ventral visual stream in perceptual tasks. Interestingly, these categories fall within broader domains of organization, with distinctions between animate, inanimate large, and inanimate small objects. While there is some evidence for category specific effects in the medial temporal lobe (MTL), it is currently unclear whether domain level organization is also present across these structures. To this end, we used fMRI with a continuous recognition memory task. Stimuli were images of objects from several different categories, which were either animate or inanimate, or large or small within the inanimate domain. We employed representational similarity analysis (RSA) to test the hypothesis that object-evoked responses in MTL structures during recognition-memory judgments also show evidence for domain-level organization along both dimensions. Our data support this hypothesis. Specifically, object representations were shaped by either animacy, real-world size, or both, in perirhinal and parahippocampal cortex, as well as the hippocampus. While sensitivity to these dimensions differed when structures when probed individually, hinting at interesting links to functional differentiation, similarities in organization across MTL structures were more prominent overall. These results argue for continuity in the organization of object representations in the ventral visual stream and the MTL.

scholarly journals A Visual Encoding Model Based on Contrastive Self-Supervised Learning for Human Brain Activity along the Ventral Visual Stream

2021 ◽  
Vol 11 (8) ◽  
pp. 1004
Author(s):  
Jingwei Li ◽  
Chi Zhang ◽  
Linyuan Wang ◽  
Penghui Ding ◽  
Lulu Hu ◽  
...  
Keyword(s):  
Human Brain ◽  
Supervised Learning ◽  
Computational Models ◽  
Brain Activity ◽  
Learning Method ◽  
Learning Mechanisms ◽  
Visual Stream ◽  
Model Based ◽  
Visual Encoding ◽  
Ventral Visual Stream

Visual encoding models are important computational models for understanding how information is processed along the visual stream. Many improved visual encoding models have been developed from the perspective of the model architecture and the learning objective, but these are limited to the supervised learning method. From the view of unsupervised learning mechanisms, this paper utilized a pre-trained neural network to construct a visual encoding model based on contrastive self-supervised learning for the ventral visual stream measured by functional magnetic resonance imaging (fMRI). We first extracted features using the ResNet50 model pre-trained in contrastive self-supervised learning (ResNet50-CSL model), trained a linear regression model for each voxel, and finally calculated the prediction accuracy of different voxels. Compared with the ResNet50 model pre-trained in a supervised classification task, the ResNet50-CSL model achieved an equal or even relatively better encoding performance in multiple visual cortical areas. Moreover, the ResNet50-CSL model performs hierarchical representation of input visual stimuli, which is similar to the human visual cortex in its hierarchical information processing. Our experimental results suggest that the encoding model based on contrastive self-supervised learning is a strong computational model to compete with supervised models, and contrastive self-supervised learning proves an effective learning method to extract human brain-like representations.

Using Convolutional Neural Networks to measure the contribution of visual features to the representation of object animacy in the brain

2019 ◽  
Author(s):  
Sushrut Thorat
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Temporal Cortex ◽  
Large Degree ◽  
Visual Features ◽  
Visual Feature ◽  
Visual Stream ◽  
Feature Information ◽  
Ventral Visual Stream ◽  
Animal Images

A mediolateral gradation in neural responses for images spanning animals to artificial objects is observed in the ventral temporal cortex (VTC). Which information streams drive this organisation is an ongoing debate. Recently, in Proklova et al. (2016), the visual shape and category (“animacy”) dimensions in a set of stimuli were dissociated using a behavioural measure of visual feature information. fMRI responses revealed a neural cluster (extra-visual animacy cluster - xVAC) which encoded category information unexplained by visual feature information, suggesting extra-visual contributions to the organisation in the ventral visual stream. We reassess these findings using Convolutional Neural Networks (CNNs) as models for the ventral visual stream. The visual features developed in the CNN layers can categorise the shape-matched stimuli from Proklova et al. (2016) in contrast to the behavioural measures used in the study. The category organisations in xVAC and VTC are explained to a large degree by the CNN visual feature differences, casting doubt over the suggestion that visual feature differences cannot account for the animacy organisation. To inform the debate further, we designed a set of stimuli with animal images to dissociate the animacy organisation driven by the CNN visual features from the degree of familiarity and agency (thoughtfulness and feelings). Preliminary results from a new fMRI experiment designed to understand the contribution of these non-visual features are presented.

scholarly journals The rapid emergence of auditory object representations in cortex reflect central acoustic attributes

2019 ◽  
Author(s):  
Mattson Ogg ◽  
Thomas A. Carlson ◽  
L. Robert Slevc
Keyword(s):  
Time Course ◽  
Music Perception ◽  
Brain Activity ◽  
Auditory Pathway ◽  
Stimulus Onset ◽  
Object Representations ◽  
Human Environment ◽  
Require Time ◽  
Acoustic Information ◽  
Auditory Object

Human listeners are bombarded by acoustic information that the brain rapidly organizes into coherent percepts of objects and events in the environment, which aids speech and music perception. The efficiency of auditory object recognition belies the critical constraint that acoustic stimuli necessarily require time to unfold. Using magentoencephalography (MEG), we studied the time course of the neural processes that transform dynamic acoustic information into auditory object representations. Participants listened to a diverse set of 36 tokens comprising everyday sounds from a typical human environment. Multivariate pattern analysis was used to decode the sound tokens from the MEG recordings. We show that sound tokens can be decoded from brain activity beginning 90 milliseconds after stimulus onset with peak decoding performance occurring at 155 milliseconds post stimulus onset. Decoding performance was primarily driven by differences between category representations (e.g., environmental vs. instrument sounds), although within-category decoding was better than chance. Representational similarity analysis revealed that these emerging neural representations were related to harmonic and spectrotemporal differences among the stimuli, which correspond to canonical acoustic features processed by the auditory pathway. Our findings begin to link the processing of physical sound properties with the perception of auditory objects and events in cortex.

scholarly journals The Efficacy of Single-Trial Multisensory Memories

2013 ◽  
Vol 26 (5) ◽  
pp. 483-502 ◽  
Author(s):  
Antonia Thelen ◽  
Micah M. Murray
Keyword(s):  
Object Recognition ◽  
Brain Activity ◽  
Memory Performance ◽  
Stimulus Onset ◽  
Object Representations ◽  
Single Trial ◽  
Multisensory Interactions ◽  
Early Processing ◽  
Opposing Effects ◽  
Task Irrelevant

This review article summarizes evidence that multisensory experiences at one point in time have long-lasting effects on subsequent unisensory visual and auditory object recognition. The efficacy of single-trial exposure to task-irrelevant multisensory events is its ability to modulate memory performance and brain activity to unisensory components of these events presented later in time. Object recognition (either visual or auditory) is enhanced if the initial multisensory experience had been semantically congruent and can be impaired if this multisensory pairing was either semantically incongruent or entailed meaningless information in the task-irrelevant modality, when compared to objects encountered exclusively in a unisensory context. Processes active during encoding cannot straightforwardly explain these effects; performance on all initial presentations was indistinguishable despite leading to opposing effects with stimulus repetitions. Brain responses to unisensory stimulus repetitions differ during early processing stages (∼100 ms post-stimulus onset) according to whether or not they had been initially paired in a multisensory context. Plus, the network exhibiting differential responses varies according to whether or not memory performance is enhanced or impaired. The collective findings we review indicate that multisensory associations formedviasingle-trial learning exert influences on later unisensory processing to promote distinct object representations that manifest as differentiable brain networks whose activity is correlated with memory performance. These influences occur incidentally, despite many intervening stimuli, and are distinguishable from the encoding/learning processes during the formation of the multisensory associations. The consequences of multisensory interactions thus persist over time to impact memory retrieval and object discrimination.

scholarly journals The Rapid Emergence of Auditory Object Representations in Cortex Reflect Central Acoustic Attributes

2020 ◽  
Vol 32 (1) ◽  
pp. 111-123 ◽  
Author(s):  
Mattson Ogg ◽  
Thomas A. Carlson ◽  
L. Robert Slevc
Keyword(s):  
Time Course ◽  
Music Perception ◽  
Brain Activity ◽  
Auditory Pathway ◽  
Stimulus Onset ◽  
Object Representations ◽  
Human Environment ◽  
Require Time ◽  
Acoustic Information ◽  
Auditory Object

Human listeners are bombarded by acoustic information that the brain rapidly organizes into coherent percepts of objects and events in the environment, which aids speech and music perception. The efficiency of auditory object recognition belies the critical constraint that acoustic stimuli necessarily require time to unfold. Using magnetoencephalography, we studied the time course of the neural processes that transform dynamic acoustic information into auditory object representations. Participants listened to a diverse set of 36 tokens comprising everyday sounds from a typical human environment. Multivariate pattern analysis was used to decode the sound tokens from the magnetoencephalographic recordings. We show that sound tokens can be decoded from brain activity beginning 90 msec after stimulus onset with peak decoding performance occurring at 155 msec poststimulus onset. Decoding performance was primarily driven by differences between category representations (e.g., environmental vs. instrument sounds), although within-category decoding was better than chance. Representational similarity analysis revealed that these emerging neural representations were related to harmonic and spectrotemporal differences among the stimuli, which correspond to canonical acoustic features processed by the auditory pathway. Our findings begin to link the processing of physical sound properties with the perception of auditory objects and events in cortex.

scholarly journals Integrative and distinctive coding of perceptual and conceptual object features in the ventral visual stream

2017 ◽  
Author(s):  
Chris B Martin ◽  
Danielle Douglas ◽  
Rachel N Newsome ◽  
Louisa LY Man ◽  
Morgan D Barense
Keyword(s):  
Cognitive Neuroscience ◽  
Occipital Cortex ◽  
Perirhinal Cortex ◽  
Visual Features ◽  
Visual Stream ◽  
Parahippocampal Cortex ◽  
Temporal Pole ◽  
Ventral Visual Stream ◽  
Object Features ◽  
Lateral Occipital Cortex

AbstractA tremendous body of research in cognitive neuroscience is aimed at understanding how object concepts are represented in the human brain. However, it remains unknown whether and where the visual and abstract conceptual features that define an object concept are integrated. We addressed this issue by comparing the neural pattern similarities among object-evoked fMRI responses with behavior-based models that independently captured the visual and conceptual similarities among these stimuli. Our results revealed evidence for distinctive coding of visual features in lateral occipital cortex, and conceptual features in the temporal pole and parahippocampal cortex. By contrast, we found evidence for integrative coding of visual and conceptual object features in perirhinal cortex. The neuroanatomical specificity of this effect was highlighted by results from a searchlight analysis. Taken together, our findings suggest that perirhinal cortex uniquely supports the representation of fully-specified object concepts through the integration of their visual and conceptual features.

Decoding human brain activity during real-world experiences

2007 ◽  
Vol 11 (8) ◽  
pp. 356-365 ◽  
Author(s):  
Hugo J. Spiers ◽  
Eleanor A. Maguire
Keyword(s):  
Human Brain ◽  
Real World ◽  
Brain Activity

Visual features of an observed agent do not modulate human brain activity during action observation

NeuroImage ◽  
2009 ◽  
Vol 46 (3) ◽  
pp. 844-853 ◽  
Author(s):  
Luca Turella ◽  
Michael Erb ◽  
Wolfgang Grodd ◽  
Umberto Castiello
Keyword(s):  
Human Brain ◽  
Brain Activity ◽  
Action Observation ◽  
Visual Features

Activation Timecourse of Ventral Visual Stream Object-recognition Areas: High Density Electrical Mapping of Perceptual Closure Processes

2000 ◽  
Vol 12 (4) ◽  
pp. 615-621 ◽  
Author(s):  
Glen M. Doniger ◽  
John J. Foxe ◽  
Micah M. Murray ◽  
Beth A. Higgins ◽  
Joan Gay Snodgrass ◽  
...  
Keyword(s):  
Object Recognition ◽  
Partial Information ◽  
Brain Activity ◽  
Event Related Potentials ◽  
High Density ◽  
Visual Stream ◽  
Density Mapping ◽  
Related Potentials ◽  
Ventral Visual Stream ◽  
Perceptual Closure

Object recognition is achieved even in circumstances when only partial information is available to the observer. Perceptual closure processes are essential in enabling such recognitions to occur. We presented successively less fragmented images while recording high-density event-related potentials (ERPs), which permitted us to monitor brain activity during the perceptual closure processes leading up to object recognition. We reveal a bilateral ERP component (Ncl) that tracks these processes (onsets ∼ 230 msec, maximal at ∼290 msec). Scalp-current density mapping of the Ncl revealed bilateral occipito-temporal scalp foci, which are consistent with generators in the human ventral visual stream, and specifically the lateral-occipital or LO complex as defined by hemodynamic studies of object recognition.

Sign in / Sign up

Export Citation Format

Share Document