scholarly journals Using neural distance to predict reaction time for categorizing the animacy, shape, and abstract properties of objects

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
J. Brendan Ritchie ◽  
Hans Op de Beeck
Keyword(s):  
Reaction Times ◽  
Visual Pathway ◽  
Neural Activation ◽  
Object Category ◽  
Asymmetric Effects ◽  
Ventral Visual Pathway ◽  
Shape Distance ◽  
Stimulus Design ◽  
High Level ◽  
The Brain

Abstract A large number of neuroimaging studies have shown that information about object category can be decoded from regions of the ventral visual pathway. One question is how this information might be functionally exploited in the brain. In an attempt to help answer this question, some studies have adopted a neural distance-to-bound approach, and shown that distance to a classifier decision boundary through neural activation space can be used to predict reaction times (RT) on animacy categorization tasks. However, these experiments have not controlled for possible visual confounds, such as shape, in their stimulus design. In the present study we sought to determine whether, when animacy and shape properties are orthogonal, neural distance in low- and high-level visual cortex would predict categorization RTs, and whether a combination of animacy and shape distance might predict RTs when categories crisscrossed the two stimulus dimensions, and so were not linearly separable. In line with previous results, we found that RTs correlated with neural distance, but only for animate stimuli, with similar, though weaker, asymmetric effects for the shape and crisscrossing tasks. Taken together, these results suggest there is potential to expand the neural distance-to-bound approach to other divisions beyond animacy and object category.

scholarly journals Using neural distance to predict reaction time for categorizing the animacy, shape, and abstract properties of objects

2018 ◽  
Author(s):  
J.Brendan Ritchie ◽  
Hans Op de Beeck
Keyword(s):  
Reaction Times ◽  
Visual Pathway ◽  
Neural Activation ◽  
Object Category ◽  
Asymmetric Effects ◽  
Ventral Visual Pathway ◽  
Shape Distance ◽  
Stimulus Design ◽  
High Level ◽  
The Brain

A large number of neuroimaging studies have shown that information about object category can be decoded from regions of the ventral visual pathway. One question is how this information might be functionally exploited in the brain. In an attempt to answer this question, some studies have adopted a neural distance-to-bound approach, and shown that distance to a classifier decision boundary through neural activation space can be used to predict reaction times (RT) on animacy categorization tasks. However, these experiments have not controlled for possible visual confounds, such as shape, in their stimulus design. In the present study we sought to determine whether, when animacy and shape properties are orthogonal, neural distance in low- and high-level visual cortex would predict categorization RTs. We also investigated whether a combination of animacy and shape distance might predict RTs when categories crisscrossed the two stimulus dimensions, and so were not linearly separable. In line with previous results, we found that RTs correlated with neural distance, but only for animate stimuli, with similar, though weaker, asymmetric effects for the shape and crisscrossing tasks. Taken together, these results suggest there is potential to expand the neural distance-to-bound approach to other divisions beyond animacy and object category.

scholarly journals Reconstructing feedback representations in ventral visual pathway with a generative adversarial autoencoder

2020 ◽  
Author(s):  
Haider Al-Tahan ◽  
Yalda Mohsenzadeh
Keyword(s):  
Neural Network ◽  
Visual Information ◽  
Visual Pathway ◽  
Functional Magnetic Resonance ◽  
Low Level ◽  
Ventral Visual Pathway ◽  
High Level ◽  
Feedback Connections ◽  
The Brain ◽  
Insight Into

AbstractWhile vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.Author summaryIt has been shown that the ventral visual cortex consists of a dense network of regions with feedforward and feedback connections. The feedforward path processes visual inputs along a hierarchy of cortical areas that starts in early visual cortex (an area tuned to low level features e.g. edges/corners) and ends in inferior temporal cortex (an area that responds to higher level categorical contents e.g. faces/objects). Alternatively, the feedback connections modulate neuronal responses in this hierarchy by broadcasting information from higher to lower areas. In recent years, deep neural network models which are trained on object recognition tasks achieved human-level performance and showed similar activation patterns to the visual brain. In this work, we developed a generative neural network model that consists of encoding and decoding sub-networks. By comparing this computational model with the human brain temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) response patterns, we found that the encoder processes resemble the brain feedforward processing dynamics and the decoder shares similarity with the brain feedback processing dynamics. These results provide an algorithmic insight into the spatiotemporal dynamics of feedforward and feedback processes in biological vision.

scholarly journals The relative coding strength of object identity and nonidentity features in human occipito-temporal cortex and convolutional neural networks

2020 ◽  
Author(s):  
Yaoda Xu ◽  
Maryam Vaziri-Pashkam
Keyword(s):  
Visual Processing ◽  
Visual Pathway ◽  
Feature Representation ◽  
Object Categorization ◽  
Object Identity ◽  
Visual Areas ◽  
Ventral Visual Pathway ◽  
Identity Representation ◽  
The Brain ◽  
Early Human

ABSTRACTAny given visual object input is characterized by multiple visual features, such as identity, position and size. Despite the usefulness of identity and nonidentity features in vision and their joint coding throughout the primate ventral visual processing pathway, they have so far been studied relatively independently. Here we document the relative coding strength of object identity and nonidentity features in a brain region and how this may change across the human ventral visual pathway. We examined a total of four nonidentity features, including two Euclidean features (position and size) and two non-Euclidean features (image statistics and spatial frequency content of an image). Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with identity outweighed the non-Euclidean features, but not the Euclidean ones, in higher levels of visual processing. A similar analysis was performed in 14 convolutional neural networks (CNNs) pretrained to perform object categorization with varying architecture, depth, and with/without recurrent processing. While the relative coding strength of object identity and nonidentity features in lower CNN layers matched well with that in early human visual areas, the match between higher CNN layers and higher human visual regions were limited. Similar results were obtained regardless of whether a CNN was trained with real-world or stylized object images that emphasized shape representation. Together, by measuring the relative coding strength of object identity and nonidentity features, our approach provided a new tool to characterize feature coding in the human brain and the correspondence between the brain and CNNs.SIGNIFICANCE STATEMENTThis study documented the relative coding strength of object identity compared to four types of nonidentity features along the human ventral visual processing pathway and compared brain responses with those of 14 CNNs pretrained to perform object categorization. Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with the coding strength of the different nonidentity features differed at higher levels of visual processing. While feature coding in lower CNN layers matched well with that of early human visual areas, the match between higher CNN layers and higher human visual regions were limited. Our approach provided a new tool to characterize feature coding in the human brain and the correspondence between the brain and CNNs.

scholarly journals Binocular depth processing in the ventral visual pathway

2016 ◽  
Vol 371 (1697) ◽  
pp. 20150259 ◽  
Author(s):  
Bram-Ernst Verhoef ◽  
Rufin Vogels ◽  
Peter Janssen
Keyword(s):  
Three Dimensional ◽  
Visual Pathway ◽  
Object Processing ◽  
Depth Processing ◽  
Ventral Visual Pathway ◽  
Ventral Pathway ◽  
Dimensional Shape ◽  
Three Dimensional Shape ◽  
The Brain ◽  
Binocular Depth

One of the most powerful forms of depth perception capitalizes on the small relative displacements, or binocular disparities, in the images projected onto each eye. The brain employs these disparities to facilitate various computations, including sensori-motor transformations (reaching, grasping), scene segmentation and object recognition. In accordance with these different functions, disparity activates a large number of regions in the brain of both humans and monkeys. Here, we review how disparity processing evolves along different regions of the ventral visual pathway of macaques, emphasizing research based on both correlational and causal techniques. We will discuss the progression in the ventral pathway from a basic absolute disparity representation to a more complex three-dimensional shape code. We will show that, in the course of this evolution, the underlying neuronal activity becomes progressively more bound to the global perceptual experience. We argue that these observations most probably extend beyond disparity processing per se , and pertain to object processing in the ventral pathway in general. We conclude by posing some important unresolved questions whose answers may significantly advance the field, and broaden its scope. This article is part of the themed issue ‘Vision in our three-dimensional world’.

Temporal Limitations in Object Processing Across the Human Ventral Visual Pathway

2007 ◽  
Vol 98 (1) ◽  
pp. 382-393 ◽  
Author(s):  
Thomas J. McKeeff ◽  
David A. Remus ◽  
Frank Tong
Keyword(s):  
Object Recognition ◽  
Temporal Processing ◽  
Visual Processing ◽  
Temporal Frequency ◽  
Visual Pathway ◽  
Processing Capacity ◽  
Complex Objects ◽  
Visual Areas ◽  
Ventral Visual Pathway ◽  
High Level

Behavioral studies have shown that object recognition becomes severely impaired at fast presentation rates, indicating a limitation in temporal processing capacity. Here, we studied whether this behavioral limit in object recognition reflects limitations in the temporal processing capacity of early visual areas tuned to basic features or high-level areas tuned to complex objects. We used functional MRI (fMRI) to measure the temporal processing capacity of multiple areas along the ventral visual pathway progressing from the primary visual cortex (V1) to high-level object-selective regions, specifically the fusiform face area (FFA) and parahippocampal place area (PPA). Subjects viewed successive images of faces or houses at presentation rates varying from 2.3 to 37.5 items/s while performing an object discrimination task. Measures of the temporal frequency response profile of each visual area revealed a systematic decline in peak tuning across the visual hierarchy. Areas V1–V3 showed peak activity at rapid presentation rates of 18–25 items/s, area V4v peaked at intermediate rates (9 items/s), and the FFA and PPA peaked at the slowest temporal rates (4–5 items/s). Our results reveal a progressive loss in the temporal processing capacity of the human visual system as information is transferred from early visual areas to higher areas. These data suggest that temporal limitations in object recognition likely result from the limited processing capacity of high-level object-selective areas rather than that of earlier stages of visual processing.

scholarly journals Data-driven functional clustering reveals dominance of face, place, and body selectivity in the ventral visual pathway

2012 ◽  
Vol 108 (8) ◽  
pp. 2306-2322 ◽  
Author(s):  
Edward Vul ◽  
Danial Lashkari ◽  
Po-Jang Hsieh ◽  
Polina Golland ◽  
Nancy Kanwisher
Keyword(s):  
Visual Pathway ◽  
Data Driven ◽  
Biased Sampling ◽  
Prior Work ◽  
Functional Clustering ◽  
Ventral Visual Pathway ◽  
Response Profiles ◽  
Discovery Method ◽  
Functional Profiles ◽  
The Brain

Regions selective for faces, places, and bodies feature prominently in the literature on the human ventral visual pathway. Are selectivities for these categories in fact the most robust response profiles in this pathway, or is their prominence an artifact of biased sampling of the hypothesis space in prior work? Here we use a data-driven structure discovery method that avoids the assumptions built into most prior work by 1) giving equal consideration to all possible response profiles over the conditions tested, 2) relaxing implicit anatomical constraints (that important functional profiles should manifest themselves in spatially contiguous voxels arising in similar locations across subjects), and 3) testing for dominant response profiles over images, rather than categories, thus enabling us to discover, rather than presume, the categories respected by the brain. Even with these assumptions relaxed, face, place, and body selectivity emerge as dominant in the ventral stream.

scholarly journals Reconstructing feedback representations in the ventral visual pathway with a generative adversarial autoencoder

2021 ◽  
Vol 17 (3) ◽  
pp. e1008775
Author(s):  
Haider Al-Tahan ◽  
Yalda Mohsenzadeh
Keyword(s):  
Visual Information ◽  
Activity Patterns ◽  
Visual Pathway ◽  
Generative Model ◽  
Data Set ◽  
Low Level ◽  
Ventral Visual Pathway ◽  
Neural Feedback ◽  
High Level ◽  
Diverse Data

While vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.

scholarly journals Temporal codes provide additional category-related information in object category decoding: a systematic comparison of informative EEG features

2020 ◽  
Author(s):  
Hamid Karimi-Rouzbahani ◽  
Mozhgan Shahmohammadi ◽  
Ehsan Vahab ◽  
Saeed Setayeshi ◽  
Thomas Carlson
Keyword(s):  
Visual Information ◽  
Temporal Dynamics ◽  
Principal Component ◽  
Event Related Potential ◽  
Neural Activation ◽  
Object Category ◽  
Category Information ◽  
Related Information ◽  
Distinct Features ◽  
The Brain

AbstractHumans are remarkably efficent at recognizing objects. Understanding how the brain performs object recognition has been challenging. Our understanding has been advanced substantially in recent years with the development of multivariate decoding methods. Most start-of-the-art decoding procedures, make use of the ‘mean’ neural activation to extract object category information, which overlooks temporal variability in the signals. Here, we studied category-related information in 30 mathematically distinct features from electroencephalography (EEG) across three independent and highly-varied datasets using multivariate decoding. While the event-related potential (ERP) components of N1 and P2a were among the most informative features, the informative original signal samples and Wavelet coefficients, selected through principal component analysis, outperformed them. The four mentioned informative features showed more pronounced decoding in the Theta frequency band, which has been suggested to support feed-forward processing of visual information in the brain. Correlational analyses showed that the features, which were most informative about object categories, could predict participants’ behavioral performance (reaction time) more accurately than the less informative features. These results suggest a new approach for studying how the human brain encodes object category information and how we can read them out more optimally to investigate the temporal dynamics of the neural code. The codes are available online at https://osf.io/wbvpn/.

Sign in / Sign up

Export Citation Format

Share Document