scholarly journals The integration of visual and target signals in V4 and IT during visual object search

2019 ◽  
Vol 122 (6) ◽  
pp. 2522-2540 ◽  
Author(s):  
Noam Roth ◽  
Nicole C. Rust
Keyword(s):  
Visual Information ◽  
Visual Pathway ◽  
Target Object ◽  
Visual Object ◽  
Neural Responses ◽  
Inferotemporal Cortex ◽  
Top Down ◽  
Feed Forward ◽  
Object Search ◽  
Ventral Visual Pathway

Searching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g., V4) or at multiple stages [e.g., both V4 and inferotemporal cortex (IT)]. To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size, and background context. We found nonvisual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT. NEW & NOTEWORTHY To find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and inferotemporal cortex (IT) during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4 but rather must be integrated directly within IT itself.

scholarly journals The integration of visual and target signals in V4 and IT during visual object search

2018 ◽  
Author(s):  
Noam Roth ◽  
Nicole C. Rust
Keyword(s):  
Visual Information ◽  
Visual Pathway ◽  
Target Object ◽  
Visual Object ◽  
Neural Responses ◽  
Top Down ◽  
Feed Forward ◽  
Object Search ◽  
Ventral Visual Pathway ◽  
Background Context

ABSTRACTSearching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g. V4) or at multiple stages (e.g. both V4 and inferotemporal cortex, IT). To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size and background context. We found non-visual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT.NEW & NOTEWORTHYTo find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and IT during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4, but rather must be integrated directly within IT itself.

scholarly journals Inferotemporal cortex multiplexes behaviorally-relevant target match signals and visual representations in a manner that minimizes their interference

2017 ◽  
Author(s):  
Noam Roth ◽  
Nicole C. Rust
Keyword(s):  
Visual Information ◽  
Visual Representations ◽  
Target Object ◽  
Object Identity ◽  
Neural Responses ◽  
Inferotemporal Cortex ◽  
Match To Sample ◽  
Sample Object ◽  
High Level ◽  
Detrimental Impact

AbstractFinding a sought visual target object requires combining visual information about a scene with a remembered representation of the target to create a “target match” signal that indicates when a target is in view. Target match signals have been reported to exist within high-level visual brain areas including inferotemporal cortex (IT), where they are mixed with representations of image and object identity. However, these signals are not well understood, particularly in the context of the real-world challenge that the objects we search for typically appear at different positions, sizes, and within different background contexts. To investigate these signals, we recorded neural responses in IT as two rhesus monkeys performed a delayed-match-to-sample object search task in which target objects could appear at a variety of identity-preserving transformations. Consistent with the existence of behaviorally-relevant target match signals in IT, we found that IT contained a linearly separable target match representation that reflected behavioral confusions on trials in which the monkeys made errors. Additionally, target match signals were highly distributed across the IT population, and while a small fraction of units reflected target match signals as target match suppression, most units reflected target match signals as target match enhancement. Finally, we found that the potentially detrimental impact of target match signals on visual representations was mitigated by target match modulation that was approximately (albeit imperfectly) multiplicative. Together, these results support the existence of a robust, behaviorally-relevant target match representation in IT that is configured to minimally interfere with IT visual representations.

Temporal Dynamics of Shape Processing Differentiate Contributions of Dorsal and Ventral Visual Pathways

2019 ◽  
Vol 31 (6) ◽  
pp. 821-836 ◽  
Author(s):  
Elliot Collins ◽  
Erez Freud ◽  
Jana M. Kainerstorfer ◽  
Jiaming Cao ◽  
Marlene Behrmann
Keyword(s):  
Visual Information ◽  
Temporal Dynamics ◽  
Object Perception ◽  
Visual Pathway ◽  
Shape Sensitivity ◽  
Shape Information ◽  
Dorsal Pathway ◽  
Ventral Visual Pathway ◽  
Shape Processing ◽  
Ventral Pathway

Although shape perception is primarily considered a function of the ventral visual pathway, previous research has shown that both dorsal and ventral pathways represent shape information. Here, we examine whether the shape-selective electrophysiological signals observed in dorsal cortex are a product of the connectivity to ventral cortex or are independently computed. We conducted multiple EEG studies in which we manipulated the input parameters of the stimuli so as to bias processing to either the dorsal or ventral visual pathway. Participants viewed displays of common objects with shape information parametrically degraded across five levels. We measured shape sensitivity by regressing the amplitude of the evoked signal against the degree of stimulus scrambling. Experiment 1, which included grayscale versions of the stimuli, served as a benchmark establishing the temporal pattern of shape processing during typical object perception. These stimuli evoked broad and sustained patterns of shape sensitivity beginning as early as 50 msec after stimulus onset. In Experiments 2 and 3, we calibrated the stimuli such that visual information was delivered primarily through parvocellular inputs, which mainly project to the ventral pathway, or through koniocellular inputs, which mainly project to the dorsal pathway. In the second and third experiments, shape sensitivity was observed, but in distinct spatio-temporal configurations from each other and from that elicited by grayscale inputs. Of particular interest, in the koniocellular condition, shape selectivity emerged earlier than in the parvocellular condition. These findings support the conclusion of distinct dorsal pathway computations of object shape, independent from the ventral pathway.

scholarly journals Reconstructing feedback representations in ventral visual pathway with a generative adversarial autoencoder

2020 ◽  
Author(s):  
Haider Al-Tahan ◽  
Yalda Mohsenzadeh
Keyword(s):  
Neural Network ◽  
Visual Information ◽  
Visual Pathway ◽  
Functional Magnetic Resonance ◽  
Low Level ◽  
Ventral Visual Pathway ◽  
High Level ◽  
Feedback Connections ◽  
The Brain ◽  
Insight Into

AbstractWhile vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.Author summaryIt has been shown that the ventral visual cortex consists of a dense network of regions with feedforward and feedback connections. The feedforward path processes visual inputs along a hierarchy of cortical areas that starts in early visual cortex (an area tuned to low level features e.g. edges/corners) and ends in inferior temporal cortex (an area that responds to higher level categorical contents e.g. faces/objects). Alternatively, the feedback connections modulate neuronal responses in this hierarchy by broadcasting information from higher to lower areas. In recent years, deep neural network models which are trained on object recognition tasks achieved human-level performance and showed similar activation patterns to the visual brain. In this work, we developed a generative neural network model that consists of encoding and decoding sub-networks. By comparing this computational model with the human brain temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) response patterns, we found that the encoder processes resemble the brain feedforward processing dynamics and the decoder shares similarity with the brain feedback processing dynamics. These results provide an algorithmic insight into the spatiotemporal dynamics of feedforward and feedback processes in biological vision.

scholarly journals Oscillatory Dynamics of Perceptual to Conceptual Transformations in the Ventral Visual Pathway

2018 ◽  
Vol 30 (11) ◽  
pp. 1590-1605 ◽  
Author(s):  
Alex Clarke ◽  
Barry J. Devereux ◽  
Lorraine K. Tyler
Keyword(s):  
Visual Information ◽  
Computational Models ◽  
Semantic Information ◽  
Low Frequency ◽  
Visual Pathway ◽  
Semantic Representations ◽  
Low Level ◽  
Oscillatory Dynamics ◽  
Incremental Model ◽  
Ventral Visual Pathway

Object recognition requires dynamic transformations of low-level visual inputs to complex semantic representations. Although this process depends on the ventral visual pathway, we lack an incremental account from low-level inputs to semantic representations and the mechanistic details of these dynamics. Here we combine computational models of vision with semantics and test the output of the incremental model against patterns of neural oscillations recorded with magnetoencephalography in humans. Representational similarity analysis showed visual information was represented in low-frequency activity throughout the ventral visual pathway, and semantic information was represented in theta activity. Furthermore, directed connectivity showed visual information travels through feedforward connections, whereas visual information is transformed into semantic representations through feedforward and feedback activity, centered on the anterior temporal lobe. Our research highlights that the complex transformations between visual and semantic information is driven by feedforward and recurrent dynamics resulting in object-specific semantics.

scholarly journals Neural representations of perceptual color experience in the human ventral visual pathway

2020 ◽  
Vol 117 (23) ◽  
pp. 13145-13150 ◽  
Author(s):  
Insub Kim ◽  
Sang Wook Hong ◽  
Steven K. Shevell ◽  
Won Mok Shim
Keyword(s):  
Light Stimulus ◽  
Visual Pathway ◽  
Cortical Processing ◽  
Neural Responses ◽  
Color Experience ◽  
Retinal Stimulation ◽  
Neural Representations ◽  
Visual Areas ◽  
Ventral Visual Pathway ◽  
Higher Visual Areas

Color is a perceptual construct that arises from neural processing in hierarchically organized cortical visual areas. Previous research, however, often failed to distinguish between neural responses driven by stimulus chromaticity versus perceptual color experience. An unsolved question is whether the neural responses at each stage of cortical processing represent a physical stimulus or a color we see. The present study dissociated the perceptual domain of color experience from the physical domain of chromatic stimulation at each stage of cortical processing by using a switch rivalry paradigm that caused the color percept to vary over time without changing the retinal stimulation. Using functional MRI (fMRI) and a model-based encoding approach, we found that neural representations in higher visual areas, such as V4 and VO1, corresponded to the perceived color, whereas responses in early visual areas V1 and V2 were modulated by the chromatic light stimulus rather than color perception. Our findings support a transition in the ascending human ventral visual pathway, from a representation of the chromatic stimulus at the retina in early visual areas to responses that correspond to perceptually experienced colors in higher visual areas.

Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex

1994 ◽  
Vol 71 (3) ◽  
pp. 856-867 ◽  
Author(s):  
E. Kobatake ◽  
K. Tanaka
Keyword(s):  
Anterior Part ◽  
Posterior Part ◽  
Visual Pathway ◽  
Complex Object ◽  
Visual Responses ◽  
Inferotemporal Cortex ◽  
Critical Feature ◽  
Ventral Visual Pathway ◽  
Object Features ◽  
Simple Features

1. To infer relative roles of cortical areas at different stages of the ventral visual pathway, we quantitatively examined visual responses of cells in V2, V4, the posterior part of the inferotemporal cortex (posterior IT), and the anterior part of the inferotemporal cortex (anterior IT), using anesthetized macaque monkeys. 2. The critical feature for the activation was first determined for each recorded cell by using a reduction method. We started from images of three-dimensional complex objects and simplified the image of effective stimuli step by step by eliminating a part of the features present in the image. The simplest feature that maximally activated the cell was determined as the critical feature. The response to the critical feature was then compared with responses of the same cell to a routine set of 32 simple stimuli, which included white and black bars of four different orientations and squares or spots of four different colors. 3. Cells that responded maximally to particular complex object features were found in posterior IT and V4 as well as in anterior IT. The cells in posterior IT and V4 were, however, different from the cells in anterior IT in that many of them responded to some extent to some simple features, that the size of the receptive field was small, and that they intermingled in single penetrations with cells that responded maximally to some simple features. The complex critical features in posterior IT and V4 varied; they consisted of complex shapes, combinations of a shape and texture, and combinations of a shape and color. 4. We suggest that local neuronal networks in V4 and posterior IT play an essential role in the formation of selective responses to complex object features.

scholarly journals Contributions of magno- and parvocellular channels to conscious and non-conscious vision

2014 ◽  
Vol 369 (1641) ◽  
pp. 20130213 ◽  
Author(s):  
Bruno G. Breitmeyer
Keyword(s):  
Prefrontal Cortex ◽  
Neural Activity ◽  
Visual Information ◽  
Masked Priming ◽  
The Other ◽  
Future Research ◽  
Top Down ◽  
Pathway Activity ◽  
Feed Forward ◽  
Conscious Processing

The dorsal and ventral cortical pathways, driven predominantly by magnocellular (M) and parvocellular (P) inputs, respectively, assume leading roles in models of visual information processing. Although in prior proposals, the dorsal and ventral pathways support non-conscious and conscious vision, respectively, recent modelling and empirical developments indicate that each pathway plays important roles in both non-conscious and conscious vision. In these models, the ventral P-pathway consists of one subpathway processing an object's contour features, e.g. curvature, the other processing its surface attributes, e.g. colour. Masked priming studies have shown that feed-forward activity in the ventral P-pathway on its own supports non-conscious processing of contour and surface features. The dorsal M-pathway activity contributes directly to conscious vision of motion and indirectly to object vision by projecting to prefrontal cortex, which in turn injects top-down neural activity into the ventral P-pathway and there ‘ignites’ feed-forward–re-entrant loops deemed necessary for conscious vision. Moreover, an object's shape or contour remains invisible without the prior conscious registration of its surface properties, which for that reason are taken to comprise fundamental visual qualia. Besides suggesting avenues for future research, these developments bear on several recent and past philosophical issues.

scholarly journals Oscillatory dynamics of perceptual to conceptual transformations in the ventral visual pathway

2018 ◽  
Author(s):  
Alex Clarke ◽  
Barry J. Devereux ◽  
Lorraine K. Tyler
Keyword(s):  
Visual Information ◽  
Computational Models ◽  
Semantic Information ◽  
Visual Pathway ◽  
Semantic Representations ◽  
Low Level ◽  
Oscillatory Dynamics ◽  
Incremental Model ◽  
Visual Inputs ◽  
Ventral Visual Pathway

AbstractObject recognition requires dynamic transformations of low-level visual inputs to complex semantic representations. While this process depends on the ventral visual pathway (VVP), we lack an incremental account from low-level inputs to semantic representations, and the mechanistic details of these dynamics. Here we combine computational models of vision with semantics, and test the output of the incremental model against patterns of neural oscillations recorded with MEG in humans. Representational Similarity Analysis showed visual information was represented in alpha activity throughout the VVP, and semantic information was represented in theta activity. Furthermore, informational connectivity showed visual information travels through feedforward connections, while visual information is transformed into semantic representations through feedforward and feedback activity, centered on the anterior temporal lobe. Our research highlights that the complex transformations between visual and semantic information is driven by feedforward and recurrent dynamics resulting in object-specific semantics.

Sign in / Sign up

Export Citation Format

Share Document