scholarly journals Inferotemporal cortex multiplexes behaviorally-relevant target match signals and visual representations in a manner that minimizes their interference

2017 ◽  
Author(s):  
Noam Roth ◽  
Nicole C. Rust
Keyword(s):  
Visual Information ◽  
Visual Representations ◽  
Target Object ◽  
Object Identity ◽  
Neural Responses ◽  
Inferotemporal Cortex ◽  
Match To Sample ◽  
Sample Object ◽  
High Level ◽  
Detrimental Impact

AbstractFinding a sought visual target object requires combining visual information about a scene with a remembered representation of the target to create a “target match” signal that indicates when a target is in view. Target match signals have been reported to exist within high-level visual brain areas including inferotemporal cortex (IT), where they are mixed with representations of image and object identity. However, these signals are not well understood, particularly in the context of the real-world challenge that the objects we search for typically appear at different positions, sizes, and within different background contexts. To investigate these signals, we recorded neural responses in IT as two rhesus monkeys performed a delayed-match-to-sample object search task in which target objects could appear at a variety of identity-preserving transformations. Consistent with the existence of behaviorally-relevant target match signals in IT, we found that IT contained a linearly separable target match representation that reflected behavioral confusions on trials in which the monkeys made errors. Additionally, target match signals were highly distributed across the IT population, and while a small fraction of units reflected target match signals as target match suppression, most units reflected target match signals as target match enhancement. Finally, we found that the potentially detrimental impact of target match signals on visual representations was mitigated by target match modulation that was approximately (albeit imperfectly) multiplicative. Together, these results support the existence of a robust, behaviorally-relevant target match representation in IT that is configured to minimally interfere with IT visual representations.

scholarly journals The integration of visual and target signals in V4 and IT during visual object search

2019 ◽  
Vol 122 (6) ◽  
pp. 2522-2540 ◽  
Author(s):  
Noam Roth ◽  
Nicole C. Rust
Keyword(s):  
Visual Information ◽  
Visual Pathway ◽  
Target Object ◽  
Visual Object ◽  
Neural Responses ◽  
Inferotemporal Cortex ◽  
Top Down ◽  
Feed Forward ◽  
Object Search ◽  
Ventral Visual Pathway

Searching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g., V4) or at multiple stages [e.g., both V4 and inferotemporal cortex (IT)]. To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size, and background context. We found nonvisual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT. NEW & NOTEWORTHY To find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and inferotemporal cortex (IT) during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4 but rather must be integrated directly within IT itself.

scholarly journals Quantifying the signals contained in heterogeneous neural responses and determining their relationships with task performance

2014 ◽  
Vol 112 (6) ◽  
pp. 1584-1598 ◽  
Author(s):  
Marino Pagan ◽  
Nicole C. Rust
Keyword(s):  
Task Performance ◽  
Single Neuron ◽  
Weighted Sums ◽  
Neural Responses ◽  
Signal Modulation ◽  
Neural Data ◽  
Match To Sample ◽  
Neural Populations ◽  
Different Types ◽  
High Level

The responses of high-level neurons tend to be mixtures of many different types of signals. While this diversity is thought to allow for flexible neural processing, it presents a challenge for understanding how neural responses relate to task performance and to neural computation. To address these challenges, we have developed a new method to parse the responses of individual neurons into weighted sums of intuitive signal components. Our method computes the weights by projecting a neuron's responses onto a predefined orthonormal basis. Once determined, these weights can be combined into measures of signal modulation; however, in their raw form these signal modulation measures are biased by noise. Here we introduce and evaluate two methods for correcting this bias, and we report that an analytically derived approach produces performance that is robust and superior to a bootstrap procedure. Using neural data recorded from inferotemporal cortex and perirhinal cortex as monkeys performed a delayed-match-to-sample target search task, we demonstrate how the method can be used to quantify the amounts of task-relevant signals in heterogeneous neural populations. We also demonstrate how these intuitive quantifications of signal modulation can be related to single-neuron measures of task performance ( d′).

Connecting the dots without top-down knowledge: Evidence for rapidly-learned low-level associations that are independent of object identity

2018 ◽  
Author(s):  
Patrick Sadil ◽  
Kevin Potter ◽  
David E. Huber ◽  
Rosemary Cowell
Keyword(s):  
Visual Information ◽  
Naming Task ◽  
Object Identification ◽  
Matching Task ◽  
Lateral Part ◽  
Object Identity ◽  
Top Down ◽  
Object Processing ◽  
Episodic Memories ◽  
High Level

Knowing the identity of an object can powerfully alter perception. Visual demonstrations of this – such as Gregory’s (1980) hidden Dalmatian – affirm the existence of both top-down and bottom-up processing. We consider a third processing pathway: lateral connections between the parts of an object. Lateral associations are assumed by theories of object processing and hierarchical theories of memory, but little evidence attests to them. If they exist, their effects should be observable even in the absence of object identity knowledge. We employed Continuous Flash Suppression (CFS) while participants studied object images, such that visual details were learned without explicit object identification. At test, lateral associations were probed using a part-to-part matching task. We also tested whether part-whole links were facilitated by prior study using a part-naming task, and included another study condition (“Word”), in which participants saw only an object’s written name. The key question was whether CFS study (which provided visual information without identity) would better support part-to-part matching (via lateral associations) whereas Word study (which provided identity without the correct visual form) would better support part-naming (via top-down processing). The predicted dissociation was found, and confirmed by state-trace analyses. Thus, lateral part-to-part associations were learned and retrieved independently of object identity representations. This establishes novel links between perception and memory, demonstrating that (1) lateral associations at lower levels of the object identification hierarchy exist and contribute to object processing, and (2) these associations are learned via rapid, episodic-like mechanisms previously observed for the high-level, arbitrary relations comprising episodic memories.

scholarly journals Differential Representations of Perceived and Retrieved Visual Information in Hippocampus and Cortex

2018 ◽  
Vol 29 (10) ◽  
pp. 4452-4461 ◽  
Author(s):  
Sue-Hyun Lee ◽  
Dwight J Kravitz ◽  
Chris I Baker
Keyword(s):  
Visual Cortex ◽  
Visual Information ◽  
Response Patterns ◽  
Object Representations ◽  
Object Identity ◽  
Retrieval Process ◽  
Ultra High Field ◽  
Neural Populations ◽  
High Level ◽  
Cortical Representations

Abstract Memory retrieval is thought to depend on interactions between hippocampus and cortex, but the nature of representation in these regions and their relationship remains unclear. Here, we performed an ultra-high field fMRI (7T) experiment, comprising perception, learning and retrieval sessions. We observed a fundamental difference between representations in hippocampus and high-level visual cortex during perception and retrieval. First, while object-selective posterior fusiform cortex showed consistent responses that allowed us to decode object identity across both perception and retrieval one day after learning, object decoding in hippocampus was much stronger during retrieval than perception. Second, in visual cortex but not hippocampus, there was consistency in response patterns between perception and retrieval, suggesting that substantial neural populations are shared for both perception and retrieval. Finally, the decoding in hippocampus during retrieval was not observed when retrieval was tested on the same day as learning suggesting that the retrieval process itself is not sufficient to elicit decodable object representations. Collectively, these findings suggest that while cortical representations are stable between perception and retrieval, hippocampal representations are much stronger during retrieval, implying some form of reorganization of the representations between perception and retrieval.

scholarly journals Children are sensitive to mutual information in intermediate-complexity face and nonface features

2019 ◽  
Author(s):  
Benjamin Balas ◽  
Amanda Auen ◽  
Alyson Saville ◽  
Jamie Schmidt ◽  
Assaf Harel
Keyword(s):  
Mutual Information ◽  
Visual Information ◽  
Diagnostic Category ◽  
Neural Responses ◽  
Information Usage ◽  
Intermediate Complexity ◽  
Developmental Theories ◽  
Specific Category ◽  
Level Information ◽  
High Level

Uncovering when children learn to use specific visual information for recognizingobject categories is essential for understanding how experience shapes recognition.Research on the development of face recognition has focused on children’s use oflow-level information (e.g. orientation sub-bands), or on children's use of high-levelinformation, namely, configural or holistic information. Do children also useintermediate complexity features for categorizing faces and objects, and if so, atwhat age? Intermediate-complexity features bridge the gap between low- and high- level processing: they have computational benefits for object detection and segmentation, and are known to drive neural responses in the ventral visual system.Here, we asked when children develop sensitivity to diagnostic category information in intermediate-complexity features. We presented children (5-10 years old) and adults with image fragments of faces (Experiment 1) and cars (Experiment 2) varying in their mutual information, which quantities a fragment's diagnosticity of a specific category. Our goal was to determine whether children were sensitive to the amount of mutual information in these fragments, and if their information usage is different from adults’. We found that despite better overall categorization performance in adults, all children were sensitive to fragment diagnosticity in both categories, suggesting that intermediate representations of appearance are established early in childhood. Moreover, children's usage of mutual information was not limited to face fragments, suggesting the extracting intermediate complexity features is a process that is not specific only to faces. We discuss the implications of our findings for developmental theories of face and object recognition.

scholarly journals The integration of visual and target signals in V4 and IT during visual object search

2018 ◽  
Author(s):  
Noam Roth ◽  
Nicole C. Rust
Keyword(s):  
Visual Information ◽  
Visual Pathway ◽  
Target Object ◽  
Visual Object ◽  
Neural Responses ◽  
Top Down ◽  
Feed Forward ◽  
Object Search ◽  
Ventral Visual Pathway ◽  
Background Context

ABSTRACTSearching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g. V4) or at multiple stages (e.g. both V4 and inferotemporal cortex, IT). To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size and background context. We found non-visual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT.NEW & NOTEWORTHYTo find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and IT during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4, but rather must be integrated directly within IT itself.

scholarly journals Neural Sensitivity to Mutual Information in Intermediate-Complexity Face Features Changes during Childhood

2019 ◽  
Vol 9 (7) ◽  
pp. 154
Author(s):  
Benjamin Balas ◽  
Assaf Harel ◽  
Amanda Auen ◽  
Alyson Saville
Keyword(s):  
Face Recognition ◽  
Mutual Information ◽  
Middle Childhood ◽  
Visual Information ◽  
Neural Responses ◽  
Low Level ◽  
Face Features ◽  
Infancy And Childhood ◽  
Recognition Judgments ◽  
High Level

One way in which face recognition develops during infancy and childhood is with regard to the visual information that contributes most to recognition judgments. Adult face recognition depends on critical features spanning a hierarchy of complexity, including low-level, intermediate, and high-level visual information. To date, the development of adult-like information biases for face recognition has focused on low-level features, which are computationally well-defined but low in complexity, and high-level features, which are high in complexity, but not defined precisely. To complement this existing literature, we examined the development of children’s neural responses to intermediate-level face features characterized using mutual information. Specifically, we examined children’s and adults’ sensitivity to varying levels of category diagnosticity at the P100 and N170 components. We found that during middle childhood, sensitivity to mutual information shifts from early components to later ones, which may indicate a critical restructuring of face recognition mechanisms that takes place over several years. This approach provides a useful bridge between the study of low- and high-level visual features for face recognition and suggests many intriguing questions for further investigation.

scholarly journals Multiplicative mixing of object identity and image attributes in single inferior temporal neurons

2018 ◽  
Vol 115 (14) ◽  
pp. E3276-E3285 ◽  
Author(s):  
N. Apurva Ratan Murty ◽  
S. P. Arun
Keyword(s):  
Deep Neural Networks ◽  
The Other ◽  
Neural Population ◽  
Object Identity ◽  
Single Neurons ◽  
Neural Responses ◽  
Subtle Effect ◽  
Visual Areas ◽  
High Level ◽  
Population Decoding

Object recognition is challenging because the same object can produce vastly different images, mixing signals related to its identity with signals due to its image attributes, such as size, position, rotation, etc. Previous studies have shown that both signals are present in high-level visual areas, but precisely how they are combined has remained unclear. One possibility is that neurons might encode identity and attribute signals multiplicatively so that each can be efficiently decoded without interference from the other. Here, we show that, in high-level visual cortex, responses of single neurons can be explained better as a product rather than a sum of tuning for object identity and tuning for image attributes. This subtle effect in single neurons produced substantially better population decoding of object identity and image attributes in the neural population as a whole. This property was absent both in low-level vision models and in deep neural networks. It was also unique to invariances: when tested with two-part objects, neural responses were explained better as a sum than as a product of part tuning. Taken together, our results indicate that signals requiring separate decoding, such as object identity and image attributes, are combined multiplicatively in IT neurons, whereas signals that require integration (such as parts in an object) are combined additively.

Exploring The Effect of Visual Information Degradation on Human Perception and Performance In A Human-Telerobot System

2020 ◽  
Vol 64 (1) ◽  
pp. 1302-1307
Author(s):  
Richard Stone ◽  
Minglu Wang ◽  
Thomas Schnieders ◽  
Esraa Abdelall
Keyword(s):  
Visual Information ◽  
Human Perception ◽  
Low Level ◽  
Hit Rate ◽  
Word Level ◽  
Level Of Automation ◽  
Significant Difference ◽  
Human Operators ◽  
And Performance ◽  
High Level

Human-robotic interaction system are increasingly becoming integrated into industrial, commercial and emergency service agencies. It is critical that human operators understand and trust automation when these systems support and even make important decisions. The following study focused on human-in-loop telerobotic system performing a reconnaissance operation. Twenty-four subjects were divided into groups based on level of automation (Low-Level Automation (LLA), and High-Level Automation (HLA)). Results indicated a significant difference between low and high word level of control in hit rate when permanent error occurred. In the LLA group, the type of error had a significant effect on the hit rate. In general, the high level of automation was better than the low level of automation, especially if it was more reliable, suggesting that subjects in the HLA group could rely on the automatic implementation to perform the task more effectively and more accurately.

scholarly journals Cortical response to naturalistic stimuli is largely predictable with deep neural networks

2021 ◽  
Vol 7 (22) ◽  
pp. eabe7547
Author(s):  
Meenakshi Khosla ◽  
Gia H. Ngo ◽  
Keith Jamison ◽  
Amy Kuceyeski ◽  
Mert R. Sabuncu
Keyword(s):  
Brain Function ◽  
Deep Neural Networks ◽  
Neural Responses ◽  
Dynamic Interactions ◽  
Hierarchical Processing ◽  
Human Connectome Project ◽  
Neural Information ◽  
Neural Information Processing ◽  
Visual Interactions ◽  
High Level

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions.

Sign in / Sign up

Export Citation Format

Share Document