Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

Frontiers in Computational Neuroscience ◽

10.3389/fncom.2021.686239 ◽

2021 ◽

Vol 15 ◽

Author(s):

Edmund T. Rolls

Keyword(s):

Visual Cortex ◽

Visual System ◽

Superior Temporal Sulcus ◽

Object Motion ◽

Eye Position ◽

Head Direction ◽

Invariant Representations ◽

Spatial View ◽

The Brain ◽

Slow Learning

First, neurophysiological evidence for the learning of invariant representations in the inferior temporal visual cortex is described. This includes object and face representations with invariance for position, size, lighting, view and morphological transforms in the temporal lobe visual cortex; global object motion in the cortex in the superior temporal sulcus; and spatial view representations in the hippocampus that are invariant with respect to eye position, head direction, and place. Second, computational mechanisms that enable the brain to learn these invariant representations are proposed. For the ventral visual system, one key adaptation is the use of information available in the statistics of the environment in slow unsupervised learning to learn transform-invariant representations of objects. This contrasts with deep supervised learning in artificial neural networks, which uses training with thousands of exemplars forced into different categories by neuronal teachers. Similar slow learning principles apply to the learning of global object motion in the dorsal visual system leading to the cortex in the superior temporal sulcus. The learning rule that has been explored in VisNet is an associative rule with a short-term memory trace. The feed-forward architecture has four stages, with convergence from stage to stage. This type of slow learning is implemented in the brain in hierarchically organized competitive neuronal networks with convergence from stage to stage, with only 4-5 stages in the hierarchy. Slow learning is also shown to help the learning of coordinate transforms using gain modulation in the dorsal visual system extending into the parietal cortex and retrosplenial cortex. Representations are learned that are in allocentric spatial view coordinates of locations in the world and that are independent of eye position, head direction, and the place where the individual is located. This enables hippocampal spatial view cells to use idiothetic, self-motion, signals for navigation when the view details are obscured for short periods.

Download Full-text

Blindsight and Unconscious Vision: What They Teach Us about the Human Visual System

The Neuroscientist ◽

10.1177/1073858416673817 ◽

2016 ◽

Vol 23 (5) ◽

pp. 529-541 ◽

Cited By ~ 18

Author(s):

Sara Ajina ◽

Holly Bridge

Keyword(s):

Visual Cortex ◽

Visual System ◽

Primary Visual Cortex ◽

Neural Activity ◽

Human Visual System ◽

Visual Information ◽

Visual Loss ◽

Physiological Conditions ◽

The World ◽

The Brain

Damage to the primary visual cortex removes the major input from the eyes to the brain, causing significant visual loss as patients are unable to perceive the side of the world contralateral to the damage. Some patients, however, retain the ability to detect visual information within this blind region; this is known as blindsight. By studying the visual pathways that underlie this residual vision in patients, we can uncover additional aspects of the human visual system that likely contribute to normal visual function but cannot be revealed under physiological conditions. In this review, we discuss the residual abilities and neural activity that have been described in blindsight and the implications of these findings for understanding the intact system.

Download Full-text

Mammalian Visual System Organization

Oxford Research Encyclopedia of Neuroscience ◽

10.1093/acrefore/9780190264086.013.66 ◽

2017 ◽

Cited By ~ 2

Author(s):

Farran Briggs

Keyword(s):

Visual Cortex ◽

Visual System ◽

Visual Information ◽

Contextual Information ◽

Sensory Information ◽

Visual Image ◽

Brain Structures ◽

Object Representations ◽

System A ◽

The Brain

Many mammals, including humans, rely primarily on vision to sense the environment. While a large proportion of the brain is devoted to vision in highly visual animals, there are not enough neurons in the visual system to support a neuron-per-object look-up table. Instead, visual animals evolved ways to rapidly and dynamically encode an enormous diversity of visual information using minimal numbers of neurons (merely hundreds of millions of neurons and billions of connections!). In the mammalian visual system, a visual image is essentially broken down into simple elements that are reconstructed through a series of processing stages, most of which occur beneath consciousness. Importantly, visual information processing is not simply a serial progression along the hierarchy of visual brain structures (e.g., retina to visual thalamus to primary visual cortex to secondary visual cortex, etc.). Instead, connections within and between visual brain structures exist in all possible directions: feedforward, feedback, and lateral. Additionally, many mammalian visual systems are organized into parallel channels, presumably to enable efficient processing of information about different and important features in the visual environment (e.g., color, motion). The overall operations of the mammalian visual system are to: (1) combine unique groups of feature detectors in order to generate object representations and (2) integrate visual sensory information with cognitive and contextual information from the rest of the brain. Together, these operations enable individuals to perceive, plan, and act within their environment.

Download Full-text

Studies of suppression of epileptic seizures with serotonin-modulating anticonsolidation protein (SMAP)

European Psychiatry ◽

10.1016/s0924-9338(11)72612-8 ◽

2011 ◽

Vol 26 (S2) ◽

pp. 907-907

Author(s):

E.N. Panahova ◽

A.A. Mekhtiev

Keyword(s):

Visual Cortex ◽

Visual System ◽

Inhibitory Activity ◽

Basolateral Amygdala ◽

Epileptic Seizures ◽

Positive Component ◽

Interictal Spikes ◽

Inverse Effect ◽

Evoked Activity ◽

The Brain

Earlier studies revealed that tandem of brain amygdala complex and visual system is related directly to development of a number of the brain pathologies. In the present study the model of amygdala epilepsy in rabbits was formed and on its background the evoked activity in the visual cortex, colliculus superior (CS) and retina was analyzed. Tentative studies showed that penicillin-induced epileptic locus in the basolateral amygdala led to sharp (to 200–300%) enhancement of the evoked potentials (EP) in the cortex and retina, whereas in the CS the responses were suppressed significantly. During seizure fit of V-VI stage, according to Racine scale, in the visual cortex, CS and (that is especially interesting) in the retina the regular interictal spikes were fixed. Intramuscular administration (1 mg/kg) of SMAP, purified earlier from rat brain and being in linear relationship with serotonin, into rabbit led to suppression of seizures 30–40 min later. Administration of polyclonal anti-SMAP antibodies into amygdala (10 μl; 1.5 mg/ml) originally induced clear interictal spikes in the visual cortex, CS and retina, while after 30 min inverse effect was observed: elimination of the initial positive component and strong enhancement of the negative phase in the visual cortex. The latter, probably, indicates to switching on compensatory SMAP synthesis after antibody-mediated SMAP inactivation. So, the results indicate to inhibitory activity of SMAP in cessation of epileptic seizures, while its downregulation may bring to seizure initiation.

Download Full-text

Unsupervised Panoptic Segmentation

10.32920/17312189.v1 ◽

2021 ◽

Author(s):

Sajeel Aziz

Keyword(s):

Deep Learning ◽

Visual Cortex ◽

Visual System ◽

Unsupervised Clustering ◽

Clustering Methods ◽

Specific Knowledge ◽

Segmentation Approach ◽

Natural Formation ◽

Learning Architectures ◽

The Brain

The contributions of this paper are two-fold. We define unsupervised techniques for the panoptic segmentation of an image. We also define clusters which encapsulate the set of features that define objects of interest inside a scene. The motivation is to provide an approach that mimics natural formation of ideas inside the brain. Fundamentally, the eyes and visual cortex constitute the visual system, which is essential for humans to detect and recognize objects. This can be done even without specific knowledge of the objects. We strongly believe that a supervisory signal should not be required to identify objects in an image. We present an algorithm that replaces the eye and visual cortex with deep learning architectures and unsupervised clustering methods. The proposed methodology may also be used as a one-click panoptic segmentation approach which promises to significantly increase annotation efficiency. We have made the code available privately for review<sup>1</sup>.<div><br></div>

Download Full-text

Neurally plausible mechanisms for learning selective and invariant representations

The Journal of Mathematical Neuroscience ◽

10.1186/s13408-020-00088-7 ◽

2020 ◽

Vol 10 (1) ◽

Author(s):

Fabio Anselmi ◽

Ankit Patel ◽

Lorenzo Rosasco

Keyword(s):

Visual Cortex ◽

Unsupervised Learning ◽

Hebbian Learning ◽

Cell Model ◽

Broad Class ◽

Object Identity ◽

Convolutional Networks ◽

Main Challenge ◽

Invariant Representations ◽

The Brain

Abstract Coding for visual stimuli in the ventral stream is known to be invariant to object identity preserving nuisance transformations. Indeed, much recent theoretical and experimental work suggests that the main challenge for the visual cortex is to build up such nuisance invariant representations. Recently, artificial convolutional networks have succeeded in both learning such invariant properties and, surprisingly, predicting cortical responses in macaque and mouse visual cortex with unprecedented accuracy. However, some of the key ingredients that enable such success—supervised learning and the backpropagation algorithm—are neurally implausible. This makes it difficult to relate advances in understanding convolutional networks to the brain. In contrast, many of the existing neurally plausible theories of invariant representations in the brain involve unsupervised learning, and have been strongly tied to specific plasticity rules. To close this gap, we study an instantiation of simple-complex cell model and show, for a broad class of unsupervised learning rules (including Hebbian learning), that we can learn object representations that are invariant to nuisance transformations belonging to a finite orthogonal group. These findings may have implications for developing neurally plausible theories and models of how the visual cortex or artificial neural networks build selectivity for discriminating objects and invariance to real-world nuisance transformations.

Download Full-text

Compressive Temporal Summation in Human Visual Cortex

10.1101/157628 ◽

2017 ◽

Cited By ~ 4

Author(s):

Jingyang Zhou ◽

Noah C. Benson ◽

Kendrick Kay ◽

Jonathan Winawer

Keyword(s):

Visual Cortex ◽

Visual System ◽

Temporal Summation ◽

Neural Response ◽

Visual World ◽

Sensory Inputs ◽

Space And Time ◽

Visual Areas ◽

Invariant Representations ◽

Over Time

AbstractCombining sensory inputs over space and time is fundamental to vision. Population receptive field models have been successful in characterizing spatial encoding throughout the human visual pathways. A parallel question—how visual areas in the human brain process information distributed over time—has received less attention. One challenge is that the most widely used neuroimaging method—fMRI—has coarse temporal resolution compared to the time-scale of neural dynamics. Here, via carefully controlled temporally modulated stimuli, we show that information about temporal processing can be readily derived from fMRI signal amplitudes in male and female subjects. We find that all visual areas exhibit sub-additive summation, whereby responses to longer stimuli are less than the linear prediction from briefer stimuli. We also find fMRI evidence that the neural response to two stimuli is reduced for brief interstimulus intervals (indicating adaptation). These effects are more pronounced in visual areas anterior to V1-V3. Finally, we develop a general model that shows how these effects can be captured with two simple operations: temporal summation followed by a compressive nonlinearity. This model operates for arbitrary temporal stimulation patterns and provides a simple and interpretable set of computations that can be used to characterize neural response properties across the visual hierarchy. Importantly, compressive temporal summation directly parallels earlier findings of compressive spatial summation in visual cortex describing responses to stimuli distributed across space. This indicates that for space and time, cortex uses a similar processing strategy to achieve higher-level and increasingly invariant representations of the visual world.Significance statementCombining sensory inputs over time is fundamental to seeing. Two important temporal phenomena are summation, the accumulation of sensory inputs over time, and adaptation, a response reduction for repeated or sustained stimuli. We investigated these phenomena in the human visual system using fMRI. We built predictive models that operate on arbitrary temporal patterns of stimulation using two simple computations: temporal summation followed by a compressive nonlinearity. Our new temporal compressive summation model captures (1) subadditive temporal summation, and (2) adaptation. We show that the model accounts for systematic differences in these phenomena across visual areas. Finally, we show that for space and time, the visual system uses a similar strategy to achieve increasingly invariant representations of the visual world.

Download Full-text

Inhibition of Labyrinthine Nystagmus by Visual Fixation: Effects of Ablation of Visual Cortex and Superior Colliculi

Annals of Otology Rhinology & Laryngology ◽

10.1177/000348947908800321 ◽

1979 ◽

Vol 88 (3) ◽

pp. 419-423

Author(s):

Emil P. Liebman ◽

Joseph U. Toglia

Keyword(s):

Visual Cortex ◽

Visual System ◽

Brain Stem ◽

Light Stimulus ◽

Visual Fixation ◽

Vestibular Nystagmus ◽

Superior Colliculi ◽

Vestibular Testing ◽

Significant Change ◽

The Brain

A study was conducted to destroy two specific areas of the cat's visual system in order to determine if these lesions would affect the visual inhibition of calorically-induced vestibular nystagmus. The occipital visual cortex was removed in eight cats and the superior colliculi were removed bilaterally in nine cats. Postoperative vestibular testing revealed no significant change in the electronystagmography tracings and response to visual fixation. These findings suggest that, in cats, the visual inhibition of labyrinthine nystagmus is not dependent upon the integrity of the visual cortex or superior colliculi. The hypothesis is brought forward that the visual inhibition of the vestibular nystagmus is merely a reflex of the brain stem to light stimulus, mediated via the cerebellum.

Download Full-text

Brain-Inspired Algorithms for Processing of Visual Data

Lecture Notes in Computer Science - Brain-Inspired Computing ◽

10.1007/978-3-030-82427-3_8 ◽

2021 ◽

pp. 105-115

Author(s):

Nicola Strisciuglio ◽

Nicolai Petkov

Keyword(s):

Image Processing ◽

Computer Vision ◽

Visual Cortex ◽

Visual System ◽

Computational Models ◽

Hierarchical Organization ◽

Visual Data ◽

Convolutional Networks ◽

Improved Stability ◽

The Brain

AbstractThe study of the visual system of the brain has attracted the attention and interest of many neuro-scientists, that derived computational models of some types of neuron that compose it. These findings inspired researchers in image processing and computer vision to deploy such models to solve problems of visual data processing.In this paper, we review approaches for image processing and computer vision, the design of which is based on neuro-scientific findings about the functions of some neurons in the visual cortex. Furthermore, we analyze the connection between the hierarchical organization of the visual system of the brain and the structure of Convolutional Networks (ConvNets). We pay particular attention to the mechanisms of inhibition of the responses of some neurons, which provide the visual system with improved stability to changing input stimuli, and discuss their implementation in image processing operators and in ConvNets.

Download Full-text