What Is the Goal of Sensory Coding?

A number of recent attempts have been made to describe early sensory coding in terms of a general information processing strategy. In this paper, two strategies are contrasted. Both strategies take advantage of the redundancy in the environment to produce more effective representations. The first is described as a “compact” coding scheme. A compact code performs a transform that allows the input to be represented with a reduced number of vectors (cells) with minimal RMS error. This approach has recently become popular in the neural network literature and is related to a process called Principal Components Analysis (PCA). A number of recent papers have suggested that the optimal “compact” code for representing natural scenes will have units with receptive field profiles much like those found in the retina and primary visual cortex. However, in this paper, it is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway. In contrast, it is proposed that the visual system is near to optimal in representing natural scenes only if optimality is defined in terms of “sparse distributed” coding. In a sparse distributed code, all cells in the code have an equal response probability across the class of images but have a low response probability for any single image. In such a code, the dimensionality is not reduced. Rather, the redundancy of the input is transformed into the redundancy of the firing pattern of cells. It is proposed that the signature for a sparse code is found in the fourth moment of the response distribution (i.e., the kurtosis). In measurements with 55 calibrated natural scenes, the kurtosis was found to peak when the bandwidths of the visual code matched those of cells in the mammalian visual cortex. Codes resembling “wavelet transforms” are proposed to be effective because the response histograms of such codes are sparse (i.e., show high kurtosis) when presented with natural scenes. It is proposed that the structure of the image that allows sparse coding is found in the phase spectrum of the image. It is suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet). Possible reasons for why sensory systems would evolve toward sparse coding are presented.

Download Full-text

A Selective Sparse Coding Model with Embedded Attention Mechanism

Novel Approaches in Cognitive Informatics and Natural Intelligence ◽

10.4018/978-1-60566-170-4.ch005 ◽

2011 ◽

pp. 78-90

Author(s):

Qingyong Li ◽

Zhiping Shi ◽

Zhongzhi Shi

Keyword(s):

Visual Cortex ◽

Primary Visual Cortex ◽

Sparse Coding ◽

Coding Theory ◽

Neural Representation ◽

Natural Scenes ◽

Processing Capacity ◽

Uniform Sampling ◽

Coding Efficiency ◽

Non Uniform Sampling

Sparse coding theory demonstrates that the neurons in the primary visual cortex form a sparse representation of natural scenes in the viewpoint of statistics, but a typical scene contains many different patterns (corresponding to neurons in cortex) competing for neural representation because of the limited processing capacity of the visual system. We propose an attention-guided sparse coding model. This model includes two modules: the non-uniform sampling module simulating the process of retina and a data-driven attention module based on the response saliency. Our experiment results show that the model notably decreases the number of coefficients which may be activated, and retains the main vision information at the same time. It provides a way to improve the coding efficiency for sparse coding model and to achieve good performance in both population sparseness and lifetime sparseness.

Download Full-text

Independent Components of Color Natural Scenes Resemble V1 Neurons in Their Spatial and Color Tuning

Journal of Neurophysiology ◽

10.1152/jn.00775.2003 ◽

2004 ◽

Vol 91 (6) ◽

pp. 2859-2873 ◽

Cited By ~ 33

Author(s):

Matthew S. Caywood ◽

Benjamin Willmore ◽

David J. Tolhurst

Keyword(s):

Visual Cortex ◽

Sensory Input ◽

Independent Component ◽

Natural Scenes ◽

Sensory Coding ◽

Color Coding ◽

Visual Code ◽

Independent Components ◽

V1 Neurons ◽

Redundancy Reduction

It has been hypothesized that mammalian sensory systems are efficient because they reduce the redundancy of natural sensory input. If correct, this theory could unify our understanding of sensory coding; here, we test its predictions for color coding in the primate primary visual cortex (V1). We apply independent component analysis (ICA) to simulated cone responses to natural scenes, obtaining a set of colored independent component (IC) filters that form a redundancy-reducing visual code. We compare IC filters with physiologically measured V1 neurons, and find great spatial similarity between IC filters and V1 simple cells. On cursory inspection, there is little chromatic similarity; however, we find that many apparent differences result from biases in the physiological measurements and ICA analysis. After correcting these biases, we find that the chromatic tuning of IC filters does indeed resemble the population of V1 neurons, supporting the redundancy-reduction hypothesis.

Download Full-text

Sparse Coding Can Predict Primary Visual Cortex Receptive Field Changes Induced by Abnormal Visual Input

PLoS Computational Biology ◽

10.1371/journal.pcbi.1003005 ◽

2013 ◽

Vol 9 (5) ◽

pp. e1003005 ◽

Cited By ~ 22

Author(s):

Jonathan J. Hunt ◽

Peter Dayan ◽

Geoffrey J. Goodhill

Keyword(s):

Visual Cortex ◽

Receptive Field ◽

Primary Visual Cortex ◽

Sparse Coding ◽

Visual Input ◽

Abnormal Visual

Download Full-text

Constrained inference in sparse coding reproduces contextual effects and predicts laminar neural dynamics

10.1101/555128 ◽

2019 ◽

Cited By ~ 3

Author(s):

Federica Capparelli ◽

Klaus Pawelzik ◽

Udo Ernst

Keyword(s):

Visual Cortex ◽

Receptive Field ◽

Primary Visual Cortex ◽

Sparse Coding ◽

Contextual Effects ◽

Direct Access ◽

Contextual Processing ◽

Classical Receptive Field ◽

Non Linear ◽

Visual Scenes

AbstractA central goal in visual neuroscience is to understand computational mechanisms and to identify neural structures responsible for integrating local visual features into global representations. When probed with complex stimuli that extend beyond their classical receptive field, neurons display non-linear behaviours indicative of such integration processes already in early stages of visual processing. Recently some progress has been made in explaining these effects from first principles by sparse coding models with a neurophysiologically realistic inference dynamics. They reproduce some of the complex response characteristics observed in primary visual cortex, but only when the context is located near the classical receptive field, since the connection scheme they propose include interactions only among neurons with overlapping input fields. Longer-range interactions required for addressing the plethora of contextual effects reaching beyond this range do not exist. Hence, a satisfactory explanation of contextual phenomena in terms of realistic interactions and dynamics in visual cortex is still missing. Here we propose an extended generative model for visual scenes that includes spatial dependencies among different features. We derive a neurophysiologically realistic inference scheme under the constraint that neurons have direct access to only local image information. The scheme can be interpreted as a network in primary visual cortex where two neural populations are organized in different layers within orientation hypercolumns that are connected by local, short-range and long-range recurrent interactions. When trained with natural images, the model predicts a connectivity structure linking neurons with similar orientation preferences matching the typical patterns found for long-ranging horizontal axons and feedback projections in visual cortex. Subjected to contextual stimuli typically used in empirical studies our model replicates several hallmark effects of contextual processing and predicts characteristic differences for surround modulation between the two model populations. In summary, our model provides a novel framework for contextual processing in the visual system proposing a well-defined functional role for horizontal axons and feedback projections.Author summaryAn influential hypothesis about how the brain processes visual information posits that each given stimulus should be efficiently encoded using only a small number of cells. This idea led to the development of a class of models that provided a functional explanation for various response properties of visual neurons, including the non-linear modulations observed when localized stimuli are placed in a broader spatial context. However, it remains to be clarified through which anatomical structures and neural connectivities a network in the cortex could perform the computations that these models require. In this paper we propose a model for encoding spatially extended visual scenes. Imposing the constraint that neurons in visual cortex have direct access only to small portions of the visual field we derive a simple yet realistic neural population dynamics. Connectivities optimized for natural scenes conform with anatomical findings and the resulting model reproduces a broad set of physiological observations, while exposing the neural mechanisms relevant for spatio-temporal information integration.

Download Full-text