Neuronal variability reflects probabilistic inference tuned to natural image statistics

Neuronal activity in sensory cortex fluctuates over time and across repetitions of the same input. This variability is often considered detrimental to neural coding. The theory of neural sampling proposes instead that variability encodes the uncertainty of perceptual inferences. In primary visual cortex (V1), modulation of variability by sensory and non-sensory factors supports this view. However, it is unknown whether V1 variability reflects the statistical structure of visual inputs, as would be required for inferences correctly tuned to the statistics of the natural environment. Here we combine analysis of image statistics and recordings in macaque V1 to show that probabilistic inference tuned to natural image statistics explains the widely observed dependence between spike count variance and mean, and the modulation of V1 activity and variability by spatial context in images. Our results show that the properties of a basic aspect of cortical responses—their variability—can be explained by a probabilistic representation tuned to naturalistic inputs.

Redundancy between spectral and higher-order texture statistics for natural image segmentation

Visual texture, defined by local image statistics, provides important information to the human visual system for perceptual segmentation. Second-order or spectral statistics (equivalent to the Fourier power spectrum) are a well-studied segmentation cue. However, the role of higher-order statistics (HOS) in segmentation remains unclear, particularly for natural images. Recent experiments indicate that, in peripheral vision, the HOS of the widely adopted Portilla-Simoncelli texture model are a weak segmentation cue compared to spectral statistics, despite the fact that both are necessary to explain other perceptual phenomena and to support high-quality texture synthesis. Here we test whether this discrepancy reflects a property of natural image statistics. First, we observe that differences in spectral statistics across segments of natural images are redundant with differences in HOS. Second, using linear and nonlinear classifiers, we show that each set of statistics individually affords high performance in natural scenes and texture segmentation tasks, but combining spectral statistics and HOS produces relatively small improvements. Third, we find that HOS improve segmentation for a subset of images, although these images are difficult to identify. We also find that different subsets of HOS improve segmentation to a different extent, in agreement with previous physiological and perceptual work. These results show that the HOS add modestly to spectral statistics for natural image segmentation. We speculate that tuning to natural image statistics under resource constraints could explain the weak contribution of HOS to perceptual segmentation in human peripheral vision.

Natural Image Statistics for Mouse Vision

The mouse has dichromatic colour vision based on two different types of opsins: short (S)-and middle (M)-wavelength-sensitive opsins with peak sensitivity to ultraviolet (UV; 360 nm) and green light (508 nm), respectively. In the mouse retina, the cone photoreceptors that predominantly express the S-opsin are more sensitive to contrasts, and denser towards the ventral retina, preferentially sampling the upper part of the visual field. In contrast, the expression of the M-opsin gradually increases towards the dorsal retina that encodes the lower visual field. Such distinct retinal organizations are assumed to arise from a selective pressure in evolution to efficiently encode the natural scenes. However, natural image statistics of UV light have never been examined beyond the spectral analysis. Here we developed a multi-spectral camera and examined the UV and green image statistics of the same natural scenes. We found that the local contrast and the spatial correlation were higher in UV than in green for images above the horizon, but lower in UV than in green for those below the horizon. This suggests that the mouse retina is not necessarily optimal for maximizing the bandwidth of information transmission. Factors besides the coding efficiency, such as visual behavioural requirements, will thus need to be considered to fully explain the characteristic organization of the mouse retina.

Reconstruction of natural images from responses of primate retinal ganglion cells

The visual message conveyed by a retinal ganglion cell (RGC) is often summarized by its spatial receptive field, but in principle also depends on the responses of other RGCs and natural image statistics. This possibility was explored by linear reconstruction of natural images from responses of the four numerically-dominant macaque RGC types. Reconstructions were highly consistent across retinas. The optimal reconstruction filter for each RGC – its visual message – reflected natural image statistics, and resembled the receptive field only when nearby, same-type cells were included. ON and OFF cells conveyed largely independent, complementary representations, and parasol and midget cells conveyed distinct features. Correlated activity and nonlinearities had statistically significant but minor effects on reconstruction. Simulated reconstructions, using linear-nonlinear cascade models of RGC light responses that incorporated measured spatial properties and nonlinearities, produced similar results. Spatiotemporal reconstructions exhibited similar spatial properties, suggesting that the results are relevant for natural vision.

Signals Related to Color in the Early Visual Cortex

Visual images can be described in terms of the illuminants and objects that are causal to the light reaching the eye, the retinal image, its neural representation, or how the image is perceived. Respecting the differences among these distinct levels of description can be challenging but is crucial for a clear understanding of color vision. This article approaches color by reviewing what is known about its neural representation in the early visual cortex, with a brief description of signals in the eye and the thalamus for context. The review focuses on the properties of single neurons and advances the general theme that experimental approaches based on knowledge of feedforward signals have promoted greater understanding of the neural code for color than approaches based on correlating single-unit responses with color perception. New data from area V1 illustrate the strength of the feedforward approach. Future directions for progress in color neurophysiology are discussed: techniques for improved single-neuron characterization, for investigations of neural populations and small circuits, and for the analysis of natural image statistics.

Predictability in natural images determines V1 firing rates and synchronization: A deep neural network approach

Feedforward deep neural networks for object recognition are a promising model of visual processing and can accurately predict firing-rate responses along the ventral stream. Yet, these networks have limitations as models of various aspects of cortical processing related to recurrent connectivity, including neuronal synchronization and the integration of sensory inputs with spatio-temporal context. We trained self-supervised, generative neural networks to predict small regions of natural images based on the spatial context (i.e. inpainting). Using these network predictions, we determined the spatial predictability of visual inputs into (macaque) V1 receptive fields (RFs), and distinguished low- from high-level predictability. Spatial predictability strongly modulated V1 activity, with distinct effects on firing rates and synchronization in gamma-(30-80Hz) and beta-bands (18-30Hz). Furthermore, firing rates, but not synchronization, were accurately predicted by a deep neural network for object recognition. Neural networks trained to specifically predict V1 gamma-band synchronization developed large, grating-like RFs in the deepest layer. These findings suggest complementary roles for firing rates and synchronization in self-supervised learning of natural-image statistics.

Neuronal variability reflects probabilistic inference tuned to natural image statistics

Neuronal activity in sensory cortex fluctuates over time and across repetitions of the same input. This variability is often considered detrimental to neural coding. The theory of neural sampling proposes instead that variability encodes the uncertainty of perceptual inferences. In primary visual cortex (V1), modulation of variability by sensory and non-sensory factors supports this view. However, it is unknown whether V1 variability reflects the statistical structure of visual inputs, as would be required for inferences correctly tuned to the statistics of the natural environment. Here we combine analysis of image statistics and recordings in macaque V1 to show that probabilistic inference tuned to natural image statistics explains Poisson-like variability, and the modulation of V1 activity and variability by spatial context in images. Our results show that the properties of a basic aspect of cortical responses—their variability—can be explained by a probabilistic representation tuned to naturalistic inputs.