scholarly journals Redundancy between spectral and higher-order texture statistics for natural image segmentation

2021 ◽  
Author(s):  
Daniel Herrera-Esposito ◽  
Leonel Gomez-Sena ◽  
Ruben Coen-Cagli
Keyword(s):  
Image Segmentation ◽  
Peripheral Vision ◽  
Higher Order ◽  
Natural Images ◽  
Natural Image ◽  
Natural Image Statistics ◽  
Image Statistics ◽  
Texture Model ◽  
Spectral Statistics ◽  
Perceptual Segmentation

Visual texture, defined by local image statistics, provides important information to the human visual system for perceptual segmentation. Second-order or spectral statistics (equivalent to the Fourier power spectrum) are a well-studied segmentation cue. However, the role of higher-order statistics (HOS) in segmentation remains unclear, particularly for natural images. Recent experiments indicate that, in peripheral vision, the HOS of the widely adopted Portilla-Simoncelli texture model are a weak segmentation cue compared to spectral statistics, despite the fact that both are necessary to explain other perceptual phenomena and to support high-quality texture synthesis. Here we test whether this discrepancy reflects a property of natural image statistics. First, we observe that differences in spectral statistics across segments of natural images are redundant with differences in HOS. Second, using linear and nonlinear classifiers, we show that each set of statistics individually affords high performance in natural scenes and texture segmentation tasks, but combining spectral statistics and HOS produces relatively small improvements. Third, we find that HOS improve segmentation for a subset of images, although these images are difficult to identify. We also find that different subsets of HOS improve segmentation to a different extent, in agreement with previous physiological and perceptual work. These results show that the HOS add modestly to spectral statistics for natural image segmentation. We speculate that tuning to natural image statistics under resource constraints could explain the weak contribution of HOS to perceptual segmentation in human peripheral vision.

scholarly journals Reconstruction of natural images from responses of primate retinal ganglion cells

2020 ◽  
Author(s):  
Nora Brackbill ◽  
Colleen Rhoades ◽  
Alexandra Kling ◽  
Nishal P. Shah ◽  
Alexander Sher ◽  
...  
Keyword(s):  
Receptive Field ◽  
Ganglion Cells ◽  
Natural Images ◽  
Natural Image ◽  
Retinal Ganglion ◽  
Natural Image Statistics ◽  
Image Statistics ◽  
Visual Message ◽  
Spatial Properties ◽  
Reconstruction Filter

AbstractThe visual message conveyed by a retinal ganglion cell (RGC) is often summarized by its spatial receptive field, but in principle also depends on the responses of other RGCs and natural image statistics. This possibility was explored by linear reconstruction of natural images from responses of the four numerically-dominant macaque RGC types. Reconstructions were highly consistent across retinas. The optimal reconstruction filter for each RGC – its visual message – reflected natural image statistics, and resembled the receptive field only when nearby, same-type cells were included. ON and OFF cells conveyed largely independent, complementary representations, and parasol and midget cells conveyed distinct and expected features. Correlated activity and nonlinearities had statistically significant but minor effects on reconstruction. Simulated reconstructions, using linear-nonlinear cascade models of RGC light responses that incorporated measured spatial properties and nonlinearities, produced similar results. Spatiotemporal reconstructions exhibited similar spatial properties, suggesting that the results are relevant for natural vision.

scholarly journals Neural Sensitivity to Natural Image Statistics Changes during Middle Childhood

2020 ◽  
Author(s):  
Benjamin Balas ◽  
Alyson Saville
Keyword(s):  
Middle Childhood ◽  
Visual Processing ◽  
Event Related Potentials ◽  
Developmental Trajectory ◽  
Natural Images ◽  
Natural Image ◽  
Natural Image Statistics ◽  
Image Perception ◽  
Image Statistics ◽  
Related Potentials

AbstractNatural images have lawful statistical properties that the adult visual system is sensitive to, both in terms of behavior and neural responses to natural images. The developmental trajectory of sensitivity to natural image statistics remains unclear, however. In behavioral tasks, children appear to slowly acquire adult-like sensitivity to natural image statistics during middle childhood (Ellemberg et al., 2012), but in other tasks, infants exhibit some sensitivity to deviations of natural image structure (Balas & Woods, 2014). Here, we used event-related potentials (ERPs) to examine how sensitivity to natural image statistics changes during childhood at distinct stages of visual processing (the P1 and N1 components). We asked children (5-10 years old) and adults to view natural texture images with either positive/negative contrast, and natural/synthetic texture appearance (Portilla & Simoncelli, 2000) to compare electrophysiological responses to images that did or did not violate natural statistics. We hypothesized that children may only acquire sensitivity to these deviations from natural texture appearance late in middle childhood. Counter to this hypothesis, we observed significant responses to unnatural contrast and texture statistics at the N1 in all age groups. At the P1, however, only young children exhibited sensitivity to contrast polarity. The latter effect suggests greater sensitivity earlier in development to some violations of natural image statistics. We discuss these results in terms of changing patterns of invariant texture processing during middle childhood and ongoing refinement of the representations supporting natural image perception.

scholarly journals Two-Dimensional Hermite Filters Simplify the Description of High-Order Statistics of Natural Images

2016 ◽  
Author(s):  
Qin Hu ◽  
Jonathan Victor
Keyword(s):  
Order Statistics ◽  
Dimensional Subspace ◽  
High Order ◽  
Natural Images ◽  
Natural Image ◽  
Two Dimensional ◽  
Natural Image Statistics ◽  
Image Statistics ◽  
High Order Statistics ◽  
Orthogonal Set

AbstractNatural image statistics play a crucial role in shaping biological visual systems, understanding their function and design principles, and designing effective computer-vision algorithms. High-order statistics are critical for conveying local features, but they are challenging to study – largely because their number and variety is large. Here, via the use of two-dimensional Hermite (TDH) functions, we identify a covert symmetry in high-order statistics of natural images that simplifies this task. This emerges from the structure of TDH functions, which are an orthogonal set of functions that are organized into a hierarchy of ranks. Specifically, we find that the shape (skewness and kurtosis) of the distribution of filter coefficients depends only on the projection of the function onto a 1-dimensional subspace specific to each rank. The characterization of natural image statistics provided by TDH filter coefficients reflects both their phase and amplitude structure, and we suggest an intuitive interpretation for the special subspace within each rank.

scholarly journals Reconstruction of natural images from responses of primate retinal ganglion cells

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Nora Brackbill ◽  
Colleen Rhoades ◽  
Alexandra Kling ◽  
Nishal P Shah ◽  
Alexander Sher ◽  
...  
Keyword(s):  
Receptive Field ◽  
Ganglion Cells ◽  
Natural Images ◽  
Natural Image ◽  
Retinal Ganglion ◽  
Natural Image Statistics ◽  
Image Statistics ◽  
Visual Message ◽  
Spatial Properties ◽  
Reconstruction Filter

The visual message conveyed by a retinal ganglion cell (RGC) is often summarized by its spatial receptive field, but in principle also depends on the responses of other RGCs and natural image statistics. This possibility was explored by linear reconstruction of natural images from responses of the four numerically-dominant macaque RGC types. Reconstructions were highly consistent across retinas. The optimal reconstruction filter for each RGC – its visual message – reflected natural image statistics, and resembled the receptive field only when nearby, same-type cells were included. ON and OFF cells conveyed largely independent, complementary representations, and parasol and midget cells conveyed distinct features. Correlated activity and nonlinearities had statistically significant but minor effects on reconstruction. Simulated reconstructions, using linear-nonlinear cascade models of RGC light responses that incorporated measured spatial properties and nonlinearities, produced similar results. Spatiotemporal reconstructions exhibited similar spatial properties, suggesting that the results are relevant for natural vision.

Sign in / Sign up

Export Citation Format

Share Document