Tuning in scene-preferring cortex for mid-level visual features gives rise to selectivity across multiple levels of stimulus complexity

The scene-preferring portion of the human ventral visual stream, known as the parahippocampal place area (PPA), responds to scenes and landmark objects, which tend to be large in real-world size, fixed in location, and inanimate. However, the PPA also exhibits preferences for low-level contour statistics, including rectilinearity and cardinal orientations, that are not directly predicted by theories of scene- and landmark-selectivity. It is unknown whether these divergent findings of both low- and high-level selectivity in the PPA can be explained by a unified computational theory. To address this issue, we fit hierarchical computational models of mid-level tuning to the image-evoked fMRI responses of the PPA, and we performed a series of high-throughput experiments on these models. Our findings show that hierarchical encoding models of the PPA exhibit emergent selectivity across multiple levels of complexity, giving rise to high-level preferences along dimensions of real-world size, fixedness, and naturalness/animacy as well as low-level preferences for rectilinear shapes and cardinal orientations. These results reconcile disparate theories of PPA function in a unified model of mid-level visual representation, and they demonstrate how multifaceted selectivity profiles naturally emerge from the hierarchical computations of visual cortex and the natural statistics of images.

Thermal Image Restoration Based on LWIR Sensor Statistics

An imaging system has natural statistics that reflect its intrinsic characteristics. For example, the gradient histogram of a visible light image generally obeys a heavy-tailed distribution, and its restoration considers natural statistics. Thermal imaging cameras detect infrared radiation, and their signal processors are specialized according to the optical and sensor systems. Thermal images, also known as long wavelength infrared (LWIR) images, suffer from distinct degradations of LWIR sensors and residual nonuniformity (RNU). However, despite the existence of various studies on the statistics of thermal images, thermal image processing has seldom attempted to incorporate natural statistics. In this study, natural statistics of thermal imaging sensors are derived, and an optimization method for restoring thermal images is proposed. To verify our hypothesis about the thermal images, high-frequency components of thermal images from various datasets are analyzed with various measures (correlation coefficient, histogram intersection, chi-squared test, Bhattacharyya distance, and Kullback–Leibler divergence), and generalized properties are derived. Furthermore, cost functions accommodating the validated natural statistics are designed and minimized by a pixel-wise optimization method. The proposed algorithm has a specialized structure for thermal images and outperforms the conventional methods. Several image quality assessments are employed for quantitatively demonstrating the performance of the proposed method. Experiments with synthesized images and real-world images are conducted, and the results are quantified by reference image assessments (peak signal-to-noise ratio and structural similarity index measure) and no-reference image assessments (Roughness (Ro) and Effective Roughness (ERo) indices).

How Ambient Environment Influences Olfactory Orientation in Search and Rescue Dogs

Under natural conditions, an animal orienting to an air-borne odor plume must contend with the shifting influence of meteorological variables, such as air temperature, humidity, and wind speed, on the location and the detectability of the plume. Despite their importance, the natural statistics of such variables are difficult to reproduce in the laboratory and hence few studies have investigated strategies of olfactory orientation by mobile animals under different meteorological conditions. Using trained search and rescue dogs, we quantified the olfactory orientation behaviors of dogs searching for a trail (aged 1–3 h) of a hidden human subject in a natural landscape, under a range of meteorological conditions. Dogs were highly successful in locating the human target hidden 800 m from the start location (93% success). Humidity and air temperature had a significant effect on search strategy: as air conditions became cooler and more humid, dogs searched significantly closer to the experimental trail. Dogs also modified their speed and head position according to their search location distance from the experimental trail. When close to the trail, dogs searched with their head up and ran quickly but when their search took them farther from the trail, they were more likely to search with their nose to the ground, moving more slowly. This study of a mammalian species responding to localized shifts in ambient conditions lays the foundation for future studies of olfactory orientation, and the development of a highly tractable mammalian species for such research.

Levels of Representation in a Deep Learning Model of Categorization

Deep convolutional neural networks (DCNNs) rival humans in object recognition. The layers (or levels of representation) in DCNNs have been successfully aligned with processing stages along the ventral stream for visual processing. Here, we propose a model of concept learning that uses visual representations from these networks to build memory representations of novel categories, which may rely on the medial temporal lobe (MTL) and medial prefrontal cortex (mPFC). Our approach opens up two possibilities:a) formal investigations can involve photographic stimuli as opposed to stimuli handcrafted and coded by the experimenter;b) model comparison can determine which level of representation within a DCNN a learner is using during categorization decisions. Pursuing the latter point, DCNNs suggest that the shape bias in children relies on representations at more advanced network layers whereas a learner that relied on lower network layers would display a color bias. These results confirm the role of natural statistics in the shape bias (i.e., shape is predictive of category membership) while highlighting that the type of statistics matter, i.e., those from lower or higher levels of representation. We use the same approach to provide evidence that pigeons performing seemingly sophisticated categorization of complex imagery may in fact be relying on representations that are very low-level (i.e., retinotopic). Although complex features, such as shape, relatively predominate at more advanced network layers, even simple features, such as spatial frequency and orientation, are better represented at the more advanced layers, contrary to a standard hierarchical view.

Blind Image Quality Assessment Based on Natural Statistics of Double-Opponency

One of the challenges in image quality assessment (IQA) is to determine the quality score without the presence of the reference image. In this paper, the authors proposed a no-reference image quality assessment method based on the natural statistics of double-opponent (DO) cells. It utilizes the statistical modeling of the three opponency channels using the generalized Gaussian distribution (GGD) and asymmetric generalized Gaussian distribution (AGGD). The parameters of GGD and AGGD are then applied to feedforward neural network to predict the image quality. Result shows that for any opposing channels, its natural statistics parameters when applied to feedforward neural network can achieve satisfactory prediction of image quality.

Natural ITD statistics predict human auditory spatial perception

A neural code adapted to the statistical structure of sensory cues may optimize perception. We investigated whether interaural time difference (ITD) statistics inherent in natural acoustic scenes are parameters determining spatial discriminability. The natural ITD rate of change across azimuth (ITDrc) and ITD variability over time (ITDv) were combined in a Fisher information statistic for assessing the amount of azimuthal information conveyed by this sensory cue. We hypothesized that natural ITD statistics drive the neural code for ITD and thus influence spatial perception. Human spatial discriminability and spatial novelty detection of sounds with invariant statistics correlated with natural ITD statistics. The lack of natural statistics in the stimuli suggests that this correlation results from natural statistics driving the neural code. Additionally, the density distribution of ITD tuning matching natural statistics is consistent with classic models of ITD coding and can explain the ITD tuning distribution observed in the mammalian brainstem.Impact statementHuman brain has incorporated natural statistics of spatial cues to the neural code supporting perception of sound location.Major subject areaNeuroscience;Research organismHuman;