Composite receptive fields in the mouse auditory cortex

A central question in sensory neuroscience is how neurons represent complex natural stimuli. This process involves multiple steps of feature extraction to obtain a condensed, categorical representation useful for classification and behavior. It has previously been shown that central auditory neurons in the starling have composite receptive fields composed of multiple features when probed with conspecific songs. Whether this property is an idiosyncratic characteristic of songbirds, a group of highly specialized vocal learners, or a generic characteristic of central auditory systems in different animals is, however, unknown. To address this question, we have recorded responses from auditory cortical neurons in mice, and characterized their receptive fields using mouse ultrasonic vocalizations (USVs) as a natural and ethologically relevant stimulus and pitch-shifted starling songs as a natural but ethologically irrelevant control stimulus. We have found that auditory cortical neurons in the mouse display composite receptive fields with multiple excitatory and inhibitory subunits. Moreover, this was the case with either the conspecific or the heterospecific vocalizations. We then trained the sparse filtering algorithm on both classes of natural stimuli to obtain statistically optimal features, and compared the natural and artificial features using UMAP, a dimensionality-reduction algorithm previously used to analyze mouse USVs and birdsongs. We have found that the receptive-field features obtained with the mouse USVs and those obtained with the pitch-shifted starling songs clustered together, as did the sparse-filtering features. However, the natural and artificial receptive-field features clustered mostly separately. These results indicate that composite receptive fields are likely a generic property of central auditory systems in different classes of animals. They further suggest that the quadratic receptive-field features of the mouse auditory cortical neurons are natural-stimulus invariant.

Wild lab: A naturalistic free viewing experiment reveals previously unknown EEG signatures of face processing.

Neural mechanisms of face perception are predominantly studied in well-controlled experimental settings that involve random stimulus sequences and fixed eye positions. While powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behavior, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying nonlinear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fERPs). Importantly, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing. This includes modulation of early fERP components, as well as category-specific adaptation effects across subsequent fixations that emerge even before fixation onset.

Ecological origins of perceptual grouping principles in the auditory system

Events and objects in the world must be inferred from sensory signals to support behavior. Because sensory measurements are temporally and spatially local, the estimation of an object or event can be viewed as the grouping of these measurements into representations of their common causes. Per-ceptual grouping is believed to reflect internalized regularities of the natural environment, yet grouping cues have traditionally been identified using informal observation, and investigated using artificial stim-uli. The relationship of grouping to natural signal statistics has thus remained unclear, and additional or alternative cues remain possible. Here we derive auditory grouping cues by measuring and summarizing statistics of natural sound features. Feature co-occurrence statistics reproduced established cues but also revealed previously unappreciated grouping principles. The results suggest that auditory grouping is adapted to natural stimulus statistics, show how these statistics can reveal novel grouping phenomena, and provide a framework for studying grouping in natural signals.

Neuronal variability and tuning are balanced to optimize naturalistic self-motion coding in primate vestibular pathways

It is commonly assumed that the brain’s neural coding strategies are adapted to the statistics of natural stimuli. Specifically, to maximize information transmission, a sensory neuron’s tuning function should effectively oppose the decaying stimulus spectral power, such that the neural response is temporally decorrelated (i.e. ‘whitened’). However, theory predicts that the structure of neuronal variability also plays an essential role in determining how coding is optimized. Here, we provide experimental evidence supporting this view by recording from neurons in early vestibular pathways during naturalistic self-motion. We found that central vestibular neurons displayed temporally whitened responses that could not be explained by their tuning alone. Rather, computational modeling and analysis revealed that neuronal variability and tuning were matched to effectively complement natural stimulus statistics, thereby achieving temporal decorrelation and optimizing information transmission. Taken together, our findings reveal a novel strategy by which neural variability contributes to optimized processing of naturalistic stimuli.

Neural responses to natural and model-matched stimuli reveal distinct computations in primary and non-primary auditory cortex

A central goal of sensory neuroscience is to construct models that can explain neural responses to complex, natural stimuli. As a consequence, sensory models are often tested by comparing neural responses to natural stimuli with model responses to those stimuli. One challenge is that distinct model features are often correlated across natural stimuli, and thus model features can predict neural responses even if they do not in fact drive them. Here we propose a simple alternative for testing a sensory model: we synthesize stimuli that yield the same model response as a natural stimulus, and test whether the natural and “model-matched” stimulus elicit the same neural response. We used this approach to test whether a common model of auditory cortex – in which spectrogram-like peripheral input is processed by linear spectrotemporal filters – can explain fMRI responses in humans to natural sounds. Prior studies have that shown that this model has good predictive power throughout auditory cortex, but this finding could reflect stimulus-driven correlations. We observed that fMRI voxel responses to natural and model-matched stimuli were nearly equivalent in primary auditory cortex, but that non-primary regions showed highly divergent responses to the two sound sets, suggesting that neurons in non-primary regions extract higher-order properties not made explicit by traditional models. This dissociation between primary and non-primary regions was not clear from model predictions due to the influence of stimulus-driven response correlations. Our methodology enables stronger tests of sensory models and could be broadly applied in other domains.Author SummaryModeling neural responses to natural stimuli is a core goal of sensory neuroscience. Here we propose a new approach for testing sensory models: we synthesize a “model-matched” stimulus that yields the same model response as a natural stimulus, and test whether it produces the same neural response. We used model-matching to test whether a standard model of auditory cortex can explain human cortical responses measured with fMRI. Model-matched stimuli produced nearly equivalent voxel responses in primary auditory cortex, but highly divergent responses in non-primary regions. This dissociation was not evident using more standard approaches for model testing, and suggests that non-primary regions compute higher-order stimulus properties not captured by traditional models. The methodology could be broadly applied in other domains.

Learning Midlevel Auditory Codes from Natural Sound Statistics

Interaction with the world requires an organism to transform sensory signals into representations in which behaviorally meaningful properties of the environment are made explicit. These representations are derived through cascades of neuronal processing stages in which neurons at each stage recode the output of preceding stages. Explanations of sensory coding may thus involve understanding how low-level patterns are combined into more complex structures. To gain insight into such midlevel representations for sound, we designed a hierarchical generative model of natural sounds that learns combinations of spectrotemporal features from natural stimulus statistics. In the first layer, the model forms a sparse convolutional code of spectrograms using a dictionary of learned spectrotemporal kernels. To generalize from specific kernel activation patterns, the second layer encodes patterns of time-varying magnitude of multiple first-layer coefficients. When trained on corpora of speech and environmental sounds, some second-layer units learned to group similar spectrotemporal features. Others instantiate opponency between distinct sets of features. Such groupings might be instantiated by neurons in the auditory cortex, providing a hypothesis for midlevel neuronal computation.

Reliability and correlation of fMRI, ECoG and EEG during natural stimulus processing

Human brain mapping relies heavily on fMRI, ECoG and EEG, which capture different physiological signals. Relationships between these signals have been established in the context of specific tasks or during resting state, often using spatially confined concurrent recordings in animals. But it is not certain whether these correlations generalize to other contexts relevant for human cognitive neuroscience. Here, we address the case of complex naturalistic stimuli and ask two basic questions. First, how reliable are the responses evoked by a naturalistic audio-visual stimulus in each of these imaging methods, and second, how similar are stimulus-related responses across methods? To this end, we investigated a wide range of brain regions and frequency bands. We presented the same movie clip twice to three different cohorts of subjects (NEEG = 45, NfMRI = 11, NECoG = 5) and assessed stimulus-driven correlations across viewings and between imaging methods, thereby ruling out task-irrelevant confounds. All three imaging methods had similar repeat-reliability across viewings when fMRI and EEG data were averaged across subjects, highlighting the potential to achieve large signal-to-noise ratio by leveraging large sample sizes. The fMRI signal correlated positively with high-frequency ECoG power across multiple task-related cortical structures but positively with low-frequency EEG and ECoG power. In contrast to previous studies, these correlations were as strong for low-frequency as for high frequency ECoG. We also observed links between fMRI and infra-slow EEG voltage fluctuations. These results extend previous findings to the case of natural stimulus processing.