scholarly journals The feature-weighted receptive field: an interpretable encoding model for complex feature spaces

2017 ◽  
Author(s):  
Ghislain St-Yves ◽  
Thomas Naselaris
Keyword(s):  
Receptive Field ◽  
Frequency Tuning ◽  
Brain Activity ◽  
Receptive Fields ◽  
Visual Space ◽  
Model Complexity ◽  
Visual Features ◽  
Natural Scenes ◽  
Feature Maps ◽  
Feature Map

AbstractWe introduce the feature-weighted receptive field (fwRF), an encoding model designed to balance expressiveness, interpretability and scalability. The fwRF is organized around the notion of a feature map—a transformation of visual stimuli into visual features that preserves the topology of visual space (but not necessarily the native resolution of the stimulus). The key assumption of the fwRF model is that activity in each voxel encodes variation in a spatially localized region across multiple feature maps. This region is fixed for all feature maps; however, the contribution of each feature map to voxel activity is weighted. Thus, the model has two separable sets of parameters: “where” parameters that characterize the location and extent of pooling over visual features, and “what” parameters that characterize tuning to visual features. The “where” parameters are analogous to classical receptive fields, while “what” parameters are analogous to classical tuning functions. By treating these as separable parameters, the fwRF model complexity is independent of the resolution of the underlying feature maps. This makes it possible to estimate models with thousands of high-resolution feature maps from relatively small amounts of data. Once a fwRF model has been estimated from data, spatial pooling and feature tuning can be read-off directly with no (or very little) additional post-processing or in-silico experimentation.We describe an optimization algorithm for estimating fwRF models from data acquired during standard visual neuroimaging experiments. We then demonstrate the model’s application to two distinct sets of features: Gabor wavelets and features supplied by a deep convolutional neural network. We show that when Gabor feature maps are used, the fwRF model recovers receptive fields and spatial frequency tuning functions consistent with known organizational principles of the visual cortex. We also show that a fwRF model can be used to regress entire deep convolutional networks against brain activity. The ability to use whole networks in a single encoding model yields state-of-the-art prediction accuracy. Our results suggest a wide variety of uses for the feature-weighted receptive field model, from retinotopic mapping with natural scenes, to regressing the activities of whole deep neural networks onto measured brain activity.

Receptive Field Size and Response Latency Are Correlated Within the Cat Visual Thalamus

2005 ◽  
Vol 93 (6) ◽  
pp. 3537-3547 ◽  
Author(s):  
Chong Weng ◽  
Chun-I Yeh ◽  
Carl R. Stoelzel ◽  
Jose-Manuel Alonso
Keyword(s):  
Receptive Field ◽  
Spatial Frequency ◽  
Response Latency ◽  
Time Course ◽  
Frequency Tuning ◽  
Receptive Fields ◽  
Cortical Cell ◽  
Field Size ◽  
Receptive Field Size ◽  
Spatial Frequency Tuning

Each point in visual space is encoded at the level of the thalamus by a group of neighboring cells with overlapping receptive fields. Here we show that the receptive fields of these cells differ in size and response latency but not at random. We have found that in the cat lateral geniculate nucleus (LGN) the receptive field size and response latency of neighboring neurons are significantly correlated: the larger the receptive field, the faster the response to visual stimuli. This correlation is widespread in LGN. It is found in groups of cells belonging to the same type (e.g., Y cells), and of different types (i.e., X and Y), within a specific layer or across different layers. These results indicate that the inputs from the multiple geniculate afferents that converge onto a cortical cell (approximately 30) are likely to arrive in a sequence determined by the receptive field size of the geniculate afferents. Recent studies have shown that the peak of the spatial frequency tuning of a cortical cell shifts toward higher frequencies as the response progresses in time. Our results are consistent with the idea that these shifts in spatial frequency tuning arise from differences in the response time course of the thalamic inputs.

scholarly journals RFaNet: Receptive Field-Aware Network with Finger Attention for Fingerspelling Recognition Using a Depth Sensor

Mathematics ◽  
2021 ◽  
Vol 9 (21) ◽  
pp. 2815
Author(s):  
Shih-Hung Yang ◽  
Yao-Mao Cheng ◽  
Jyun-We Huang ◽  
Yon-Ping Chen
Keyword(s):  
Receptive Field ◽  
American Sign Language ◽  
Multiple Scales ◽  
Receptive Fields ◽  
Depth Image ◽  
Small Scale ◽  
Feature Maps ◽  
Depth Sensor ◽  
Depth Images ◽  
Effective Transfer

Automatic fingerspelling recognition tackles the communication barrier between deaf and hearing individuals. However, the accuracy of fingerspelling recognition is reduced by high intra-class variability and low inter-class variability. In the existing methods, regular convolutional kernels, which have limited receptive fields (RFs) and often cannot detect subtle discriminative details, are applied to learn features. In this study, we propose a receptive field-aware network with finger attention (RFaNet) that highlights the finger regions and builds inter-finger relations. To highlight the discriminative details of these fingers, RFaNet reweights the low-level features of the hand depth image with those of the non-forearm image and improves finger localization, even when the wrist is occluded. RFaNet captures neighboring and inter-region dependencies between fingers in high-level features. An atrous convolution procedure enlarges the RFs at multiple scales and a non-local operation computes the interactions between multi-scale feature maps, thereby facilitating the building of inter-finger relations. Thus, the representation of a sign is invariant to viewpoint changes, which are primarily responsible for intra-class variability. On an American Sign Language fingerspelling dataset, RFaNet achieved 1.77% higher classification accuracy than state-of-the-art methods. RFaNet achieved effective transfer learning when the number of labeled depth images was insufficient. The fingerspelling representation of a depth image can be effectively transferred from large- to small-scale datasets via highlighting the finger regions and building inter-finger relations, thereby reducing the requirement for expensive fingerspelling annotations.

scholarly journals Strobe Rearing Reduces Direction Selectivity in Area 17 by Altering Spatiotemporal Receptive-Field Structure

1998 ◽  
Vol 80 (6) ◽  
pp. 2991-3004 ◽  
Author(s):  
Allen L. Humphrey ◽  
Alan B. Saul
Keyword(s):  
Receptive Field ◽  
Frequency Tuning ◽  
Temporal Frequency ◽  
Receptive Fields ◽  
Direction Selectivity ◽  
Field Structure ◽  
Area 17 ◽  
Directional Tuning ◽  
Simple Cells ◽  
Spatiotemporal Receptive Field

Humphrey, Allen L. and Alan B. Saul. Strobe rearing reduces direction selectivity in area 17 by altering spatiotemporal receptive-field structure. J. Neurophysiol. 80: 2991–3004, 1998. Direction selectivity in simple cells of cat area 17 is linked to spatiotemporal (S-T) receptive-field structure. S-T inseparable receptive fields display gradients of response timing across the receptive field that confer a preferred direction of motion. Receptive fields that are not direction selective lack gradients; they are S-T separable, displaying uniform timing across the field. Here we further examine this link using a developmental paradigm that disrupts direction selectivity. Cats were reared from birth to 8 mo of age in 8-Hz stroboscopic illumination. Direction selectivity in simple cells was then measured using gratings drifting at different temporal frequencies (0.25–16 Hz). S-T structure was assessed using stationary bars presented at different receptive-field positions, with bar luminance being modulated sinusoidally at different temporal frequencies. For each cell, plots of response phase versus bar position were fit by lines to characterize S-T inseparability at each temporal frequency. Strobe rearing produced a profound loss of direction selectivity at all temporal frequencies; only 10% of cells were selective compared with 80% in normal cats. The few remaining directional cells were selective over a narrower than normal range of temporal frequencies and exhibited weaker than normal direction selectivity. Importantly, the directional loss was accompanied by a virtual elimination of S-T inseparability. Nearly all cells were S-T separable, like nondirectional cells in normal cats. The loss was clearest in layer 4. Normally, inseparability is greatest there, and it correlates well ( r = 0.77) with direction selectivity; strobe rearing reduced inseparability and direction selectivity to very low values. The few remaining directional cells were inseparable. In layer 6 of normal cats, most direction-selective cells are only weakly inseparable, and there is no consistent relationship between the two measures. However, after strobe rearing, even the weak inseparability was eliminated along with direction selectivity. The correlated changes in S-T structure and direction selectivity were confirmed using conventional linear predictions of directional tuning based on responses to counterphasing bars and white noise stimuli. The developmental changes were permanent, being observed up to 12 yr after strobe rearing. The deficits were remarkably specific; strobe rearing did not affect spatial receptive-field structure, orientation selectivity, spatial or temporal frequency tuning, or general responsiveness to visual stimuli. These results provide further support for a critical role of S-T structure in determining direction selectivity in simple cells. Strobe rearing eliminates directional tuning by altering the timing of responses within the receptive field.

Length and width tuning of neurons in the cat's primary visual cortex

1994 ◽  
Vol 71 (1) ◽  
pp. 347-374 ◽  
Author(s):  
G. C. DeAngelis ◽  
R. D. Freeman ◽  
I. Ohzawa
Keyword(s):  
Receptive Field ◽  
Spatial Frequency ◽  
Spatial Organization ◽  
Striate Cortex ◽  
Frequency Tuning ◽  
Variable Parameter ◽  
Visual Space ◽  
Similar Degree ◽  
Tuning Curves ◽  
Number Of Cycles

1. The classically defined receptive field of a visual neuron is the area of visual space over which the cell responds to visual stimuli. It is well established, however, that the discharge produced by an optimal stimulus can be modulated by the presence of additional stimuli that by themselves do not produce any response. This study examines inhibitory influences that originate from areas located outside of the classical (i.e., excitatory) receptive field. Previous work has shown that for some cells the response to a properly oriented bar of light becomes attenuated when the bar extends beyond the receptive field, a phenomenon known as end-inhibition (or length tuning). Analogously, it has been shown that increasing the number of cycles of a drifting grating stimulus may also inhibit the firing of some cells, an effect known as side-inhibition (or width tuning). Very little information is available, however, about the relationship between end- and side-inhibition. We have examined the spatial organization and tuning characteristics of these inhibitory effects by recording extracellularly from single neurons in the cat's striate cortex (Area 17). 2. For each cortical neuron, length and width tuning curves were obtained with the use of rectangular patches of drifting sinusoidal gratings that have variable length and width. Results from 82 cells show that the strengths of end- and side-inhibition tend to be correlated. Most cells that exhibit clear end-inhibition also show a similar degree of side-inhibition. For these cells, the excitatory receptive field is surrounded on all sides by inhibitory zones. Some cells exhibit only end- or side-inhibition, but not both. Data for 28 binocular cells show that length and width tuning curves for the dominant and nondominant eyes tend to be closely matched. 3. We also measured tuning characteristics of end- and side-inhibition. To obtain these data, the excitatory receptive field was stimulated with a grating patch having optimal orientation, spatial frequency, and size, whereas the end- or side-inhibitory regions were stimulated with patches of gratings that had a variable parameter (such as orientation). Results show that end- and side-inhibition tend to be strongest at the orientation and spatial frequency that yield maximal excitation. However, orientation and spatial frequency tuning curves for inhibition are considerably broader than those for excitation, suggesting that inhibition is mediated by a pool of neurons.(ABSTRACT TRUNCATED AT 400 WORDS)

Visual receptive fields and their images in superior colliculus of the cat

1975 ◽  
Vol 38 (2) ◽  
pp. 219-230 ◽  
Author(s):  
J. T. McIlwain
Keyword(s):  
Receptive Field ◽  
Coordinate System ◽  
Coordinate Transformation ◽  
Receptive Fields ◽  
Visual Space ◽  
Oculomotor System ◽  
Large Field ◽  
Visual Receptive Fields ◽  
Visual Orienting ◽  
Area Centralis

1. The receptive fields of collicular neurons in the cat, recorded in a single microelectrode penetration, were not centered on a point in visual space, but nested eccentrically with the smaller fields displaced toward the area centralis. The eccentric nesting was not eliminated by correcting the fields for the tangent screen distortion or by making penetrations normal to the collicular surface in coronal and parasagittal planes. These findings do not support the idea that collicular cells form topographically organized columns oriented normal to the collicular surface. 2. When the receptive fields were plotted in the visual coordinate system of the collicular map, the nesting became much more concentric, suggesting that the eccentric nesting of the receptive fields in visual space was largely a product of the retinotectal coordinate transformation. 3. The profile of a collicular receptive field, plotted in the collicular visual coordinate system is called the receptive-field image. Receptive-field images tended to have oval shapes with the long axis oriented mediolaterally. Clusters of receptive-field images, plotted for single penetrations, appeared similar wherever they occurred in the collicular map, suggesting that a common pattern of neural convergence determines the geometry of the receptive-field images in all parts of the colliculus. 4. The neural substrate of the receptive-field images was examined by tracing the theoretical patterns of neural activity which a point stimulus would produce in the retinotectal system. This analysis suggested that the shape and dimensions of the receptive-field images, and consequently the receptive fields, might be accounted for in large part by the geometry of collicular dendritic fields, the dimensions of the visual receptive fields of afferent fibers, and the retinotectal coordinate transformation. 5. Because it adjusts for the retinotectal distortion of visual space, the receptive-field image may be used to outline the distribution of collicular cells excited by a point stimulus. This makes it possible to show that a point stimulus activates large-field cells in the superficial gray layer over an area of about 2.5 by 1.5 mm in the central parts of the colliculus. It is suggested that such cells may organize the directional signals required by the oculomotor system for visual orienting behavior.

Virtual evolution for visual search in natural images results in behavioral receptive fields with inhibitory surrounds

2009 ◽  
Vol 26 (1) ◽  
pp. 93-108 ◽  
Author(s):  
SHENG ZHANG ◽  
CRAIG K. ABBEY ◽  
MIGUEL P. ECKSTEIN
Keyword(s):  
Receptive Field ◽  
White Noise ◽  
Receptive Fields ◽  
Image Data ◽  
Optimization Method ◽  
Natural Images ◽  
Natural Image ◽  
Natural Scenes ◽  
Linear Combinations ◽  
Data Set

AbstractThe neural mechanisms driving perception and saccades during search use information about the target but are also based on an inhibitory surround not present in the target luminance profile (e.g., Eckstein et al., 2007). Here, we ask whether these inhibitory surrounds might reflect a strategy that the brain has adapted to optimize the search for targets in natural scenes. To test this hypothesis, we sought to estimate the best linear template (behavioral receptive field), built from linear combinations of Gabor channels representing V1 simple cells in search for an additive Gaussian target embedded in natural images. Statistically nonstationary and non-Gaussian properties of natural scenes preclude calculation of the best linear template from analytic expressions and require an iterative optimization method such as a virtual evolution via a genetic algorithm. Evolved linear receptive fields built from linear combinations of Gabor functions include substantial inhibitory surround, larger than those found in humans performing target search in white noise. The inhibitory surrounds were robust to changes in the contrast of the signal, generalized to a larger calibrated natural image data set, and tasks in which the signal occluded other objects in the image. We show that channel nonlinearities can have strong effects on the observed linear behavioral receptive field but preserve the inhibitory surrounds. Together, the results suggest that the apparent suboptimality of inhibitory surrounds in human behavioral receptive fields when searching for a target in white noise might reflect a strategy to optimize detection of signals in natural scenes. Finally, we contend that optimized linear detection of spatially compact signals in natural images might be a new possible hypothesis, distinct from decorrelation of visual input and sparse representations (e.g., Graham et al., 2006), to explain the evolution of center–surround organization of receptive fields in early vision.

scholarly journals Enet Semantic Segmentation Combined with Attention Mechanism

2021 ◽  
Author(s):  
Wei Bai
Keyword(s):  
Feature Fusion ◽  
Receptive Fields ◽  
Design Feature ◽  
Semantic Segmentation ◽  
Attention Mechanism ◽  
Segmentation Algorithm ◽  
Feature Maps ◽  
Model Learning ◽  
Simple Method ◽  
Feature Map

Abstract Image semantic segmentation is one of the core tasks of computer vision. It is widely used in fields such as unmanned driving, medical image processing, geographic information systems and intelligent robots. Aiming at the problem that the existing semantic segmentation algorithm ignores the different channel and location features of the feature map and the simple method when the feature map is fused, this paper designs a semantic segmentation algorithm that combines the attention mechanism. Firstly, dilated convolution is used, and a smaller downsampling factor is used to maintain the resolution of the image and obtain the detailed information of the image. Secondly, the attention mechanism module is introduced to assign weights to different parts of the feature map, which reduces the accuracy loss. The design feature fusion module assigns weights to the feature maps of different receptive fields obtained by the two paths, and merges them together to obtain the final segmentation result. Finally, through experiments, it was verified on the Camvid, Cityscapes and PASCAL VOC2012 datasets. Mean intersection over union (MIoU) and mean pixel accuracy (MPA) are used as metrics. The method in this paper can make up for the loss of accuracy caused by downsampling while ensuring the receptive field and improving the resolution, which can better guide the model learning. And the proposed feature fusion module can better integrate the features of different receptive fields. Therefore, the proposed method can significantly improve the segmentation performance compared to the traditional method.

scholarly journals Emergence of local and global synaptic organization on cortical dendrites

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Jan H. Kirchner ◽  
Julijana Gjorgjieva
Keyword(s):  
Receptive Field ◽  
Spatial Scales ◽  
Back Propagation ◽  
Receptive Fields ◽  
Visual Space ◽  
Synaptic Organization ◽  
Early Postnatal Development ◽  
Cortical Magnification ◽  
Stimulus Features ◽  
Species Specific

AbstractSynaptic inputs on cortical dendrites are organized with remarkable subcellular precision at the micron level. This organization emerges during early postnatal development through patterned spontaneous activity and manifests both locally where nearby synapses are significantly correlated, and globally with distance to the soma. We propose a biophysically motivated synaptic plasticity model to dissect the mechanistic origins of this organization during development and elucidate synaptic clustering of different stimulus features in the adult. Our model captures local clustering of orientation in ferret and receptive field overlap in mouse visual cortex based on the receptive field diameter and the cortical magnification of visual space. Including action potential back-propagation explains branch clustering heterogeneity in the ferret and produces a global retinotopy gradient from soma to dendrite in the mouse. Therefore, by combining activity-dependent synaptic competition and species-specific receptive fields, our framework explains different aspects of synaptic organization regarding stimulus features and spatial scales.

Efficient coding correlates with spatial frequency tuning in a model of V1 receptive field organization

2009 ◽  
Vol 26 (1) ◽  
pp. 21-34 ◽  
Author(s):  
JAN WILTSCHUT ◽  
FRED H. HAMKER
Keyword(s):  
Experimental Data ◽  
Receptive Field ◽  
Visual Processing ◽  
Hebbian Learning ◽  
Frequency Tuning ◽  
Receptive Fields ◽  
Efficient Coding ◽  
Early Visual Processing ◽  
Field Organization ◽  
Receptive Field Organization

AbstractEfficient coding has been proposed to play an essential role in early visual processing. While several approaches used an objective function to optimize a particular aspect of efficient coding, such as the minimization of mutual information or the maximization of sparseness, we here explore how different estimates of efficient coding in a model with nonlinear dynamics and Hebbian learning determine the similarity of model receptive fields to V1 data with respect to spatial tuning. Our simulation results indicate that most measures of efficient coding correlate with the similarity of model receptive field data to V1 data, that is, optimizing the estimate of efficient coding increases the similarity of the model data to experimental data. However, the degree of the correlation varies with the different estimates of efficient coding, and in particular, the variance in the firing pattern of each cell does not predict a similarity of model and experimental data.

scholarly journals Coherent alpha oscillations link current and future receptive fields during saccades

2017 ◽  
Vol 114 (29) ◽  
pp. E5979-E5985 ◽  
Author(s):  
Sujaya Neupane ◽  
Daniel Guitton ◽  
Christopher C. Pack
Keyword(s):  
Receptive Field ◽  
Neural Coding ◽  
Visual Stimulation ◽  
Receptive Fields ◽  
Visual Space ◽  
Spatial Arrangement ◽  
Brain Regions ◽  
Alpha Oscillations ◽  
Area V4 ◽  
Before And After

Oscillations are ubiquitous in the brain, and they can powerfully influence neural coding. In particular, when oscillations at distinct sites are coherent, they provide a means of gating the flow of neural signals between different cortical regions. Coherent oscillations also occur within individual brain regions, although the purpose of this coherence is not well understood. Here, we report that within a single brain region, coherent alpha oscillations link stimulus representations as they change in space and time. Specifically, in primate cortical area V4, alpha coherence links sites that encode the retinal location of a visual stimulus before and after a saccade. These coherence changes exhibit properties similar to those of receptive field remapping, a phenomenon in which individual neurons change their receptive fields according to the metrics of each saccade. In particular, alpha coherence, like remapping, is highly dependent on the saccade vector and the spatial arrangement of current and future receptive fields. Moreover, although visual stimulation plays a modulatory role, it is neither necessary nor sufficient to elicit alpha coherence. Indeed, a similar pattern of coherence is observed even when saccades are made in darkness. Together, these results show that the pattern of alpha coherence across the retinotopic map in V4 matches many of the properties of receptive field remapping. Thus, oscillatory coherence might play a role in constructing the stable representation of visual space that is an essential aspect of conscious perception.

Sign in / Sign up

Export Citation Format

Share Document