scholarly journals The contrasting shape representations that support object recognition in humans and CNNs

2021 ◽  
Author(s):  
Gaurav Malhotra ◽  
Marin Dujmovic ◽  
John Hummel ◽  
Jeffrey S Bowers
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Fundamental Difference ◽  
Human Vision ◽  
Shape Representations ◽  
Enhanced Sensitivity ◽  
Human Object ◽  
Proximal Stimulus ◽  
Human Participants ◽  
Object Features

The success of Convolutional Neural Networks (CNNs) in classifying objects has led to a surge of interest in using these systems to understand human vision. Recent studies have argued that when CNNs are trained in the correct learning environment, they can emulate a key property of human vision -- learning to classify objects based on their shape. While showing a shape-bias is indeed a desirable property for any model of human object recognition, it is unclear whether the resulting shape representations learned by these networks are human-like. We explored this question in the context of a well-known observation from psychology showing that humans encode the shape of objects in terms of relations between object features. To check whether this is also true for the representations of CNNs, we ran a series of simulations where we trained CNNs on datasets of novel shapes and tested them on a set of controlled deformations of these shapes. We found that CNNs do not show any enhanced sensitivity to deformations which alter relations between features, even when explicitly trained on such deformations. This behaviour contrasted with human participants in previous studies as well as in a new experiment. We argue that these results are a consequence of a fundamental difference between how humans and CNNs learn to recognise objects: while CNNs select features that allow them to optimally classify the proximal stimulus, humans select features that they infer to be properties of the distal stimulus. This makes human representations more generalisable to novel contexts and tasks.

A solution to the tag-assignment problem for neural networks

1989 ◽  
Vol 12 (3) ◽  
pp. 381-397 ◽  
Author(s):  
Gary W. Strong ◽  
Bruce A. Whitehead
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Assignment Problem ◽  
Model Object ◽  
Parallel Processes ◽  
Illusory Conjunctions ◽  
Feature Processing ◽  
Human Object ◽  
Processing Activity ◽  
Normal Perception

AbstractPurely parallel neural networks can model object recognition in brief displays – the same conditions under which illusory conjunctions (the incorrect combination of features into perceived objects in a stimulus array) have been demonstrated empirically (Treisman 1986; Treisman & Gelade 1980). Correcting errors of illusory conjunction is the “tag-assignment” problem for a purely parallel processor: the problem of assigning a spatial tag to nonspatial features, feature combinations, and objects. This problem must be solved to model human object recognition over a longer time scale. Our model simulates both the parallel processes that may underlie illusory conjunctions and the serial processes that may solve the tag-assignment problem in normal perception. One component of the model extracts pooled features and another provides attentional tags that correct illusory conjunctions. Our approach addresses two questions: (i) How can objects be identified from simultaneously attended features in a parallel, distributed representation? (ii) How can the spatial selectional requirements of such an attentional process be met by a separation of pathways for spatial and nonspatial processing? Our analysis of these questions yields a neurally plausible simulation of tag assignment based on synchronizing feature processing activity in a spatial focus of attention.

scholarly journals Feature blindness: a challenge for understanding and modelling visual object recognition

2021 ◽  
Author(s):  
Gaurav Malhotra ◽  
Marin Dujmovic ◽  
Jeffrey S Bowers
Keyword(s):  
Object Recognition ◽  
Statistical Inference ◽  
Computational Cost ◽  
Fundamental Difference ◽  
Human Vision ◽  
Visual Object ◽  
Visual Object Recognition ◽  
Global Features ◽  
Diagnostic Features ◽  
Inference Models

A central problem in vision sciences is to understand how humans recognise objects under novel viewing conditions. Recently, statistical inference models such as Convolutional Neural Networks (CNNs) seem to have reproduced this ability by incorporating some architectural constraints of biological vision systems into machine learning models. This has led to the proposal that, like CNNs, humans solve the problem of object recognition by performing a statistical inference over their observations. This hypothesis remains difficult to test as models and humans learn in vastly different environments. Accordingly, any differences in performance could be attributed to the training environment rather than reflect any fundamental difference between statistical inference models and human vision. To overcome these limitations, we conducted a series of experiments and simulations where humans and models had no prior experience with the stimuli. The stimuli contained multiple features that varied in the extent to which they predicted category membership. We observed that human participants frequently ignored features that were highly predictive and clearly visible. Instead, they learned to rely on global features such as colour or shape, even when these features were not the most predictive. When these features were absent they failed to learn the task entirely. By contrast, ideal inference models as well as CNNs always learned to categorise objects based on the most predictive feature. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results highlight a fundamental difference between statistical inference models and humans: while statistical inference models such as CNNs learn most diagnostic features with little regard for the computational cost of learning these features, humans are highly constrained by their limited cognitive capacities which results in a qualitatively different approach to object recognition.

scholarly journals Characterizing the temporal dynamics of object recognition by deep neural networks : role of depth

2017 ◽  
Author(s):  
Kandan Ramakrishnan ◽  
Iris I.A. Groen ◽  
Arnold W.M. Smeulders ◽  
H. Steven Scholte ◽  
Sennay Ghebreab
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Visual Processing ◽  
Temporal Dynamics ◽  
Occipital Cortex ◽  
Stimulus Onset ◽  
Human Vision ◽  
Visual Object ◽  
Visual Object Recognition ◽  
Brain Responses

AbstractConvolutional neural networks (CNNs) have recently emerged as promising models of human vision based on their ability to predict hemodynamic brain responses to visual stimuli measured with functional magnetic resonance imaging (fMRI). However, the degree to which CNNs can predict temporal dynamics of visual object recognition reflected in neural measures with millisecond precision is less understood. Additionally, while deeper CNNs with higher numbers of layers perform better on automated object recognition, it is unclear if this also results into better correlation to brain responses. Here, we examined 1) to what extent CNN layers predict visual evoked responses in the human brain over time and 2) whether deeper CNNs better model brain responses. Specifically, we tested how well CNN architectures with 7 (CNN-7) and 15 (CNN-15) layers predicted electro-encephalography (EEG) responses to several thousands of natural images. Our results show that both CNN architectures correspond to EEG responses in a hierarchical spatio-temporal manner, with lower layers explaining responses early in time at electrodes overlying early visual cortex, and higher layers explaining responses later in time at electrodes overlying lateral-occipital cortex. While the explained variance of neural responses by individual layers did not differ between CNN-7 and CNN-15, combining the representations across layers resulted in improved performance of CNN-15 compared to CNN-7, but only after 150 ms after stimulus-onset. This suggests that CNN representations reflect both early (feed-forward) and late (feedback) stages of visual processing. Overall, our results show that depth of CNNs indeed plays a role in explaining time-resolved EEG responses.

scholarly journals Contextual associations represented both in neural networks and human behavior

2022 ◽  
Author(s):  
Elissa M Aminoff ◽  
Shira Baror ◽  
Eric W Roginek ◽  
Daniel D Leeds
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Human Behavior ◽  
Image Understanding ◽  
Human Vision ◽  
Artificial Vision ◽  
White Background ◽  
Bicycle Helmet ◽  
Contextual Associations

Contextual associations facilitate object recognition in human vision. However, the role of context in artificial vision remains elusive as does the characteristics that humans use to define context. We investigated whether contextually related objects (bicycle-helmet) are represented more similarly in convolutional neural networks (CNNs) used for image understanding than unrelated objects (bicycle-fork). Stimuli were of objects against a white background and consisted of a diverse set of contexts (N=73). CNN representations of contextually related objects were more similar to one another than to unrelated objects across all CNN layers. Critically, the similarity found in CNNs correlated with human behavior across three experiments assessing contextual relatedness, emerging significant only in the later layers. The results demonstrate that context is inherently represented in CNNs as a result of object recognition training, and that the representation in the later layers of the network tap into the contextual regularities that predict human behavior.

scholarly journals Convolutional neural networks trained with a developmental sequence of blurry to clear images reveal core differences between face and object processing

2021 ◽  
Author(s):  
Hojin Jang ◽  
Frank Tong
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Frequency Tuning ◽  
Human Vision ◽  
Object Processing ◽  
Training Images ◽  
Early Experiences ◽  
Spatial Frequencies ◽  
Blurry Vision

Although convolutional neural networks (CNNs) provide a promising model for understanding human vision, most CNNs lack robustness to challenging viewing conditions such as image blur, whereas human vision is much more reliable. Might robustness to blur be attributable to vision during infancy, given that acuity is initially poor but improves considerably over the first several months of life? Here, we evaluated the potential consequences of such early experiences by training CNN models on face and object recognition tasks while gradually reducing the amount of blur applied to the training images. For CNNs trained on blurry to clear faces, we observed sustained robustness to blur, consistent with a recent report by Vogelsang and colleagues (2018). By contrast, CNNs trained with blurry to clear objects failed to retain robustness to blur. Further analyses revealed that the spatial frequency tuning of the two CNNs was profoundly different. The blurry to clear face-trained network successfully retained a preference for low spatial frequencies, whereas the blurry to clear object-trained CNN exhibited a progressive shift toward higher spatial frequencies. Our findings provide novel computational evidence showing how face recognition, unlike object recognition, allows for more holistic processing. Moreover, our results suggest that blurry vision during infancy is insufficient to account for the robustness of adult vision to blurry objects.

scholarly journals Human and bird detection and classification based on Doppler radar spectrograms and vision images using convolutional neural networks

2021 ◽  
Vol 18 (3) ◽  
pp. 172988142110105
Author(s):  
Jnana Sai Abhishek Varma Gokaraju ◽  
Weon Keun Song ◽  
Min-Ho Ka ◽  
Somyot Kaitwanidvilai
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Doppler Radar ◽  
Autonomous Vehicle ◽  
Sampling Point ◽  
Body Segment ◽  
Kinematic Modeling ◽  
Deep Convolutional Neural Networks ◽  
Kinematic Models

The study investigated object detection and classification based on both Doppler radar spectrograms and vision images using two deep convolutional neural networks. The kinematic models for a walking human and a bird flapping its wings were incorporated into MATLAB simulations to create data sets. The dynamic simulator identified the final position of each ellipsoidal body segment taking its rotational motion into consideration in addition to its bulk motion at each sampling point to describe its specific motion naturally. The total motion induced a micro-Doppler effect and created a micro-Doppler signature that varied in response to changes in the input parameters, such as varying body segment size, velocity, and radar location. Micro-Doppler signature identification of the radar signals returned from the target objects that were animated by the simulator required kinematic modeling based on a short-time Fourier transform analysis of the signals. Both You Only Look Once V3 and Inception V3 were used for the detection and classification of the objects with different red, green, blue colors on black or white backgrounds. The results suggested that clear micro-Doppler signature image-based object recognition could be achieved in low-visibility conditions. This feasibility study demonstrated the application possibility of Doppler radar to autonomous vehicle driving as a backup sensor for cameras in darkness. In this study, the first successful attempt of animated kinematic models and their synchronized radar spectrograms to object recognition was made.

Sign in / Sign up

Export Citation Format

Share Document