Representation of adversarial images in deep neural networks and the human brain

AbstractRecent studies showed agreement between how the human brain and neural networks represent objects, suggesting that we might start to understand the underlying computations. However, we know that the human brain is prone to biases at many perceptual and cognitive levels, often shaped by learning history and evolutionary constraints. Here we explore one such bias, namely the bias to perceive animacy, and used the performance of neural networks as a benchmark. We performed an fMRI study that dissociated object appearance (how an object looks like) from object category (animate or inanimate) by constructing a stimulus set that includes animate objects (e.g., a cow), typical inanimate objects (e.g., a mug), and, crucially, inanimate objects that look like the animate objects (e.g., a cow-mug). Behavioral judgments and deep neural networks categorized images mainly by animacy, setting all objects (lookalike and inanimate) apart from the animate ones. In contrast, activity patterns in ventral occipitotemporal cortex (VTC) were strongly biased towards object appearance: animals and lookalikes were similarly represented and separated from the inanimate objects. Furthermore, this bias interfered with proper object identification, such as failing to signal that a cow-mug is a mug. The bias in VTC to represent a lookalike as animate was even present when participants performed a task requiring them to report the lookalikes as inanimate. In conclusion, VTC representations, in contrast to neural networks, fail to veridically represent objects when visual appearance is dissociated from animacy, probably due to a biased processing of visual features typical of animate objects.

Download Full-text

Network attributes describe a similarity between deep neural networks and large scale brain networks

Journal of Complex Networks ◽

10.1093/comnet/cnz044 ◽

2019 ◽

Author(s):

Kosuke Takagi

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Human Brain ◽

Large Scale ◽

Deep Neural Networks ◽

Distribution Model ◽

Common Mechanism ◽

Learning Models ◽

Connection Weight ◽

Wide Range

Abstract Despite the recent success of deep learning models in solving various problems, their ability is still limited compared with human intelligence, which has the flexibility to adapt to a changing environment. To obtain a model which achieves adaptability to a wide range of problems and tasks is a challenging problem. To achieve this, an issue that must be addressed is identification of the similarities and differences between the human brain and deep neural networks. In this article, inspired by the human flexibility which might suggest the existence of a common mechanism allowing solution of different kinds of tasks, we consider a general learning process in neural networks, on which no specific conditions and constraints are imposed. Subsequently, we theoretically show that, according to the learning progress, the network structure converges to the state, which is characterized by a unique distribution model with respect to network quantities such as the connection weight and node strength. Noting that the empirical data indicate that this state emerges in the large scale network in the human brain, we show that the same state can be reproduced in a simple example of deep learning models. Although further research is needed, our findings provide an insight into the common inherent mechanism underlying the human brain and deep learning. Thus, our findings provide suggestions for designing efficient learning algorithms for solving a wide variety of tasks in the future.

Download Full-text

Beyond Core Object Recognition: Recurrent processes account for object recognition under occlusion

10.1101/302034 ◽

2018 ◽

Cited By ~ 8

Author(s):

Karim Rajaei ◽

Yalda Mohsenzadeh ◽

Reza Ebrahimpour ◽

Seyed-Mahdi Khaligh-Razavi

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Object Recognition ◽

Human Brain ◽

Deep Neural Networks ◽

Temporal Dynamics ◽

Computational Modelling ◽

Mechanistic Explanation ◽

The Core ◽

Level Performance

AbstractCore object recognition, the ability to rapidly recognize objects despite variations in their appearance, is largely solved through the feedforward processing of visual information. Deep neural networks are shown to achieve human-level performance in these tasks, and explain the primate brain representation. On the other hand, object recognition under more challenging conditions (i.e. beyond the core recognition problem) is less characterized. One such example is object recognition under occlusion. It is unclear to what extent feedforward and recurrent processes contribute in object recognition under occlusion. Furthermore, we do not know whether the conventional deep neural networks, such as AlexNet, which were shown to be successful in solving core object recognition, can perform similarly well in problems that go beyond the core recognition. Here, we characterize neural dynamics of object recognition under occlusion, using magnetoencephalography (MEG), while participants were presented with images of objects with various levels of occlusion. We provide evidence from multivariate analysis of MEG data, behavioral data, and computational modelling, demonstrating an essential role for recurrent processes in object recognition under occlusion. Furthermore, the computational model with local recurrent connections, used here, suggests a mechanistic explanation of how the human brain might be solving this problem.Author SummaryIn recent years, deep-learning-based computer vision algorithms have been able to achieve human-level performance in several object recognition tasks. This has also contributed in our understanding of how our brain may be solving these recognition tasks. However, object recognition under more challenging conditions, such as occlusion, is less characterized. Temporal dynamics of object recognition under occlusion is largely unknown in the human brain. Furthermore, we do not know if the previously successful deep-learning algorithms can similarly achieve human-level performance in these more challenging object recognition tasks. By linking brain data with behavior, and computational modeling, we characterized temporal dynamics of object recognition under occlusion, and proposed a computational mechanism that explains both behavioral and the neural data in humans. This provides a plausible mechanistic explanation for how our brain might be solving object recognition under more challenging conditions.

Download Full-text

Brain-inspired Multimodal Learning Based on Neural Networks

Brain Science Advances ◽

10.26599/bsa.2018.9050004 ◽

2018 ◽

Vol 4 (1) ◽

pp. 61-72 ◽

Cited By ~ 4

Author(s):

Chang Liu ◽

Fuchun Sun ◽

Bo Zhang

Keyword(s):

Neural Networks ◽

Human Brain ◽

Computational Models ◽

Deep Neural Networks ◽

Brain Research ◽

Visual Signals ◽

Multimodal Learning ◽

Image Captioning ◽

Unified Framework ◽

Learning Tasks

Modern computational models have leveraged biological advances in human brain research. This study addresses the problem of multimodal learning with the help of brain-inspired models. Specifically, a unified multimodal learning architecture is proposed based on deep neural networks, which are inspired by the biology of the visual cortex of the human brain. This unified framework is validated by two practical multimodal learning tasks: image captioning, involving visual and natural language signals, and visual-haptic fusion, involving haptic and visual signals. Extensive experiments are conducted under the framework, and competitive results are achieved.

Download Full-text

Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

NeuroImage ◽

10.1016/j.neuroimage.2016.03.063 ◽

2017 ◽

Vol 153 ◽

pp. 346-358 ◽

Cited By ~ 70

Author(s):

Radoslaw Martin Cichy ◽

Aditya Khosla ◽

Dimitrios Pantazis ◽

Aude Oliva

Keyword(s):

Neural Networks ◽

Human Brain ◽

Deep Neural Networks

Download Full-text

The Representation of Speech and Its Processing in the Human Brain and Deep Neural Networks

Speech and Computer - Lecture Notes in Computer Science ◽

10.1007/978-3-030-26061-3_1 ◽

2019 ◽

pp. 1-8

Author(s):

Odette Scharenborg

Keyword(s):

Neural Networks ◽

Human Brain ◽

Deep Neural Networks

Download Full-text

Shared Spatiotemporal Category Representations in Biological and Artificial Deep Neural Networks

10.1101/225607 ◽

2017 ◽

Author(s):

Michelle R. Greene ◽

Bruce C. Hansen

Keyword(s):

Neural Networks ◽

Human Brain ◽

Cognitive Neuroscience ◽

Deep Neural Networks ◽

Event Related Potentials ◽

Deep Convolutional Neural Networks ◽

Scene Categorization ◽

Time Resolved ◽

Visual Categorization ◽

Related Potentials

AbstractUnderstanding the computational transformations that enable invariant visual categorization is a fundamental challenge in both systems and cognitive neuroscience. Recently developed deep convolutional neural networks (CNNs) perform visual categorization at accuracies that rival humans, providing neuroscientists with the opportunity to interrogate the series of representational transformations that enable categorization in silico. The goal of the current study is to assess the extent to which sequential visual representations built by a CNN map onto those built in the human brain as assessed by high-density, time-resolved event-related potentials (ERPs). We found correspondence both over time and across the scalp: earlier ERP activity was best explained by early CNN layers at all electrodes. Later neural activity was best explained by the later, conceptual layers of the CNN. This effect was especially true both in frontal and right occipital sites. Together, we conclude that deep artificial neural networks trained to perform scene categorization traverse similar representational stages as the human brain. Thus, examining these networks will allow neuroscientists to better understand the transformations that enable invariant visual categorization.

Download Full-text

Recent advances in understanding object recognition in the human brain: deep neural networks, temporal dynamics, and context

F1000Research ◽

10.12688/f1000research.22296.1 ◽

2020 ◽

Vol 9 ◽

pp. 590

Author(s):

Susan G. Wardle ◽

Chris I. Baker

Keyword(s):

Neural Networks ◽

Object Recognition ◽

Human Brain ◽

Spatial Organization ◽

Deep Neural Networks ◽

Temporal Dynamics ◽

Brain Regions ◽

Visual Features ◽

Recent Advances ◽

Ambient Lighting

Object recognition is the ability to identify an object or category based on the combination of visual features observed. It is a remarkable feat of the human brain, given that the patterns of light received by the eye associated with the properties of a given object vary widely with simple changes in viewing angle, ambient lighting, and distance. Furthermore, different exemplars of a specific object category can vary widely in visual appearance, such that successful categorization requires generalization across disparate visual features. In this review, we discuss recent advances in understanding the neural representations underlying object recognition in the human brain. We highlight three current trends in the approach towards this goal within the field of cognitive neuroscience. Firstly, we consider the influence of deep neural networks both as potential models of object vision and in how their representations relate to those in the human brain. Secondly, we review the contribution that time-series neuroimaging methods have made towards understanding the temporal dynamics of object representations beyond their spatial organization within different brain regions. Finally, we argue that an increasing emphasis on the context (both visual and task) within which object recognition occurs has led to a broader conceptualization of what constitutes an object representation for the brain. We conclude by identifying some current challenges facing the experimental pursuit of understanding object recognition and outline some emerging directions that are likely to yield new insight into this complex cognitive process.

Download Full-text