P4-037: THE USE OF CONVOLUTIONAL NEURAL NETWORKS TO QUANTIFY AMYLOID PLAQUES IN POSTMORTEM HUMAN BRAIN

Despite the remarkable similarities between convolutional neural networks (CNN) and the human brain, CNNs still fall behind humans in many visual tasks, indicating that there still exist considerable differences between the two systems. Here, we leverage adversarial noise (AN) and adversarial interference (AI) images to quantify the consistency between neural representations and perceptual outcomes in the two systems. Humans can successfully recognize AI images as the same categories as their corresponding regular images but perceive AN images as meaningless noise. In contrast, CNNs can recognize AN images similar as corresponding regular images but classify AI images into wrong categories with surprisingly high confidence. We use functional magnetic resonance imaging to measure brain activity evoked by regular and adversarial images in the human brain, and compare it to the activity of artificial neurons in a prototypical CNN—AlexNet. In the human brain, we find that the representational similarity between regular and adversarial images largely echoes their perceptual similarity in all early visual areas. In AlexNet, however, the neural representations of adversarial images are inconsistent with network outputs in all intermediate processing layers, providing no neural foundations for the similarities at the perceptual level. Furthermore, we show that voxel-encoding models trained on regular images can successfully generalize to the neural responses to AI images but not AN images. These remarkable differences between the human brain and AlexNet in representation-perception association suggest that future CNNs should emulate both behavior and the internal neural presentations of the human brain.

Download Full-text

Application Research of Deep Convolutional Neural Network in Computer Vision

Journal of Networking and Telecommunications ◽

10.18282/jnt.v2i2.886 ◽

2020 ◽

Vol 2 (2) ◽

pp. 23

Author(s):

Lei Wang

Keyword(s):

Neural Network ◽

Neural Networks ◽

Computer Vision ◽

Face Recognition ◽

Human Brain ◽

Image Classification ◽

Convolutional Neural Networks ◽

Convolution Neural Network ◽

Data Set ◽

Deep Convolution Neural Network

<p>As an important research achievement in the field of brain like computing, deep convolution neural network has been widely used in many fields such as computer vision, natural language processing, information retrieval, speech recognition, semantic understanding and so on. It has set off a wave of neural network research in industry and academia and promoted the development of artificial intelligence. At present, the deep convolution neural network mainly simulates the complex hierarchical cognitive laws of the human brain by increasing the number of layers of the network, using a larger training data set, and improving the network structure or training learning algorithm of the existing neural network, so as to narrow the gap with the visual system of the human brain and enable the machine to acquire the capability of "abstract concepts". Deep convolution neural network has achieved great success in many computer vision tasks such as image classification, target detection, face recognition, pedestrian recognition, etc. Firstly, this paper reviews the development history of convolutional neural networks. Then, the working principle of the deep convolution neural network is analyzed in detail. Then, this paper mainly introduces the representative achievements of convolution neural network from the following two aspects, and shows the improvement effect of various technical methods on image classification accuracy through examples. From the aspect of adding network layers, the structures of classical convolutional neural networks such as AlexNet, ZF-Net, VGG, GoogLeNet and ResNet are discussed and analyzed. From the aspect of increasing the size of data set, the difficulties of manually adding labeled samples and the effect of using data amplification technology on improving the performance of neural network are introduced. This paper focuses on the latest research progress of convolution neural network in image classification and face recognition. Finally, the problems and challenges to be solved in future brain-like intelligence research based on deep convolution neural network are proposed.</p>

Download Full-text

Not so fast: Limited validity of deep convolutional neural networks as in silico models for human naturalistic face processing

10.1101/2021.11.17.469009 ◽

2021 ◽

Author(s):

Guo Jiahui ◽

Ma Feilong ◽

Matteo Visconti di Oleggio Castello ◽

Samuel A Nastase ◽

James V Haxby ◽

...

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Face Processing ◽

In Silico ◽

Processing System ◽

Human Face ◽

Deep Convolutional Neural Networks ◽

Neural Representations ◽

Fully Connected

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The relationships between internal representations learned by DCNNs and those of the primate face processing system are not well understood, especially in naturalistic settings. We developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces) and used representational similarity analysis to investigate how well the representations learned by high-performing DCNNs match human brain representations across the entire distributed face processing system. DCNN representational geometries were strikingly consistent across diverse architectures and captured meaningful variance among faces. Similarly, representational geometries throughout the human face network were highly consistent across subjects. Nonetheless, correlations between DCNN and neural representations were very weak overall—DCNNs captured 3% of variance in the neural representational geometries at best. Intermediate DCNN layers better matched visual and face-selective cortices than the final fully-connected layers. Behavioral ratings of face similarity were highly correlated with intermediate layers of DCNNs, but also failed to capture representational geometry in the human brain. Our results suggest that the correspondence between intermediate DCNN layers and neural representations of naturalistic human face processing is weak at best, and diverges even further in the later fully-connected layers. This poor correspondence can be attributed, at least in part, to the dynamic and cognitive information that plays an essential role in human face processing but is not modeled by DCNNs. These mismatches indicate that current DCNNs have limited validity as in silico models of dynamic, naturalistic face processing in humans.

Download Full-text

Deriving tumor detection models using convolutional neural networks from MRI of human brain scans

International Journal of Information Technology ◽

10.1007/s41870-020-00438-4 ◽

2020 ◽

Vol 12 (2) ◽

pp. 403-408

Author(s):

T. Kalaiselvi ◽

S. T. Padmapriya ◽

P. Sriramakrishnan ◽

Karuppanagounder Somasundaram

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Tumor Detection ◽

Brain Scans

Download Full-text

Limits to visual representational correspondence between convolutional neural networks and the human brain

Nature Communications ◽

10.1038/s41467-021-22244-7 ◽

2021 ◽

Vol 12 (1) ◽

Author(s):

Yaoda Xu ◽

Maryam Vaziri-Pashkam

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Real World ◽

Visual Information ◽

Visual Representations ◽

Human Vision ◽

Brain Responses ◽

General Correspondence ◽

The Brain

AbstractConvolutional neural networks (CNNs) are increasingly used to model human vision due to their high object categorization capabilities and general correspondence with human brain responses. Here we evaluate the performance of 14 different CNNs compared with human fMRI responses to natural and artificial images using representational similarity analysis. Despite the presence of some CNN-brain correspondence and CNNs’ impressive ability to fully capture lower level visual representation of real-world objects, we show that CNNs do not fully capture higher level visual representations of real-world objects, nor those of artificial objects, either at lower or higher levels of visual representations. The latter is particularly critical, as the processing of both real-world and artificial visual stimuli engages the same neural circuits. We report similar results regardless of differences in CNN architecture, training, or the presence of recurrent processing. This indicates some fundamental differences exist in how the brain and CNNs represent visual information.

Download Full-text

Emergence of Visual Center-Periphery Spatial Organization in Deep Convolutional Neural Networks

10.1101/2020.02.19.956748 ◽

2020 ◽

Author(s):

Yalda Mohsenzadeh ◽

Caitlin Mullin ◽

Benjamin Lahner ◽

Aude Oliva

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Spatial Organization ◽

Brain Regions ◽

Deep Convolutional Neural Networks ◽

Visual Center ◽

Neural Representations ◽

Parahippocampal Cortex ◽

Cortical Regions

AbstractResearch at the intersection of computer vision and neuroscience has revealed hierarchical correspondence between layers of deep convolutional neural networks (DCNNs) and cascade of regions along human ventral visual cortex. Recently, studies have uncovered emergence of human interpretable concepts within DCNNs layers trained to identify visual objects and scenes. Here, we asked whether an artificial neural network (with convolutional structure) trained for visual categorization would demonstrate spatial correspondences with human brain regions showing central/peripheral biases. Using representational similarity analysis, we compared activations of convolutional layers of a DCNN trained for object and scene categorization with neural representations in human brain visual regions. Results reveal a brain-like topographical organization in the layers of the DCNN, such that activations of layer-units with central-bias were associated with brain regions with foveal tendencies (e.g. fusiform gyrus), and activations of layer-units with selectivity for image backgrounds were associated with cortical regions showing peripheral preference (e.g. parahippocampal cortex). The emergence of a categorical topographical correspondence between DCNNs and brain regions suggests these models are a good approximation of the perceptual representation generated by biological neural networks.

Download Full-text

Brain-Mediated Transfer Learning of Convolutional Neural Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.5974 ◽

2020 ◽

Vol 34 (04) ◽

pp. 5281-5288 ◽

Cited By ~ 2

Author(s):

Satoshi Nishida ◽

Yusuke Nakano ◽

Antoine Blanc ◽

Naoya Maeda ◽

Masataka Kado ◽

...

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Human Brain ◽

Transfer Learning ◽

Convolutional Neural Networks ◽

Individual Variability ◽

Generalization Ability ◽

Feature Representations ◽

And Behavior ◽

The Brain

The human brain can effectively learn a new task from a small number of samples, which indicates that the brain can transfer its prior knowledge to solve tasks in different domains. This function is analogous to transfer learning (TL) in the field of machine learning. TL uses a well-trained feature space in a specific task domain to improve performance in new tasks with insufficient training data. TL with rich feature representations, such as features of convolutional neural networks (CNNs), shows high generalization ability across different task domains. However, such TL is still insufficient in making machine learning attain generalization ability comparable to that of the human brain. To examine if the internal representation of the brain could be used to achieve more efficient TL, we introduce a method for TL mediated by human brains. Our method transforms feature representations of audiovisual inputs in CNNs into those in activation patterns of individual brains via their association learned ahead using measured brain responses. Then, to estimate labels reflecting human cognition and behavior induced by the audiovisual inputs, the transformed representations are used for TL. We demonstrate that our brain-mediated TL (BTL) shows higher performance in the label estimation than the standard TL. In addition, we illustrate that the estimations mediated by different brains vary from brain to brain, and the variability reflects the individual variability in perception. Thus, our BTL provides a framework to improve the generalization ability of machine-learning feature representations and enable machine learning to estimate human-like cognition and behavior, including individual variability.

Download Full-text