Not so fast: Limited validity of deep convolutional neural networks as in silico models for human naturalistic face processing

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The relationships between internal representations learned by DCNNs and those of the primate face processing system are not well understood, especially in naturalistic settings. We developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces) and used representational similarity analysis to investigate how well the representations learned by high-performing DCNNs match human brain representations across the entire distributed face processing system. DCNN representational geometries were strikingly consistent across diverse architectures and captured meaningful variance among faces. Similarly, representational geometries throughout the human face network were highly consistent across subjects. Nonetheless, correlations between DCNN and neural representations were very weak overall—DCNNs captured 3% of variance in the neural representational geometries at best. Intermediate DCNN layers better matched visual and face-selective cortices than the final fully-connected layers. Behavioral ratings of face similarity were highly correlated with intermediate layers of DCNNs, but also failed to capture representational geometry in the human brain. Our results suggest that the correspondence between intermediate DCNN layers and neural representations of naturalistic human face processing is weak at best, and diverges even further in the later fully-connected layers. This poor correspondence can be attributed, at least in part, to the dynamic and cognitive information that plays an essential role in human face processing but is not modeled by DCNNs. These mismatches indicate that current DCNNs have limited validity as in silico models of dynamic, naturalistic face processing in humans.

Download Full-text

Emergence of Visual Center-Periphery Spatial Organization in Deep Convolutional Neural Networks

10.1101/2020.02.19.956748 ◽

2020 ◽

Author(s):

Yalda Mohsenzadeh ◽

Caitlin Mullin ◽

Benjamin Lahner ◽

Aude Oliva

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Spatial Organization ◽

Brain Regions ◽

Deep Convolutional Neural Networks ◽

Visual Center ◽

Neural Representations ◽

Parahippocampal Cortex ◽

Cortical Regions

AbstractResearch at the intersection of computer vision and neuroscience has revealed hierarchical correspondence between layers of deep convolutional neural networks (DCNNs) and cascade of regions along human ventral visual cortex. Recently, studies have uncovered emergence of human interpretable concepts within DCNNs layers trained to identify visual objects and scenes. Here, we asked whether an artificial neural network (with convolutional structure) trained for visual categorization would demonstrate spatial correspondences with human brain regions showing central/peripheral biases. Using representational similarity analysis, we compared activations of convolutional layers of a DCNN trained for object and scene categorization with neural representations in human brain visual regions. Results reveal a brain-like topographical organization in the layers of the DCNN, such that activations of layer-units with central-bias were associated with brain regions with foveal tendencies (e.g. fusiform gyrus), and activations of layer-units with selectivity for image backgrounds were associated with cortical regions showing peripheral preference (e.g. parahippocampal cortex). The emergence of a categorical topographical correspondence between DCNNs and brain regions suggests these models are a good approximation of the perceptual representation generated by biological neural networks.

Download Full-text

Dissociable Neural Representations of Adversarially Perturbed Images in Convolutional Neural Networks and the Human Brain

Frontiers in Neuroinformatics ◽

10.3389/fninf.2021.677925 ◽

2021 ◽

Vol 15 ◽

Author(s):

Chi Zhang ◽

Xiao-Han Duan ◽

Lin-Yuan Wang ◽

Yong-Li Li ◽

Bin Yan ◽

...

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Brain Activity ◽

Neural Responses ◽

Neural Representations ◽

Artificial Neurons ◽

Visual Areas ◽

Two Systems ◽

Visual Tasks

Despite the remarkable similarities between convolutional neural networks (CNN) and the human brain, CNNs still fall behind humans in many visual tasks, indicating that there still exist considerable differences between the two systems. Here, we leverage adversarial noise (AN) and adversarial interference (AI) images to quantify the consistency between neural representations and perceptual outcomes in the two systems. Humans can successfully recognize AI images as the same categories as their corresponding regular images but perceive AN images as meaningless noise. In contrast, CNNs can recognize AN images similar as corresponding regular images but classify AI images into wrong categories with surprisingly high confidence. We use functional magnetic resonance imaging to measure brain activity evoked by regular and adversarial images in the human brain, and compare it to the activity of artificial neurons in a prototypical CNN—AlexNet. In the human brain, we find that the representational similarity between regular and adversarial images largely echoes their perceptual similarity in all early visual areas. In AlexNet, however, the neural representations of adversarial images are inconsistent with network outputs in all intermediate processing layers, providing no neural foundations for the similarities at the perceptual level. Furthermore, we show that voxel-encoding models trained on regular images can successfully generalize to the neural responses to AI images but not AN images. These remarkable differences between the human brain and AlexNet in representation-perception association suggest that future CNNs should emulate both behavior and the internal neural presentations of the human brain.

Download Full-text

Human Face Sentiment Classification Using Synthetic Sentiment Images with Deep Convolutional Neural Networks

2019 International Conference on Machine Learning and Cybernetics (ICMLC) ◽

10.1109/icmlc48188.2019.8949240 ◽

2019 ◽

Author(s):

Chen-Chun Huang ◽

Yi-Leh Wu ◽

Cheng-Yuan Tang

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Sentiment Classification ◽

Human Face ◽

Deep Convolutional Neural Networks

Download Full-text

Inferring Skin Lesion Segmentation With Fully Connected CRFs Based on Multiple Deep Convolutional Neural Networks

IEEE Access ◽

10.1109/access.2020.3014787 ◽

2020 ◽

Vol 8 ◽

pp. 144246-144258

Author(s):

Yuming Qiu ◽

Jingyong Cai ◽

Xiaolin Qin ◽

Ju Zhang

Keyword(s):

Neural Networks ◽

Skin Lesion ◽

Convolutional Neural Networks ◽

Lesion Segmentation ◽

Deep Convolutional Neural Networks ◽

Fully Connected

Download Full-text

Kayu7net: Identifikasi dan Evaluasi F-Measure Citra Kayu berbasis Deep Convolutional Neural Network (DCNN)

Jurnal Teknologi Informasi dan Ilmu Komputer ◽

10.25126/jtiik.2020712663 ◽

2020 ◽

Vol 7 (6) ◽

pp. 1089

Author(s):

Iwan Muhammad Erwin ◽

Risnandar Risnandar ◽

Esa Prakarsa ◽

Bambang Sugiarto

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Computer Vision ◽

Convolutional Neural Networks ◽

Iteration Process ◽

Batch Size ◽

Wood Identification ◽

Deep Convolutional Neural Networks ◽

Fully Connected ◽

F Measure

Identifikasi kayu salah satu kebutuhan untuk mendukung pemerintah dan kalangan bisnis kayu untuk melakukan perdagangan kayu secara legal. Keahlian khusus dan waktu yang cukup dibutuhkan untuk memproses identifikasi kayu di laboratorium. Beberapa metodologi penelitian sebelumnya, proses identifikasi kayu masih dengan cara menggabungkan sistem manual menggunakan anatomi DNA kayu. Sedangkan penggunaan sistem komputer diperoleh dari citra penampamg melintang kayu secara proses mikrokopis dan makroskopis. Saat ini, telah berkembang teknologi computer vision dan machine learning untuk mengidentifikasi berbagai jenis objek, salah satunya citra kayu. Penelitian ini berkontribusi dalam mengklasifikasi beberapa spesies kayu yang diperdagangkan menggunakan Deep Convolutional Neural Networks (DCNN). Kebaruan penelitian ini terletak pada arsitektur DCNN yang bernama Kayu7Net. Arsitektur Kayu7Net yang diusulkan memiliki tiga lapisan konvolusi terhadap tujuh spesies dataset citra kayu. Pengujian dengan merubah citra input menjadi berukuran 600×600, 300×300, dan 128×128 piksel serta masing-masing diulang pada epoch 50 dan 100. DCNN yang diusulkan menggunakan fungsi aktivasi ReLU dengan batch size 32. ReLU bersifat lebih konvergen dan cepat saat proses iterasi. Sedangkan Fully-Connected (FC) berjumlah 4 lapisan akan menghasilkan proses training yang lebih efisien. Hasil eksperimen memperlihatkan bahwa Kayu7Net yang diusulkan memiliki nilai akurasi sebesar 95,54%, precision sebesar 95,99%, recall sebesar 95,54%, specificity sebesar 99,26% dan terakhir, nilai F-measure sebesar 95,46%. Hasil ini menunjukkan bahwa arsitektur Kayu7Net lebih unggul sebesar 1,49% pada akurasi, 2,49% pada precision, dan 5,26% pada specificity dibandingkan penelitian sebelumnya. AbstractWood identification is one of the needs to support the government and the wood business community for a legally wood trading system. Special expertise and sufficient time are needed to process wood identification in the laboratory. Some previous research works show that the process of identifying wood combines a manual system using a wood DNA anatomy. While, the use of a computer system is obtained from the wood image of microscopic and macroscopic process. Recently, the latest technology has developed by using the machine learning and computer vision to identify many objects, the one of them is wood image. This research contributes to classify several the traded wood species by using Deep Convolutional Neural Networks (DCNN). The novelty of this research is in the DCNN architecture, namely Kayu7Net. The proposed of Kayu7Net Architecture has three convolution layers of the seven species wood image dataset. The testing changes the wood image input to 600×600, 300×300, and 128×128 pixel, respectively, and each of them repeated until 50 and 100 epoches, respectively. The proposed DCNN uses the ReLU activation function and batch size 32. The ReLU is more convergent and faster during the iteration process. Whereas, the 4 layers of Fully-Connected (FC) will produce a more efficient training process. The experimental results show that the proposed Kayu7Net has an accuracy value of 95.54%, a precision of 95.99%, a recall of 95.54%, a specificity of 99.26% and finally, an F-measure value of 95.46%. These results indicate that Kayu7Net is superior by 1.49% of accuracy, 2.49% of precision, and 5.26% of specificity compared to the previous work.

Download Full-text

Deep distributed convolutional neural networks: Universality

Analysis and Applications ◽

10.1142/s0219530518500124 ◽

2018 ◽

Vol 16 (06) ◽

pp. 895-919 ◽

Cited By ~ 19

Author(s):

Ding-Xuan Zhou

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Mathematical Foundation ◽

Deep Convolutional Neural Networks ◽

The Family ◽

Fully Connected

Deep learning based on structured deep neural networks has provided powerful applications in various fields. The structures imposed on the deep neural networks are crucial, which makes deep learning essentially different from classical schemes based on fully connected neural networks. One of the commonly used deep neural network structures is generated by convolutions. The produced deep learning algorithms form the family of deep convolutional neural networks. Despite of their power in some practical domains, little is known about the mathematical foundation of deep convolutional neural networks such as universality of approximation. In this paper, we propose a family of new structured deep neural networks: deep distributed convolutional neural networks. We show that these deep neural networks have the same order of computational complexity as the deep convolutional neural networks, and we prove their universality of approximation. Some ideas of our analysis are from ridge approximation, wavelets, and learning theory.

Download Full-text

Learning Class-Specific Features with Class Regularization for Videos

Applied Sciences ◽

10.3390/app10186241 ◽

2020 ◽

Vol 10 (18) ◽

pp. 6241

Author(s):

Alexandros Stergiou ◽

Ronald Poppe ◽

Remco C. Veltkamp

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

General Feature ◽

Deep Convolutional Neural Networks ◽

Systematic Improvement ◽

Novel Method ◽

Different Depths ◽

Explicit Correlation ◽

Fully Connected ◽

The Given

One of the main principles of Deep Convolutional Neural Networks (CNNs) is the extraction of useful features through a hierarchy of kernels operations. The kernels are not explicitly tailored to address specific target classes but are rather optimized as general feature extractors. Distinction between classes is typically left until the very last fully-connected layers. Consequently, variances between classes that are relatively similar are treated the same way as variations between classes that exhibit great dissimilarities. In order to directly address this problem, we introduce Class Regularization, a novel method that can regularize feature map activations based on the classes of the examples used. Essentially, we amplify or suppress activations based on an educated guess of the given class. We can apply this step to each minibatch of activation maps, at different depths in the network. We demonstrate that this improves feature search during training, leading to systematic improvement gains on the Kinetics, UCF-101, and HMDB-51 datasets. Moreover, Class Regularization establishes an explicit correlation between features and class, which makes it a perfect tool to visualize class-specific features at various network depths.

Download Full-text