scholarly journals How does the brain represent the semantic content of an image?

2021 ◽  
Author(s):  
Huawei Xu ◽  
Ming Liu ◽  
Delong Zhang
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Semantic Content ◽  
Grounded Cognition ◽  
Similarity Analysis ◽  
Style Transfer ◽  
The Core ◽  
Representational Similarity Analysis ◽  
Visual Areas ◽  
The Brain

Using deep neural networks (DNNs) as models to explore the biological brain is controversial, which is mainly due to the impenetrability of DNNs. Inspired by neural style transfer, we circumvented this problem by using deep features that were given a clear meaning--the representation of the semantic content of an image. Using encoding models and the representational similarity analysis, we quantitatively showed that the deep features which represented the semantic content of an image mainly modulated the activity of voxels in the early visual areas (V1, V2, and V3) and these features were essentially depictive but also propositional. This result is in line with the core viewpoint of the grounded cognition to some extent, which suggested that the representation of information in our brain is essentially depictive and can implement symbolic functions naturally.

scholarly journals Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

2020 ◽  
Author(s):  
Soma Nonaka ◽  
Kei Majima ◽  
Shuntaro C. Aoki ◽  
Yukiyasu Kamitani
Keyword(s):  
Neural Networks ◽  
Image Recognition ◽  
High Performance ◽  
Deep Neural Networks ◽  
Recognition Performance ◽  
Brain Activity ◽  
Spatial Integration ◽  
Hierarchical Processing ◽  
Visual Areas ◽  
The Brain

SummaryAchievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pretrained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.

scholarly journals Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 39
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Objective Function ◽  
Learning Process ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The Core ◽  
Learning Tasks ◽  
Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

scholarly journals Lessons From Deep Neural Networks for Studying the Coding Principles of Biological Neural Networks

2021 ◽  
Vol 14 ◽  
Author(s):  
Hyojin Bae ◽  
Sang Jeong Kim ◽  
Chang-Eop Kim
Keyword(s):  
Neural Networks ◽  
Neural Coding ◽  
Deep Neural Networks ◽  
Neural Response ◽  
Feature Representation ◽  
Network Feature ◽  
Biological Neural Networks ◽  
Stimulus Features ◽  
The Comparative Study ◽  
The Brain

One of the central goals in systems neuroscience is to understand how information is encoded in the brain, and the standard approach is to identify the relation between a stimulus and a neural response. However, the feature of a stimulus is typically defined by the researcher's hypothesis, which may cause biases in the research conclusion. To demonstrate potential biases, we simulate four likely scenarios using deep neural networks trained on the image classification dataset CIFAR-10 and demonstrate the possibility of selecting suboptimal/irrelevant features or overestimating the network feature representation/noise correlation. Additionally, we present studies investigating neural coding principles in biological neural networks to which our points can be applied. This study aims to not only highlight the importance of careful assumptions and interpretations regarding the neural response to stimulus features but also suggest that the comparative study between deep and biological neural networks from the perspective of machine learning can be an effective strategy for understanding the coding principles of the brain.

scholarly journals BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization

2021 ◽  
Vol 15 ◽  
Author(s):  
Yuyang Gao ◽  
Giorgio A. Ascoli ◽  
Liang Zhao
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Model Performance ◽  
Semantic Content ◽  
Population Coding ◽  
Crucial Issue ◽  
Synaptic Interactions ◽  
Training Samples ◽  
Efficient Learning ◽  
Neuronal Assembly

Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization1 to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited.

scholarly journals Inter-subject representational similarity analysis reveals individual variations in affective experience when watching erotic movies

2019 ◽  
Author(s):  
Pin-Hao A. Chen ◽  
Eshin Jolly ◽  
Jin Hyun Cheong ◽  
Luke J. Chang
Keyword(s):  
Executive Control ◽  
The Self ◽  
Self Control ◽  
Similarity Analysis ◽  
Affective Experience ◽  
Representational Similarity Analysis ◽  
Neural Processes ◽  
Individual Variations ◽  
Affective Experiences ◽  
The Brain

AbstractWe spend much of our life pursuing or avoiding affective experiences. However, surprisingly little is known about how these experiences are represented in the brain and if they are shared across individuals. Here, we explore variations in the construction of an affective experience during a naturalistic viewing paradigm based on subjective preferences in sociosexual desire and self-control using intersubject representational similarity analysis (IS-RSA). We found that when watching erotic movies, intersubject variations in sociosexual desire preferences of 26 heterosexual males were associated with similarly structured fluctuations in the cortico-striatal reward, default mode, and mentalizing networks. In contrast, variations in the self-control preferences were associated with shared dynamics in the fronto-parietal executive control and cingulo-insula salience networks. Importantly, these results were specific to the affective experience, as we did not observe any relationship with variation in preferences when individuals watched neutral movies. Moreover, these results appear to require multivariate representations of preferences as we did not observe any significant results using single summary scores. Our findings demonstrate that multidimensional variations in individual preferences can be used to uncover unique dimensions of an affective experience, and that IS-RSA can provide new insights into the neural processes underlying psychological experiences elicited through naturalistic experimental designs.

scholarly journals Comparing the functional structure of neural networks from representational similarity analysis with those from functional connectivity and univariate analyses

2018 ◽  
Author(s):  
Ineke Pillet ◽  
Hans Op de Beeck ◽  
Haemy Lee Masson
Keyword(s):  
Neural Networks ◽  
Functional Connectivity ◽  
Univariate Analysis ◽  
Brain Regions ◽  
Functional Structure ◽  
Functional Networks ◽  
Similarity Analysis ◽  
Network Nodes ◽  
Representational Similarity Analysis ◽  
The Neural Network

AbstractThe invention of representational similarity analysis (RSA, following multi-voxel pattern analysis (MVPA)) has allowed cognitive neuroscientists to identify the representational structure of multiple brain regions, moving beyond functional localization. By comparing these structures, cognitive neuroscientists can characterize how brain areas form functional networks. Univariate analysis (UNIVAR) and functional connectivity analysis (FCA) are two other popular methods to identify the functional structure of brain networks. Despite their popularity, few studies have examined the relationship between the structure of the networks from RSA with those from UNIVAR and FCA. Thus, the aim of the current study is to examine the similarities between neural networks derived from RSA with those from UNIVAR and FCA to explore how these methods relate to each other. We analyzed the data of a previously published study with the three methods and compared the results by performing (partial) correlation and multiple regression analysis. Our findings reveal that neural networks resulting from RSA, UNIVAR, and FCA methods are highly similar to each other even after ruling out the effect of anatomical proximity between the network nodes. Nevertheless, the neural network from each method shows idiosyncratic structure that cannot be explained by any of the other methods. Thus, we conclude that the RSA, UNIVAR and FCA methods provide similar but not identical information on how brain regions are organized in functional networks.

scholarly journals Interactive Artistic Multi-style Transfer

2021 ◽  
Vol 14 (1) ◽  
Author(s):  
Xiaohui Wang ◽  
Yiran Lyu ◽  
Junfeng Huang ◽  
Ziying Wang ◽  
Jingyan Qin
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Deep Neural Networks ◽  
Subjective Evaluation ◽  
User Interaction ◽  
Evaluation Framework ◽  
Transfer System ◽  
Style Transfer ◽  
Artistic Style ◽  
Artistic Styles

AbstractArtistic style transfer is to render an image in the style of another image, which is a challenge problem in both image processing and arts. Deep neural networks are adopted to artistic style transfer and achieve remarkable success, such as AdaIN (adaptive instance normalization), WCT (whitening and coloring transforms), MST (multimodal style transfer), and SEMST (structure-emphasized multimodal style transfer). These algorithms modify the content image as a whole using only one style and one algorithm, which is easy to cause the foreground and background to be blurred together. In this paper, an iterative artistic multi-style transfer system is built to edit the image with multiple styles by flexible user interaction. First, a subjective evaluation experiment with art professionals is conducted to build an open evaluation framework for style transfer, including the universal evaluation questions and personalized answers for ten typical artistic styles. Then, we propose the interactive artistic multi-style transfer system, in which an interactive image crop tool is designed to cut a content image into several parts. For each part, users select a style image and an algorithm from AdaIN, WCT, MST, and SEMST by referring to the characteristics of styles and algorithms summarized by the evaluation experiments. To obtain richer results, the system provides a semantic-based parameter adjustment mode and the function of preserving colors of content image. Finally, case studies show the effectiveness and flexibility of the system.

scholarly journals Understanding a Deep Learning Technique through a Neuromorphic System a Case Study with SpiNNaker Neuromorphic Platform

2018 ◽  
Vol 164 ◽  
pp. 01015
Author(s):  
Indar Sugiarto ◽  
Felix Pasila
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Cellular Level ◽  
System A ◽  
Learning Technique ◽  
Neuromorphic System ◽  
The Brain ◽  
Potential Use

Deep learning (DL) has been considered as a breakthrough technique in the field of artificial intelligence and machine learning. Conceptually, it relies on a many-layer network that exhibits a hierarchically non-linear processing capability. Some DL architectures such as deep neural networks, deep belief networks and recurrent neural networks have been developed and applied to many fields with incredible results, even comparable to human intelligence. However, many researchers are still sceptical about its true capability: can the intelligence demonstrated by deep learning technique be applied for general tasks? This question motivates the emergence of another research discipline: neuromorphic computing (NC). In NC, researchers try to identify the most fundamental ingredients that construct intelligence behaviour produced by the brain itself. To achieve this, neuromorphic systems are developed to mimic the brain functionality down to cellular level. In this paper, a neuromorphic platform called SpiNNaker is described and evaluated in order to understand its potential use as a platform for a deep learning approach. This paper is a literature review that contains comparative study on algorithms that have been implemented in SpiNNaker.

The architecture challenge: Future artificial-intelligence systems will require sophisticated architectures, and knowledge of the brain might guide their construction

2017 ◽  
Vol 40 ◽  
Author(s):  
Gianluca Baldassarre ◽  
Vieri Giuliano Santucci ◽  
Emilio Cartoni ◽  
Daniele Caligiore
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Human Cognition ◽  
Important Means ◽  
The Brain ◽  
Artificial Intelligence Systems

AbstractIn this commentary, we highlight a crucial challenge posed by the proposal of Lake et al. to introduce key elements of human cognition into deep neural networks and future artificial-intelligence systems: the need to design effective sophisticated architectures. We propose that looking at the brain is an important means of facing this great challenge.

scholarly journals Beyond Core Object Recognition: Recurrent processes account for object recognition under occlusion

2018 ◽  
Author(s):  
Karim Rajaei ◽  
Yalda Mohsenzadeh ◽  
Reza Ebrahimpour ◽  
Seyed-Mahdi Khaligh-Razavi
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Object Recognition ◽  
Human Brain ◽  
Deep Neural Networks ◽  
Temporal Dynamics ◽  
Computational Modelling ◽  
Mechanistic Explanation ◽  
The Core ◽  
Level Performance

AbstractCore object recognition, the ability to rapidly recognize objects despite variations in their appearance, is largely solved through the feedforward processing of visual information. Deep neural networks are shown to achieve human-level performance in these tasks, and explain the primate brain representation. On the other hand, object recognition under more challenging conditions (i.e. beyond the core recognition problem) is less characterized. One such example is object recognition under occlusion. It is unclear to what extent feedforward and recurrent processes contribute in object recognition under occlusion. Furthermore, we do not know whether the conventional deep neural networks, such as AlexNet, which were shown to be successful in solving core object recognition, can perform similarly well in problems that go beyond the core recognition. Here, we characterize neural dynamics of object recognition under occlusion, using magnetoencephalography (MEG), while participants were presented with images of objects with various levels of occlusion. We provide evidence from multivariate analysis of MEG data, behavioral data, and computational modelling, demonstrating an essential role for recurrent processes in object recognition under occlusion. Furthermore, the computational model with local recurrent connections, used here, suggests a mechanistic explanation of how the human brain might be solving this problem.Author SummaryIn recent years, deep-learning-based computer vision algorithms have been able to achieve human-level performance in several object recognition tasks. This has also contributed in our understanding of how our brain may be solving these recognition tasks. However, object recognition under more challenging conditions, such as occlusion, is less characterized. Temporal dynamics of object recognition under occlusion is largely unknown in the human brain. Furthermore, we do not know if the previously successful deep-learning algorithms can similarly achieve human-level performance in these more challenging object recognition tasks. By linking brain data with behavior, and computational modeling, we characterized temporal dynamics of object recognition under occlusion, and proposed a computational mechanism that explains both behavioral and the neural data in humans. This provides a plausible mechanistic explanation for how our brain might be solving object recognition under more challenging conditions.

Sign in / Sign up

Export Citation Format

Share Document