Exploring Internal Representations of Deep Neural Networks

2019 ◽  
pp. 119-138
Author(s):  
Jérémie Despraz ◽  
Stéphane Gomez ◽  
Héctor F. Satizábal ◽  
Carlos Andrés Peña-Reyes
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Internal Representations

scholarly journals DNNBrain: A Unifying Toolbox for Mapping Deep Neural Networks and Brains

2020 ◽  
Vol 14 ◽  
Author(s):  
Xiayu Chen ◽  
Ming Zhou ◽  
Zhengxin Gong ◽  
Wei Xu ◽  
Xingyu Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Cognitive Neuroscience ◽  
Deep Neural Networks ◽  
Learning Strategy ◽  
Neural Systems ◽  
Levels Of Abstraction ◽  
Internal Representations ◽  
Internal Operations ◽  
High Level

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Deep learning's success is appealing to neuroscientists not only as a method for applying DNNs to model biological neural systems but also as a means of adopting concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks, such as PyTorch and TensorFlow, could be used to allow such cross-disciplinary investigations, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed as a mechanism for cognitive neuroscientists to map both DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring the internal representations of DNNs as well as brains. Through the integration of DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios. These include extracting DNN activation, probing and visualizing DNN representations, and mapping DNN representations onto the brain. We expect that our toolbox will accelerate scientific research by both applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.

scholarly journals DNNBrain: a unifying toolbox for mapping deep neural networks and brains

2020 ◽  
Author(s):  
Xiayu Chen ◽  
Ming Zhou ◽  
Zhengxin Gong ◽  
Wei Xu ◽  
Xingyu Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Cognitive Neuroscience ◽  
Deep Neural Networks ◽  
Learning Strategy ◽  
Neural Systems ◽  
Levels Of Abstraction ◽  
Internal Representations ◽  
High Level ◽  
The Brain

ABSTRACTDeep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks through an end-to-end deep learning strategy. Deep learning gives rise to data representations with multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Its success appeals to neuroscientists not only to apply DNNs to model biological neural systems, but also to adopt concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks such as PyTorch and TensorFlow could be used to allow such cross-disciplinary studies, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed for cognitive neuroscientists to map DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring internal representations in both DNNs and the brain. By integrating DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios, such as extracting DNN activation, probing DNN representations, mapping DNN representations onto the brain, and visualizing DNN representations. We expect that our toolbox will accelerate scientific research in applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.

Deep neural networks trained with heavier data augmentation learn features closer to representations in hIT

2018 ◽  
Author(s):  
Alex Hernández-García ◽  
Johannes Mehrer ◽  
Nikolaus Kriegeskorte ◽  
Peter König ◽  
Tim C. Kietzmann
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Data Augmentation

Representation of adversarial images in deep neural networks and the human brain

2018 ◽  
Author(s):  
Chi Zhang ◽  
Xiaohan Duan ◽  
Ruyuan Zhang ◽  
Li Tong
Keyword(s):  
Neural Networks ◽  
Human Brain ◽  
Deep Neural Networks

Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks

2018 ◽  
Author(s):  
Daniel Povey ◽  
Gaofeng Cheng ◽  
Yiming Wang ◽  
Ke Li ◽  
Hainan Xu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Matrix Factorization ◽  
Deep Neural Networks ◽  
Low Rank ◽  
Rank Matrix ◽  
Low Rank Matrix
Sign in / Sign up

Export Citation Format

Share Document