Deep neural networks and visuo-semantic models explain complementary components of human ventral-stream representational dynamics

Deep neural networks (DNNs) are promising models of the cortical computations supporting human object recognition. However, despite their ability to explain a significant portion of variance in neural data, the agreement between models and brain representational dynamics is far from perfect. Here, we address this issue by asking which representational features are currently unaccounted for in neural timeseries data, estimated for multiple areas of the human ventral stream via source-reconstructed magnetoencephalography (MEG) data. In particular, we focus on the ability of visuo-semantic models, consisting of human-generated labels of higher-level object features and categories, to explain variance beyond the explanatory power of DNNs alone. We report a gradual transition in the importance of visuo-semantic features from early to higher-level areas along the ventral stream. While early visual areas are better explained by DNN features, higher-level cortical dynamics are best accounted for by visuo-semantic models. These results suggest that current DNNs fail to fully capture the visuo-semantic features represented in higher-level human visual cortex and suggest a path towards more accurate models of ventral stream computations.

Download Full-text

Faculty Opinions recommendation of Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.726413891.793534418 ◽

2017 ◽

Author(s):

Odelia Schwartz

Keyword(s):

Neural Networks ◽

Object Recognition ◽

Deep Neural Networks ◽

Visual Object ◽

Visual Object Recognition ◽

Cortical Dynamics ◽

Spatio Temporal

Download Full-text

Identifying cross-lingual plagiarism using rich semantic features and deep neural networks: A study on Arabic-English plagiarism cases

Journal of King Saud University - Computer and Information Sciences ◽

10.1016/j.jksuci.2020.04.009 ◽

2020 ◽

Author(s):

Salha Alzahrani ◽

Hanan Aljuaid

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Semantic Features ◽

Cross Lingual

Download Full-text

Deep neural networks design and analysis for automatic phase pickers from three-component microseismic recordings

Geophysical Journal International ◽

10.1093/gji/ggz441 ◽

2019 ◽

Vol 220 (1) ◽

pp. 323-334

Author(s):

Jing Zheng ◽

Shuaishuai Shen ◽

Tianqi Jiang ◽

Weiqiang Zhu

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Semantic Segmentation ◽

Structure Design ◽

P Wave ◽

Negative Slope ◽

Semantic Features ◽

S Wave ◽

Automatic Phase ◽

Rectified Linear Unit

SUMMARY It is essential to pick P-wave and S-wave arrival times rapidly and accurately for the microseismic monitoring systems. Meanwhile, it is not easy to identify the arrivals at a true phase automatically using traditional picking method. This is one of the reasons that many researchers are trying to introduce deep neural networks to solve these problems. Convolutional neural networks (CNNs) are very attractive for designing automatic phase pickers especially after introducing the fundamental network structure from semantic segmentation field, which can give the probability outputs for every labelled phase at every sample in the recordings. The typical segmentation architecture consists of two main parts: (1) an encoder part trained to extracting coarse semantic features; (2) a decoder part responsible not only for recovering the input resolution at the output but also for obtaining sparse representation of the objects. The fundamental segmentation structure performs well; however, the influence of the parameters in the structure on the pickers has not been investigated. It means that the structure design just depends on experience and tests. In this paper, we solve two main questions to give some guidance on network design. First, we show what sparse features will learn from the three-component microseismic recordings using CNNs. Second, the influence of two key parameters in the network on pickers, namely, the depth of decoder and activation functions, is analysed. Increasing the number of levels for a certain layer in the decoder will increase the burden of demand on trainable parameters, but it is beneficial to the accuracy of the model. Reasonable depth of the decoder can balance prediction accuracy and the demand of labelled data, which is important for microseismic systems because manual labelling process will decrease the real-time performance in monitoring tasks. Standard rectified linear unit (ReLU) and leaky rectified linear unit (Leaky ReLU) with different negative slopes are compared for the analysis. Leaky ReLU with a small negative slope can improve the performance of a given model than ReLU activation function by keeping some information about the negative parts.

Download Full-text