scholarly journals Noise-trained deep neural networks effectively predict human vision and its neural responses to challenging images

PLoS Biology ◽  
2021 ◽  
Vol 19 (12) ◽  
pp. e3001418
Author(s):  
Hojin Jang ◽  
Devin McCormack ◽  
Frank Tong
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Functional Neuroimaging ◽  
Visual Learning ◽  
Human Vision ◽  
Noise Robustness ◽  
Training Procedure ◽  
Correlated Noise ◽  
Neural Responses ◽  
Viable Model

Deep neural networks (DNNs) for object classification have been argued to provide the most promising model of the visual system, accompanied by claims that they have attained or even surpassed human-level performance. Here, we evaluated whether DNNs provide a viable model of human vision when tested with challenging noisy images of objects, sometimes presented at the very limits of visibility. We show that popular state-of-the-art DNNs perform in a qualitatively different manner than humans—they are unusually susceptible to spatially uncorrelated white noise and less impaired by spatially correlated noise. We implemented a noise training procedure to determine whether noise-trained DNNs exhibit more robust responses that better match human behavioral and neural performance. We found that noise-trained DNNs provide a better qualitative match to human performance; moreover, they reliably predict human recognition thresholds on an image-by-image basis. Functional neuroimaging revealed that noise-trained DNNs provide a better correspondence to the pattern-specific neural representations found in both early visual areas and high-level object areas. A layer-specific analysis of the DNNs indicated that noise training led to broad-ranging modifications throughout the network, with greater benefits of noise robustness accruing in progressively higher layers. Our findings demonstrate that noise-trained DNNs provide a viable model to account for human behavioral and neural responses to objects in challenging noisy viewing conditions. Further, they suggest that robustness to noise may be acquired through a process of visual learning.

scholarly journals Noise-robust recognition of objects by humans and deep neural networks

2020 ◽  
Author(s):  
Hojin Jang ◽  
Devin McCormack ◽  
Frank Tong
Keyword(s):  
Neural Networks ◽  
Visual Processing ◽  
Deep Neural Networks ◽  
Signal To Noise Ratio ◽  
Human Vision ◽  
Training Procedure ◽  
Robust Recognition ◽  
Recognition Of Objects ◽  
Noise Robust ◽  
Level Performance

ABSTRACTDeep neural networks (DNNs) can accurately recognize objects in clear viewing conditions, leading to claims that they have attained or surpassed human-level performance. However, standard DNNs are severely impaired at recognizing objects in visual noise, whereas human vision remains robust. We developed a noise-training procedure, generating noisy images of objects with low signal-to-noise ratio, to investigate whether DNNs can acquire robustness that better matches human vision. After noise training, DNNs outperformed human observers while exhibiting more similar patterns of performance, and provided a better model for predicting human recognition thresholds on an image-by-image basis. Noise training also improved DNN recognition of vehicles in noisy weather. Layer-specific analyses revealed that the contaminating effects of noise were dampened, rather than amplified, across successive stages of the noise-trained network, with greater benefit at higher levels of the network. Our findings indicate that DNNs can learn noise-robust representations that better approximate human visual processing.

scholarly journals Effective Ensemble of Deep Neural Networks Predicts Neural Responses to Naturalistic Videos

2021 ◽  
Author(s):  
Huzheng Yang ◽  
Shanghang Zhang ◽  
Yifan Wu ◽  
Yuanning Li ◽  
Shi Gu
Keyword(s):  
Neural Networks ◽  
Visual Cortex ◽  
Deep Neural Networks ◽  
Functional Neuroimaging ◽  
State Of The Art ◽  
Prediction Performance ◽  
Neural Responses ◽  
Neural Signals ◽  
Art Performance ◽  
Image Streams

This report provides a review of our submissions to the Algonauts Challenge 2021. In this challenge, neural responses in the visual cortex were recorded using functional neuroimaging when participants were watching naturalistic videos. The goal of the challenge is to develop voxel-wise encoding models which predict such neural signals based on the input videos. Here we built an ensemble of models that extract representations based on the input videos from 4 perspectives: image streams, motion, edges, and audio. We showed that adding new modules into the ensemble consistently improved our prediction performance. Our methods achieved state-of-the-art performance on both the mini track and the full track tasks.

scholarly journals How well do Deep Neural Networks model Human Vision?

2016 ◽  
Vol 16 (12) ◽  
pp. 176
Author(s):  
John Clevenger ◽  
Diane Beck
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Human Vision

Pruning for Hardware-Based Deep Spiking Neural Networks Using Gated Schottky Diode as Synaptic Devices

2020 ◽  
Vol 20 (11) ◽  
pp. 6603-6608 ◽  
Author(s):  
Sung-Tae Lee ◽  
Suhwan Lim ◽  
Jong-Ho Bae ◽  
Dongseok Kwon ◽  
Hyeong-Su Kim ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Schottky Diodes ◽  
Computational Cost ◽  
Spiking Neural Networks ◽  
Training Procedure ◽  
Learning Tasks ◽  
L1 Regularization ◽  
The Cost ◽  
High Computational Cost

Deep learning represents state-of-the-art results in various machine learning tasks, but for applications that require real-time inference, the high computational cost of deep neural networks becomes a bottleneck for the efficiency. To overcome the high computational cost of deep neural networks, spiking neural networks (SNN) have been proposed. Herein, we propose a hardware implementation of the SNN with gated Schottky diodes as synaptic devices. In addition, we apply L1 regularization for connection pruning of the deep spiking neural networks using gated Schottky diodes as synap-tic devices. Applying L1 regularization eliminates the need for a re-training procedure because it prunes the weights based on the cost function. The compressed hardware-based SNN is energy efficient while achieving a classification accuracy of 97.85% which is comparable to 98.13% of the software deep neural networks (DNN).

scholarly journals Deep learning: Using machine learning to study biological vision

2017 ◽  
Author(s):  
Najib J. Majaj ◽  
Denis G. Pelli
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Brain Activity ◽  
Neural Responses ◽  
Statistical Tool ◽  
Biological Vision ◽  
Vision Science ◽  
Living Organisms

ABSTRACTToday many vision-science presentations employ machine learning, especially the version called “deep learning”. Many neuroscientists use machine learning to decode neural responses. Many perception scientists try to understand how living organisms recognize objects. To them, deep neural networks offer benchmark accuracies for recognition of learned stimuli. Originally machine learning was inspired by the brain. Today, machine learning is used as a statistical tool to decode brain activity. Tomorrow, deep neural networks might become our best model of brain function. This brief overview of the use of machine learning in biological vision touches on its strengths, weaknesses, milestones, controversies, and current directions. Here, we hope to help vision scientists assess what role machine learning should play in their research.

scholarly journals An efficient test method for noise robustness of deep neural networks

2019 ◽  
Vol 10 (2) ◽  
pp. 221-235
Author(s):  
Muneki Yasuda ◽  
Hironori Sakata ◽  
Seung-Il Cho ◽  
Tomochika Harada ◽  
Atushi Tanaka ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Test Method ◽  
Noise Robustness ◽  
Efficient Test

Outperforming Dermatologist-Level Skin Cancer Classification via Enhanced Training of Deep Neural Networks (Preprint)

2018 ◽  
Author(s):  
Titus Josef Brinker ◽  
Achim Hekler ◽  
Christof von Kalle
Keyword(s):  
Neural Networks ◽  
Skin Cancer ◽  
Deep Neural Networks ◽  
Challenge Test ◽  
Image Data ◽  
Cancer Classification ◽  
Receiver Operating Curve ◽  
Training Procedure ◽  
Average Precision ◽  
Test Sets

BACKGROUND In recent months, multiple publications have demonstrated the use of convolutional neural networks (CNN) to classify images of skin cancer as precisely as dermatologists. These CNNs failed to outperform the International Symposium on Biomedical Imaging (ISBI) 2016 challenge in terms of average precision, however, so the technical progress represented by these studies is limited. In addition, the available reports are difficult to reproduce, due to incomplete descriptions of training procedures and the use of proprietary image databases. These factors prevent the comparison of various CNN classifiers in equal terms. OBJECTIVE To demonstrate the training of an image-classifier CNN that outperforms the winner of the ISBI 2016 challenge by using open source images exclusively. METHODS A detailed description of the training procedure is reported while the used images and test sets are disclosed fully, to insure the reproducibility of our work. RESULTS Our CNN classifier outperforms all recent attempts to classify the original ISBI 2016 challenge test data (full set of 379 test images), with an average precision of 0.709 (vs. 0.637 of the ISBI winner) and with an area under the receiver operating curve of 0.85. CONCLUSIONS This work illustrates the potential for improving skin cancer classification with enhanced training procedures for CNNs, while avoiding the use of costly equipment or proprietary image data.

scholarly journals Deep Neural Networks in Computational Neuroscience

2019 ◽  
Author(s):  
Tim C. Kietzmann ◽  
Patrick McClure ◽  
Nikolaus Kriegeskorte
Keyword(s):  
Neural Networks ◽  
Computational Neuroscience ◽  
Computer Games ◽  
Deep Neural Networks ◽  
Learning Algorithm ◽  
Visual Object ◽  
Robot Arm ◽  
Neural Responses ◽  
Sensory Stimuli ◽  
Model Complex

The goal of computational neuroscience is to find mechanistic explanations of how the nervous system processes information to give rise to cognitive function and behavior. At the heart of the field are its models, that is, mathematical and computational descriptions of the system being studied, which map sensory stimuli to neural responses and/or neural to behavioral responses. These models range from simple to complex. Recently, deep neural networks (DNNs) have come to dominate several domains of artificial intelligence (AI). As the term “neural network” suggests, these models are inspired by biological brains. However, current DNNs neglect many details of biological neural networks. These simplifications contribute to their computational efficiency, enabling them to perform complex feats of intelligence, ranging from perceptual (e.g., visual object and auditory speech recognition) to cognitive tasks (e.g., machine translation), and on to motor control (e.g., playing computer games or controlling a robot arm). In addition to their ability to model complex intelligent behaviors, DNNs excel at predicting neural responses to novel sensory stimuli with accuracies well beyond any other currently available model type. DNNs can have millions of parameters, which are required to capture the domain knowledge needed for successful task performance. Contrary to the intuition that this renders them into impenetrable black boxes, the computational properties of the network units are the result of four directly manipulable elements: input statistics, network structure, functional objective, and learning algorithm. With full access to the activity and connectivity of all units, advanced visualization techniques, and analytic tools to map network representations to neural data, DNNs represent a powerful framework for building task-performing models and will drive substantial insights in computational neuroscience.

Sign in / Sign up

Export Citation Format

Share Document