scholarly journals Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-linear Activations

2020 ◽  
pp. 399-414
Author(s):  
Saima Sharmin ◽  
Nitin Rathi ◽  
Priyadarshini Panda ◽  
Kaushik Roy
Keyword(s):  
Neural Networks ◽  
Spiking Neural Networks ◽  
Non Linear

scholarly journals Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks

2021 ◽  
Vol 15 ◽  
Author(s):  
Stefano Brivio ◽  
Denys R. B. Ly ◽  
Elisa Vianello ◽  
Sabina Spiga
Keyword(s):  
Neural Networks ◽  
Classification Performance ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
System Level ◽  
Training Time ◽  
Background Theory ◽  
Non Linear ◽  
Synaptic Dynamics ◽  
The Impact

Spiking neural networks (SNNs) are a computational tool in which the information is coded into spikes, as in some parts of the brain, differently from conventional neural networks (NNs) that compute over real-numbers. Therefore, SNNs can implement intelligent information extraction in real-time at the edge of data acquisition and correspond to a complementary solution to conventional NNs working for cloud-computing. Both NN classes face hardware constraints due to limited computing parallelism and separation of logic and memory. Emerging memory devices, like resistive switching memories, phase change memories, or memristive devices in general are strong candidates to remove these hurdles for NN applications. The well-established training procedures of conventional NNs helped in defining the desiderata for memristive device dynamics implementing synaptic units. The generally agreed requirements are a linear evolution of memristive conductance upon stimulation with train of identical pulses and a symmetric conductance change for conductance increase and decrease. Conversely, little work has been done to understand the main properties of memristive devices supporting efficient SNN operation. The reason lies in the lack of a background theory for their training. As a consequence, requirements for NNs have been taken as a reference to develop memristive devices for SNNs. In the present work, we show that, for efficient CMOS/memristive SNNs, the requirements for synaptic memristive dynamics are very different from the needs of a conventional NN. System-level simulations of a SNN trained to classify hand-written digit images through a spike timing dependent plasticity protocol are performed considering various linear and non-linear plausible synaptic memristive dynamics. We consider memristive dynamics bounded by artificial hard conductance values and limited by the natural dynamics evolution toward asymptotic values (soft-boundaries). We quantitatively analyze the impact of resolution and non-linearity properties of the synapses on the network training and classification performance. Finally, we demonstrate that the non-linear synapses with hard boundary values enable higher classification performance and realize the best trade-off between classification accuracy and required training time. With reference to the obtained results, we discuss how memristive devices with non-linear dynamics constitute a technologically convenient solution for the development of on-line SNN training.

A Developmental Method for Evolving Large-Scale Spiking Neural Networks

2012 ◽  
Vol 35 (12) ◽  
pp. 2633 ◽  
Author(s):  
Xiang-Hong LIN ◽  
Tian-Wen ZHANG ◽  
Gui-Cang ZHANG
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Spiking Neural Networks

Analysis of Non-Linear Activation Functions for Classification Tasks Using Convolutional Neural Networks

2019 ◽  
Vol 12 (3) ◽  
pp. 156-161 ◽  
Author(s):  
Aman Dureja ◽  
Payal Pahwa
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Activation Function ◽  
Primary Objective ◽  
Experimental Comparison ◽  
Activation Functions ◽  
Practical Applications ◽  
Network Activation ◽  
Non Linear ◽  
Hidden Layer

Background: In making the deep neural network, activation functions play an important role. But the choice of activation functions also affects the network in term of optimization and to retrieve the better results. Several activation functions have been introduced in machine learning for many practical applications. But which activation function should use at hidden layer of deep neural networks was not identified. Objective: The primary objective of this analysis was to describe which activation function must be used at hidden layers for deep neural networks to solve complex non-linear problems. Methods: The configuration for this comparative model was used by using the datasets of 2 classes (Cat/Dog). The number of Convolutional layer used in this network was 3 and the pooling layer was also introduced after each layer of CNN layer. The total of the dataset was divided into the two parts. The first 8000 images were mainly used for training the network and the next 2000 images were used for testing the network. Results: The experimental comparison was done by analyzing the network by taking different activation functions on each layer of CNN network. The validation error and accuracy on Cat/Dog dataset were analyzed using activation functions (ReLU, Tanh, Selu, PRelu, Elu) at number of hidden layers. Overall the Relu gave best performance with the validation loss at 25th Epoch 0.3912 and validation accuracy at 25th Epoch 0.8320. Conclusion: It is found that a CNN model with ReLU hidden layers (3 hidden layers here) gives best results and improve overall performance better in term of accuracy and speed. These advantages of ReLU in CNN at number of hidden layers are helpful to effectively and fast retrieval of images from the databases.

scholarly journals Spiking Neural Networks and online learning: An overview and perspectives

2020 ◽  
Vol 121 ◽  
pp. 88-100 ◽  
Author(s):  
Jesus L. Lobo ◽  
Javier Del Ser ◽  
Albert Bifet ◽  
Nikola Kasabov
Keyword(s):  
Neural Networks ◽  
Online Learning ◽  
Spiking Neural Networks

scholarly journals Uncertainty Due to Infectious Diseases and Stock–Bond Correlation

Econometrics ◽  
2021 ◽  
Vol 9 (2) ◽  
pp. 17
Author(s):  
Konstantinos Gkillas ◽  
Christoforos Konstantatos ◽  
Costas Siriopoulos
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Infectious Diseases ◽  
Granger Causality ◽  
Predictive Value ◽  
High Frequency ◽  
Causal Relation ◽  
Granger Causality Test ◽  
Causality Test ◽  
Non Linear

We study the non-linear causal relation between uncertainty-due-to-infectious-diseases and stock–bond correlation. To this end, we use high-frequency 1-min data to compute daily realized measures of correlation and jumps, and then, we employ a nonlinear Granger causality test with the use of artificial neural networks so as to investigate the predictability of this type of uncertainty on realized stock–bond correlation and jumps. Our findings reveal that uncertainty-due-to-infectious-diseases has significant predictive value on the changes of the stock–bond relation.

Sign in / Sign up

Export Citation Format

Share Document