scholarly journals Low‐Voltage Electrochemical Li x WO 3 Synapses with Temporal Dynamics for Spiking Neural Networks

2021 ◽  
pp. 2100021
Author(s):  
Qingzhou Wan ◽  
Marco Rasetto ◽  
Mohammad T. Sharbati ◽  
John R. Erickson ◽  
Sridhar Reddy Velagala ◽  
...  
Keyword(s):  
Neural Networks ◽  
Temporal Dynamics ◽  
Low Voltage ◽  
Spiking Neural Networks

scholarly journals Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch

2021 ◽  
Vol 15 ◽  
Author(s):  
Youngeun Kim ◽  
Priyadarshini Panda
Keyword(s):  
Neural Networks ◽  
Temporal Dynamics ◽  
Temporal Characteristic ◽  
Spiking Neural Networks ◽  
Time Step ◽  
Intelligent Computing ◽  
Early Exit ◽  
Batch Normalization ◽  
Binary Event ◽  
Neuromorphic Hardware

Spiking Neural Networks (SNNs) have recently emerged as an alternative to deep learning owing to sparse, asynchronous and binary event (or spike) driven processing, that can yield huge energy efficiency benefits on neuromorphic hardware. However, SNNs convey temporally-varying spike activation through time that is likely to induce a large variation of forward activation and backward gradients, resulting in unstable training. To address this training issue in SNNs, we revisit Batch Normalization (BN) and propose a temporal Batch Normalization Through Time (BNTT) technique. Different from previous BN techniques with SNNs, we find that varying the BN parameters at every time-step allows the model to learn the time-varying input distribution better. Specifically, our proposed BNTT decouples the parameters in a BNTT layer along the time axis to capture the temporal dynamics of spikes. We demonstrate BNTT on CIFAR-10, CIFAR-100, Tiny-ImageNet, event-driven DVS-CIFAR10 datasets, and Sequential MNIST and show near state-of-the-art performance. We conduct comprehensive analysis on the temporal characteristic of BNTT and showcase interesting benefits toward robustness against random and adversarial noise. Further, by monitoring the learnt parameters of BNTT, we find that we can do temporal early exit. That is, we can reduce the inference latency by ~5 − 20 time-steps from the original training latency. The code has been released at https://github.com/Intelligent-Computing-Lab-Yale/BNTT-Batch-Normalization-Through-Time.

A Developmental Method for Evolving Large-Scale Spiking Neural Networks

2012 ◽  
Vol 35 (12) ◽  
pp. 2633 ◽  
Author(s):  
Xiang-Hong LIN ◽  
Tian-Wen ZHANG ◽  
Gui-Cang ZHANG
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Spiking Neural Networks

scholarly journals Spiking Neural Networks and online learning: An overview and perspectives

2020 ◽  
Vol 121 ◽  
pp. 88-100 ◽  
Author(s):  
Jesus L. Lobo ◽  
Javier Del Ser ◽  
Albert Bifet ◽  
Nikola Kasabov
Keyword(s):  
Neural Networks ◽  
Online Learning ◽  
Spiking Neural Networks
Sign in / Sign up

Export Citation Format

Share Document