Darwin: a neuromorphic hardware co-processor based on Spiking Neural Networks

2015 ◽  
Vol 59 (2) ◽  
pp. 1-5 ◽  
Author(s):  
Juncheng Shen ◽  
De Ma ◽  
Zonghua Gu ◽  
Ming Zhang ◽  
Xiaolei Zhu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Spiking Neural Networks ◽  
Neuromorphic Hardware

scholarly journals Exploring Optimized Spiking Neural Network Architectures for Classification Tasks on Embedded Platforms

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3240
Author(s):  
Tehreem Syed ◽  
Vijay Kakani ◽  
Xuenan Cui ◽  
Hakil Kim
Keyword(s):  
Neural Networks ◽  
Gradient Descent ◽  
Spiking Neural Networks ◽  
License Plate ◽  
Training Techniques ◽  
Neuromorphic Hardware ◽  
Private And Public ◽  
Embedded Platforms ◽  
Public Datasets ◽  
Event Based

In recent times, the usage of modern neuromorphic hardware for brain-inspired SNNs has grown exponentially. In the context of sparse input data, they are undertaking low power consumption for event-based neuromorphic hardware, specifically in the deeper layers. However, using deep ANNs for training spiking models is still considered as a tedious task. Until recently, various ANN to SNN conversion methods in the literature have been proposed to train deep SNN models. Nevertheless, these methods require hundreds to thousands of time-steps for training and still cannot attain good SNN performance. This work proposes a customized model (VGG, ResNet) architecture to train deep convolutional spiking neural networks. In this current study, the training is carried out using deep convolutional spiking neural networks with surrogate gradient descent backpropagation in a customized layer architecture similar to deep artificial neural networks. Moreover, this work also proposes fewer time-steps for training SNNs with surrogate gradient descent. During the training with surrogate gradient descent backpropagation, overfitting problems have been encountered. To overcome these problems, this work refines the SNN based dropout technique with surrogate gradient descent. The proposed customized SNN models achieve good classification results on both private and public datasets. In this work, several experiments have been carried out on an embedded platform (NVIDIA JETSON TX2 board), where the deployment of customized SNN models has been extensively conducted. Performance validations have been carried out in terms of processing time and inference accuracy between PC and embedded platforms, showing that the proposed customized models and training techniques are feasible for achieving a better performance on various datasets such as CIFAR-10, MNIST, SVHN, and private KITTI and Korean License plate dataset.

Chapter 9. Spike-Based Symbolic Computations on Bit Strings and Numbers

2021 ◽  
Author(s):  
Ceca Kraišniković ◽  
Wolfgang Maass ◽  
Robert Legenstein
Keyword(s):  
Neural Networks ◽  
Energy Efficient ◽  
Learning Rule ◽  
Spiking Neural Networks ◽  
New Paradigm ◽  
Symbolic Computations ◽  
Neuromorphic Hardware ◽  
State Of Research ◽  
Higher Cognitive Functions ◽  
The Brain

The brain uses recurrent spiking neural networks for higher cognitive functions such as symbolic computations, in particular, mathematical computations. We review the current state of research on spike-based symbolic computations of this type. In addition, we present new results which show that surprisingly small spiking neural networks can perform symbolic computations on bit sequences and numbers and even learn such computations using a biologically plausible learning rule. The resulting networks operate in a rather low firing rate regime, where they could not simply emulate artificial neural networks by encoding continuous values through firing rates. Thus, we propose here a new paradigm for symbolic computation in neural networks that provides concrete hypotheses about the organization of symbolic computations in the brain. The employed spike-based network models are the basis for drastically more energy-efficient computer hardware – neuromorphic hardware. Hence, our results can be seen as creating a bridge from symbolic artificial intelligence to energy-efficient implementation in spike-based neuromorphic hardware.

scholarly journals Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch

2021 ◽  
Vol 15 ◽  
Author(s):  
Youngeun Kim ◽  
Priyadarshini Panda
Keyword(s):  
Neural Networks ◽  
Temporal Dynamics ◽  
Temporal Characteristic ◽  
Spiking Neural Networks ◽  
Time Step ◽  
Intelligent Computing ◽  
Early Exit ◽  
Batch Normalization ◽  
Binary Event ◽  
Neuromorphic Hardware

Spiking Neural Networks (SNNs) have recently emerged as an alternative to deep learning owing to sparse, asynchronous and binary event (or spike) driven processing, that can yield huge energy efficiency benefits on neuromorphic hardware. However, SNNs convey temporally-varying spike activation through time that is likely to induce a large variation of forward activation and backward gradients, resulting in unstable training. To address this training issue in SNNs, we revisit Batch Normalization (BN) and propose a temporal Batch Normalization Through Time (BNTT) technique. Different from previous BN techniques with SNNs, we find that varying the BN parameters at every time-step allows the model to learn the time-varying input distribution better. Specifically, our proposed BNTT decouples the parameters in a BNTT layer along the time axis to capture the temporal dynamics of spikes. We demonstrate BNTT on CIFAR-10, CIFAR-100, Tiny-ImageNet, event-driven DVS-CIFAR10 datasets, and Sequential MNIST and show near state-of-the-art performance. We conduct comprehensive analysis on the temporal characteristic of BNTT and showcase interesting benefits toward robustness against random and adversarial noise. Further, by monitoring the learnt parameters of BNTT, we find that we can do temporal early exit. That is, we can reduce the inference latency by ~5 − 20 time-steps from the original training latency. The code has been released at https://github.com/Intelligent-Computing-Lab-Yale/BNTT-Batch-Normalization-Through-Time.

scholarly journals Run-time Mapping of Spiking Neural Networks to Neuromorphic Hardware

2020 ◽  
Vol 92 (11) ◽  
pp. 1293-1302
Author(s):  
Adarsha Balaji ◽  
Thibaut Marty ◽  
Anup Das ◽  
Francky Catthoor
Keyword(s):  
Neural Networks ◽  
Spiking Neural Networks ◽  
Neuromorphic Hardware ◽  
Run Time

scholarly journals Mapping Spiking Neural Networks to Neuromorphic Hardware

2020 ◽  
Vol 28 (1) ◽  
pp. 76-86 ◽  
Author(s):  
Adarsha Balaji ◽  
Francky Catthoor ◽  
Anup Das ◽  
Yuefeng Wu ◽  
Khanh Huynh ◽  
...  
Keyword(s):  
Neural Networks ◽  
Spiking Neural Networks ◽  
Neuromorphic Hardware

scholarly journals Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Julian Büchel ◽  
Dmitrii Zendrikov ◽  
Sergio Solinas ◽  
Giacomo Indiveri ◽  
Dylan R. Muir
Keyword(s):  
Neural Networks ◽  
Analog Circuits ◽  
Learning Rule ◽  
High Energy ◽  
Spiking Neural Networks ◽  
Mixed Signal ◽  
Device Mismatch ◽  
Linear Control Theory ◽  
Neuromorphic Hardware ◽  
On Chip

AbstractMixed-signal analog/digital circuits emulate spiking neurons and synapses with extremely high energy efficiency, an approach known as “neuromorphic engineering”. However, analog circuits are sensitive to process-induced variation among transistors in a chip (“device mismatch”). For neuromorphic implementation of Spiking Neural Networks (SNNs), mismatch causes parameter variation between identically-configured neurons and synapses. Each chip exhibits a different distribution of neural parameters, causing deployed networks to respond differently between chips. Current solutions to mitigate mismatch based on per-chip calibration or on-chip learning entail increased design complexity, area and cost, making deployment of neuromorphic devices expensive and difficult. Here we present a supervised learning approach that produces SNNs with high robustness to mismatch and other common sources of noise. Our method trains SNNs to perform temporal classification tasks by mimicking a pre-trained dynamical system, using a local learning rule from non-linear control theory. We demonstrate our method on two tasks requiring temporal memory, and measure the robustness of our approach to several forms of noise and mismatch. We show that our approach is more robust than common alternatives for training SNNs. Our method provides robust deployment of pre-trained networks on mixed-signal neuromorphic hardware, without requiring per-device training or calibration.

Sign in / Sign up

Export Citation Format

Share Document