scholarly journals SpikePropamine: Differentiable Plasticity in Spiking Neural Networks

2021 ◽  
Vol 15 ◽  
Author(s):  
Samuel Schmidgall ◽  
Julia Ashkanazy ◽  
Wallace Lawson ◽  
Joe Hays
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
Critical Role ◽  
Learning Task ◽  
Spiking Neural Networks ◽  
Training Period ◽  
Initial Training ◽  
Learning Tasks ◽  
Robotic Learning ◽  
Learning Focused

The adaptive changes in synaptic efficacy that occur between spiking neurons have been demonstrated to play a critical role in learning for biological neural networks. Despite this source of inspiration, many learning focused applications using Spiking Neural Networks (SNNs) retain static synaptic connections, preventing additional learning after the initial training period. Here, we introduce a framework for simultaneously learning the underlying fixed-weights and the rules governing the dynamics of synaptic plasticity and neuromodulated synaptic plasticity in SNNs through gradient descent. We further demonstrate the capabilities of this framework on a series of challenging benchmarks, learning the parameters of several plasticity rules including BCM, Oja's, and their respective set of neuromodulatory variants. The experimental results display that SNNs augmented with differentiable plasticity are sufficient for solving a set of challenging temporal learning tasks that a traditional SNN fails to solve, even in the presence of significant noise. These networks are also shown to be capable of producing locomotion on a high-dimensional robotic learning task, where near-minimal degradation in performance is observed in the presence of novel conditions not seen during the initial training period.

Neuromorphic Functional Modules of Spiking Neural Network

2021 ◽  
Vol 23 (6) ◽  
pp. 317-326
Author(s):  
E.A. Ryndin ◽  
◽  
N.V. Andreeva ◽  
V.V. Luchinin ◽  
K.S. Goncharov ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Synaptic Plasticity ◽  
Data Processing ◽  
Circuit Design ◽  
Spiking Neural Networks ◽  
Spiking Neural Network ◽  
Wide Range ◽  
Artificial Neural

In the current era, design and development of artificial neural networks exploiting the architecture of the human brain have evolved rapidly. Artificial neural networks effectively solve a wide range of common for artificial intelligence tasks involving data classification and recognition, prediction, forecasting and adaptive control of object behavior. Biologically inspired underlying principles of ANN operation have certain advantages over the conventional von Neumann architecture including unsupervised learning, architectural flexibility and adaptability to environmental change and high performance under significantly reduced power consumption due to heavy parallel and asynchronous data processing. In this paper, we present the circuit design of main functional blocks (neurons and synapses) intended for hardware implementation of a perceptron-based feedforward spiking neural network. As the third generation of artificial neural networks, spiking neural networks perform data processing utilizing spikes, which are discrete events (or functions) that take place at points in time. Neurons in spiking neural networks initiate precisely timing spikes and communicate with each other via spikes transmitted through synaptic connections or synapses with adaptable scalable weight. One of the prospective approach to emulate the synaptic behavior in hardware implemented spiking neural networks is to use non-volatile memory devices with analog conduction modulation (or memristive structures). Here we propose a circuit design for functional analogues of memristive structure to mimic a synaptic plasticity, pre- and postsynaptic neurons which could be used for developing circuit design of spiking neural network architectures with different training algorithms including spike-timing dependent plasticity learning rule. Two different circuits of electronic synapse were developed. The first one is an analog synapse with photoresistive optocoupler used to ensure the tunable conductivity for synaptic plasticity emulation. While the second one is a digital synapse, in which the synaptic weight is stored in a digital code with its direct conversion into conductivity (without digital-to-analog converter andphotoresistive optocoupler). The results of the prototyping of developed circuits for electronic analogues of synapses, pre- and postsynaptic neurons and the study of transient processes are presented. The developed approach could provide a basis for ASIC design of spiking neural networks based on CMOS (complementary metal oxide semiconductor) design technology.

Pruning for Hardware-Based Deep Spiking Neural Networks Using Gated Schottky Diode as Synaptic Devices

2020 ◽  
Vol 20 (11) ◽  
pp. 6603-6608 ◽  
Author(s):  
Sung-Tae Lee ◽  
Suhwan Lim ◽  
Jong-Ho Bae ◽  
Dongseok Kwon ◽  
Hyeong-Su Kim ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Schottky Diodes ◽  
Computational Cost ◽  
Spiking Neural Networks ◽  
Training Procedure ◽  
Learning Tasks ◽  
L1 Regularization ◽  
The Cost ◽  
High Computational Cost

Deep learning represents state-of-the-art results in various machine learning tasks, but for applications that require real-time inference, the high computational cost of deep neural networks becomes a bottleneck for the efficiency. To overcome the high computational cost of deep neural networks, spiking neural networks (SNN) have been proposed. Herein, we propose a hardware implementation of the SNN with gated Schottky diodes as synaptic devices. In addition, we apply L1 regularization for connection pruning of the deep spiking neural networks using gated Schottky diodes as synap-tic devices. Applying L1 regularization eliminates the need for a re-training procedure because it prunes the weights based on the cost function. The compressed hardware-based SNN is energy efficient while achieving a classification accuracy of 97.85% which is comparable to 98.13% of the software deep neural networks (DNN).

Synaptic plasticity in spiking neural networks (SP/sup 2/INN): a system approach

2003 ◽  
Vol 14 (5) ◽  
pp. 980-992 ◽  
Author(s):  
N. Mehrtash ◽  
Dietmar Jung ◽  
H.H. Hellmich ◽  
T. Schoenauer ◽  
Vi Thanh Lu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
System Approach ◽  
Spiking Neural Networks

Ensemble Learning for Regression

2011 ◽  
pp. 777-782
Author(s):  
Niall Rooney
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Neural Networks ◽  
Ensemble Learning ◽  
Learning Task ◽  
General Definition ◽  
Learning Tasks ◽  
Continuous Output ◽  
Input Variables ◽  
The Given

The concept of ensemble learning has its origins in research from the late 1980s/early 1990s into combining a number of artificial neural networks (ANNs) models for regression tasks. Ensemble learning is now a widely deployed and researched topic within the area of machine learning and data mining. Ensemble learning, as a general definition, refers to the concept of being able to apply more than one learning model to a particular machine learning problem using some method of integration. The desired goal of course is that the ensemble as a unit will outperform any of its individual members for the given learning task. Ensemble learning has been extended to cover other learning tasks such as classification (refer to Kuncheva, 2004 for a detailed overview of this area), online learning (Fern & Givan, 2003) and clustering (Strehl & Ghosh, 2003). The focus of this article is to review ensemble learning with respect to regression, where by regression, we refer to the supervised learning task of creating a model that relates a continuous output variable to a vector of input variables.

scholarly journals Homeostatic structural plasticity leads to the formation of memory engrams through synaptic rewiring in recurrent networks

2020 ◽  
Author(s):  
Júlia V. Gallinaro ◽  
Nebojša Gašparović ◽  
Stefan Rotter
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
Diffusion Process ◽  
Structural Plasticity ◽  
Memory Formation ◽  
Homeostatic Plasticity ◽  
Spiking Neural Networks ◽  
Recurrent Networks ◽  
Hebbian Plasticity ◽  
Stimulation Of

AbstractBrain networks store new memories using functional and structural synaptic plasticity. Memory formation is generally attributed to Hebbian plasticity, while homeostatic plasticity is thought to have an ancillary role in stabilizing network dynamics. Here we report that homeostatic plasticity alone can also lead to the formation of stable memories. We analyze this phenomenon using a new theory of network remodeling, combined with numerical simulations of recurrent spiking neural networks that exhibit structural plasticity based on firing rate homeostasis. These networks are able to store repeatedly presented patterns and recall them upon the presentation of incomplete cues. Storing is fast, governed by the homeostatic drift. In contrast, forgetting is slow, driven by a diffusion process. Joint stimulation of neurons induces the growth of associative connections between them, leading to the formation of memory engrams. In conclusion, homeostatic structural plasticity induces a specific type of “silent memories”, different from conventional attractor states.

scholarly journals ASP: Learning to Forget With Adaptive Synaptic Plasticity in Spiking Neural Networks

2018 ◽  
Vol 8 (1) ◽  
pp. 51-64 ◽  
Author(s):  
Priyadarshini Panda ◽  
Jason M. Allred ◽  
Shriram Ramanathan ◽  
Kaushik Roy
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
Spiking Neural Networks
Sign in / Sign up

Export Citation Format

Share Document