scholarly journals Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations

2017 ◽  
Author(s):  
Shihui Yin ◽  
Shreyas K. Venkataramanaiah ◽  
Gregory K. Chen ◽  
Ram Krishnamurthy ◽  
Yu Cao ◽  
...  
Keyword(s):  
Neural Networks ◽  
Discrete Time ◽  
Back Propagation ◽  
Hardware Design ◽  
Spiking Neural Networks

Quantized state simulation of spiking neural networks

SIMULATION ◽  
2011 ◽  
Vol 88 (3) ◽  
pp. 299-313 ◽  
Author(s):  
Guillermo L Grinblat ◽  
Hernán Ahumada ◽  
Ernesto Kofman
Keyword(s):  
Neural Networks ◽  
Discrete Time ◽  
Discrete Event ◽  
State System ◽  
Spiking Neurons ◽  
Spiking Neural Networks ◽  
Computational Costs ◽  
Simulation Results ◽  
Neuroscience Community

In this work, we explore the usage of quantized state system (QSS) methods in the simulation of networks of spiking neurons. We compare the simulation results obtained by these discrete-event algorithms with the results of the discrete time methods in use by the neuroscience community. We found that the computational costs of the QSS methods grow almost linearly with the size of the network, while they grows at least quadratically in the discrete time algorithms. We show that this advantage is mainly due to the fact that QSS methods only perform calculations in the components of the system that experience activity.

scholarly journals Modelling an Adaptive Learning System Using Artificial Intelligence

Webology ◽  
2021 ◽  
Vol 19 (1) ◽  
pp. 01-18
Author(s):  
Hayder Rahm Dakheel AL-Fayyadh ◽  
Salam Abdulabbas Ganim Ali ◽  
Dr. Basim Abood
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Adaptive Learning ◽  
Back Propagation ◽  
Learning System ◽  
Spiking Neural Networks ◽  
Domain Specific Language ◽  
Domain Specific ◽  
Adaptive Learning System ◽  
Artificial Neural

The goal of this paper is to use artificial intelligence to build and evaluate an adaptive learning system where we adopt the basic approaches of spiking neural networks as well as artificial neural networks. Spiking neural networks receive increasing attention due to their advantages over traditional artificial neural networks. They have proven to be energy efficient, biological plausible, and up to 105 times faster if they are simulated on analogue traditional learning systems. Artificial neural network libraries use computational graphs as a pervasive representation, however, spiking models remain heterogeneous and difficult to train. Using the artificial intelligence deductive method, the paper posits two hypotheses that examines whether 1) there exists a common representation for both neural networks paradigms for tutorial mentoring, and whether 2) spiking and non-spiking models can learn a simple recognition task for learning activities for adaptive learning. The first hypothesis is confirmed by specifying and implementing a domain-specific language that generates semantically similar spiking and non-spiking neural networks for tutorial mentoring. Through three classification experiments, the second hypothesis is shown to hold for non-spiking models, but cannot be proven for the spiking models. The paper contributes three findings: 1) a domain-specific language for modelling neural network topologies in adaptive tutorial mentoring for students, 2) a preliminary model for generalizable learning through back-propagation in spiking neural networks for learning activities for students also represented in results section, and 3) a method for transferring optimised non-spiking parameters to spiking neural networks has also been developed for adaptive learning system. The latter contribution is promising because the vast machine learning literature can spill-over to the emerging field of spiking neural networks and adaptive learning computing. Future work includes improving the back-propagation model, exploring time-dependent models for learning, and adding support for adaptive learning systems.

Compact Hardware Synthesis of Stochastic Spiking Neural Networks

2019 ◽  
Vol 29 (08) ◽  
pp. 1950004 ◽  
Author(s):  
Fabio Galán-Prado ◽  
Alejandro Morán ◽  
Joan Font ◽  
Miquel Roca ◽  
Josep L. Rosselló
Keyword(s):  
Neural Networks ◽  
Processing Speed ◽  
Back Propagation ◽  
High Volume ◽  
High Density ◽  
Digital Design ◽  
Spiking Neural Networks ◽  
Biologically Inspired ◽  
Proposed Model ◽  
Technical Interest

Spiking neural networks (SNN) are able to emulate real neural behavior with high confidence due to their bio-inspired nature. Many designs have been proposed for the implementation of SNN in hardware, although the realization of high-density and biologically-inspired SNN is currently a complex challenge of high scientific and technical interest. In this work, we propose a compact digital design for the implementation of high-volume SNN that considers the intrinsic stochastic processes present in biological neurons and enables high-density hardware implementation. The proposed stochastic SNN model (SSNN) is compared with previous SSNN models, achieving a higher processing speed. We also show how the proposed model can be scaled to high-volume neural networks trained by using back propagation and applied to a pattern classification task. The proposed model achieves better results compared with other recently-published SNN models configured with unsupervised STDP learning.

scholarly journals Brain-inspired Balanced Tuning for Spiking Neural Networks

2018 ◽  
Author(s):  
Tielin Zhang ◽  
Yi Zeng ◽  
Dongcheng Zhao ◽  
Bo Xu
Keyword(s):  
Neural Networks ◽  
Membrane Potential ◽  
Back Propagation ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Learning Ability ◽  
Inhibitory Neurons ◽  
Final Layer ◽  
Weakly Supervised ◽  
Learning Principles

Due to the nature of Spiking Neural Networks (SNNs), it is challenging to be trained by biologically plausible learning principles. The multi-layered SNNs are with non-differential neurons, temporary-centric synapses, which make them nearly impossible to be directly tuned by back propagation. Here we propose an alternative biological inspired balanced tuning approach to train SNNs. The approach contains three main inspirations from the brain: Firstly, the biological network will usually be trained towards the state where the temporal update of variables are equilibrium (e.g. membrane potential); Secondly, specific proportions of excitatory and inhibitory neurons usually contribute to stable representations; Thirdly, the short-term plasticity (STP) is a general principle to keep the input and output of synapses balanced towards a better learning convergence. With these inspirations, we train SNNs with three steps: Firstly, the SNN model is trained with three brain-inspired principles; then weakly supervised learning is used to tune the membrane potential in the final layer for network classification; finally the learned information is consolidated from membrane potential into the weights of synapses by Spike-Timing Dependent Plasticity (STDP). The proposed approach is verified on the MNIST hand-written digit recognition dataset and the performance (the accuracy of 98.64%) indicates that the ideas of balancing state could indeed improve the learning ability of SNNs, which shows the power of proposed brain-inspired approach on the tuning of biological plausible SNNs.

scholarly journals Toward One-Shot Learning in Neuroscience-Inspired Deep Spiking Neural Networks

2019 ◽  
Author(s):  
Faramarz Faghihi ◽  
Hossein Molhem ◽  
Ahmed A. Moustafa
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Learning Rule ◽  
Spiking Neural Networks ◽  
Spiking Neural Network ◽  
Essential Information ◽  
Information Channels ◽  
Training Examples

AbstractConventional deep neural networks capture essential information processing stages in perception. Deep neural networks often require very large volume of training examples, whereas children can learn concepts such as hand-written digits with few examples. The goal of this project is to develop a deep spiking neural network that can learn from few training trials. Using known neuronal mechanisms, a spiking neural network model is developed and trained to recognize hand-written digits with presenting one to four training examples for each digit taken from the MNIST database. The model detects and learns geometric features of the images from MNIST database. In this work, a novel biological back-propagation based learning rule is developed and used to a train the network to detect basic features of different digits. For this purpose, randomly initialized synaptic weights between the layers are being updated. By using a neuroscience inspired mechanism named ‘synaptic pruning’ and a predefined threshold, some of the synapses through the training are deleted. Hence, information channels are constructed that are highly specific for each digit as matrix of synaptic connections between two layers of spiking neural networks. These connection matrixes named ‘information channels’ are used in the test phase to assign a digit class to each test image. As similar to humans’ abilities to learn from small training trials, the developed spiking neural network needs a very small dataset for training, compared to conventional deep learning methods checked on MNIST dataset.

Fuzzy ART Neural Network Model for Automated Detection of Freeway Incidents

1998 ◽  
Vol 1634 (1) ◽  
pp. 56-63 ◽  
Author(s):  
Sherif S. Ishak ◽  
Haitham M. Al-Deek
Keyword(s):  
Neural Networks ◽  
False Alarm ◽  
False Alarm Rate ◽  
Back Propagation ◽  
Incident Detection ◽  
Traffic Patterns ◽  
Traffic Pattern ◽  
Detection Algorithms ◽  
Fuzzy Art ◽  
Freeway Incidents

Pattern recognition techniques such as artificial neural networks continue to offer potential solutions to many of the existing problems associated with freeway incident-detection algorithms. This study focuses on the application of Fuzzy ART neural networks to incident detection on freeways. Unlike back-propagation models, Fuzzy ART is capable of fast, stable learning of recognition categories. It is an incremental approach that has the potential for on-line implementation. Fuzzy ART is trained with traffic patterns that are represented by 30-s loop-detector data of occupancy, speed, or a combination of both. Traffic patterns observed at the incident time and location are mapped to a group of categories. Each incident category maps incidents with similar traffic pattern characteristics, which are affected by the type and severity of the incident and the prevailing traffic conditions. Detection rate and false alarm rate are used to measure the performance of the Fuzzy ART algorithm. To reduce the false alarm rate that results from occasional misclassification of traffic patterns, a persistence time period of 3 min was arbitrarily selected. The algorithm performance improves when the temporal size of traffic patterns increases from one to two 30-s periods for all traffic parameters. An interesting finding is that the speed patterns produced better results than did the occupancy patterns. However, when combined, occupancy–speed patterns produced the best results. When compared with California algorithms 7 and 8, the Fuzzy ART model produced better performance.

Sign in / Sign up

Export Citation Format

Share Document