scholarly journals Brain-inspired Balanced Tuning for Spiking Neural Networks

2018 ◽  
Author(s):  
Tielin Zhang ◽  
Yi Zeng ◽  
Dongcheng Zhao ◽  
Bo Xu
Keyword(s):  
Neural Networks ◽  
Membrane Potential ◽  
Back Propagation ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Learning Ability ◽  
Inhibitory Neurons ◽  
Final Layer ◽  
Weakly Supervised ◽  
Learning Principles

Due to the nature of Spiking Neural Networks (SNNs), it is challenging to be trained by biologically plausible learning principles. The multi-layered SNNs are with non-differential neurons, temporary-centric synapses, which make them nearly impossible to be directly tuned by back propagation. Here we propose an alternative biological inspired balanced tuning approach to train SNNs. The approach contains three main inspirations from the brain: Firstly, the biological network will usually be trained towards the state where the temporal update of variables are equilibrium (e.g. membrane potential); Secondly, specific proportions of excitatory and inhibitory neurons usually contribute to stable representations; Thirdly, the short-term plasticity (STP) is a general principle to keep the input and output of synapses balanced towards a better learning convergence. With these inspirations, we train SNNs with three steps: Firstly, the SNN model is trained with three brain-inspired principles; then weakly supervised learning is used to tune the membrane potential in the final layer for network classification; finally the learned information is consolidated from membrane potential into the weights of synapses by Spike-Timing Dependent Plasticity (STDP). The proposed approach is verified on the MNIST hand-written digit recognition dataset and the performance (the accuracy of 98.64%) indicates that the ideas of balancing state could indeed improve the learning ability of SNNs, which shows the power of proposed brain-inspired approach on the tuning of biological plausible SNNs.

scholarly journals The Effects of Radiation on Memristor-Based Electronic Spiking Neural Networks

2020 ◽  
Author(s):  
Sumedha Gandharava Dahl
Keyword(s):  
Neural Networks ◽  
Radiation Effects ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Learning Ability ◽  
Pattern Learning ◽  
Neuron Death ◽  
Learning Capability ◽  
Spatio Temporal ◽  
The Impact

In this dissertation, memristor-based spiking neural networks (SNNs) are used to analyze the effect of radiation on the spatio-temporal pattern recognition (STPR) capability of the networks. Two-terminal resistive memory devices (memristors) are used as synapses to manipulate conductivity paths in the network. Spike-timing-dependent plasticity (STDP) learning behavior results in pattern learning and is achieved using biphasic shaped pre- and post-synaptic spikes. A TiO2 based non-linear drift memristor model designed in Verilog-A implements synaptic behavior and is modified to include experimentally observed effects of state-altering, ionizing, and off-state degradation radiation on the device. The impact of neuron "death" (disabled neuron circuits) due to radiation is also examined. In general, radiation interaction events distort the STDP learning curve undesirably, favoring synaptic potentiation. At lower short-term flux, the network is able to recover and relearn the pattern with consistent training, although some pixels may be affected due to stability issues. As the radiation flux and duration increases, it can overwhelm the leaky integrate-and-fire (LIF) post-synaptic neuron circuit, and the network does not learn the pattern. On the other hand, in the absence of the pattern, the radiation effects cumulate, and the system never regains stability. Neuron-death simulation results emphasize the importance of non-participating neurons during the learning process, concluding that non-participating afferents contribute to improving the learning ability of the neural network. Instantaneous neuron death proves to be more detrimental for the network compared to when the afferents die over time thus, retaining the network's pattern learning capability.

scholarly journals Biological modelling of a computational spiking neural network with neuronal avalanches

2017 ◽  
Vol 375 (2096) ◽  
pp. 20160286 ◽  
Author(s):  
Xiumin Li ◽  
Qing Chen ◽  
Fangzheng Xue
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Critical State ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Mathematical Methods ◽  
Computational Accuracy ◽  
Active Neuron ◽  
Computational Performance ◽  
Neuronal Avalanches

In recent years, an increasing number of studies have demonstrated that networks in the brain can self-organize into a critical state where dynamics exhibit a mixture of ordered and disordered patterns. This critical branching phenomenon is termed neuronal avalanches. It has been hypothesized that the homeostatic level balanced between stability and plasticity of this critical state may be the optimal state for performing diverse neural computational tasks. However, the critical region for high performance is narrow and sensitive for spiking neural networks (SNNs). In this paper, we investigated the role of the critical state in neural computations based on liquid-state machines, a biologically plausible computational neural network model for real-time computing. The computational performance of an SNN when operating at the critical state and, in particular, with spike-timing-dependent plasticity for updating synaptic weights is investigated. The network is found to show the best computational performance when it is subjected to critical dynamic states. Moreover, the active-neuron-dominant structure refined from synaptic learning can remarkably enhance the robustness of the critical state and further improve computational accuracy. These results may have important implications in the modelling of spiking neural networks with optimal computational performance. This article is part of the themed issue ‘Mathematical methods in medicine: neuroscience, cardiology and pathology’.

scholarly journals Biologically Plausible Sequence Learning with Spiking Neural Networks

2020 ◽  
Vol 34 (02) ◽  
pp. 1316-1323
Author(s):  
Zuozhu Liu ◽  
Thiparat Chotibut ◽  
Christopher Hillar ◽  
Shaowei Lin
Keyword(s):  
Neural Networks ◽  
Sequence Learning ◽  
Nervous Activity ◽  
Learning Rule ◽  
Generative Models ◽  
Spike Timing ◽  
Hopfield Network ◽  
Spiking Neural Networks ◽  
Theoretical Ground ◽  
Time Model

Motivated by the celebrated discrete-time model of nervous activity outlined by McCulloch and Pitts in 1943, we propose a novel continuous-time model, the McCulloch-Pitts network (MPN), for sequence learning in spiking neural networks. Our model has a local learning rule, such that the synaptic weight updates depend only on the information directly accessible by the synapse. By exploiting asymmetry in the connections between binary neurons, we show that MPN can be trained to robustly memorize multiple spatiotemporal patterns of binary vectors, generalizing the ability of the symmetric Hopfield network to memorize static spatial patterns. In addition, we demonstrate that the model can efficiently learn sequences of binary pictures as well as generative models for experimental neural spike-train data. Our learning rule is consistent with spike-timing-dependent plasticity (STDP), thus providing a theoretical ground for the systematic design of biologically inspired networks with large and robust long-range sequence storage capacity.

Neural Autopoiesis: Organizing Self-Boundaries by Stimulus Avoidance in Biological and Artificial Neural Networks

2020 ◽  
Vol 26 (1) ◽  
pp. 130-151 ◽  
Author(s):  
Atsushi Masumori ◽  
Lana Sinapayen ◽  
Norihiro Maruyama ◽  
Takeshi Mita ◽  
Douglas Bakkum ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Environmental Changes ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Cultured Neurons ◽  
Autonomous Regulation ◽  
Artificial Neural ◽  
Living Organisms ◽  
External Stimuli

Living organisms must actively maintain themselves in order to continue existing. Autopoiesis is a key concept in the study of living organisms, where the boundaries of the organism are not static but dynamically regulated by the system itself. To study the autonomous regulation of a self-boundary, we focus on neural homeodynamic responses to environmental changes using both biological and artificial neural networks. Previous studies showed that embodied cultured neural networks and spiking neural networks with spike-timing dependent plasticity (STDP) learn an action as they avoid stimulation from outside. In this article, as a result of our experiments using embodied cultured neurons, we find that there is also a second property allowing the network to avoid stimulation: If the agent cannot learn an action to avoid the external stimuli, it tends to decrease the stimulus-evoked spikes, as if to ignore the uncontrollable input. We also show such a behavior is reproduced by spiking neural networks with asymmetric STDP. We consider that these properties are to be regarded as autonomous regulation of self and nonself for the network, in which a controllable neuron is regarded as self, and an uncontrollable neuron is regarded as nonself. Finally, we introduce neural autopoiesis by proposing the principle of stimulus avoidance.

scholarly journals Memristive Spiking Neural Networks Trained with Unsupervised STDP

Electronics ◽  
2018 ◽  
Vol 7 (12) ◽  
pp. 396 ◽  
Author(s):  
Errui Zhou ◽  
Liang Fang ◽  
Binbin Yang
Keyword(s):  
Neural Networks ◽  
Feature Learning ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Learning Method ◽  
Hardware Complexity ◽  
Neuromorphic Computing ◽  
Computing Systems ◽  
Von Neumann ◽  
Complex Architectures

Neuromorphic computing systems are promising alternatives in the fields of pattern recognition, image processing, etc. especially when conventional von Neumann architectures face several bottlenecks. Memristors play vital roles in neuromorphic computing systems and are usually used as synaptic devices. Memristive spiking neural networks (MSNNs) are considered to be more efficient and biologically plausible than other systems due to their spike-based working mechanism. In contrast to previous SNNs with complex architectures, we propose a hardware-friendly architecture and an unsupervised spike-timing dependent plasticity (STDP) learning method for MSNNs in this paper. The architecture, which is friendly to hardware implementation, includes an input layer, a feature learning layer and a voting circuit. To reduce hardware complexity, some constraints are enforced: the proposed architecture has no lateral inhibition and is purely feedforward; it uses the voting circuit as a classifier and does not use additional classifiers; all neurons can generate at most one spike and do not need to consider firing rates and refractory periods; all neurons have the same fixed threshold voltage for classification. The presented unsupervised STDP learning method is time-dependent and uses no homeostatic mechanism. The MNIST dataset is used to demonstrate our proposed architecture and learning method. Simulation results show that our proposed architecture with the learning method achieves a classification accuracy of 94.6%, which outperforms other unsupervised SNNs that use time-based encoding schemes.

scholarly journals LogicSNN: A Unified Spiking Neural Networks Logical Operation Paradigm

Electronics ◽  
2021 ◽  
Vol 10 (17) ◽  
pp. 2123 ◽  
Author(s):  
Lingfei Mo ◽  
Minghao Wang
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Network Structure ◽  
Large Scale ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Logical Operation ◽  
Large Scale Network ◽  
Learning Capabilities ◽  
Scale Network

LogicSNN, a unified spiking neural networks (SNN) logical operation paradigm is proposed in this paper. First, we define the logical variables under the semantics of SNN. Then, we design the network structure of this paradigm and use spike-timing-dependent plasticity for training. According to this paradigm, six kinds of basic SNN binary logical operation modules and three kinds of combined logical networks based on these basic modules are implemented. Through these experiments, the rationality, cascading characteristics and the potential of building large-scale network of this paradigm are verified. This study fills in the blanks of the logical operation of SNN and provides a possible way to realize more complex machine learning capabilities.

scholarly journals Modelling an Adaptive Learning System Using Artificial Intelligence

Webology ◽  
2021 ◽  
Vol 19 (1) ◽  
pp. 01-18
Author(s):  
Hayder Rahm Dakheel AL-Fayyadh ◽  
Salam Abdulabbas Ganim Ali ◽  
Dr. Basim Abood
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Adaptive Learning ◽  
Back Propagation ◽  
Learning System ◽  
Spiking Neural Networks ◽  
Domain Specific Language ◽  
Domain Specific ◽  
Adaptive Learning System ◽  
Artificial Neural

The goal of this paper is to use artificial intelligence to build and evaluate an adaptive learning system where we adopt the basic approaches of spiking neural networks as well as artificial neural networks. Spiking neural networks receive increasing attention due to their advantages over traditional artificial neural networks. They have proven to be energy efficient, biological plausible, and up to 105 times faster if they are simulated on analogue traditional learning systems. Artificial neural network libraries use computational graphs as a pervasive representation, however, spiking models remain heterogeneous and difficult to train. Using the artificial intelligence deductive method, the paper posits two hypotheses that examines whether 1) there exists a common representation for both neural networks paradigms for tutorial mentoring, and whether 2) spiking and non-spiking models can learn a simple recognition task for learning activities for adaptive learning. The first hypothesis is confirmed by specifying and implementing a domain-specific language that generates semantically similar spiking and non-spiking neural networks for tutorial mentoring. Through three classification experiments, the second hypothesis is shown to hold for non-spiking models, but cannot be proven for the spiking models. The paper contributes three findings: 1) a domain-specific language for modelling neural network topologies in adaptive tutorial mentoring for students, 2) a preliminary model for generalizable learning through back-propagation in spiking neural networks for learning activities for students also represented in results section, and 3) a method for transferring optimised non-spiking parameters to spiking neural networks has also been developed for adaptive learning system. The latter contribution is promising because the vast machine learning literature can spill-over to the emerging field of spiking neural networks and adaptive learning computing. Future work includes improving the back-propagation model, exploring time-dependent models for learning, and adding support for adaptive learning systems.

Sign in / Sign up

Export Citation Format

Share Document