scholarly journals Multi-Objective Differential Evolution of Evolving Spiking Neural Networks for Classification Problems

2015 ◽  
pp. 351-368 ◽  
Author(s):  
Abdulrazak Yahya Saleh ◽  
Siti Mariyam Shamsuddin ◽  
Haza Nuzly Abdull Hamed
Keyword(s):  
Neural Networks ◽  
Differential Evolution ◽  
Spiking Neural Networks ◽  
Classification Problems ◽  
Multi Objective

SPIKING NEURAL NETWORKS

2009 ◽  
Vol 19 (04) ◽  
pp. 295-308 ◽  
Author(s):  
SAMANWOY GHOSH-DASTIDAR ◽  
HOJJAT ADELI
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Information Transfer ◽  
Spiking Neurons ◽  
Spiking Neural Networks ◽  
Classification Problems ◽  
Temporal Dimension ◽  
Information Encoding ◽  
Dynamic Representation ◽  
Insight Into

Most current Artificial Neural Network (ANN) models are based on highly simplified brain dynamics. They have been used as powerful computational tools to solve complex pattern recognition, function estimation, and classification problems. ANNs have been evolving towards more powerful and more biologically realistic models. In the past decade, Spiking Neural Networks (SNNs) have been developed which comprise of spiking neurons. Information transfer in these neurons mimics the information transfer in biological neurons, i.e., via the precise timing of spikes or a sequence of spikes. To facilitate learning in such networks, new learning algorithms based on varying degrees of biological plausibility have also been developed recently. Addition of the temporal dimension for information encoding in SNNs yields new insight into the dynamics of the human brain and could result in compact representations of large neural networks. As such, SNNs have great potential for solving complicated time-dependent pattern recognition problems because of their inherent dynamic representation. This article presents a state-of-the-art review of the development of spiking neurons and SNNs, and provides insight into their evolution as the third generation neural networks.

scholarly journals The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

2020 ◽  
Author(s):  
Friedemann Zenke ◽  
Tim P. Vogels
Keyword(s):  
Neural Networks ◽  
Complex Function ◽  
Spiking Neural Networks ◽  
Learning Performance ◽  
Design Parameters ◽  
Classification Problems ◽  
Systematic Account ◽  
Practical Algorithms ◽  
Spiking Networks ◽  
Gradient Learning

AbstractBrains process information in spiking neural networks. Their intricate connections shape the diverse functions these networks perform. In comparison, the functional capabilities of models of spiking networks are still rudimentary. This shortcoming is mainly due to the lack of insight and practical algorithms to construct the necessary connectivity. Any such algorithm typically attempts to build networks by iteratively reducing the error compared to a desired output. But assigning credit to hidden units in multi-layered spiking networks has remained challenging due to the non-differentiable nonlinearity of spikes. To avoid this issue, one can employ surrogate gradients to discover the required connectivity in spiking network models. However, the choice of a surrogate is not unique, raising the question of how its implementation influences the effectiveness of the method. Here, we use numerical simulations to systematically study how essential design parameters of surrogate gradients impact learning performance on a range of classification problems. We show that surrogate gradient learning is robust to different shapes of underlying surrogate derivatives, but the choice of the derivative’s scale can substantially affect learning performance. When we combine surrogate gradients with a suitable activity regularization technique, robust information processing can be achieved in spiking networks even at the sparse activity limit. Our study provides a systematic account of the remarkable robustness of surrogate gradient learning and serves as a practical guide to model functional spiking neural networks.

scholarly journals Evolutionary Spiking Neural Networks for Solving Supervised Classification Problems

2019 ◽  
Vol 2019 ◽  
pp. 1-13 ◽  
Author(s):  
G. López-Vázquez ◽  
M. Ornelas-Rodriguez ◽  
A. Espinal ◽  
J. A. Soria-Alcaraz ◽  
A. Rojas-Domínguez ◽  
...  
Keyword(s):  
Neural Networks ◽  
Supervised Classification ◽  
Fitness Function ◽  
Statistical Tests ◽  
Search Space ◽  
Spiking Neural Networks ◽  
Classification Problems ◽  
Benchmark Datasets ◽  
Explicit Training ◽  
Better Than

This paper presents a grammatical evolution (GE)-based methodology to automatically design third generation artificial neural networks (ANNs), also known as spiking neural networks (SNNs), for solving supervised classification problems. The proposal performs the SNN design by exploring the search space of three-layered feedforward topologies with configured synaptic connections (weights and delays) so that no explicit training is carried out. Besides, the designed SNNs have partial connections between input and hidden layers which may contribute to avoid redundancies and reduce the dimensionality of input feature vectors. The proposal was tested on several well-known benchmark datasets from the UCI repository and statistically compared against a similar design methodology for second generation ANNs and an adapted version of that methodology for SNNs; also, the results of the two methodologies and the proposed one were improved by changing the fitness function in the design process. The proposed methodology shows competitive and consistent results, and the statistical tests support the conclusion that the designs produced by the proposal perform better than those produced by other methodologies.

scholarly journals Differential Evolution for Neural Networks Optimization

Mathematics ◽  
2020 ◽  
Vol 8 (1) ◽  
pp. 69 ◽  
Author(s):  
Marco Baioletti ◽  
Gabriele Di Bari ◽  
Alfredo Milani ◽  
Valentina Poggioni
Keyword(s):  
Neural Networks ◽  
Comparative Study ◽  
Differential Evolution ◽  
Adaptive Methods ◽  
Experimental Results ◽  
Classification Problems ◽  
Adaptive Differential Evolution ◽  
Neural Networks Optimization ◽  
Better Than ◽  
Self Adaptive

In this paper, a Neural Networks optimizer based on Self-adaptive Differential Evolution is presented. This optimizer applies mutation and crossover operators in a new way, taking into account the structure of the network according to a per layer strategy. Moreover, a new crossover called interm is proposed, and a new self-adaptive version of DE called MAB-ShaDE is suggested to reduce the number of parameters. The framework has been tested on some well-known classification problems and a comparative study on the various combinations of self-adaptive methods, mutation, and crossover operators available in literature is performed. Experimental results show that DENN reaches good performances in terms of accuracy, better than or at least comparable with those obtained by backpropagation.

scholarly journals Supervised Learning With First-to-Spike Decoding in Multilayer Spiking Neural Networks

2021 ◽  
Vol 15 ◽  
Author(s):  
Brian Gardner ◽  
André Grüning
Keyword(s):  
Neural Networks ◽  
Supervised Learning ◽  
Experimental Studies ◽  
Learning Rule ◽  
Image Data ◽  
Classification Performance ◽  
Spiking Neural Networks ◽  
Classification Problems ◽  
Sensory Stimuli ◽  
Wide Range

Experimental studies support the notion of spike-based neuronal information processing in the brain, with neural circuits exhibiting a wide range of temporally-based coding strategies to rapidly and efficiently represent sensory stimuli. Accordingly, it would be desirable to apply spike-based computation to tackling real-world challenges, and in particular transferring such theory to neuromorphic systems for low-power embedded applications. Motivated by this, we propose a new supervised learning method that can train multilayer spiking neural networks to solve classification problems based on a rapid, first-to-spike decoding strategy. The proposed learning rule supports multiple spikes fired by stochastic hidden neurons, and yet is stable by relying on first-spike responses generated by a deterministic output layer. In addition to this, we also explore several distinct, spike-based encoding strategies in order to form compact representations of presented input data. We demonstrate the classification performance of the learning rule as applied to several benchmark datasets, including MNIST. The learning rule is capable of generalizing from the data, and is successful even when used with constrained network architectures containing few input and hidden layer neurons. Furthermore, we highlight a novel encoding strategy, termed “scanline encoding,” that can transform image data into compact spatiotemporal patterns for subsequent network processing. Designing constrained, but optimized, network structures and performing input dimensionality reduction has strong implications for neuromorphic applications.

Sign in / Sign up

Export Citation Format

Share Document