scholarly journals A heuristic approach to the hyperparameters in training spiking neural networks using spike-timing-dependent plasticity

2021 ◽  
Author(s):  
Dawid Połap ◽  
Marcin Woźniak ◽  
Waldemar Hołubowski ◽  
Robertas Damaševičius
Keyword(s):  
Neural Networks ◽  
Optimization Problem ◽  
Heuristic Algorithms ◽  
Search Algorithm ◽  
Cuckoo Search ◽  
Spike Timing ◽  
Heuristic Approach ◽  
Spiking Neural Networks ◽  
Grasshopper Optimization Algorithm ◽  
Selection Of

AbstractThe third type of neural network called spiking is developed due to a more accurate representation of neuronal activity in living organisms. Spiking neural networks have many different parameters that can be difficult to adjust manually to the current classification problem. The analysis and selection of coefficients’ values in the network can be analyzed as an optimization problem. A practical method for automatic selection of them can decrease the time needed to develop such a model. In this paper, we propose the use of a heuristic approach to analyze and select coefficients with the idea of collaborative working. The proposed idea is based on parallel analyzing of different coefficients and choosing the best of them or average ones. This type of optimization problem allows the selection of all variables, which can significantly affect the convergence of the accuracy. Our proposal was tested using network simulators and popular databases to indicate the possibilities of the described approach. Five different heuristic algorithms were tested and the best results were reached by Cuckoo Search Algorithm, Grasshopper Optimization Algorithm, and Polar Bears Algorithm.

scholarly journals Memory Based Cuckoo Search Algorithm for Feature Selection of Gene Expression Dataset

2021 ◽  
pp. 100572
Author(s):  
Malek Alzaqebah ◽  
Khaoula Briki ◽  
Nashat Alrefai ◽  
Sami Brini ◽  
Sana Jawarneh ◽  
...  
Keyword(s):  
Gene Expression ◽  
Feature Selection ◽  
Search Algorithm ◽  
Cuckoo Search ◽  
Cuckoo Search Algorithm ◽  
Gene Expression Dataset ◽  
Selection Of

scholarly journals Biological modelling of a computational spiking neural network with neuronal avalanches

2017 ◽  
Vol 375 (2096) ◽  
pp. 20160286 ◽  
Author(s):  
Xiumin Li ◽  
Qing Chen ◽  
Fangzheng Xue
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Critical State ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Mathematical Methods ◽  
Computational Accuracy ◽  
Active Neuron ◽  
Computational Performance ◽  
Neuronal Avalanches

In recent years, an increasing number of studies have demonstrated that networks in the brain can self-organize into a critical state where dynamics exhibit a mixture of ordered and disordered patterns. This critical branching phenomenon is termed neuronal avalanches. It has been hypothesized that the homeostatic level balanced between stability and plasticity of this critical state may be the optimal state for performing diverse neural computational tasks. However, the critical region for high performance is narrow and sensitive for spiking neural networks (SNNs). In this paper, we investigated the role of the critical state in neural computations based on liquid-state machines, a biologically plausible computational neural network model for real-time computing. The computational performance of an SNN when operating at the critical state and, in particular, with spike-timing-dependent plasticity for updating synaptic weights is investigated. The network is found to show the best computational performance when it is subjected to critical dynamic states. Moreover, the active-neuron-dominant structure refined from synaptic learning can remarkably enhance the robustness of the critical state and further improve computational accuracy. These results may have important implications in the modelling of spiking neural networks with optimal computational performance. This article is part of the themed issue ‘Mathematical methods in medicine: neuroscience, cardiology and pathology’.

An improved sparrow search algorithm based on levy flight and opposition-based learning

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Danni Chen ◽  
JianDong Zhao ◽  
Peng Huang ◽  
Xiongna Deng ◽  
Tingting Lu
Keyword(s):  
Parameter Optimization ◽  
Optimization Problem ◽  
Heuristic Algorithms ◽  
Search Algorithm ◽  
Convergence Speed ◽  
Support Vector ◽  
Lévy Flight ◽  
Levy Flight ◽  
Content Type ◽  
Opposition Based Learning

Purpose Sparrow search algorithm (SSA) is a novel global optimization method, but it is easy to fall into local optimization, which leads to its poor search accuracy and stability. The purpose of this study is to propose an improved SSA algorithm, called levy flight and opposition-based learning (LOSSA), based on LOSSA strategy. The LOSSA shows better search accuracy, faster convergence speed and stronger stability. Design/methodology/approach To further enhance the optimization performance of the algorithm, The Levy flight operation is introduced into the producers search process of the original SSA to enhance the ability of the algorithm to jump out of the local optimum. The opposition-based learning strategy generates better solutions for SSA, which is beneficial to accelerate the convergence speed of the algorithm. On the one hand, the performance of the LOSSA is evaluated by a set of numerical experiments based on classical benchmark functions. On the other hand, the hyper-parameter optimization problem of the Support Vector Machine (SVM) is also used to test the ability of LOSSA to solve practical problems. Findings First of all, the effectiveness of the two improved methods is verified by Wilcoxon signed rank test. Second, the statistical results of the numerical experiment show the significant improvement of the LOSSA compared with the original algorithm and other natural heuristic algorithms. Finally, the feasibility and effectiveness of the LOSSA in solving the hyper-parameter optimization problem of machine learning algorithms are demonstrated. Originality/value An improved SSA based on LOSSA is proposed in this paper. The experimental results show that the overall performance of the LOSSA is satisfactory. Compared with the SSA and other natural heuristic algorithms, the LOSSA shows better search accuracy, faster convergence speed and stronger stability. Moreover, the LOSSA also showed great optimization performance in the hyper-parameter optimization of the SVM model.

scholarly journals Biologically Plausible Sequence Learning with Spiking Neural Networks

2020 ◽  
Vol 34 (02) ◽  
pp. 1316-1323
Author(s):  
Zuozhu Liu ◽  
Thiparat Chotibut ◽  
Christopher Hillar ◽  
Shaowei Lin
Keyword(s):  
Neural Networks ◽  
Sequence Learning ◽  
Nervous Activity ◽  
Learning Rule ◽  
Generative Models ◽  
Spike Timing ◽  
Hopfield Network ◽  
Spiking Neural Networks ◽  
Theoretical Ground ◽  
Time Model

Motivated by the celebrated discrete-time model of nervous activity outlined by McCulloch and Pitts in 1943, we propose a novel continuous-time model, the McCulloch-Pitts network (MPN), for sequence learning in spiking neural networks. Our model has a local learning rule, such that the synaptic weight updates depend only on the information directly accessible by the synapse. By exploiting asymmetry in the connections between binary neurons, we show that MPN can be trained to robustly memorize multiple spatiotemporal patterns of binary vectors, generalizing the ability of the symmetric Hopfield network to memorize static spatial patterns. In addition, we demonstrate that the model can efficiently learn sequences of binary pictures as well as generative models for experimental neural spike-train data. Our learning rule is consistent with spike-timing-dependent plasticity (STDP), thus providing a theoretical ground for the systematic design of biologically inspired networks with large and robust long-range sequence storage capacity.

Neural Autopoiesis: Organizing Self-Boundaries by Stimulus Avoidance in Biological and Artificial Neural Networks

2020 ◽  
Vol 26 (1) ◽  
pp. 130-151 ◽  
Author(s):  
Atsushi Masumori ◽  
Lana Sinapayen ◽  
Norihiro Maruyama ◽  
Takeshi Mita ◽  
Douglas Bakkum ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Environmental Changes ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Cultured Neurons ◽  
Autonomous Regulation ◽  
Artificial Neural ◽  
Living Organisms ◽  
External Stimuli

Living organisms must actively maintain themselves in order to continue existing. Autopoiesis is a key concept in the study of living organisms, where the boundaries of the organism are not static but dynamically regulated by the system itself. To study the autonomous regulation of a self-boundary, we focus on neural homeodynamic responses to environmental changes using both biological and artificial neural networks. Previous studies showed that embodied cultured neural networks and spiking neural networks with spike-timing dependent plasticity (STDP) learn an action as they avoid stimulation from outside. In this article, as a result of our experiments using embodied cultured neurons, we find that there is also a second property allowing the network to avoid stimulation: If the agent cannot learn an action to avoid the external stimuli, it tends to decrease the stimulus-evoked spikes, as if to ignore the uncontrollable input. We also show such a behavior is reproduced by spiking neural networks with asymmetric STDP. We consider that these properties are to be regarded as autonomous regulation of self and nonself for the network, in which a controllable neuron is regarded as self, and an uncontrollable neuron is regarded as nonself. Finally, we introduce neural autopoiesis by proposing the principle of stimulus avoidance.

scholarly journals Memristive Spiking Neural Networks Trained with Unsupervised STDP

Electronics ◽  
2018 ◽  
Vol 7 (12) ◽  
pp. 396 ◽  
Author(s):  
Errui Zhou ◽  
Liang Fang ◽  
Binbin Yang
Keyword(s):  
Neural Networks ◽  
Feature Learning ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Learning Method ◽  
Hardware Complexity ◽  
Neuromorphic Computing ◽  
Computing Systems ◽  
Von Neumann ◽  
Complex Architectures

Neuromorphic computing systems are promising alternatives in the fields of pattern recognition, image processing, etc. especially when conventional von Neumann architectures face several bottlenecks. Memristors play vital roles in neuromorphic computing systems and are usually used as synaptic devices. Memristive spiking neural networks (MSNNs) are considered to be more efficient and biologically plausible than other systems due to their spike-based working mechanism. In contrast to previous SNNs with complex architectures, we propose a hardware-friendly architecture and an unsupervised spike-timing dependent plasticity (STDP) learning method for MSNNs in this paper. The architecture, which is friendly to hardware implementation, includes an input layer, a feature learning layer and a voting circuit. To reduce hardware complexity, some constraints are enforced: the proposed architecture has no lateral inhibition and is purely feedforward; it uses the voting circuit as a classifier and does not use additional classifiers; all neurons can generate at most one spike and do not need to consider firing rates and refractory periods; all neurons have the same fixed threshold voltage for classification. The presented unsupervised STDP learning method is time-dependent and uses no homeostatic mechanism. The MNIST dataset is used to demonstrate our proposed architecture and learning method. Simulation results show that our proposed architecture with the learning method achieves a classification accuracy of 94.6%, which outperforms other unsupervised SNNs that use time-based encoding schemes.

scholarly journals Identification of suitable websites for digital marketing – an approach using bio-inspired computing

2017 ◽  
Vol 7 (1.2) ◽  
pp. 239 ◽  
Author(s):  
B. Suresh Kumar ◽  
Deepshikha Bharghava ◽  
Arpan Kumar Kar ◽  
Chinwe Peace Igiri
Keyword(s):  
Search Algorithm ◽  
Cuckoo Search ◽  
Integrated Approach ◽  
Cuckoo Search Algorithm ◽  
Internet Usage ◽  
Digital Marketing ◽  
Assessment Criteria ◽  
Fast Pace ◽  
The Web ◽  
Selection Of

Due to the immense growth of Internet usage, the point of convergence has moved from physical to the web. The size of the web is increasing at a very fast pace to cater to the fast-evolving needs of the businesses, governments, and societies. However, selecting or identifying the best website is challenging. The practical issue to solve the problem comprises two parts. The first part is to identify the assessment criteria for appraising websites. Second is to evaluate the websites in the context of these assessment criteria and screen them to address a specific need. However, this objective is extremely complex and computationally extremely expensive. This research proposes an approach to identify websites from the Internet. The proposed integrated approach uses the Henry Garrett ranking method and cuckoo search algorithm for ranking and selection of websites for planning digital marketing campaigns.

Sign in / Sign up

Export Citation Format

Share Document