Training oscillatory neural networks using natural gradient particle swarm optimization

Robotica ◽  
2014 ◽  
Vol 33 (7) ◽  
pp. 1551-1567 ◽  
Author(s):  
Hamed Shahbazi ◽  
Kamal Jamshidi ◽  
Amir Hasan Monadjemi ◽  
Hafez Eslami Manoochehri
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Transfer Functions ◽  
Learning Algorithm ◽  
Particle Swarm ◽  
Humanoid Robots ◽  
Unknown Parameters ◽  
Swarm Optimization ◽  
Oscillatory Neural Network

SUMMARYIn this paper, a new design of neural networks is introduced, which is able to generate oscillatory patterns in its output. The oscillatory neural network is used in a biped robot to enable it to learn to walk. The fundamental building block of the neural network proposed in this paper is O-neurons, which can generate oscillations in its transfer functions. O-neurons are connected and coupled with each other in order to shape a network, and their unknown parameters are found by a particle swarm optimization method. The main contribution of this paper is the learning algorithm that can combine natural policy gradient with particle swarm optimization methods. The oscillatory neural network has six outputs that determine set points for proportional-integral-derivative controllers in 6-DOF humanoid robots. Our experiment on the simulated humanoid robot presents smooth and flexible walking.

Using Particle Swarm Optimization Algorithm as an Optimization Tool Within Developed Neural Networks

2018 ◽  
pp. 215-244
Author(s):  
Goran Klepac
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Evolutionary Algorithms ◽  
Optimization Algorithm ◽  
Recommendation System ◽  
Particle Swarm Optimization Algorithm ◽  
Particle Swarm ◽  
Optimal Combination ◽  
Swarm Optimization

Developed neural networks as an output could have numerous potential outputs caused by numerous combinations of input values. When we are in position to find optimal combination of input values for achieving specific output value within neural network model it is not a trivial task. This request comes from profiling purposes if, for example, neural network gives information of specific profile regarding input or recommendation system realized by neural networks, etc. Utilizing evolutionary algorithms like particle swarm optimization algorithm, which will be illustrated in this chapter, can solve these problems.

scholarly journals GENERALIZED CELLULAR NEURAL NETWORKS (GCNNs) CONSTRUCTED USING PARTICLE SWARM OPTIMIZATION FOR SPATIO-TEMPORAL EVOLUTIONARY PATTERN IDENTIFICATION

2008 ◽  
Vol 18 (12) ◽  
pp. 3611-3624 ◽  
Author(s):  
H. L. WEI ◽  
S. A. BILLINGS
Keyword(s):  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Orthogonal Projection ◽  
Learning Algorithm ◽  
Projection Pursuit ◽  
Particle Swarm ◽  
Cellular Neural Networks ◽  
Modeling Framework ◽  
Swarm Optimization ◽  
Spatio Temporal

Particle swarm optimization (PSO) is introduced to implement a new constructive learning algorithm for training generalized cellular neural networks (GCNNs) for the identification of spatio-temporal evolutionary (STE) systems. The basic idea of the new PSO-based learning algorithm is to successively approximate the desired signal by progressively pursuing relevant orthogonal projections. This new algorithm will thus be referred to as the orthogonal projection pursuit (OPP) algorithm, which is in mechanism similar to the conventional projection pursuit approach. A novel two-stage hybrid training scheme is proposed for constructing a parsimonious GCNN model. In the first stage, the orthogonal projection pursuit algorithm is applied to adaptively and successively augment the network, where adjustable parameters of the associated units are optimized using a particle swarm optimizer. The resultant network model produced at the first stage may be redundant. In the second stage, a forward orthogonal regression (FOR) algorithm, aided by mutual information estimation, is applied to refine and improve the initially trained network. The effectiveness and performance of the proposed method is validated by applying the new modeling framework to a spatio-temporal evolutionary system identification problem.

scholarly journals Designing Artificial Neural Networks Using Particle Swarm Optimization Algorithms

2015 ◽  
Vol 2015 ◽  
pp. 1-20 ◽  
Author(s):  
Beatriz A. Garro ◽  
Roberto A. Vázquez
Keyword(s):  
Particle Swarm Optimization ◽  
Transfer Functions ◽  
Learning Algorithm ◽  
Optimization Algorithms ◽  
Back Propagation ◽  
Particle Swarm ◽  
Classification Error ◽  
Classification Problems ◽  
Swarm Optimization ◽  
Artificial Neural

Artificial Neural Network (ANN) design is a complex task because its performance depends on the architecture, the selected transfer function, and the learning algorithm used to train the set of synaptic weights. In this paper we present a methodology that automatically designs an ANN using particle swarm optimization algorithms such as Basic Particle Swarm Optimization (PSO), Second Generation of Particle Swarm Optimization (SGPSO), and a New Model of PSO called NMPSO. The aim of these algorithms is to evolve, at the same time, the three principal components of an ANN: the set of synaptic weights, the connections or architecture, and the transfer functions for each neuron. Eight different fitness functions were proposed to evaluate the fitness of each solution and find the best design. These functions are based on the mean square error (MSE) and the classification error (CER) and implement a strategy to avoid overtraining and to reduce the number of connections in the ANN. In addition, the ANN designed with the proposed methodology is compared with those designed manually using the well-known Back-Propagation and Levenberg-Marquardt Learning Algorithms. Finally, the accuracy of the method is tested with different nonlinear pattern classification problems.

scholarly journals Unstable System Control Using an Improved Particle Swarm Optimization-Based Neural Network Controller

Electronics ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. 1302 ◽  
Author(s):  
Cheng-Jian Lin ◽  
Xin-You Lin ◽  
Jyun-Yu Jhang
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Learning Algorithm ◽  
Particle Swarm ◽  
Swarm Optimization ◽  
Pendulum System ◽  
Improved Particle Swarm Optimization ◽  
Inverted Pendulum System ◽  
Neural Network Controller ◽  
Network Controller

In this study, an improved particle swarm optimization (IPSO)-based neural network controller (NNC) is proposed for solving a real unstable control problem. The proposed IPSO automatically determines an NNC structure by a hierarchical approach and optimizes the parameters of the NNC by chaos particle swarm optimization. The proposed NNC based on an IPSO learning algorithm is used for controlling a practical planetary train-type inverted pendulum system. Experimental results show that the robustness and effectiveness of the proposed NNC based on IPSO are superior to those of other methods.

A neuro-particle swarm optimization logistic model fitting algorithm for software reliability analysis

2019 ◽  
Vol 233 (6) ◽  
pp. 958-971 ◽  
Author(s):  
Pooja Rani ◽  
GS Mahapatra
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Optimization Algorithm ◽  
Software Reliability ◽  
Particle Swarm Optimization Algorithm ◽  
Learning Algorithm ◽  
Particle Swarm ◽  
Swarm Optimization ◽  
Inertia Weight ◽  
Text Model

This article develops a particle swarm optimization algorithm based on a feed-forward neural network architecture to fit software reliability growth models. We employ adaptive inertia weight within the proposed particle swarm optimization in consideration of learning algorithm. The dynamic adaptive nature of proposed prior best particle swarm optimization prevents the algorithm from becoming trapped in local optima. These neuro-prior best particle swarm optimization algorithms were applied to a popular flexible logistic growth curve as the [Formula: see text] model based on the weights derived by the artificial neural network learning algorithm. We propose the prior best particle swarm optimization algorithm to train the network for application to three different software failure data sets. The new search strategy improves the rate of convergence because it retains information on the prior particle, thereby enabling better predictions. The results are verified through testing approaching of constant, modified, and linear inertia weight. We assess the fitness of each particle according to the normalized root mean squared error which updates the best particle and velocity to accelerate convergence to an optimal solution. Experimental results demonstrate that the proposed [Formula: see text] model based prior best Particle Swarm Optimization based on Neural Network (pPSONN) improves predictive quality over the [Formula: see text], [Formula: see text], and existing model.

scholarly journals Concepts, Methods, and Performances of Particle Swarm Optimization, Backpropagation, and Neural Networks

2018 ◽  
Vol 2018 ◽  
pp. 1-7 ◽  
Author(s):  
Leke Zajmi ◽  
Falah Y. H. Ahmed ◽  
Adam Amril Jaharadak
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Particle Swarm Optimization ◽  
Real World ◽  
Particle Swarm ◽  
Swarm Optimization ◽  
Advantages And Disadvantages ◽  
Artificial Neural ◽  
Real World Problems

With the advancement of Machine Learning, since its beginning and over the last years, a special attention has been given to the Artificial Neural Network. As an inspiration from natural selection of animal groups and human’s neural system, the Artificial Neural Network also known as Neural Networks has become the new computational power which is used for solving real world problems. Neural Networks alone as a concept involve various methods for achieving their success; thus, this review paper describes an overview of such methods called Particle Swarm Optimization, Backpropagation, and Neural Network itself, respectively. A brief explanation of the concepts, history, performances, advantages, and disadvantages is given, followed by the latest researches done on these methods. A description of solutions and applications on various industrial sectors such as Medicine or Information Technology has been provided. The last part briefly discusses the directions, current, and future challenges of Neural Networks towards achieving the highest success rate in solving real world problems.

Sign in / Sign up

Export Citation Format

Share Document