A new fast learning algorithm with promising global convergence capability for feed-forward neural networks

2013 ◽  
Author(s):  
Chi-Chung Cheung ◽  
Sin-Chun Ng ◽  
Andrew K Lui ◽  
Sean Shensheng Xu
Keyword(s):  
Neural Networks ◽  
Global Convergence ◽  
Learning Algorithm ◽  
Feed Forward ◽  
Fast Learning ◽  
Feed Forward Neural Networks

A New Type of Hybrid Learning Algorithm for Three-Layered Feed-Forward Neural Networks

2014 ◽  
Vol 1030-1032 ◽  
pp. 1627-1632
Author(s):  
Yun Jun Yu ◽  
Sui Peng ◽  
Zhi Chuan Wu ◽  
Peng Liang He
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Least Square Method ◽  
Hybrid Learning ◽  
Least Square ◽  
Feed Forward ◽  
Hybrid Learning Algorithm ◽  
Layer Function ◽  
Feed Forward Neural Networks ◽  
New Type

The problem of local minimum cannot be avoided when it comes to nonlinear optimization in the learning algorithm of neural network parameters, and the larger the optimization space is, the more obvious the problem becomes. This paper proposes a new type of hybrid learning algorithm for three-layered feed-forward neural networks. This algorithm is based on three-layered feed-forward neural networks with output layer function, namely linear function, combining a quasi Newton algorithm with adaptive decoupled step and momentum (QNADSM) and iterative least square method to export. Simulation proves that this hybrid algorithm has strong self-adaptive capability, small calculation amount and fast convergence speed. It is an effective engineer practical algorithm.

DISCRETE LEARNING IN FEED-FORWARD NEURAL NETWORKS

1991 ◽  
Vol 02 (04) ◽  
pp. 323-329 ◽  
Author(s):  
C.J. Pérez Vicente ◽  
J. Carrabina ◽  
E. Valderrama
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Arbitrary Distribution ◽  
Feed Forward ◽  
Learning Techniques ◽  
Feed Forward Neural Networks ◽  
Discrete Learning

We introduce a learning algorithm for feed-forward neural networks with synapses which can only take a discrete number of values. Taking into account the inherent limitations associated to these networks, we think that the performance of the method is quite efficient as we have shown through some simple results. The main novelty with respect to other discrete learning techniques is a different strategy in the search for solutions. Generalizations to any arbitrary distribution of discrete weights are straightforward.

scholarly journals Generalized Dai-Yuan conjugate gradient algorithm for training multi-layer feed-forward neural networks

2019 ◽  
Vol 24 (1) ◽  
pp. 115
Author(s):  
Hind H. Mohammed
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Global Convergence ◽  
Conjugate Gradient ◽  
Conjugate Gradient Algorithm ◽  
Gradient Algorithm ◽  
Conjugacy Condition ◽  
Feed Forward ◽  
Descent Property ◽  
Feed Forward Neural Networks

In this paper, we will present different type of CG algorithms depending on Peary conjugacy condition. The new conjugate gradient training (GDY) algorithm using to train MFNNs and prove it's descent property and global convergence for it and then we tested the behavior of this algorithm in the training of artificial neural networks and compared it with known algorithms in this field through two types of issues   http://dx.doi.org/10.25130/tjps.24.2019.020

Optimizing and Learning Algorithm for Feed-forward Neural Networks

2001 ◽  
Vol 5 (1) ◽  
pp. 51-57
Author(s):  
Pilar Bachiller ◽  
◽  
Julia González
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Neural Model ◽  
Training Set ◽  
Feed Forward Neural Network ◽  
Feed Forward ◽  
Training Time ◽  
Network Pruning ◽  
Feed Forward Neural Networks ◽  
Pruning Techniques

Feed-forward neural networks have emerged as a good solution for many problems, such as classification, recognition and identification, and signal processing. However, the importance of selecting an adequate hidden structure for this neural model should not be underestimated. When the hidden structure of the network is too large and complex for the model being developed, the network may tend to memorize input and output sets rather than learning relationships between them. Such a network may train well but test poorly when inputs outside the training set are presented. In addition, training time will significantly increase when the network is unnecessarily large and complex. Most of the proposed solutions to this problem consist of training a larger than necessary network, pruning unnecessary links and nodes and retraining the reduced network. We propose a new method to optimize the size of a feed-forward neural network using orthogonal transformations. This approach prunes unnecessary nodes during the training process, avoiding the retraining phase of the reduced network, which is necessary in most pruning techniques.

scholarly journals Using an extended Kalman filter learning algorithm for feed-forward neural networks to describe tracer correlations

2004 ◽  
Vol 4 (3) ◽  
pp. 3653-3667 ◽  
Author(s):  
D. J. Lary ◽  
H. Y. Mussa
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Kalman Filter ◽  
Extended Kalman Filter ◽  
Learning Algorithm ◽  
Feed Forward ◽  
The Neural Network ◽  
Fortran Code ◽  
Feed Forward Neural Networks ◽  
Time Of Year

Abstract. In this study a new extended Kalman filter (EKF) learning algorithm for feed-forward neural networks (FFN) is used. With the EKF approach, the training of the FFN can be seen as state estimation for a non-linear stationary process. The EKF method gives excellent convergence performances provided that there is enough computer core memory and that the machine precision is high. Neural networks are ideally suited to describe the spatial and temporal dependence of tracer-tracer correlations. The neural network performs well even in regions where the correlations are less compact and normally a family of correlation curves would be required. For example, the CH4-N2O correlation can be well described using a neural network trained with the latitude, pressure, time of year, and CH4 volume mixing ratio (v.m.r.). The neural network was able to reproduce the CH4-N2O correlation with a correlation coefficient between simulated and training values of 0.9997. The neural network Fortran code used is available for download.

Sign in / Sign up

Export Citation Format

Share Document