BACKPROPAGATION ALGORITHM ADAPTATION PARAMETERS USING LEARNING AUTOMATA

2001 ◽  
Vol 11 (03) ◽  
pp. 219-228 ◽  
Author(s):  
HAMID BEIGY ◽  
MOHAMMAD R. MEYBODI
Keyword(s):  
Neural Networks ◽  
Structure Learning ◽  
Learning Automata ◽  
Feedforward Neural Networks ◽  
Adaptation Algorithm ◽  
Training Time ◽  
Long Time ◽  
Rate Of Learning ◽  
The Many ◽  
Exclusive Or

Despite of the many successful applications of backpropagation for training multi–layer neural networks, it has many drawbacks. For complex problems it may require a long time to train the networks, and it may not train at all. Long training time can be the result of the non-optimal parameters. It is not easy to choose appropriate value of the parameters for a particular problem. In this paper, by interconnection of fixed structure learning automata (FSLA) to the feedforward neural networks, we apply learning automata (LA) scheme for adjusting these parameters based on the observation of random response of neural networks. The main motivation in using learning automata as an adaptation algorithm is to use its capability of global optimization when dealing with multi-modal surface. The feasibility of proposed method is shown through simulations on three learning problems: exclusive-or, encoding problem, and digit recognition. The simulation results show that the adaptation of these parameters using this method not only increases the convergence rate of learning but it increases the likelihood of escaping from the local minima.

NEW LEARNING AUTOMATA BASED ALGORITHMS FOR ADAPTATION OF BACKPROPAGATION ALGORITHM PARAMETERS

2002 ◽  
Vol 12 (01) ◽  
pp. 45-67 ◽  
Author(s):  
M. R. MEYBODI ◽  
H. BEIGY
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Learning Automata ◽  
Feedforward Neural Networks ◽  
Bp Algorithm ◽  
Backpropagation Algorithm ◽  
New Learning ◽  
The Neural Networks ◽  
Factor Α ◽  
High Possibility

One popular learning algorithm for feedforward neural networks is the backpropagation (BP) algorithm which includes parameters, learning rate (η), momentum factor (α) and steepness parameter (λ). The appropriate selections of these parameters have large effects on the convergence of the algorithm. Many techniques that adaptively adjust these parameters have been developed to increase speed of convergence. In this paper, we shall present several classes of learning automata based solutions to the problem of adaptation of BP algorithm parameters. By interconnection of learning automata to the feedforward neural networks, we use learning automata scheme for adjusting the parameters η, α, and λ based on the observation of random response of the neural networks. One of the important aspects of the proposed schemes is its ability to escape from local minima with high possibility during the training period. The feasibility of proposed methods is shown through simulations on several problems.

TONE RECOGNITION OF CONTINUOUS THAI SPEECH UNDER TONAL ASSIMILATION AND DECLINATION EFFECTS USING HALF-TONE MODEL

2001 ◽  
Vol 09 (06) ◽  
pp. 815-825 ◽  
Author(s):  
NUTTAKORN THUBTHONG ◽  
BOONSERM KIJSIRIKUL
Keyword(s):  
Neural Networks ◽  
Recognition Rate ◽  
Feedforward Neural Networks ◽  
Training Time ◽  
Tone Recognition ◽  
Context Dependent ◽  
Novel Model

This paper presents a method for continuous Thai tone recognition. One of the main problems in tone recognition is that several interacting factors affect F0realization of tones. In this paper, we focus on the tonal assimilation and declination effects. These effects are compensated by the tone information of neighboring syllables, the F0downdrift and the context-dependent tone model. However, the context-dependent tone model is too large and its training time is very long. To overcome these problems, we propose a novel model called the half-tone model. The experiments, which compare all tone features and all tone models, were simulated by feedforward neural networks. The results show that the proposed tone features increase the recognition rates and the half-tone model outperforms conventional tone models, i.e. context-independent and context-dependent tone models, in terms of recognition rate and speed. The best results are 94.77% and 93.82% for the inside test and outside test, respectively.

scholarly journals Speech Emotion Recognition Using Deep Learning LSTM for Tamil Language

2021 ◽  
Vol 29 (3) ◽  
Author(s):  
Bennilo Fernandes ◽  
Kasiprasad Mannepalli
Keyword(s):  
Neural Networks ◽  
Speech Processing ◽  
Performance Test ◽  
Short Term Memory ◽  
Feedforward Neural Networks ◽  
Speech Emotion Recognition ◽  
Training Time ◽  
Hierarchical Learning ◽  
And Training

Deep Neural Networks (DNN) are more than just neural networks with several hidden units that gives better results with classification algorithm in automated voice recognition activities. Then spatial correlation was considered in traditional feedforward neural networks and which do not manage speech signal properly to it extend, so recurrent neural networks (RNNs) were implemented. Long Short-Term Memory (LSTM) systems is a unique case of RNNs for speech processing, thus considering long-term dependencies Deep Hierarchical LSTM and BiLSTM is designed with dropout layers to reduce the gradient and long-term learning error in emotional speech analysis. Thus, four different combinations of deep hierarchical learning architecture Deep Hierarchical LSTM and LSTM (DHLL), Deep Hierarchical LSTM and BiLSTM (DHLB), Deep Hierarchical BiLSTM and LSTM (DHBL) and Deep Hierarchical dual BiLSTM (DHBB) is designed with dropout layers to improve the networks. The performance test of all four model were compared in this paper and better efficiency of classification is attained with minimal dataset of Tamil Language. The experimental results show that DHLB reaches the best precision of about 84% in recognition of emotions for Tamil database, however, the DHBL gives 83% of efficiency. Other design layers also show equal performance but less than the above models DHLL & DHBB shows 81% of efficiency for lesser dataset and minimal execution and training time.

scholarly journals Variable Chromosome Genetic Algorithm for Structure Learning in Neural Networks to Imitate Human Brain

2019 ◽  
Vol 9 (15) ◽  
pp. 3176 ◽  
Author(s):  
Kang-moon Park ◽  
Donghoon Shin ◽  
Sung-do Chi
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Structure Learning ◽  
Structural Parameters ◽  
Future Research ◽  
Xor Problem ◽  
Chromosome Attachment ◽  
Artificial Neural Network Ann ◽  
Exclusive Or

This paper proposes the variable chromosome genetic algorithm (VCGA) for structure learning in neural networks. Currently, the structural parameters of neural networks, i.e., number of neurons, coupling relations, number of layers, etc., have mostly been designed on the basis of heuristic knowledge of an artificial intelligence (AI) expert. To overcome this limitation, in this study evolutionary approach (EA) has been utilized to automatically generate the proper artificial neural network (ANN) structures. VCGA has a new genetic operation called a chromosome attachment. By applying the VCGA, the initial ANN structures can be flexibly evolved toward the proper structure. The case study applied to the typical exclusive or (XOR) problem shows the feasibility of our methodology. Our approach is differentiated with others in that it uses a variable chromosome in the genetic algorithm. It makes a neural network structure vary naturally, both constructively and destructively. It has been shown that the XOR problem is successfully optimized using a VCGA with a chromosome attachment to learn the structure of neural networks. Research on the structure learning of more complex problems is the topic of our future research.

scholarly journals A Novel Fast Feedforward Neural Networks Training Algorithm

2021 ◽  
Vol 11 (4) ◽  
pp. 287-306
Author(s):  
Jarosław Bilski ◽  
Bartosz Kowalczyk ◽  
Andrzej Marjański ◽  
Michał Gandor ◽  
Jacek Zurada
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Recursive Least Squares ◽  
Convergence Time ◽  
Normal Equation ◽  
Training Algorithms ◽  
Training Algorithm ◽  
Training Time ◽  
Satisfactory Outcome ◽  
Accelerated Gradient

Abstract In this paper1 a new neural networks training algorithm is presented. The algorithm originates from the Recursive Least Squares (RLS) method commonly used in adaptive filtering. It uses the QR decomposition in conjunction with the Givens rotations for solving a normal equation - resulting from minimization of the loss function. An important parameter in neural networks is training time. Many commonly used algorithms require a big number of iterations in order to achieve a satisfactory outcome while other algorithms are effective only for small neural networks. The proposed solution is characterized by a very short convergence time compared to the well-known backpropagation method and its variants. The paper contains a complete mathematical derivation of the proposed algorithm. There are presented extensive simulation results using various benchmarks including function approximation, classification, encoder, and parity problems. Obtained results show the advantages of the featured algorithm which outperforms commonly used recent state-of-the-art neural networks training algorithms, including the Adam optimizer and the Nesterov’s accelerated gradient.

A Structure Learning Method for Block-Based Neural Networks

2013 ◽  
Vol 133 (10) ◽  
pp. 1976-1982 ◽  
Author(s):  
Hidetaka Watanabe ◽  
Seiichi Koakutsu ◽  
Takashi Okamoto ◽  
Hironori Hirata
Keyword(s):  
Neural Networks ◽  
Structure Learning ◽  
Learning Method ◽  
Block Based
Sign in / Sign up

Export Citation Format

Share Document