scholarly journals Local Levenberg-Marquardt Algorithm for Learning Feedforwad Neural Networks

2020 ◽  
Vol 10 (4) ◽  
pp. 299-316 ◽  
Author(s):  
Jarosław Bilski ◽  
Bartosz Kowalczyk ◽  
Alina Marchlewska ◽  
Jacek M. Zurada
Keyword(s):  
Neural Networks ◽  
Computational Complexity ◽  
Function Approximation ◽  
Classification Problems ◽  
Local Computation ◽  
Method Performance ◽  
Marquardt Algorithm ◽  
Levenberg Marquardt ◽  
Local Modification ◽  
Lm Algorithm

AbstractThis paper presents a local modification of the Levenberg-Marquardt algorithm (LM). First, the mathematical basics of the classic LM method are shown. The classic LM algorithm is very efficient for learning small neural networks. For bigger neural networks, whose computational complexity grows significantly, it makes this method practically inefficient. In order to overcome this limitation, local modification of the LM is introduced in this paper. The main goal of this paper is to develop a more complexity efficient modification of the LM method by using a local computation. The introduced modification has been tested on the following benchmarks: the function approximation and classification problems. The obtained results have been compared to the classic LM method performance. The paper shows that the local modification of the LM method significantly improves the algorithm’s performance for bigger networks. Several possible proposals for future works are suggested.

scholarly journals Adaptive Levenberg–Marquardt Algorithm: A New Optimization Strategy for Levenberg–Marquardt Neural Networks

Mathematics ◽  
2021 ◽  
Vol 9 (17) ◽  
pp. 2176
Author(s):  
Zhiqi Yan ◽  
Shisheng Zhong ◽  
Lin Lin ◽  
Zhiquan Cui
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Frequency Noise ◽  
Optimization Strategy ◽  
Engineering Data ◽  
Marquardt Algorithm ◽  
Highly Nonlinear ◽  
Rectified Linear Unit ◽  
Levenberg Marquardt ◽  
Lm Algorithm

Engineering data are often highly nonlinear and contain high-frequency noise, so the Levenberg–Marquardt (LM) algorithm may not converge when a neural network optimized by the algorithm is trained with engineering data. In this work, we analyzed the reasons for the LM neural network’s poor convergence commonly associated with the LM algorithm. Specifically, the effects of different activation functions such as Sigmoid, Tanh, Rectified Linear Unit (RELU) and Parametric Rectified Linear Unit (PRLU) were evaluated on the general performance of LM neural networks, and special values of LM neural network parameters were found that could make the LM algorithm converge poorly. We proposed an adaptive LM (AdaLM) algorithm to solve the problem of the LM algorithm. The algorithm coordinates the descent direction and the descent step by the iteration number, which can prevent falling into the local minimum value and avoid the influence of the parameter state of LM neural networks. We compared the AdaLM algorithm with the traditional LM algorithm and its variants in terms of accuracy and speed in the context of testing common datasets and aero-engine data, and the results verified the effectiveness of the AdaLM algorithm.

A FAST AND CONVERGENT STOCHASTIC MLP LEARNING ALGORITHM

2001 ◽  
Vol 11 (06) ◽  
pp. 573-583
Author(s):  
AKITO SAKURAI
Keyword(s):  
Learning Algorithm ◽  
Classification Problem ◽  
Threshold Function ◽  
Medium Size ◽  
Multilayer Perceptrons ◽  
Classification Problems ◽  
Marquardt Algorithm ◽  
Parity Function ◽  
Levenberg Marquardt ◽  
Generalization Accuracy

We propose a stochastic learning algorithm for multilayer perceptrons of linear-threshold function units, which theoretically converges with probability one and experimentally exhibits 100% convergence rate and remarkable speed on parity and classification problems with typical generalization accuracy. For learning the n bit parity function with n hidden units, the algorithm converged on all the trials we tested (n=2 to 12) after 5.8· 4.1n presentations for 0.23· 4.0n-6 seconds on a 533MHz Alpha 21164A chip on average, which is five to ten times faster than Levenberg-Marquardt algorithm with restarts. For a medium size classification problem known as Thyroid in UCI repository, the algorithm is faster in speed and comparative in generalization accuracy than the standard backpropagation and Levenberg-Marquardt algorithms.

scholarly journals Processing and Interpretation of UAV Magnetic Data: A Workflow Based on Improved Variational Mode Decomposition and Levenberg-Marquardt Algorithm

Drones ◽  
2022 ◽  
Vol 6 (1) ◽  
pp. 11
Author(s):  
Yaoxin Zheng ◽  
Shiyan Li ◽  
Kang Xing ◽  
Xiaojuan Zhang
Keyword(s):  
Data Processing ◽  
Original Data ◽  
Magnetic Data ◽  
Variational Mode Decomposition ◽  
Euler Deconvolution ◽  
Near Surface ◽  
Mode Decomposition ◽  
Marquardt Algorithm ◽  
Levenberg Marquardt ◽  
Lm Algorithm

Unmanned aerial vehicles (UAVs) have become a research hotspot in the field of magnetic exploration because of their unique advantages, e.g., low cost, high safety, and easy to operate. However, the lack of effective data processing and interpretation method limits their further deployment. In view of this situation, a complete workflow of UAV magnetic data processing and interpretation is proposed in this paper, which can be divided into two steps: (1) the improved variational mode decomposition (VMD) is applied to the original data to improve its signal-to-noise ratio as much as possible, and the decomposition modes number K is determined adaptively according to the mode characteristics; (2) the parameters of target position and magnetic moment are obtained by Euler deconvolution first, and then used as the prior information of the Levenberg–Marquardt (LM) algorithm to further improve its accuracy. Experiments are carried out to verify the effectiveness of the proposed method. Results show that the proposed method can significantly improve the quality of the original data; by combining the Euler deconvolution and LM algorithm, the horizontal positioning error can be reduced from 15.31 cm to 4.05 cm, and the depth estimation error can be reduced from 16.2 cm to 5.4 cm. Moreover, the proposed method can be used not only for the detection and location of near-surface targets, but also for the follow-up work, such as the clearance of targets (e.g., the unexploded ordnance).

Sign in / Sign up

Export Citation Format

Share Document