An Improved BP Algorithm Based on Steepness Factor and Adaptive Learning Rate Adjustment Factor

2011 ◽  
Vol 121-126 ◽  
pp. 705-709
Author(s):  
Xiu Hua Zhang ◽  
Fu Jun Ren ◽  
Yong Cheng Jiang
Keyword(s):  
Adaptive Learning ◽  
Convergence Speed ◽  
Learning Rate ◽  
Bp Algorithm ◽  
Neural Net ◽  
Adjustment Factor ◽  
Bp Network ◽  
Adaptive Learning Rate ◽  
Rate Adjustment ◽  
Work Model

BP network is the most widely used of the neural net work model, but there are many problems of slow convergence speed and easily getting into the local minimum in the conventional BP algorithm. For this, an improved algorithm is proposed. Momentum term is added, steepness factors are introduced and adaptive learning rate adjustment factor is added. In the Matlab platform simulations are carried out by each improvement methods on the same BP neural network. The results show that: Convergence of improved BP network is decreased from 1000 to 49 and the error is decreased from 10-2 to 10-6. The convergence speed has been significantly improved and the error has been decreased. Using the synthesis improvement method effect is obvious and it provides a good theoretical basis for the practical application.

A Study on Approximation Performances of Improved Bp Neural Networks Based on LM Algorithms

2013 ◽  
Vol 411-414 ◽  
pp. 1935-1938 ◽  
Author(s):  
Shuo Ding ◽  
Xiao Heng Chang ◽  
Qing Hui Wu
Keyword(s):  
Adaptive Learning ◽  
Numerical Optimization ◽  
Nonlinear Function ◽  
Learning Rate ◽  
Bp Algorithm ◽  
Convergence Time ◽  
Nonlinear Functions ◽  
Adaptive Learning Rate ◽  
Marquardt Algorithm ◽  
Poor Convergence

When approximating nonlinear functions, standard BP algorithms and traditional improved BP algorithms have low convergence rate and tend to be stuck in local minimums. In this paper, standard BP algorithm is improved by numerical optimization algorithm. Firstly, the principle of Levenberg-Marquardt algorithm is introduced. Secondly, to test its approximation performance, LMBP neural network is programmed via MATLAB7.0 taking specific nonlinear function as an example. Thirdly, its approximation result is compared with those of standard BP algorithm and adaptive learning rate algorithm. Simulation results indicate that compared with standard BP algorithm and adaptive learning rate algorithm, LMBP algorithm overcomes deficiencies ranging from poor convergence ability, prolonged convergence time, increasing iteration steps to nonconvergence. Thus with its good approximation ability, LMBP algorithm is the most suitable for medium-sized networks.

Performance-Based Adaptive Learning Rate Scheduler Algorithm

2021 ◽  
pp. 417-433
Author(s):  
Vakada Naveen ◽  
Yaswanth Mareedu ◽  
Neeharika Sai Mandava ◽  
Sravya Kaveti ◽  
G. Krishna Kishore
Keyword(s):  
Adaptive Learning ◽  
Learning Rate ◽  
Adaptive Learning Rate

Correlation filter tracking based on adaptive learning rate and location refiner

2018 ◽  
Vol 26 (8) ◽  
pp. 2100-2111 ◽  
Author(s):  
刘教民 LIU Jiao-min ◽  
郭剑威 GUO Jian-wei ◽  
师 硕 SHI Shuo
Keyword(s):  
Adaptive Learning ◽  
Learning Rate ◽  
Correlation Filter ◽  
Adaptive Learning Rate

MEMS Inertial Sensor Fault Diagnosis Using a CNN-Based Data-Driven Method

2020 ◽  
Vol 34 (14) ◽  
pp. 2059048
Author(s):  
Tong Gao ◽  
Wei Sheng ◽  
Mingliang Zhou ◽  
Bin Fang ◽  
Liping Zheng
Keyword(s):  
Fault Diagnosis ◽  
Adaptive Learning ◽  
Inertial Sensors ◽  
Inertial Sensor ◽  
Optimization Method ◽  
Learning Rate ◽  
Data Driven ◽  
Adaptive Learning Rate ◽  
Improved Performance ◽  
Rate Optimization

In this paper, we propose a novel fault diagnosis (FD) approach for micro-electromechanical systems (MEMS) inertial sensors that recognize the fault patterns of MEMS inertial sensors in an end-to-end manner. We use a convolutional neural network (CNN)-based data-driven method to classify the temperature-related sensor faults in unmanned aerial vehicles (UAVs). First, we formulate the FD problem for MEMS inertial sensors into a deep learning framework. Second, we design a multi-scale CNN which uses the raw data of MEMS inertial sensors as input and which outputs classification results indicating faults. Then we extract fault features in the temperature domain to solve the non-uniform sampling problem. Finally, we propose an improved adaptive learning rate optimization method which accelerates the loss convergence by using the Kalman filter (KF) to train the network efficiently with a small dataset. Our experimental results show that our method achieved high fault recognition accuracy and that our proposed adaptive learning rate method improved performance in terms of loss convergence and robustness on a small training batch.

Sign in / Sign up

Export Citation Format

Share Document