A modified limited memory steepest descent method motivated by an inexact super-linear convergence rate analysis

2020 ◽  
Author(s):  
Ran Gu ◽  
Qiang Du
Keyword(s):  
Convergence Rate ◽  
Linear Convergence ◽  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Step Size ◽  
Linear Convergence Rate ◽  
Limited Memory ◽  
Rate Analysis ◽  
Convergence Rate Analysis

Abstract How to choose the step size of gradient descent method has been a popular subject of research. In this paper we propose a modified limited memory steepest descent method (MLMSD). In each iteration we propose a selection rule to pick a unique step size from a candidate set, which is calculated by Fletcher’s limited memory steepest descent method (LMSD), instead of going through all the step sizes in a sweep, as in Fletcher’s original LMSD algorithm. MLMSD is motivated by an inexact super-linear convergence rate analysis. The R-linear convergence of MLMSD is proved for a strictly convex quadratic minimization problem. Numerical tests are presented to show that our algorithm is efficient and robust.

THE METHOD OF STEEPEST DESCENT

1982 ◽  
pp. 443-473
Author(s):  
L.V. KANTOROVICH ◽  
G.P. AKILOV
Keyword(s):  
Steepest Descent ◽  
Method Of Steepest Descent
Sign in / Sign up

Export Citation Format

Share Document