scholarly journals TIME BOUNDS OF BASIC STEEPEST DESCENT ALGORITHMS FOR M-CONVEX FUNCTION MINIMIZATION AND RELATED PROBLEMS

2021 ◽  
Vol 64 (2) ◽  
pp. 45-60
Author(s):  
Norito Minamikawa ◽  
Akiyoshi Shioura
Keyword(s):  
Convex Function ◽  
Steepest Descent ◽  
Function Minimization ◽  
Descent Algorithms ◽  
Time Bounds

A Study on the Optimal Double Parameters for Steepest Descent with Momentum

2015 ◽  
Vol 27 (4) ◽  
pp. 982-1004 ◽  
Author(s):  
Naimin Zhang
Keyword(s):  
Stability Analysis ◽  
Steepest Descent ◽  
Quadratic Functions ◽  
Optimal Parameters ◽  
Optimal Learning ◽  
Descent Algorithms ◽  
Learning Rates ◽  
Global Optimal ◽  
The Stability

This letter presents the stability analysis for two steepest descent algorithms with momentum for quadratic functions. The corresponding local optimal parameters in Torii and Hagan ( 2002 ) and Zhang ( 2013 ) are extended to the global optimal parameters, that is, both the optimal learning rates and the optimal momentum factors are obtained simultaneously which make for the fastest convergence.

Semistability of Steepest Descent with Momentum for Quadratic Functions

2013 ◽  
Vol 25 (5) ◽  
pp. 1277-1301 ◽  
Author(s):  
Naimin Zhang
Keyword(s):  
Necessary Conditions ◽  
Learning Rate ◽  
Steepest Descent ◽  
Quadratic Functions ◽  
Descent Algorithms ◽  
Sufficient And Necessary Conditions

Two steepest-descent algorithms with momentum for quadratic functions are considered. For a given learning rate, the sufficient and necessary conditions for the semistability of the steepest-descent algorithms with momentum are established. Moreover, the optimal momentum factors that generally make for the fastest convergence are obtained.

Sign in / Sign up

Export Citation Format

Share Document