Discrete Steepest Descent Algorithms And Their Realization On Optical Analog Processors

1989 ◽  
Author(s):  
Eugene Pochapsky ◽  
David Casasent
Keyword(s):  
Steepest Descent ◽  
Descent Algorithms ◽  
Optical Analog

A Study on the Optimal Double Parameters for Steepest Descent with Momentum

2015 ◽  
Vol 27 (4) ◽  
pp. 982-1004 ◽  
Author(s):  
Naimin Zhang
Keyword(s):  
Stability Analysis ◽  
Steepest Descent ◽  
Quadratic Functions ◽  
Optimal Parameters ◽  
Optimal Learning ◽  
Descent Algorithms ◽  
Learning Rates ◽  
Global Optimal ◽  
The Stability

This letter presents the stability analysis for two steepest descent algorithms with momentum for quadratic functions. The corresponding local optimal parameters in Torii and Hagan ( 2002 ) and Zhang ( 2013 ) are extended to the global optimal parameters, that is, both the optimal learning rates and the optimal momentum factors are obtained simultaneously which make for the fastest convergence.

Semistability of Steepest Descent with Momentum for Quadratic Functions

2013 ◽  
Vol 25 (5) ◽  
pp. 1277-1301 ◽  
Author(s):  
Naimin Zhang
Keyword(s):  
Necessary Conditions ◽  
Learning Rate ◽  
Steepest Descent ◽  
Quadratic Functions ◽  
Descent Algorithms ◽  
Sufficient And Necessary Conditions

Two steepest-descent algorithms with momentum for quadratic functions are considered. For a given learning rate, the sufficient and necessary conditions for the semistability of the steepest-descent algorithms with momentum are established. Moreover, the optimal momentum factors that generally make for the fastest convergence are obtained.

Steepest Descent Algorithms for Optimization Under Unitary Matrix Constraint

2008 ◽  
Vol 56 (3) ◽  
pp. 1134-1147 ◽  
Author(s):  
Traian E. Abrudan ◽  
Jan Eriksson ◽  
Visa Koivunen
Keyword(s):  
Steepest Descent ◽  
Unitary Matrix ◽  
Descent Algorithms
Sign in / Sign up

Export Citation Format

Share Document