Stochastic Approximation with Averaging and Feedback: Faster Convergence

1995 ◽  
pp. 205-228 ◽  
Author(s):  
Harold J. Kushner ◽  
Jichuan Yang
Keyword(s):  
Stochastic Approximation

scholarly journals An Introduction to Development of Centralized and Distributed Stochastic Approximation Algorithm with Expanding Truncations

Algorithms ◽  
2021 ◽  
Vol 14 (6) ◽  
pp. 174
Author(s):  
Wenxiao Zhao
Keyword(s):  
Approximation Algorithm ◽  
Stochastic Approximation ◽  
Probabilistic Method ◽  
Principal Component ◽  
Stochastic Approximation Algorithm ◽  
The Past ◽  
Systems And Control ◽  
Ode Method ◽  
Centralized Algorithm ◽  
And Control

The stochastic approximation algorithm (SAA), starting from the pioneer work by Robbins and Monro in 1950s, has been successfully applied in systems and control, statistics, machine learning, and so forth. In this paper, we will review the development of SAA in China, to be specific, the stochastic approximation algorithm with expanding truncations (SAAWET) developed by Han-Fu Chen and his colleagues during the past 35 years. We first review the historical development for the centralized algorithm including the probabilistic method (PM) and the ordinary differential equation (ODE) method for SAA and the trajectory-subsequence method for SAAWET. Then, we will give an application example of SAAWET to the recursive principal component analysis. We will also introduce the recent progress on SAAWET in a networked and distributed setting, named the distributed SAAWET (DSAAWET).

Sign in / Sign up

Export Citation Format

Share Document