The rate of convergence of optimization algorithms obtained via discretizations of heavy ball dynamical systems for convex optimization problems

Optimization ◽  
2021 ◽  
pp. 1-31
Author(s):  
Cristian Daniel Alecsa
Keyword(s):  
Dynamical Systems ◽  
Convex Optimization ◽  
Rate Of Convergence ◽  
Optimization Problems ◽  
Optimization Algorithms ◽  
Convex Optimization Problems

scholarly journals Continuous Dynamics Related to Monotone Inclusions and Non-Smooth Optimization Problems

2020 ◽  
Vol 28 (4) ◽  
pp. 611-642 ◽  
Author(s):  
Ernö Robert Csetnek
Keyword(s):  
Dynamical Systems ◽  
Convex Optimization ◽  
Optimization Problems ◽  
Decisive Role ◽  
Energy Functional ◽  
Lyapunov Theory ◽  
Smooth Functions ◽  
Monotone Inclusions ◽  
Convex Optimization Problems ◽  
Smooth Optimization

Abstract The aim of this survey is to present the main important techniques and tools from variational analysis used for first and second order dynamical systems of implicit type for solving monotone inclusions and non-smooth optimization problems. The differential equations are expressed by means of the resolvent (in case of a maximally monotone set valued operator) or the proximal operator for non-smooth functions. The asymptotic analysis of the trajectories generated relies on Lyapunov theory, where the appropriate energy functional plays a decisive role. While the most part of the paper is related to monotone inclusions and convex optimization problems in the variational case, we present also results for dynamical systems for solving non-convex optimization problems, where the Kurdyka-Łojasiewicz property is used.

scholarly journals Decentralized and parallel primal and dual accelerated methods for stochastic convex programming problems

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Darina Dvinskikh ◽  
Alexander Gasnikov
Keyword(s):  
Convex Optimization ◽  
Inverse Problems ◽  
Convex Programming ◽  
Data Science ◽  
Optimization Problems ◽  
Stochastic Gradient ◽  
Logarithmic Factor ◽  
Accelerated Methods ◽  
Convex Optimization Problems ◽  
Primal And Dual

Abstract We introduce primal and dual stochastic gradient oracle methods for decentralized convex optimization problems. Both for primal and dual oracles, the proposed methods are optimal in terms of the number of communication steps. However, for all classes of the objective, the optimality in terms of the number of oracle calls per node takes place only up to a logarithmic factor and the notion of smoothness. By using mini-batching technique, we show that the proposed methods with stochastic oracle can be additionally parallelized at each node. The considered algorithms can be applied to many data science problems and inverse problems.

Sign in / Sign up

Export Citation Format

Share Document