scholarly journals On the rate of convergence of alternating minimization for non-smooth non-strongly convex optimization in Banach spaces

2021 ◽  
Author(s):  
Jakub Wiktor Both
Keyword(s):  
Convex Optimization ◽  
Banach Spaces ◽  
Optimization Problems ◽  
Linear Convergence ◽  
Rates Of Convergence ◽  
Strong Convexity ◽  
Quadratic Functional ◽  
Alternating Minimization ◽  
Strongly Convex ◽  
Strongly Convex Optimization

AbstractIn this paper, the convergence of the fundamental alternating minimization is established for non-smooth non-strongly convex optimization problems in Banach spaces, and novel rates of convergence are provided. As objective function a composition of a smooth, and a block-separable, non-smooth part is considered, covering a large range of applications. For the former, three different relaxations of strong convexity are considered: (i) quasi-strong convexity; (ii) quadratic functional growth; and (iii) plain convexity. With new and improved rates benefiting from both separate steps of the scheme, linear convergence is proved for (i) and (ii), whereas sublinear convergence is showed for (iii).

scholarly journals Alternating minimization methods for strongly convex optimization

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Nazarii Tupitsa ◽  
Pavel Dvurechensky ◽  
Alexander Gasnikov ◽  
Sergey Guminov
Keyword(s):  
Convex Optimization ◽  
Convergence Rate ◽  
Condition Number ◽  
Optimization Problems ◽  
Linear Convergence ◽  
Strong Convexity ◽  
Alternating Minimization ◽  
Linear Convergence Rate ◽  
Convexity Assumption ◽  
Minimization Procedure

Abstract We consider alternating minimization procedures for convex and non-convex optimization problems with the vector of variables divided into several blocks, each block being amenable for minimization with respect to its variables while maintaining other variables blocks constant. In the case of two blocks, we prove a linear convergence rate for an alternating minimization procedure under the Polyak–Łojasiewicz (PL) condition, which can be seen as a relaxation of the strong convexity assumption. Under the strong convexity assumption in the many-blocks setting, we provide an accelerated alternating minimization procedure with linear convergence rate depending on the square root of the condition number as opposed to just the condition number for the non-accelerated method. We also consider the problem of finding an approximate non-negative solution to a linear system of equations A ⁢ x = y {Ax=y} with alternating minimization of Kullback–Leibler (KL) divergence between Ax and y.

Sign in / Sign up

Export Citation Format

Share Document