scholarly journals PROXIMAL ALGORITHMS FOR BI-LEVEL CONVEX OPTIMIZATION PROBLEMS

2021 ◽  
pp. 145-150
Author(s):  
A. V. Luita ◽  
S. O. Zhilina ◽  
V. V. Semenov
Keyword(s):  
Weak Convergence ◽  
Convex Function ◽  
Gradient Method ◽  
Optimization Problems ◽  
Rates Of Convergence ◽  
Convex Minimization ◽  
Gradient Algorithm ◽  
Penalty Function Method ◽  
Proximal Gradient Method ◽  
Proximal Algorithms

In this paper, problems of bi-level convex minimization in a Hilbert space are considered. The bi-level convex minimization problem is to minimize the first convex function on the set of minima of the second convex function. This setting has many applications, but the implicit constraints generated by the internal problem make it difficult to obtain optimality conditions and construct algorithms. Multilevel optimization problems are formulated in a similar way, the source of which is the operation research problems (optimization according to sequentially specified criteria or lexicographic optimization). Attention is focused on problem solving using two proximal methods. The main theoretical results are theorems on the convergence of methods in various situations. The first of the methods is obtained by combining the penalty function method and the proximal method. Strong convergence is proved in the case of strong convexity of the function of the exterior problem. In the general case, only weak convergence has been proved. The second, the so-called proximal-gradient method, is a combination of one of the variants of the fast proximal-gradient algorithm with the method of penalty functions. The rates of convergence of the proximal-gradient method and its weak convergence are proved.

DISTRIBUTED PROXIMAL-GRADIENT METHOD FOR CONVEX OPTIMIZATION WITH INEQUALITY CONSTRAINTS

2014 ◽  
Vol 56 (2) ◽  
pp. 160-178 ◽  
Author(s):  
JUEYOU LI ◽  
CHANGZHI WU ◽  
ZHIYOU WU ◽  
QIANG LONG ◽  
XIANGYU WANG
Keyword(s):  
Convergence Rate ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Distributed Optimization ◽  
Inequality Constraints ◽  
Gradient Algorithm ◽  
Subgradient Methods ◽  
Penalty Function Method ◽  
Proximal Gradient Method ◽  
Primal Dual

AbstractWe consider a distributed optimization problem over a multi-agent network, in which the sum of several local convex objective functions is minimized subject to global convex inequality constraints. We first transform the constrained optimization problem to an unconstrained one, using the exact penalty function method. Our transformed problem has a smaller number of variables and a simpler structure than the existing distributed primal–dual subgradient methods for constrained distributed optimization problems. Using the special structure of this problem, we then propose a distributed proximal-gradient algorithm over a time-changing connectivity network, and establish a convergence rate depending on the number of iterations, the network topology and the number of agents. Although the transformed problem is nonsmooth by nature, our method can still achieve a convergence rate, ${\mathcal{O}}(1/k)$, after $k$ iterations, which is faster than the rate, ${\mathcal{O}}(1/\sqrt{k})$, of existing distributed subgradient-based methods. Simulation experiments on a distributed state estimation problem illustrate the excellent performance of our proposed method.

scholarly journals A Modified Three-Term Type CD Conjugate Gradient Algorithm for Unconstrained Optimization Problems

2020 ◽  
Vol 2020 ◽  
pp. 1-14
Author(s):  
Zhan Wang ◽  
Pengyuan Li ◽  
Xiangrong Li ◽  
Hongtruong Pham
Keyword(s):  
Conjugate Gradient Method ◽  
Conjugate Gradient ◽  
Gradient Method ◽  
Optimization Problems ◽  
Gradient Methods ◽  
Trust Region ◽  
Conjugate Gradient Algorithm ◽  
Gradient Algorithm ◽  
General Function ◽  
Term Type

Conjugate gradient methods are well-known methods which are widely applied in many practical fields. CD conjugate gradient method is one of the classical types. In this paper, a modified three-term type CD conjugate gradient algorithm is proposed. Some good features are presented as follows: (i) A modified three-term type CD conjugate gradient formula is presented. (ii) The given algorithm possesses sufficient descent property and trust region property. (iii) The algorithm has global convergence with the modified weak Wolfe–Powell (MWWP) line search technique and projection technique for general function. The new algorithm has made great progress in numerical experiments. It shows that the modified three-term type CD conjugate gradient method is more competitive than the classical CD conjugate gradient method.

scholarly journals Stochastic proximal gradient methods for nonconvex problems in Hilbert spaces

2021 ◽  
Vol 78 (3) ◽  
pp. 705-740
Author(s):  
Caroline Geiersbach ◽  
Teresa Scarinci
Keyword(s):  
Hilbert Spaces ◽  
Optimization Problems ◽  
Random Sequence ◽  
Gradient Methods ◽  
Almost Sure Convergence ◽  
Gradient Algorithm ◽  
Stationary Points ◽  
Proximal Gradient Method ◽  
Penalty Term ◽  
Lipschitz Continuous

AbstractFor finite-dimensional problems, stochastic approximation methods have long been used to solve stochastic optimization problems. Their application to infinite-dimensional problems is less understood, particularly for nonconvex objectives. This paper presents convergence results for the stochastic proximal gradient method applied to Hilbert spaces, motivated by optimization problems with partial differential equation (PDE) constraints with random inputs and coefficients. We study stochastic algorithms for nonconvex and nonsmooth problems, where the nonsmooth part is convex and the nonconvex part is the expectation, which is assumed to have a Lipschitz continuous gradient. The optimization variable is an element of a Hilbert space. We show almost sure convergence of strong limit points of the random sequence generated by the algorithm to stationary points. We demonstrate the stochastic proximal gradient algorithm on a tracking-type functional with a $$L^1$$ L 1 -penalty term constrained by a semilinear PDE and box constraints, where input terms and coefficients are subject to uncertainty. We verify conditions for ensuring convergence of the algorithm and show a simulation.

scholarly journals Proximal-Like Incremental Aggregated Gradient Method with Linear Convergence Under Bregman Distance Growth Conditions

2019 ◽  
Author(s):  
Hui Zhang ◽  
Yu-Hong Dai ◽  
Lei Guo ◽  
Wei Peng
Keyword(s):  
Growth Condition ◽  
Gradient Method ◽  
Legendre Function ◽  
Lower Semicontinuous ◽  
Gradient Algorithm ◽  
Bregman Distance ◽  
Proximal Gradient Method ◽  
Step Size ◽  
Convergence Results ◽  
Special Cases

We introduce a unified algorithmic framework, called the proximal-like incremental aggregated gradient (PLIAG) method, for minimizing the sum of a convex function that consists of additive relatively smooth convex components and a proper lower semicontinuous convex regularization function over an abstract feasible set whose geometry can be captured by using the domain of a Legendre function. The PLIAG method includes many existing algorithms in the literature as special cases, such as the proximal gradient method, the Bregman proximal gradient method (also called the NoLips algorithm), the incremental aggregated gradient method, the incremental aggregated proximal method, and the proximal incremental aggregated gradient method. It also includes some novel interesting iteration schemes. First, we show that the PLIAG method is globally sublinearly convergent without requiring a growth condition, which extends the sublinear convergence result for the proximal gradient algorithm to incremental aggregated-type first-order methods. Then, by embedding a so-called Bregman distance growth condition into a descent-type lemma to construct a special Lyapunov function, we show that the PLIAG method is globally linearly convergent in terms of both function values and Bregman distances to the optimal solution set, provided that the step size is not greater than some positive constant. The convergence results derived in this paper are all established beyond the standard assumptions in the literature (i.e., without requiring the strong convexity and the Lipschitz gradient continuity of the smooth part of the objective). When specialized to many existing algorithms, our results recover or supplement their convergence results under strictly weaker conditions.

scholarly journals New inertial proximal gradient methods for unconstrained convex optimization problems

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Peichao Duan ◽  
Yiqun Zhang ◽  
Qinxiong Bu
Keyword(s):  
Convex Optimization ◽  
Optimization Problems ◽  
Gradient Methods ◽  
Gradient Algorithm ◽  
Proximal Gradient Method ◽  
Proximal Point ◽  
Convex Optimization Problem ◽  
Convex Optimization Problems ◽  
Acceleration Methods ◽  
Weak Convergence Theorem

AbstractThe proximal gradient method is a highly powerful tool for solving the composite convex optimization problem. In this paper, firstly, we propose inexact inertial acceleration methods based on the viscosity approximation and proximal scaled gradient algorithm to accelerate the convergence of the algorithm. Under reasonable parameters, we prove that our algorithms strongly converge to some solution of the problem, which is the unique solution of a variational inequality problem. Secondly, we propose an inexact alternated inertial proximal point algorithm. Under suitable conditions, the weak convergence theorem is proved. Finally, numerical results illustrate the performances of our algorithms and present a comparison with related algorithms. Our results improve and extend the corresponding results reported by many authors recently.

scholarly journals A dual Bregman proximal gradient method for relatively-strongly convex optimization

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Jin-Zan Liu ◽  
Xin-Wei Liu
Keyword(s):  
Convex Optimization ◽  
Convergence Rate ◽  
Convex Function ◽  
Minimization Problem ◽  
Gradient Method ◽  
Bregman Distance ◽  
Proximal Gradient Method ◽  
Strongly Convex ◽  
Proper Convex ◽  
Strongly Convex Optimization

<p style='text-indent:20px;'>We consider a convex composite minimization problem, whose objective is the sum of a relatively-strongly convex function and a closed proper convex function. A dual Bregman proximal gradient method is proposed for solving this problem and is shown that the convergence rate of the primal sequence is <inline-formula><tex-math id="M1">\begin{document}$ O(\frac{1}{k}) $\end{document}</tex-math></inline-formula>. Moreover, based on the acceleration scheme, we prove that the convergence rate of the primal sequence is <inline-formula><tex-math id="M2">\begin{document}$ O(\frac{1}{k^{\gamma}}) $\end{document}</tex-math></inline-formula>, where <inline-formula><tex-math id="M3">\begin{document}$ \gamma\in[1,2] $\end{document}</tex-math></inline-formula> is determined by the triangle scaling property of the Bregman distance.</p>

Sign in / Sign up

Export Citation Format

Share Document