A Linearized Alternating Direction Method of Multipliers with Substitution Procedure

2015 ◽  
Vol 32 (03) ◽  
pp. 1550011 ◽  
Author(s):  
Miantao Chao ◽  
Caozong Cheng ◽  
Haibin Zhang
Keyword(s):  
Alternating Direction Method ◽  
Convex Programming Problem ◽  
Method Of Multipliers ◽  
Worst Case ◽  
Alternating Direction ◽  
Separable Convex Programming ◽  
Contractive Type ◽  
Linearly Constrained ◽  
Contraction Type ◽  
Substitution Procedure

We consider the linearly constrained separable convex programming problem whose objective function is separable into m individual convex functions with non-overlapping variables. The alternating direction method of multipliers (ADMM) has been well studied in the literature for the special case m = 2, but the direct extension of ADMM for the general case m ≥ 2 is not necessarily convergent. In this paper, we propose a new linearized ADMM-based contraction type algorithms for the general case m ≥ 2. For the proposed algorithm, we prove its convergence via the analytic framework of contractive type methods and we derive a worst-case O(1/t) convergence rate in ergodic sense. Finally, numerical results are reported to demonstrate the effectiveness of the proposed algorithm.

scholarly journals A Proximal Alternating Direction Method of Multipliers with a Substitution Procedure

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Miantao Chao ◽  
Yongxin Zhao ◽  
Dongying Liang
Keyword(s):  
Linear Constraints ◽  
Alternating Direction Method ◽  
Convex Programming Problem ◽  
Method Of Multipliers ◽  
Worst Case ◽  
Alternating Direction ◽  
Separable Convex Programming ◽  
Contractive Type ◽  
Gradient Based ◽  
Substitution Procedure

In this paper, we considers the separable convex programming problem with linear constraints. Its objective function is the sum of m individual blocks with nonoverlapping variables and each block consists of two functions: one is smooth convex and the other one is convex. For the general case m≥3, we present a gradient-based alternating direction method of multipliers with a substitution. For the proposed algorithm, we prove its convergence via the analytic framework of contractive-type methods and derive a worst-case O1/t convergence rate in nonergodic sense. Finally, some preliminary numerical results are reported to support the efficiency of the proposed algorithm.

scholarly journals On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

2013 ◽  
Vol 2013 ◽  
pp. 1-7 ◽  
Author(s):  
Caihua Chen ◽  
Yuan Shen ◽  
Yanfei You
Keyword(s):  
Alternating Direction Method ◽  
Method Of Multipliers ◽  
Objective Functions ◽  
Step Size ◽  
Mild Conditions ◽  
Alternating Direction ◽  
Separable Convex Programming ◽  
Optimal Step Size ◽  
Linearly Constrained ◽  
Kkt Points

We consider a class of linearly constrained separable convex programming problems whose objective functions are the sum of three convex functions without coupled variables. For those problems, Han and Yuan (2012) have shown that the sequence generated by the alternating direction method of multipliers (ADMM) with three blocks converges globally to their KKT points under some technical conditions. In this paper, a new proof of this result is found under new conditions which are much weaker than Han and Yuan’s assumptions. Moreover, in order to accelerate the ADMM with three blocks, we also propose a relaxed ADMM involving an additional computation of optimal step size and establish its global convergence under mild conditions.

scholarly journals A Dynamical Alternating Direction Multiplier Method for Two-Block Optimization Problems

2021 ◽  
Author(s):  
Miantao Chao ◽  
Liqun Liu
Keyword(s):  
Dynamical System ◽  
Optimization Problems ◽  
Suitable Condition ◽  
Time Discretization ◽  
Alternating Direction Method ◽  
Lagrangian Function ◽  
Method Of Multipliers ◽  
Worst Case ◽  
Alternating Direction ◽  
Separable Optimization

Abstract In this paper, we propose a dynamic alternating direction method of multipliers for two-block separable optimization problems. The well-known classical ADMM can be obtained after the time discretization of the dynamical system. Under suitable condition, we prove that the trajectory asymptotically converges to a saddle point of the Lagrangian function of the problems. When the coefficient matrices in the constraint are identiy matrices, we prove the worst-case O(1/t) convergence rate in ergodic sense.

Sign in / Sign up

Export Citation Format

Share Document