On the Convergence Properties of a Majorized Alternating Direction Method of Multipliers for Linearly Constrained Convex Optimization Problems with Coupled Objective Functions

2016 ◽  
Vol 169 (3) ◽  
pp. 1013-1041 ◽  
Author(s):  
Ying Cui ◽  
Xudong Li ◽  
Defeng Sun ◽  
Kim-Chuan Toh
Keyword(s):  
Convex Optimization ◽  
Optimization Problems ◽  
Alternating Direction Method ◽  
Method Of Multipliers ◽  
Objective Functions ◽  
Convergence Properties ◽  
Convex Optimization Problems ◽  
Alternating Direction ◽  
Linearly Constrained ◽  
Linearly Constrained Convex Optimization

scholarly journals On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

2013 ◽  
Vol 2013 ◽  
pp. 1-7 ◽  
Author(s):  
Caihua Chen ◽  
Yuan Shen ◽  
Yanfei You
Keyword(s):  
Alternating Direction Method ◽  
Method Of Multipliers ◽  
Objective Functions ◽  
Step Size ◽  
Mild Conditions ◽  
Alternating Direction ◽  
Separable Convex Programming ◽  
Optimal Step Size ◽  
Linearly Constrained ◽  
Kkt Points

We consider a class of linearly constrained separable convex programming problems whose objective functions are the sum of three convex functions without coupled variables. For those problems, Han and Yuan (2012) have shown that the sequence generated by the alternating direction method of multipliers (ADMM) with three blocks converges globally to their KKT points under some technical conditions. In this paper, a new proof of this result is found under new conditions which are much weaker than Han and Yuan’s assumptions. Moreover, in order to accelerate the ADMM with three blocks, we also propose a relaxed ADMM involving an additional computation of optimal step size and establish its global convergence under mild conditions.

scholarly journals Managing randomization in the multi-block alternating direction method of multipliers for quadratic optimization

2020 ◽  
Author(s):  
Krešimir Mihić ◽  
Mingxi Zhu ◽  
Yinyu Ye
Keyword(s):  
Quadratic Programming ◽  
Large Scale ◽  
Optimization Problems ◽  
Quadratic Optimization ◽  
Alternating Direction Method ◽  
Convex Quadratic Programming ◽  
Method Of Multipliers ◽  
Numerical Tests ◽  
Alternating Direction ◽  
Decision Variables

Abstract The Alternating Direction Method of Multipliers (ADMM) has gained a lot of attention for solving large-scale and objective-separable constrained optimization. However, the two-block variable structure of the ADMM still limits the practical computational efficiency of the method, because one big matrix factorization is needed at least once even for linear and convex quadratic programming. This drawback may be overcome by enforcing a multi-block structure of the decision variables in the original optimization problem. Unfortunately, the multi-block ADMM, with more than two blocks, is not guaranteed to be convergent. On the other hand, two positive developments have been made: first, if in each cyclic loop one randomly permutes the updating order of the multiple blocks, then the method converges in expectation for solving any system of linear equations with any number of blocks. Secondly, such a randomly permuted ADMM also works for equality-constrained convex quadratic programming even when the objective function is not separable. The goal of this paper is twofold. First, we add more randomness into the ADMM by developing a randomly assembled cyclic ADMM (RAC-ADMM) where the decision variables in each block are randomly assembled. We discuss the theoretical properties of RAC-ADMM and show when random assembling helps and when it hurts, and develop a criterion to guarantee that it converges almost surely. Secondly, using the theoretical guidance on RAC-ADMM, we conduct multiple numerical tests on solving both randomly generated and large-scale benchmark quadratic optimization problems, which include continuous, and binary graph-partition and quadratic assignment, and selected machine learning problems. Our numerical tests show that the RAC-ADMM, with a variable-grouping strategy, could significantly improve the computation efficiency on solving most quadratic optimization problems.

Sign in / Sign up

Export Citation Format

Share Document