Dual and Cutting Plane Methods

We consider the problem of best [Formula: see text]-subset convex regression using [Formula: see text] observations in [Formula: see text] variables. For the case without sparsity, we develop a scalable algorithm for obtaining high quality solutions in practical times that compare favorably with other state of the art methods. We show that by using a cutting plane method, the least squares convex regression problem can be solved for sizes [Formula: see text] in minutes and [Formula: see text] in hours. Our algorithm can be adapted to solve variants such as finding the best convex or concave functions with coordinate-wise monotonicity, norm-bounded subgradients, and minimize the [Formula: see text] loss—all with similar scalability to the least squares convex regression problem. Under sparsity, we propose algorithms which iteratively solve for the best subset of features based on first order and cutting plane methods. We show that our methods scale for sizes [Formula: see text] in minutes and [Formula: see text] in hours. We demonstrate that these methods control for the false discovery rate effectively.

Download Full-text

Cutting Plane Methods for Global Optimization

Encyclopedia of Optimization ◽

10.1007/0-306-48332-7_79 ◽

2006 ◽

pp. 366-371 ◽

Cited By ~ 1

Author(s):

Hoang Tuy

Keyword(s):

Global Optimization ◽

Cutting Plane ◽

Cutting Plane Methods

Download Full-text

Cutting Plane Methods for Analytical Target Cascading With Augmented Lagrangian Coordination

Journal of Mechanical Design ◽

10.1115/1.4024847 ◽

2013 ◽

Vol 135 (10) ◽

Cited By ~ 5

Author(s):

Wenshan Wang ◽

Vincent Y. Blouin ◽

Melissa K. Gardenghi ◽

Georges M. Fadel ◽

Margaret M. Wiecek ◽

...

Keyword(s):

Large Scale ◽

Augmented Lagrangian ◽

Optimization Problems ◽

Cutting Plane ◽

Analytical Target Cascading ◽

Cutting Plane Methods ◽

Subgradient Optimization ◽

Decomposition Approach ◽

Target Cascading ◽

Augmented Lagrangian Coordination

Analytical target cascading (ATC), a hierarchical, multilevel, multidisciplinary coordination method, has proven to be an effective decomposition approach for large-scale engineering optimization problems. In recent years, augmented Lagrangian relaxation methods have received renewed interest as dual update methods for solving ATC decomposed problems. These problems can be solved using the subgradient optimization algorithm, the application of which includes three schemes for updating dual variables. To address the convergence efficiency disadvantages of the existing dual update schemes, this paper investigates two new schemes, the linear and the proximal cutting plane methods, which are implemented in conjunction with augmented Lagrangian coordination for ATC-decomposed problems. Three nonconvex nonlinear example problems are used to show that these two cutting plane methods can significantly reduce the number of iterations and the number of function evaluations when compared to the traditional subgradient update methods. In addition, these methods are also compared to the method of multipliers and its variants, showing similar performance.

Download Full-text