Diagonal bundle method with convex and concave updates for large-scale nonconvex and nonsmooth optimization

Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving optimization problems, but the knowledge of PCD methods in the nonconvex setting is very limited. On the other hand, the asynchronous proximal coordinate descent (APCD) recently have received much attention in order to solve large-scale problems. However, the accelerated variants of APCD algorithms are rarely studied. In this paper, we extend APCD method to the accelerated algorithm (AAPCD) for nonsmooth and nonconvex problems that satisfies the sufficient descent property, by comparing between the function values at proximal update and a linear extrapolated point using a delay-aware momentum value. To the best of our knowledge, we are the first to provide stochastic and deterministic accelerated extension of APCD algorithms for general nonconvex and nonsmooth problems ensuring that for both bounded delays and unbounded delays every limit point is a critical point. By leveraging Kurdyka-Łojasiewicz property, we will show linear and sublinear convergence rates for the deterministic AAPCD with bounded delays. Numerical results demonstrate the practical efficiency of our algorithm in speed.

Download Full-text

An alternating linearization bundle method for a class of nonconvex nonsmooth optimization problems

Journal of Inequalities and Applications ◽

10.1186/s13660-018-1683-1 ◽

2018 ◽

Vol 2018 (1) ◽

Author(s):

Chunming Tang ◽

Jinman Lv ◽

Jinbao Jian

Keyword(s):

Nonsmooth Optimization ◽

Optimization Problems ◽

Bundle Method ◽

Nonconvex Nonsmooth Optimization

Download Full-text

Solving the Short-Term Scheduling Problem of Hydrothermal Systems via Lagrangian Relaxation and Augmented Lagrangian

Mathematical Problems in Engineering ◽

10.1155/2012/856178 ◽

2012 ◽

Vol 2012 ◽

pp. 1-18 ◽

Cited By ~ 5

Author(s):

Rafael N. Rodrigues ◽

Edson L. da Silva ◽

Erlon C. Finardi ◽

Fabricio Y. K. Takigawa

Keyword(s):

Lagrangian Relaxation ◽

Large Scale ◽

Augmented Lagrangian ◽

Augmented Lagrangian Method ◽

Real Life ◽

Hydrothermal Systems ◽

Bundle Method ◽

Mixed Integer ◽

Scheduling Problem ◽

Short Term

This paper addresses the short-term scheduling problem of hydrothermal power systems, which results in a large-scale mixed-integer nonlinear programming problem. The objective consists in minimizing the operation cost over a two-day horizon with a one-hour time resolution. To solve this difficult problem, a Lagrangian Relaxation (LR) based on variable splitting is designed where the resulting dual problem is solved by a Bundle method. Given that the LR usually fails to find a feasible solution, we use an inexact Augmented Lagrangian method to improve the quality of the solution supplied by the LR. We assess our approach by using a real-life hydrothermal configuration extracted from the Brazilian power system, proving the conceptual and practical feasibility of the proposed algorithm. In summary, the main contributions of this paper are (i) a detailed and compatible modelling for this problem is presented; (ii) in order to solve efficiently the entire problem, a suitable decomposition strategy is presented. As a result of these contributions, the proposed model is able to find practical solutions with moderate computational burden, which is absolutely necessary in the modern power industry.

Download Full-text

Efficient Optimization ofF-Measure with Cost-Sensitive SVM

Mathematical Problems in Engineering ◽

10.1155/2016/5873769 ◽

2016 ◽

Vol 2016 ◽

pp. 1-11 ◽

Cited By ~ 2

Author(s):

Fan Cheng ◽

Yuan Zhou ◽

Jian Gao ◽

Shuangqiu Zheng

Keyword(s):

Large Scale ◽

Line Search ◽

Performance Metrics ◽

Empirical Studies ◽

Closed Form Solution ◽

Form Solution ◽

Bundle Method ◽

Additional Line ◽

Direct Optimization ◽

Current State

F-measure is one of the most commonly used performance metrics in classification, particularly when the classes are highly imbalanced. Direct optimization of this measure is often challenging, since no closed form solution exists. Current algorithms design the classifiers by using the approximations to theF-measure. These algorithms are not efficient and do not scale well to the large datasets. To fill the gap, in this paper, we propose a novel algorithm, which can efficiently optimizeF-measure with cost-sensitive SVM. First of all, we present an explicit transformation from the optimization ofF-measure to cost-sensitive SVM. Then we adopt bundle method to solve the inner optimization. For the problem where the existing bundle method may have the fluctuations in the primal objective during iterations, an additional line search procedure is involved, which can alleviate the fluctuations problem and make our algorithm more efficient. Empirical studies on the large-scale datasets demonstrate that our algorithm can provide significant speedups over current state-of-the-artF-measure based learners, while obtaining better (or comparable) precise solutions.

Download Full-text