On Exchange Methods for Nonlinear Semi-Infinite Programs

A new exchange method is presented for semi-infinite optimization problems with polyhedron constraints. The basic idea is to use an active set strategy as exchange rule to construct an approximate problem with finitely many constraints at each iteration. Under mild conditions, we prove that the proposed algorithm terminates in a finite number of iterations and guarantees that the solution of the resulting approximate problem at final iteration converges to the solution of the original problem within arbitrarily given tolerance. Numerical results indicate that the proposed algorithm is efficient and promising.

Accelerated Asynchronous Greedy Coordinate Descent Algorithm for SVMs

Support vector machines play an important role in machine learning in the last two decades. Traditional SVM solvers (e.g. LIBSVM) are not scalable in the current big data era. Recently, a state of the art solver was proposed based on the asynchronous greedy coordinate descent (AsyGCD) algorithm. However, AsyGCD is still not scalable enough, and is limited to binary classification. To address these issues, in this paper we propose an asynchronous accelerated greedy coordinate descent algorithm (AsyAGCD) for SVMs. Compared with AsyGCD, our AsyAGCD has the following two-fold advantages: 1) our AsyAGCD is an accelerated version of AsyGCD because active set strategy is used. Specifically, our AsyAGCD can converge much faster than AsyGCD for the second half of iterations. 2) Our AsyAGCD can handle more SVM formulations (including binary classification and regression SVMs) than AsyGCD. We provide the comparison of computational complexity of AsyGCD and our AsyAGCD. Experiment results on a variety of datasets and learning applications confirm that our AsyAGCD is much faster than the existing SVM solvers (including AsyGCD).

Augmented Lagrangian Method for Nonlinear Programming with many Complex Constraints

A method of multiplier is presented for solving optimization problems. For large-scale constraint problems, combining the active set strategy, we use the aggregate function to approximate the max-value function. Only a few of functions are involved at each iteration, so the computation for gradient is significantly reduced. The numerical results show that the method is effective.