LMBOPT: a limited memory method for bound-constrained optimization

Recently, Neumaier and Azmi gave a comprehensive convergence theory for a generic algorithm for bound constrained optimization problems with a continuously differentiable objective function. The algorithm combines an active set strategy with a gradient-free line search along a piecewise linear search path defined by directions chosen to reduce zigzagging. This paper describes , an efficient implementation of this scheme. It employs new limited memory techniques for computing the search directions, improves by adding various safeguards relevant when finite precision arithmetic is used, and adds many practical enhancements in other details. The paper compares and several other solvers on the unconstrained and bound constrained problems from the collection and makes recommendations on which solver to use and when. Depending on the problem class, the problem dimension, and the precise goal, the best solvers are , , and .

Neural network method of the decision Of the nonlinear problem of optimum distribution of the non-uniform resource

Given article is devoted features of the decision of a problem of integer nonlinear programming, by means of developed neural network method and algorithm of nonlinear optimization of means «decision Search» tabular processor Microsoft Excel. In offered neural network method the task in view decision is made by means of a recurrent neural network (RNN) matrix architecture with m neurons in each line and n neurons in each column. All neurons such network are connected with each other by communications, and the signal from an exit neuron can move on its input. Neural network method is characterized by that on inputs mentioned RNN the entrance vector of values of parameters of optimized nonlinear criterion function of a problem of distribution of a non-uniform resource moves, calculation of values of weight factors connected among themselves neurons is carried out and signal RNN is formed. This signal by means of nonlinear function will be transformed to the discrete target signal characterizing values quasi-optimal of the decision of the mentioned problem which size changes from 0 to 1. The estimation of efficiency of the decision of a considered problem was carried out at its various values of an indicator of efficiency on the basis of developed imitating model RNN. As indicators of efficiency of application offered neural network method were used – an average relative error and time of the decision of a problem. The value received by means of algorithm of nonlinear optimization of means was accepted to the exact decision «decision Search» tabular processor Microsoft Excel. The analysis of the received results of the experimental researches, offered neural network method, has allowed to make the conclusion that in comparison with an existing method of nonlinear optimization of tabular processor Microsoft Excel use offered neural network method allows essentially (in 9,4 times) to lower time of the decision of a problem dimension 10 × 8 (m × n) and thus to provide accuracy of its decision not less than 99,8 %.

On the Locally Polynomial Complexity of the Projection-Gradient Method for Solving Piecewise Quadratic Optimisation Problems

This paper proposes a method for solving optimisation problems involving piecewise quadratic functions. The method provides a solution in a finite number of iterations, and the computational complexity of the proposed method is locally polynomial of the problem dimension, i.e., if the initial point belongs to the sufficiently small neighbourhood of the solution set. Proposed method could be applied for solving large systems of linear inequalities.

A Diversity Model Based on Dimension Entropy and Its Application to Swarm Intelligence Algorithm

The swarm intelligence algorithm has become an important method to solve optimization problems because of its excellent self-organization, self-adaptation, and self-learning characteristics. However, when a traditional swarm intelligence algorithm faces high and complex multi-peak problems, population diversity is quickly lost, which leads to the premature convergence of the algorithm. In order to solve this problem, dimension entropy is proposed as a measure of population diversity, and a diversity control mechanism is proposed to guide the updating of the swarm intelligence algorithm. It maintains the diversity of the algorithm in the early stage and ensures the convergence of the algorithm in the later stage. Experimental results show that the performance of the improved algorithm is better than that of the original algorithm.

Interaction-Transformation Evolutionary Algorithm for Symbolic Regression

Interaction-Transformation (IT) is a new representation for Symbolic Regression that reduces the space of solutions to a set of expressions that follow a specific structure. The potential of this representation was illustrated in prior work with the algorithm called SymTree. This algorithm starts with a simple linear model and incrementally introduces new transformed features until a stop criterion is met. While the results obtained by this algorithm were competitive with the literature, it had the drawback of not scaling well with the problem dimension. This paper introduces a mutation only Evolutionary Algorithm, called ITEA, capable of evolving a population of IT expressions. One advantage of this algorithm is that it enables the user to specify the maximum number of terms in an expression. In order to verify the competitiveness of this approach, ITEA is compared to linear, nonlinear and Symbolic Regression models from the literature. The results indicate that ITEA is capable of finding equal or better approximations than other Symbolic Regression models while being competitive to state-of-the-art non-linear models. Additionally, since this representation follows a specific structure, it is possible to extract the importance of each original feature of a data set as an analytical function, enabling us to automate the explanation of any prediction. In conclusion, ITEA is competitive when comparing to regression models with the additional benefit of automating the extraction of additional information of the generated models.

Coupling BEM and the Local Point Interpolation for the Solution of Anisotropic Elastic Nonlinear, Multi-Physics and Multi-Fields Problems

The pure boundary element method (BEM) is effective for the solution of a large class of problems. The main appeal of this BEM (reduction of the problem dimension by one) is tarnished to some extent when a fundamental solution to the governing equations does not exist as in the case of nonlinear problems. The easy to implement local point interpolation method applied to the strong form of differential equations is an attractive numerical approach. Its accuracy deteriorates in the presence of Neumann-type boundary conditions which are practically inevitable in solid mechanics. The main appeal of the BEM can be maintained by a judicious coupling of the pure BEM with the local point interpolation method. The resulting approach, named the LPI-BEM, seems versatile and effective. This is demonstrated by considering some linear and nonlinear elasticity problems including multi-physics and multi-field problems.

Differentially Private Iterative Gradient Hard Thresholding for Sparse Learning

We consider the differentially private sparse learning problem, where the goal is to estimate the underlying sparse parameter vector of a statistical model in the high-dimensional regime while preserving the privacy of each training example. We propose a generic differentially private iterative gradient hard threshoding algorithm with a linear convergence rate and strong utility guarantee. We demonstrate the superiority of our algorithm through two specific applications: sparse linear regression and sparse logistic regression. Specifically, for sparse linear regression, our algorithm can achieve the best known utility guarantee without any extra support selection procedure used in previous work \cite{kifer2012private}. For sparse logistic regression, our algorithm can obtain the utility guarantee with a logarithmic dependence on the problem dimension. Experiments on both synthetic data and real world datasets verify the effectiveness of our proposed algorithm.