Deterministic global optimization with Gaussian processes embedded

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the “MeLOn—MachineLearning Models for Optimization” toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

A deterministic semi-infinite global optimization method to design slotless permanent magnet machines

Purpose In this work, a method to design a slotless permanent magnet machine (SPMM) based on the joint use of an analytical model and deterministic global optimization algorithms is addressed. The purpose of this study is to propose to include torque ripples as an extra constraint in the optimization phase involving de facto the study of a semi-infinite optimization problem. Design/methodology/approach Based on the use of a well-known analytical model describing the electromagnetic behavior of an SPMM, this analytical model has been supplemented by the calculus of the dynamic torque and its ripples to carry out a more accurate optimized sizing method of such an electromechanical converter. As a consequence, the calculated torque depends on a continuous variable, namely, the rotor angular position, resulting in the definition of a semi-infinite optimization problem. The way to solve this kind of semi-infinite problem by discretizing the rotor angular position by using a deterministic global optimization solver, that is to say COUENNE, via the AMPL modeling language is addressed. Findings In this study, the proposed approach is validated on some numerical tests based on the minimization of the magnet volume. Efficient global optimal solutions with torque ripples about 5% (instead of 30%) can be so obtained. Research limitations/implications The analytical model does not use results from the solution of two-dimensional field equations. A strong assumption is put forward to approximate the distribution of the magnetic flux density in the air gap of the SPMM. Originality/value The problem to design an SPMM can be efficiently formulated as a semi-infinite global optimization problem. This kind of optimization problems are hard to solve because they involve an infinity of constraints (coming from a constraint on the torque ripple). The authors show in this paper that by using analytical models, a discretization method and a deterministic global optimization code COUENNE, this problem is efficiently tackled. Some numerical results show that the deterministic global solution of the design can be reached even if the step of discretization is small.

Linearization of McCormick relaxations and hybridization with the auxiliary variable method

The computation of lower bounds via the solution of convex lower bounding problems depicts current state-of-the-art in deterministic global optimization. Typically, the nonlinear convex relaxations are further underestimated through linearizations of the convex underestimators at one or several points resulting in a lower bounding linear optimization problem. The selection of linearization points substantially affects the tightness of the lower bounding linear problem. Established methods for the computation of such linearization points, e.g., the sandwich algorithm, are already available for the auxiliary variable method used in state-of-the-art deterministic global optimization solvers. In contrast, no such methods have been proposed for the (multivariate) McCormick relaxations. The difficulty of determining a good set of linearization points for the McCormick technique lies in the fact that no auxiliary variables are introduced and thus, the linearization points have to be determined in the space of original optimization variables. We propose algorithms for the computation of linearization points for convex relaxations constructed via the (multivariate) McCormick theorems. We discuss alternative approaches based on an adaptation of Kelley’s algorithm; computation of all vertices of an n-simplex; a combination of the two; and random selection. All algorithms provide substantial speed ups when compared to the single point strategy used in our previous works. Moreover, we provide first results on the hybridization of the auxiliary variable method with the McCormick technique benefiting from the presented linearization strategies resulting in additional computational advantages.

Automatic Convexity Deduction for Efficient Function’s Range Bounding

Reliable bounding of a function’s range is essential for deterministic global optimization, approximation, locating roots of nonlinear equations, and several other computational mathematics areas. Despite years of extensive research in this direction, there is still room for improvement. The traditional and compelling approach to this problem is interval analysis. We show that accounting convexity/concavity can significantly tighten the bounds computed by interval analysis. To make our approach applicable to a broad range of functions, we also develop the techniques for handling nondifferentiable composite functions. Traditional ways to ensure the convexity fail in such cases. Experimental evaluation showed the remarkable potential of the proposed methods.

Multi-scale membrane process optimization with high-fidelity ion transport models through machine learning

Innovative membrane technologies optimally integrated into large separation process plants are essential for economical water treatment and disposal. However, the mass transport through membranes is commonly described by nonlinear differential-algebraic mechanistic models at the nano-scale, while the process and its economics range up to large-scale. Thus, the optimal design of membranes in process plants requires decision making across multiple scales, which is not tractable using standard tools. In this work, we embed artificial neural networks (ANNs) as surrogate models in the deterministic global optimization to bridge the gap of scales. This methodology allows for deterministic global optimization of membrane processes with accurate transport models – avoiding the utilization of inaccurate approximations through heuristics or short-cut models. The ANNs are trained based on data generated by a one-dimensional extended Nernst-Planck ion transport model and extended to a more accurate two-dimensional distribution of the membrane module, that captures the filtration-related decreasing retention of salt. We simultaneously design the membrane and plant layout yielding optimal membrane module synthesis properties along with the optimal plant design for multiple objectives, feed concentrations, filtration stages, and salt mixtures. The developed process models and the optimization solver are available open-source, enabling computational resource-efficient multi-scale optimization in membrane science.