Expensive Black-Box Model Optimization via a Gold Rush Policy

2018 ◽  
Author(s):  
Benson Isaac ◽  
Douglas Allaire
Keyword(s):  
Global Optimization ◽  
Structural Information ◽  
Optimization Methods ◽  
Gold Rush ◽  
Black Box ◽  
Optimization Techniques ◽  
Benchmark Problems ◽  
Expensive Function ◽  
Box Models ◽  
Bayesian Global Optimization

The optimization of expensive black-box models is a challenging task owing to the lack of analytic gradient information and structural information about the underlying function, and also due to the sheer computational expense. A common approach to tackling such problems is the implementation of Bayesian global optimization techniques. However, these techniques often rely on surrogate modeling strategies that endow the approximation of the underlying expensive function with nonexistent features. Further, these techniques tend to push new queries away from previously queried design points, making it difficult to locate an optimum point that rests near a previous model evaluation. To overcome these issues, we propose a gold rush policy that relies on purely local information to identify the next best design alternative to query. The method employs a surrogate constructed pointwise, that adds no additional features to the approximation. The result is a policy that performs well in comparison to state of the art Bayesian global optimization methods on several benchmark problems. The policy is also demonstrated on a constrained optimization problem using a penalty method.

Expensive Black-Box Model Optimization Via a Gold Rush Policy

2019 ◽  
Vol 141 (3) ◽  
Author(s):  
Benson Isaac ◽  
Douglas Allaire
Keyword(s):  
Global Optimization ◽  
Structural Information ◽  
Optimization Methods ◽  
Gold Rush ◽  
Black Box ◽  
Optimization Techniques ◽  
Benchmark Problems ◽  
Expensive Function ◽  
Box Models ◽  
Bayesian Global Optimization

The optimization of black-box models is a challenging task owing to the lack of analytic gradient information and structural information about the underlying function, and also due often to significant run times. A common approach to tackling such problems is the implementation of Bayesian global optimization techniques. However, these techniques often rely on surrogate modeling strategies that endow the approximation of the underlying expensive function with nonexistent features. Further, these techniques tend to push new queries away from previously queried design points, making it difficult to locate an optimum point that rests near a previous model evaluation. To overcome these issues, we propose a gold rush (GR) policy that relies on purely local information to identify the next best design alternative to query. The method employs a surrogate constructed pointwise, that adds no additional features to the approximation. The result is a policy that performs well in comparison to state of the art Bayesian global optimization methods on several benchmark problems. The policy is also demonstrated on a constrained optimization problem using a penalty method.

Adaptive Dimensionality Reduction for Fast Sequential Optimization With Gaussian Processes

2019 ◽  
Vol 141 (7) ◽  
Author(s):  
Seyede Fatemeh Ghoreishi ◽  
Samuel Friedman ◽  
Douglas L. Allaire
Keyword(s):  
Global Optimization ◽  
Computational Models ◽  
Black Box ◽  
Optimization Techniques ◽  
Sequential Optimization ◽  
Benchmark Problems ◽  
Box Models ◽  
Gradient Based ◽  
Bayesian Global Optimization ◽  
Knowledge Gradient

Available computational models for many engineering design applications are both expensive and and of a black-box nature. This renders traditional optimization techniques difficult to apply, including gradient-based optimization and expensive heuristic approaches. For such situations, Bayesian global optimization approaches, that both explore and exploit a true function while building a metamodel of it, are applied. These methods often rely on a set of alternative candidate designs over which a querying policy is designed to search. For even modestly high-dimensional problems, such an alternative set approach can be computationally intractable, due to the reliance on excessive exploration of the design space. To overcome this, we have developed a framework for the optimization of expensive black-box models, which is based on active subspace exploitation and a two-step knowledge gradient policy. We demonstrate our approach on three benchmark problems and a practical aerostructural wing design problem, where our method performs well against traditional direct application of Bayesian global optimization techniques.

A New Global Optimization Method for Simultaneous Computation on Expensive Black-Box Functions

2003 ◽  
Author(s):  
Liqun Wang ◽  
Songqing Shan ◽  
G. Gary Wang
Keyword(s):  
Global Optimization ◽  
Optimization Problems ◽  
Optimization Methods ◽  
Search Space ◽  
Optimization Method ◽  
Global Optimum ◽  
Black Box ◽  
Efficient Global Optimization ◽  
Global Optimization Method ◽  
Global Optimization Methods

The presence of black-box functions in engineering design, which are usually computation-intensive, demands efficient global optimization methods. This work proposes a new global optimization method for black-box functions. The global optimization method is based on a novel mode-pursuing sampling (MPS) method which systematically generates more sample points in the neighborhood of the function mode while statistically covers the entire search space. Quadratic regression is performed to detect the region containing the global optimum. The sampling and detection process iterates until the global optimum is obtained. Through intensive testing, this method is found to be effective, efficient, robust, and applicable to both continuous and discontinuous functions. It supports simultaneous computation and applies to both unconstrained and constrained optimization problems. Because it does not call any existing global optimization tool, it can be used as a standalone global optimization method for inexpensive problems as well. Limitation of the method is also identified and discussed.

Prediction of Pareto Dominance Using an Attribute Tendency Model for Expensive Multi-Objective Optimization

2019 ◽  
Vol 29 (02) ◽  
pp. 2050021
Author(s):  
Wenbin Li ◽  
Junqiang Jiang ◽  
Xi Chen ◽  
Guanqi Guo ◽  
Jianjun He
Keyword(s):  
Surrogate Model ◽  
Prediction Method ◽  
Optimization Methods ◽  
Black Box ◽  
Benchmark Problems ◽  
Pareto Dominance ◽  
Decision Space ◽  
Multi Objective ◽  
Decision Attributes ◽  
Coarse Model

This paper proposes a novel surrogate-assisted multi-objective evolutionary algorithm, MOEA-ATCM, to solve expensive or black-box multi-objective problems with small evaluation budgets. The proposed approach encompasses a state-of-the-art MOEA based on a nondominated sorting genetic algorithm assisted by multi-fidelity optimization methods. A high-fidelity attribute tendency (AT) surrogate model was used to construct a linear decision space by introducing the knowledge of the objective space. A coarse model (CM) based on the AT model and correlation analyses of the objective functions and decision attributes were used to predict the Pareto dominance for candidates in the new decision space constructed by the AT model. Two major roles of MOEA-ATCM were identified: (1) the development of a new multi-fidelity surrogate-model-based method to predict Pareto dominance in a decision space that was then applied to MOEA, which does not need to dynamically update surrogate models in the optimization process and (2) the development of a Pareto dominance prediction method to obtain good nondominated solutions of expensive or black box problems with relatively few objective function evaluations. The advantages of MOEA-ATCM were verified by mathematical benchmark problems and a real-world multi-objective parameter optimization problem.

Hybrid Optimization Techniques for Industrial Production Planning

2014 ◽  
pp. 41-68 ◽  
Author(s):  
P. Vasant
Keyword(s):  
Global Optimization ◽  
Production Planning ◽  
Industrial Production ◽  
Optimization Problems ◽  
Hybrid Methods ◽  
Optimization Methods ◽  
Optimization Techniques ◽  
Search Pattern ◽  
Future Research ◽  
Hybrid Optimization Techniques

This chapter provides a review of new hybrid methods that deal with the continuous local and global optimization problems for constrained industrial production planning problems. In this chapter, details about all types of optimization methods and approaches for the local and global optimization are highlighted. Altogether there are eight famous methods in hybrid evolutionary optimization. In this research, the hybridization between evolutionary algorithms and other heuristic approaches such as simulated annealing, line search, pattern search, and mesh adaptive direct search are adopted. A particular evolutionary computation approach of genetic algorithm is used in this hybridization process. An intelligent performance analysis table is suggested in this chapter which is significantly important for decision makers and implementers in the industrial engineering of production planning. A brief summary on the conclusions of the main contributions and achievements in this chapter as well as future research directions are highlighted.

scholarly journals Calibration of the modified Bartlett-Lewis model using global optimization techniques and alternative objective functions

2012 ◽  
Vol 16 (3) ◽  
pp. 873-891 ◽  
Author(s):  
W. J. Vanhaute ◽  
S. Vandenberghe ◽  
K. Scheerlinck ◽  
B. De Baets ◽  
N. E. C. Verhoest
Keyword(s):  
Global Optimization ◽  
Optimization Methods ◽  
Optimization Method ◽  
Optimization Techniques ◽  
Objective Functions ◽  
Local Search Algorithms ◽  
Stochastic Point Process ◽  
Shuffled Complex Evolution ◽  
Global Optimization Methods ◽  
Downhill Simplex

Abstract. The calibration of stochastic point process rainfall models, such as of the Bartlett-Lewis type, suffers from the presence of multiple local minima which local search algorithms usually fail to avoid. To meet this shortcoming, four relatively new global optimization methods are presented and tested for their ability to calibrate the Modified Bartlett-Lewis Model. The list of tested methods consists of: the Downhill Simplex Method, Simplex-Simulated Annealing, Particle Swarm Optimization and Shuffled Complex Evolution. The parameters of these algorithms are first optimized to ensure optimal performance, after which they are used for calibration of the Modified Bartlett-Lewis model. Furthermore, this paper addresses the choice of weights in the objective function. Three alternative weighing methods are compared to determine whether or not simulation results (obtained after calibration with the best optimization method) are influenced by the choice of weights.

Evaluation of global optimization methods for conceptual rainfall-runoff model calibration

1997 ◽  
Vol 36 (5) ◽  
pp. 53-60 ◽  
Author(s):  
V. A. Cooper ◽  
V. T. V. Nguyen ◽  
J. A. Nicell
Keyword(s):  
Global Optimization ◽  
Optimal Solution ◽  
Optimization Methods ◽  
Global Optimum ◽  
Hydrologic Model ◽  
Optimization Techniques ◽  
Model Parameters ◽  
Rainfall Runoff ◽  
Global Optimization Methods ◽  
Optimization Search

The calibration of conceptual rainfall runoff (CRR) models is an optimization problem whose objective is to determine the values of the model parameters which provide the best fit between observed and estimated flows. This study investigated the performance of three probabilistic optimization techniques for calibrating the Tank model, a hydrologic model typical of CRR models. These methods were the Shuffled Complex Evolution (SCE), genetic algorithms (GA) and simulated annealing (SA) methods. It was found that performances depended on the choice of the objective function considered and also an the position of the start of the optimization search relative to the global optimum. Of the three global optimization methods (GOM) in the study, the SCE method provided better estimates of the optimal solution than the GA and SA methods. Regarding the efficiency of the GOMs, as expressed by the number of iterations for convergence, the ranking in order of decreasing performance was the SCE, the GA and the SA methods.

AN ASSESSMENT OF THE PERFORMANCE OF GLOBAL OPTIMIZATION METHODS FOR GEO-ACOUSTIC INVERSION

2008 ◽  
Vol 16 (02) ◽  
pp. 199-223 ◽  
Author(s):  
MIRJAM SNELLEN ◽  
DICK G. SIMONS
Keyword(s):  
Global Optimization ◽  
Optimization Methods ◽  
Simplex Algorithm ◽  
Optimization Techniques ◽  
Acoustic Properties ◽  
Superior Performance ◽  
Efficient Global Optimization ◽  
Acoustic Inversion ◽  
Global Optimization Methods ◽  
Downhill Simplex

Having available efficient global optimization methods is of high importance when going to a practical application of geo-acoustic inversion, where fast processing of the data is an essential requirement. A series of global optimization techniques are available and have been described in literature. In this paper three optimization techniques are considered, being a genetic algorithm (GA), differential evolution (DE), and the downhill simplex algorithm (DHS). The performance of these three methods is assessed using a test function, demonstrating superior performance of DE. Additionally, the DE optimal setting is determined. As a next step DE is applied for determining the geo-acoustic properties of the upper seabed sediments from simulated seabed reflection loss, indicating good DE performance also for real geo-acoustic inversion problems.

scholarly journals Calibration of the modified Bartlett-Lewis model using global optimization techniques and alternative objective functions

2011 ◽  
Vol 8 (6) ◽  
pp. 9707-9756 ◽  
Author(s):  
W. J. Vanhaute ◽  
S. Vandenberghe ◽  
K. Scheerlinck ◽  
B. De Baets ◽  
N. E. C. Verhoest
Keyword(s):  
Time Series ◽  
Global Optimization ◽  
Optimization Methods ◽  
Optimization Method ◽  
Optimization Techniques ◽  
Rainfall Time Series ◽  
Stochastic Point Process ◽  
Shuffled Complex Evolution ◽  
Global Optimization Methods ◽  
Downhill Simplex

Abstract. The use of rainfall time series for various applications is widespread. However, in many cases historical rainfall records lack in length or quality for certain practical purposes, resulting in a reliance on rainfall models to supply simulated rainfall time series, e.g., in the design of hydraulic structures. One way to obtain such simulations is by means of stochastic point process rainfall models, such as the Bartlett-Lewis type of model. It is widely acknowledged that the calibration of such models suffers from the presence of multiple local minima which local search algorithms usually fail to avoid. To meet this shortcoming, four relatively new global optimization methods are presented and tested for their abilities to calibrate the Modified Bartlett-Lewis Model (MBL). The list of tested methods consists of: the Downhill Simplex Method (DSM), Simplex-Simulated Annealing (SIMPSA), Particle Swarm Optimization (PSO) and Shuffled Complex Evolution (SCE-UA). The parameters of these algorithms are first optimized to ensure optimal performance, after which they are used for calibration of the MBL model. Furthermore, this paper addresses the issue of subjectivity in the choice of weights in the objective function. Three alternative weighing methods are compared to determine whether or not simulation results (obtained after calibration with the best optimization method) are influenced by the choice of weights.

Sign in / Sign up

Export Citation Format

Share Document