scholarly journals Combined Global and Local Search for Optimization with Gaussian Process Models

2021 ◽  
Author(s):  
Qun Meng ◽  
Songhao Wang ◽  
Szu Hui Ng
Keyword(s):  
Local Search ◽  
Gaussian Process ◽  
Design Space ◽  
Global Optimum ◽  
Process Models ◽  
Computationally Efficient ◽  
Model Based ◽  
Probability Of Collision ◽  
Global And Local ◽  
Gp Model

Gaussian process (GP) model based optimization is widely applied in simulation and machine learning. In general, it first estimates a GP model based on a few observations from the true response and then uses this model to guide the search, aiming to quickly locate the global optimum. Despite its successful applications, it has several limitations that may hinder its broader use. First, building an accurate GP model can be difficult and computationally expensive, especially when the response function is multimodal or varies significantly over the design space. Second, even with an appropriate model, the search process can be trapped in suboptimal regions before moving to the global optimum because of the excessive effort spent around the current best solution. In this work, we adopt the additive global and local GP (AGLGP) model in the optimization framework. The model is rooted in the inducing points based GP sparse approximations and is combined with independent local models in different regions. With these properties, the AGLGP model is suitable for multimodal responses with relatively large data sizes. Based on this AGLGP model, we propose a combined global and local search for optimization (CGLO) algorithm. It first divides the whole design space into disjoint local regions and identifies a promising region with the global model. Next, a local model in the selected region is fit to guide detailed search within this region. The algorithm then switches back to the global step when a good local solution is found. The global and local natures of CGLO enable it to enjoy the benefits of both global and local search to efficiently locate the global optimum. Summary of Contribution: This work proposes a new Gaussian process based algorithm for stochastic simulation optimization, which is an important area in operations research. This type of algorithm is also regarded as one of the state-of-the-art optimization algorithms for black-box functions in computer science. The aim of this work is to provide a computationally efficient optimization algorithm when the baseline functions are highly nonstationary (the function values change dramatically across the design space). Such nonstationary surfaces are very common in reality, such as the case in the maritime traffic safety problem considered here. In this problem, agent-based simulation is used to simulate the probability of collision of one vessel with the others on a given trajectory, and the decision maker needs to choose the trajectory with the minimum probability of collision quickly. Typically, in a high-congestion region, a small turn of the vessel can result in a very different conflict environment, and thus the response is highly nonstationary. Through our study, we find that the proposed algorithm can provide safer choices within a limited time compared with other methods. We believe the proposed algorithm is very computationally efficient and has large potential in such operational problems.

Methodology for Global Optimization of Computationally Expensive Design Problems

2014 ◽  
Vol 136 (8) ◽  
Author(s):  
Stefanos Koullias ◽  
Dimitri N. Mavris
Keyword(s):  
Global Optimization ◽  
Objective Function ◽  
Surrogate Model ◽  
Design Space ◽  
Design Problems ◽  
Early Design ◽  
Model Based ◽  
Computationally Expensive ◽  
Fully Bayesian ◽  
Gp Model

The design of unconventional systems requires early use of high-fidelity physics-based tools to search the design space for improved and potentially optimum designs. Current methods for incorporating these computationally expensive tools into early design for the purpose of reducing uncertainty are inadequate due to the limited computational resources that are available in early design. Furthermore, the lack of finite difference derivatives, unknown design space properties, and the possibility of code failures motivates the need for a robust and efficient global optimization (EGO) algorithm. A novel surrogate model-based global optimization algorithm capable of efficiently searching challenging design spaces for improved designs is presented. The algorithm, called fBcEGO for fully Bayesian constrained EGO, constructs a fully Bayesian Gaussian process (GP) model through a set of observations and then uses the model to make new observations in promising areas where improvements are likely to occur. This model remedies the inadequacies of likelihood-based approaches, which may provide an incomplete inference of the underlying function when function evaluations are expensive and therefore scarce. A challenge in the construction of the fully Bayesian GP model is the selection of the prior distribution placed on the model hyperparameters. Previous work employs static priors, which may not capture a sufficient number of interpretations of the data to make any useful inferences about the underlying function. An iterative method that dynamically assigns hyperparameter priors by exploiting the mechanics of Bayesian penalization is presented. fBcEGO is incorporated into a methodology that generates relatively few infeasible designs and provides large reductions in the objective function values of design problems. This new algorithm, upon implementation, was found to solve more nonlinearly constrained algebraic test problems to higher accuracies relative to the global minimum than other popular surrogate model-based global optimization algorithms and obtained the largest reduction in the takeoff gross weight objective function for the case study of a notional 70-passenger regional jet when compared with competing design methods.

Gaussian Process Meta-Models for Efficient Probabilistic Design in Complex Engineering Design Spaces

2005 ◽  
Author(s):  
Liping Wang ◽  
Don Beeson ◽  
Srikanth Akkaram ◽  
Gene Wiggs
Keyword(s):  
Gaussian Process ◽  
Design Space ◽  
Computer Experiments ◽  
Response Surfaces ◽  
Probabilistic Design ◽  
Original Design ◽  
Meta Model ◽  
Point Estimate Method ◽  
Highly Nonlinear ◽  
Gp Model

Probabilistic design in complex design spaces is often a computationally expensive and difficult task because of the highly nonlinear and noisy nature of those spaces. Approximate probabilistic methods, such as, First-Order Second-Moments (FOSM) and Point Estimate Method (PEM) have been developed to alleviate the high computational cost issue. However, both methods have difficulty with non-monotonic spaces and FOSM may have convergence problems if noise on the space makes it difficult to calculate accurate numerical partial derivatives. Use of design and Analysis of Computer Experiments (DACE) methods to build polynomial meta-models is a common approach which both smoothes the design space and significantly improves the computational efficiency. However, this type of model is inherently limited by the properties of the polynomial function and its transformations. Therefore, polynomial meta-models may not accurately represent the portion of the design space that is of interest to the engineer. The objective of this paper is to utilize Gaussian Process (GP) techniques to build an alternative meta-model that retains the properties of smoothness and fast execution but has a much higher level of accuracy. If available, this high quality GP model can then be used for fast probabilistic analysis based on a function that much more closely represents the original design space. Achieving the GP goal of a highly accurate meta-model requires a level of mathematics that is much more complex than the mathematics required for regular linear and quadratic response surfaces. Many difficult mathematical issues encountered in the implementation of the Gaussian Process meta-model are addressed in this paper. Several selected examples demonstrate the accuracy of the GP models and efficiency improvements related to probabilistic design.

An Enhanced Squared Exponential Kernel With Manhattan Similarity Measure for High Dimensional Gaussian Process Models

2021 ◽  
Author(s):  
Yanwen Xu ◽  
Pingfeng Wang
Keyword(s):  
Gaussian Process ◽  
Process Model ◽  
Similarity Measures ◽  
Kriging Model ◽  
Process Models ◽  
Superior Performance ◽  
High Dimensional ◽  
Manhattan Distance ◽  
Exponential Kernel ◽  
Gp Model

Abstract The Gaussian Process (GP) model has become one of the most popular methods and exhibits superior performance among surrogate models in many engineering design applications. However, the standard Gaussian process model is not able to deal with high dimensional applications. The root of the problem comes from the similarity measurements of the GP model that relies on the Euclidean distance, which becomes uninformative in the high-dimensional cases, and causes accuracy and efficiency issues. Limited studies explore this issue. In this study, thereby, we propose an enhanced squared exponential kernel using Manhattan distance that is more effective at preserving the meaningfulness of proximity measures and preferred to be used in the GP model for high-dimensional cases. The experiments show that the proposed approach has obtained a superior performance in high-dimensional problems. Based on the analysis and experimental results of similarity metrics, a guide to choosing the desirable similarity measures which result in the most accurate and efficient results for the Kriging model with respect to different sample sizes and dimension levels is provided in this paper.

Minimum variance lower bound estimation with Gaussian Process models

2017 ◽  
Vol 40 (6) ◽  
pp. 1799-1807 ◽  
Author(s):  
Mehdi Ghasemi Naraghi ◽  
Yousef Alipouri
Keyword(s):  
Lower Bound ◽  
Gaussian Process ◽  
Real Life ◽  
Predictive Distribution ◽  
Minimum Variance ◽  
Process Models ◽  
Estimation Uncertainty ◽  
Gaussian Process Models ◽  
Gp Model ◽  
Bound Estimation

In this paper, we utilize the probability density function of the data to estimate the minimum variance lower bound (MVLB) of a nonlinear system. For this purpose, the Gaussian Process (GP) model has been used. With this model, given a new input and based on past observations, we naturally obtained the variance of the predictive distribution of the future output, which enabled us to estimate MVLB as well as estimation uncertainty. Also, an advantage of the proposed method over others is its ability to estimate MVLB recursively. The application of this method to the real-life dynamic process (experimental four-tank process) indicates that this approach gives very credible estimates of the MVLB.

scholarly journals GAUSSIAN PROCESS MODELS FOR MORTALITY RATES AND IMPROVEMENT FACTORS

2018 ◽  
Vol 48 (3) ◽  
pp. 1307-1347 ◽  
Author(s):  
Mike Ludkovski ◽  
Jimmy Risk ◽  
Howard Zail
Keyword(s):  
Gaussian Process ◽  
Model Performance ◽  
Mortality Rates ◽  
Process Models ◽  
Model Parameters ◽  
General Decline ◽  
Out Of Sample ◽  
Data Driven Approach ◽  
Mortality Improvement ◽  
Gp Model

AbstractWe develop a Gaussian process (GP) framework for modeling mortality rates and mortality improvement factors. GP regression is a nonparametric, data-driven approach for determining the spatial dependence in mortality rates and jointly smoothing raw rates across dimensions, such as calendar year and age. The GP model quantifies uncertainty associated with smoothed historical experience and generates full stochastic trajectories for out-of-sample forecasts. Our framework is well suited for updating projections when newly available data arrives, and for dealing with “edge” issues where credibility is lower. We present a detailed analysis of GP model performance for US mortality experience based on the CDC (Center for Disease Control) datasets. We investigate the interaction between mean and residual modeling, Bayesian and non-Bayesian GP methodologies, accuracy of in-sample and out-of-sample forecasting, and stability of model parameters. We also document the general decline, along with strong age-dependency, in mortality improvement factors over the past few years, contrasting our findings with the Society of Actuaries (SOA) MP-2014 and -2015 models that do not fully reflect these recent trends.

Adaptive Critic Design with Local Gaussian Process Models

2016 ◽  
Vol 20 (7) ◽  
pp. 1135-1140
Author(s):  
Wei Wang ◽  
◽  
Xin Chen ◽  
Jianxin He
Keyword(s):  
Gaussian Process ◽  
Adaptive Dynamic Programming ◽  
Process Models ◽  
Gaussian Kernel ◽  
Value Functions ◽  
Value Iteration ◽  
Two Phase ◽  
State Action ◽  
Adaptive Critic Design ◽  
Gp Model

In this paper, local Gaussian process (GP) approximation is introduced to build the critic network of adaptive dynamic programming (ADP). The sample data are partitioned into local regions, and for each region, an individual GP model is utilized. The nearest local model is used to predict a given state-action point. With the two-phase value iteration method for a Gaussian-kernel (GK)-based critic network which realizes the update of the hyper-parameters and value functions simultaneously, fast value function approximation can be achieved. Combining this critic network with an actor network, we present a local GK-based ADP approach. Simulations were carried out to demonstrate the feasibility of the proposed approach.

Gaussian Process-Driven, Nested Experimental Co-Design: Theoretical Framework and Application to an Airborne Wind Energy System

2020 ◽  
Vol 143 (5) ◽  
Author(s):  
Joe Deese ◽  
Peter Tkacik ◽  
Chris Vermillion
Keyword(s):  
Wind Energy ◽  
Gaussian Process ◽  
Design Space ◽  
Energy System ◽  
Bayesian Optimization ◽  
Plant Design ◽  
Original Design ◽  
Space Reduction ◽  
And Control ◽  
Gp Model

Abstract This paper presents and experimentally evaluates a nested combined plant and controller optimization (co-design) strategy that is applicable to complex systems that require extensive simulations or experiments to evaluate performance. The proposed implementation leverages principles from Gaussian process (GP) modeling to simultaneously characterize performance and uncertainty over the design space within each loop of the co-design framework. Specifically, the outer loop uses a GP model and batch Bayesian optimization to generate a batch of candidate plant designs. The inner loop utilizes recursive GP modeling and a statistically driven adaptation procedure to optimize control parameters for each candidate plant design in real-time, during each experiment. The characterizations of uncertainty made available through the GP models are used to reduce both the plant and control parameter design space as the process proceeds, and the optimization process is terminated once sufficient design space reduction has been achieved. The process is validated in this work on a lab-scale experimental platform for characterizing the flight dynamics and control of an airborne wind energy (AWE) system. The proposed co-design process converges to a design space that is less than 8% of the original design space and results in more than a 50% increase in performance.

scholarly journals Sparse Gaussian Process Emulators for Surrogate Design Modelling

2018 ◽  
Vol 885 ◽  
pp. 18-31 ◽  
Author(s):  
Paul Gardner ◽  
Timothy J. Rogers ◽  
Charles Lord ◽  
Rob J. Barthorpe
Keyword(s):  
Gaussian Process ◽  
Structural Dynamics ◽  
Design Space ◽  
Design Stage ◽  
Probabilistic Framework ◽  
Computationally Efficient ◽  
Early Design ◽  
Early Design Stage ◽  
Sparse Approximations ◽  
Data Points

Efficient surrogate modelling of computer models (herein defined as simulators) becomes of increasing importance as more complex simulators and non-deterministic methods, such as Monte Carlo simulations, are utilised. This is especially true in large multidimensional design spaces. In order for these technologies to be feasible in an early design stage context, the surrogate model (oremulator) must create an accurate prediction of the simulator in the proposed design space. Gaussian Processes (GPs) are a powerful non-parametric Bayesian approach that can be used as emulators. The probabilistic framework means that predictive distributions are inferred, providing an understanding of the uncertainty introduced by replacing the simulator with an emulator, known as code uncertainty. An issue with GPs is that they have a computational complexity of O(N3) (where N is the number of data points), which can be reduced to O(NM2) by using various sparse approximations, calculated from a subset of inducing points (where M is the number of inducing points). This paper explores the use of sparse Gaussian process emulators as a computationally efficient method for creating surrogate models of structural dynamics simulators. Discussions on the performance of these methods are presented along with comments regarding key applications to the early design stage.

Sign in / Sign up

Export Citation Format

Share Document