scholarly journals Iterative construction of Gaussian process surrogate models for Bayesian inference

2020 ◽  
Vol 207 ◽  
pp. 55-72
Author(s):  
Leen Alawieh ◽  
Jonathan Goodman ◽  
John B. Bell
Keyword(s):  
Bayesian Inference ◽  
Gaussian Process ◽  
Surrogate Models ◽  
Iterative Construction

scholarly journals Adaptive Gaussian Process Approximation for Bayesian Inference with Expensive Likelihood Functions

2018 ◽  
Vol 30 (11) ◽  
pp. 3072-3094 ◽  
Author(s):  
Hongqiao Wang ◽  
Jinglai Li
Keyword(s):  
Bayesian Inference ◽  
Gaussian Process ◽  
Adaptive Algorithm ◽  
Posterior Density ◽  
Unknown Parameters ◽  
Likelihood Functions ◽  
Inference Problems ◽  
Active Learning Method ◽  
Computationally Intensive ◽  
Process Approximation

We consider Bayesian inference problems with computationally intensive likelihood functions. We propose a Gaussian process (GP)–based method to approximate the joint distribution of the unknown parameters and the data, built on recent work (Kandasamy, Schneider, & Póczos, 2015 ). In particular, we write the joint density approximately as a product of an approximate posterior density and an exponentiated GP surrogate. We then provide an adaptive algorithm to construct such an approximation, where an active learning method is used to choose the design points. With numerical examples, we illustrate that the proposed method has competitive performance against existing approaches for Bayesian computation.

scholarly journals Bayesian3 Active Learning for the Gaussian Process Emulator Using Information Theory

Entropy ◽  
2020 ◽  
Vol 22 (8) ◽  
pp. 890
Author(s):  
Sergey Oladyshkin ◽  
Farid Mohammadi ◽  
Ilja Kroeker ◽  
Wolfgang Nowak
Keyword(s):  
Bayesian Inference ◽  
Active Learning ◽  
Gaussian Process ◽  
Relative Entropy ◽  
Bayesian Model ◽  
Information Gain ◽  
Measurement Data ◽  
Superior Performance ◽  
Evidence Based ◽  
Model Evidence

Gaussian process emulators (GPE) are a machine learning approach that replicates computational demanding models using training runs of that model. Constructing such a surrogate is very challenging and, in the context of Bayesian inference, the training runs should be well invested. The current paper offers a fully Bayesian view on GPEs for Bayesian inference accompanied by Bayesian active learning (BAL). We introduce three BAL strategies that adaptively identify training sets for the GPE using information-theoretic arguments. The first strategy relies on Bayesian model evidence that indicates the GPE’s quality of matching the measurement data, the second strategy is based on relative entropy that indicates the relative information gain for the GPE, and the third is founded on information entropy that indicates the missing information in the GPE. We illustrate the performance of our three strategies using analytical- and carbon-dioxide benchmarks. The paper shows evidence of convergence against a reference solution and demonstrates quantification of post-calibration uncertainty by comparing the introduced three strategies. We conclude that Bayesian model evidence-based and relative entropy-based strategies outperform the entropy-based strategy because the latter can be misleading during the BAL. The relative entropy-based strategy demonstrates superior performance to the Bayesian model evidence-based strategy.

A Comparison of Numerical Optimizers in Developing High Dimensional Surrogate Models

2019 ◽  
Author(s):  
Yanwen Xu ◽  
Pingfeng Wang
Keyword(s):  
Gaussian Process ◽  
Process Model ◽  
Likelihood Function ◽  
Surrogate Models ◽  
Kernel Functions ◽  
Function Optimization ◽  
High Dimensional ◽  
Model Parameters ◽  
Gradient Based ◽  
Gp Model

Abstract The Gaussian Process (GP) model has become one of the most popular methods to develop computationally efficient surrogate models in many engineering design applications, including simulation-based design optimization and uncertainty analysis. When more observations are used for high dimensional problems, estimating the best model parameters of Gaussian Process model is still an essential yet challenging task due to considerable computation cost. One of the most commonly used methods to estimate model parameters is Maximum Likelihood Estimation (MLE). A common bottleneck arising in MLE is computing a log determinant and inverse over a large positive definite matrix. In this paper, a comparison of five commonly used gradient based and non-gradient based optimizers including Sequential Quadratic Programming (SQP), Quasi-Newton method, Interior Point method, Trust Region method and Pattern Line Search for likelihood function optimization of high dimension GP surrogate modeling problem is conducted. The comparison has been focused on the accuracy of estimation, the efficiency of computation and robustness of the method for different types of Kernel functions.

Sign in / Sign up

Export Citation Format

Share Document