High-Dimensional Bayesian Optimization of Personalized Cardiac Model Parameters via an Embedded Generative Model

Abstract Simulating the resting-state brain dynamics via mathematical whole-brain models requires an optimal selection of parameters, which determine the model’s capability to replicate empirical data. Since the parameter optimization via a grid search (GS) becomes unfeasible for high-dimensional models, we evaluate several alternative approaches to maximize the correspondence between simulated and empirical functional connectivity. A dense GS serves as a benchmark to assess the performance of four optimization schemes: Nelder-Mead Algorithm (NMA), Particle Swarm Optimization (PSO), Covariance Matrix Adaptation Evolution Strategy (CMAES) and Bayesian Optimization (BO). To compare them, we employ an ensemble of coupled phase oscillators built upon individual empirical structural connectivity of 105 healthy subjects. We determine optimal model parameters from two- and three-dimensional parameter spaces and show that the overall fitting quality of the tested methods can compete with the GS. There are, however, marked differences in the required computational resources and stability properties, which we also investigate before proposing CMAES and BO as efficient alternatives to a high-dimensional GS. For the three-dimensional case, these methods generated similar results as the GS, but within less than 6% of the computation time. Our results contribute to an efficient validation of models for personalized simulations of brain dynamics.

Download Full-text

Bayesian Optimization on Large Graphs via a Graph Convolutional Generative Model: Application in Cardiac Model Personalization

Lecture Notes in Computer Science - Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 ◽

10.1007/978-3-030-32245-8_51 ◽

2019 ◽

pp. 458-467 ◽

Cited By ~ 1

Author(s):

Jwala Dhamala ◽

Sandesh Ghimire ◽

John L. Sapp ◽

B. Milan Horáček ◽

Linwei Wang

Keyword(s):

Generative Model ◽

Bayesian Optimization ◽

Model Application ◽

Large Graphs ◽

Cardiac Model ◽

Model Personalization

Download Full-text

Gaussian Processes Proxy Model with Latent Variable Models and Variogram-Based Sensitivity Analysis for Assisted History Matching

Energies ◽

10.3390/en13174290 ◽

2020 ◽

Vol 13 (17) ◽

pp. 4290

Author(s):

Dongmei Zhang ◽

Yuyang Zhang ◽

Bohou Jiang ◽

Xinwei Jiang ◽

Zhijiang Kang

Keyword(s):

Sensitivity Analysis ◽

Gaussian Processes ◽

Latent Variable ◽

History Matching ◽

Latent Variable Models ◽

High Dimensional ◽

Model Parameters ◽

Variable Model ◽

Assisted History Matching ◽

Proxy Models

Reservoir history matching is a well-known inverse problem for production prediction where enormous uncertain reservoir parameters of a reservoir numerical model are optimized by minimizing the misfit between the simulated and history production data. Gaussian Process (GP) has shown promising performance for assisted history matching due to the efficient nonparametric and nonlinear model with few model parameters to be tuned automatically. Recently introduced Gaussian Processes proxy models and Variogram Analysis of Response Surface-based sensitivity analysis (GP-VARS) uses forward and inverse Gaussian Processes (GP) based proxy models with the VARS-based sensitivity analysis to optimize the high-dimensional reservoir parameters. However, the inverse GP solution (GPIS) in GP-VARS are unsatisfactory especially for enormous reservoir parameters where the mapping from low-dimensional misfits to high-dimensional uncertain reservoir parameters could be poorly modeled by GP. To improve the performance of GP-VARS, in this paper we propose the Gaussian Processes proxy models with Latent Variable Models and VARS-based sensitivity analysis (GPLVM-VARS) where Gaussian Processes Latent Variable Model (GPLVM)-based inverse solution (GPLVMIS) instead of GP-based GPIS is provided with the inputs and outputs of GPIS reversed. The experimental results demonstrate the effectiveness of the proposed GPLVM-VARS in terms of accuracy and complexity. The source code of the proposed GPLVM-VARS is available at https://github.com/XinweiJiang/GPLVM-VARS.

Download Full-text

Bayesian Optimization in High-Dimensional Spaces: A Brief Survey

10.1109/iisa52424.2021.9555522 ◽

2021 ◽

Author(s):

Mohit Malu ◽

Gautam Dasarathy ◽

Andreas Spanias

Keyword(s):

Bayesian Optimization ◽

High Dimensional

Download Full-text

DNILMF-LDA: Prediction of lncRNA-Disease Associations by Dual-Network Integrated Logistic Matrix Factorization and Bayesian Optimization

Genes ◽

10.3390/genes10080608 ◽

2019 ◽

Vol 10 (8) ◽

pp. 608 ◽

Cited By ~ 3

Author(s):

Yan Li ◽

Junyi Li ◽

Naizheng Bian

Keyword(s):

Matrix Factorization ◽

Disease Diagnosis ◽

Disease Association ◽

Bayesian Optimization ◽

Model Parameters ◽

Target Interaction ◽

Disease Associations ◽

Auc Value ◽

Similarity Networks ◽

Fold Cross Validation

Identifying associations between lncRNAs and diseases can help understand disease-related lncRNAs and facilitate disease diagnosis and treatment. The dual-network integrated logistic matrix factorization (DNILMF) model has been used for drug–target interaction prediction, and good results have been achieved. We firstly applied DNILMF to lncRNA–disease association prediction (DNILMF-LDA). We combined different similarity kernel matrices of lncRNAs and diseases by using nonlinear fusion to extract the most important information in fused matrices. Then, lncRNA–disease association networks and similarity networks were built simultaneously. Finally, the Gaussian process mutual information (GP-MI) algorithm of Bayesian optimization was adopted to optimize the model parameters. The 10-fold cross-validation result showed that the area under receiving operating characteristic (ROC) curve (AUC) value of DNILMF-LDA was 0.9202, and the area under precision-recall (PR) curve (AUPR) was 0.5610. Compared with LRLSLDA, SIMCLDA, BiwalkLDA, and TPGLDA, the AUC value of our method increased by 38.81%, 13.07%, 8.35%, and 6.75%, respectively. The AUPR value of our method increased by 52.66%, 40.05%, 37.01%, and 44.25%. These results indicate that DNILMF-LDA is an effective method for predicting the associations between lncRNAs and diseases.

Download Full-text

A Comparison of Numerical Optimizers in Developing High Dimensional Surrogate Models

Volume 2B: 45th Design Automation Conference ◽

10.1115/detc2019-97499 ◽

2019 ◽

Author(s):

Yanwen Xu ◽

Pingfeng Wang

Keyword(s):

Gaussian Process ◽

Process Model ◽

Likelihood Function ◽

Surrogate Models ◽

Kernel Functions ◽

Function Optimization ◽

High Dimensional ◽

Model Parameters ◽

Gradient Based ◽

Gp Model

Abstract The Gaussian Process (GP) model has become one of the most popular methods to develop computationally efficient surrogate models in many engineering design applications, including simulation-based design optimization and uncertainty analysis. When more observations are used for high dimensional problems, estimating the best model parameters of Gaussian Process model is still an essential yet challenging task due to considerable computation cost. One of the most commonly used methods to estimate model parameters is Maximum Likelihood Estimation (MLE). A common bottleneck arising in MLE is computing a log determinant and inverse over a large positive definite matrix. In this paper, a comparison of five commonly used gradient based and non-gradient based optimizers including Sequential Quadratic Programming (SQP), Quasi-Newton method, Interior Point method, Trust Region method and Pattern Line Search for likelihood function optimization of high dimension GP surrogate modeling problem is conducted. The comparison has been focused on the accuracy of estimation, the efficiency of computation and robustness of the method for different types of Kernel functions.

Download Full-text

Variance Regularization in Sequential Bayesian Optimization

Mathematics of Operations Research ◽

10.1287/moor.2019.1019 ◽

2020 ◽

Vol 45 (3) ◽

pp. 966-992

Author(s):

Michael Jong Kim

Keyword(s):

Error Bounds ◽

Broad Class ◽

Planning Horizon ◽

Bayesian Optimization ◽

Model Parameters ◽

Regularization Term ◽

Problem Class ◽

Special Cases ◽

The Impact ◽

Bayesian Dynamic Programming

Sequential Bayesian optimization constitutes an important and broad class of problems where model parameters are not known a priori but need to be learned over time using Bayesian updating. It is known that the solution to these problems can in principle be obtained by solving the Bayesian dynamic programming (BDP) equation. Although the BDP equation can be solved in certain special cases (for example, when posteriors have low-dimensional representations), solving this equation in general is computationally intractable and remains an open problem. A second unresolved issue with the BDP equation lies in its (rather generic) interpretation. Beyond the standard narrative of balancing immediate versus future costs—an interpretation common to all dynamic programs with or without learning—the BDP equation does not provide much insight into the underlying mechanism by which sequential Bayesian optimization trades off between learning (exploration) and optimization (exploitation), the distinguishing feature of this problem class. The goal of this paper is to develop good approximations (with error bounds) to the BDP equation that help address the issues of computation and interpretation. To this end, we show how the BDP equation can be represented as a tractable single-stage optimization problem that trades off between a myopic term and a “variance regularization” term that measures the total solution variability over the remaining planning horizon. Intuitively, the myopic term can be regarded as a pure exploitation objective that ignores the impact of future learning, whereas the variance regularization term captures a pure exploration objective that only puts value on solutions that resolve statistical uncertainty. We develop quantitative error bounds for this representation and prove that the error tends to zero like o(n-1) almost surely in the number of stages n, which as a corollary, establishes strong consistency of the approximate solution.

Download Full-text

Bayesian recurrent neural network as a tool for reconstructing dynamical systems from multidimensional data

10.5194/egusphere-egu2020-8787 ◽

2020 ◽

Author(s):

Alexander Feigin ◽

Aleksei Seleznev ◽

Dmitry Mukhin ◽

Andrey Gavrilov ◽

Evgeny Loskutov

Keyword(s):

Neural Network ◽

Time Series ◽

Cost Function ◽

Recurrent Neural Network ◽

Bayesian Optimization ◽

Data Driven ◽

High Dimensional ◽

Model Learning ◽

Low Dimensional ◽

The Cost

We suggest a new method for construction of data-driven dynamical models from observed multidimensional time series. The method is based on a recurrent neural network (RNN) with specific structure, which allows for the joint reconstruction of both a low-dimensional embedding for dynamical components in the data and an operator describing the low-dimensional evolution of the system. The key link of the method is a Bayesian optimization of both model structure and the hypothesis about the data generating law, which is needed for constructing the cost function for model learning. &#160;The form of the model we propose allows us to construct a stochastic dynamical system of moderate dimension that copies dynamical properties of the original high-dimensional system. An advantage of the proposed method is the data-adaptive properties of the RNN model: it is based on the adjustable nonlinear elements and has easily scalable structure. The combination of the RNN with the Bayesian optimization procedure efficiently provides the model with statistically significant nonlinearity and dimension. The method developed for the model optimization aims to detect the long-term connections between system&#8217;s states &#8211; the memory of the system: the cost-function used for model learning is constructed taking into account this factor. In particular, in the case of absence of interaction between the dynamical component and noise, the method provides unbiased reconstruction of the hidden deterministic system. In the opposite case when the noise has strong impact on the dynamics, the method yield a model in the form of a nonlinear stochastic map determining the Markovian process with memory. Bayesian approach used for selecting both the optimal model&#8217;s structure and the appropriate cost function allows to obtain the statistically significant inferences about the dynamical signal in data as well as its interaction with the noise components. Data driven model derived from the relatively short time series of the QG3 model &#8211; the high dimensional nonlinear system producing chaotic behavior &#8211; is shown be able to serve as a good simulator for the QG3 LFV components. The statistically significant recurrent states of the QG3 model, i.e. the well-known teleconnections in NH, are all reproduced by the model obtained. Moreover, statistics of the residence times of the model near these states is very close to the corresponding statistics of the original QG3 model. These results demonstrate that the method can be useful in modeling the variability of the real atmosphere.The work was supported by the Russian Science Foundation (Grant No. 19-42-04121).

Download Full-text

Efficient ensemble generation for uncertain correlated parameters in atmospheric chemical models

10.5194/gmd-2021-26 ◽

2021 ◽

Author(s):

Annika Vogel ◽

Hendrik Elbern

Keyword(s):

Forecast Model ◽

High Dimensional ◽

Biogenic Emissions ◽

Model Parameters ◽

Emission Rates ◽

Model Uncertainties ◽

Dimensional Parameter ◽

Novel Approach ◽

Correlated Parameters ◽

Ensemble Generation

Abstract. Atmospheric chemical forecasts highly rely on various model parameters, which are often insufficiently known, as emission rates and deposition velocities. However, a reliable estimation of resulting uncertainties by an ensemble of forecasts is impaired by the high-dimensionality of the system. This study presents a novel approach to efficiently perturb atmospheric-chemical model parameters according to their leading coupled uncertainties. The algorithm is based on the idea that the forecast model acts as a dynamical system inducing multi-variational correlations of model uncertainties. The specific algorithm presented in this study is designed for parameters which depend on local environmental conditions and consists of three major steps: (1) an efficient assessment of various sources of model uncertainties spanned by independent sensitivities, (2) an efficient extraction of leading coupled uncertainties using eigenmode decomposition, and (3) an efficient generation of perturbations for high-dimensional parameter fields by the Karhunen-Loéve expansion. Due to their perceived simulation challenge the method has been applied to biogenic emissions of five trace gases, considering state-dependent sensitivities to local atmospheric and terrestrial conditions. Rapidly decreasing eigenvalues state high spatial- and cross-correlations of regional biogenic emissions, which are represented by a low number of dominating components. Consequently, leading uncertainties can be covered by low number of perturbations enabling ensemble sizes of the order of 10 members. This demonstrates the suitability of the algorithm for efficient ensemble generation for high-dimensional atmospheric chemical parameters.

Download Full-text