An introduction to thermodynamic integration and application to dynamic causal models

AbstractIn generative modeling of neuroimaging data, such as dynamic causal modeling (DCM), one typically considers several alternative models, either to determine the most plausible explanation for observed data (Bayesian model selection) or to account for model uncertainty (Bayesian model averaging). Both procedures rest on estimates of the model evidence, a principled trade-off between model accuracy and complexity. In the context of DCM, the log evidence is usually approximated using variational Bayes. Although this approach is highly efficient, it makes distributional assumptions and is vulnerable to local extrema. This paper introduces the use of thermodynamic integration (TI) for Bayesian model selection and averaging in the context of DCM. TI is based on Markov chain Monte Carlo sampling which is asymptotically exact but orders of magnitude slower than variational Bayes. In this paper, we explain the theoretical foundations of TI, covering key concepts such as the free energy and its origins in statistical physics. Our aim is to convey an in-depth understanding of the method starting from its historical origin in statistical physics. In addition, we demonstrate the practical application of TI via a series of examples which serve to guide the user in applying this method. Furthermore, these examples demonstrate that, given an efficient implementation and hardware capable of parallel processing, the challenge of high computational demand can be overcome successfully. The TI implementation presented in this paper is freely available as part of the open source software TAPAS.

Download Full-text

An Introduction to Thermodynamic Integration and Application to Dynamic Causal Models

10.1101/2020.12.21.423807 ◽

2020 ◽

Author(s):

Eduardo Aponte ◽

Yu Yao ◽

Sudhir Raman ◽

Stefan Frassle ◽

Jakob Heinzle ◽

...

Keyword(s):

Model Selection ◽

Bayesian Model ◽

Statistical Physics ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Bayesian Model Selection ◽

Monte Carlo Sampling ◽

Thermodynamic Integration ◽

Causal Modeling ◽

Variational Bayes

In generative modeling of neuroimaging data, such as dynamic causal modeling (DCM), one typically considers several alternative models, either to determine the most plausible explanation for observed data (Bayesian model selection) or to account for model uncertainty (Bayesian model averaging). Both procedures rest on estimates of the model evidence, a principled trade-off between model accuracy and complexity. In the context of DCM, the log evidence is usually approximated using variational Bayes. Although this approach is highly efficient, it makes distributional assumptions and is vulnerable to local extrema. This paper introduces the use of thermodynamic integration (TI) for Bayesian model selection and averaging in the context of DCM. TI is based on Markov chain Monte Carlo sampling which is asymptotically exact but orders of magnitude slower than variational Bayes. In this paper, we explain the theoretical foundations of TI, covering key concepts such as the free energy and its origins in statistical physics. Our aim is to convey an in-depth understanding of the method starting from its historical origin in statistical physics. In addition, we demonstrate the practical application of TI via a series of examples which serve to guide the user in applying this method. Furthermore, these examples demonstrate that, given an efficient implementation and hardware capable of parallel processing, the challenge of high computational demand can be overcome successfully. The TI implementation presented in this paper is freely available as part of the open source software TAPAS.

Download Full-text

Thermodynamic integration for dynamic causal models

10.1101/471417 ◽

2018 ◽

Author(s):

Eduardo A. Aponte ◽

Sudhir Raman ◽

Stefan Frässle ◽

Jakob Heinzle ◽

Will D. Penny ◽

...

Keyword(s):

Model Selection ◽

Bayesian Model ◽

Model Comparison ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Computation Time ◽

Thermodynamic Integration ◽

Causal Modeling ◽

Single Subject ◽

Model Evidence

AbstractIn generative modeling of neuroimaging data, such as dynamic causal modeling (DCM), one typically considers several alternative models, either to determine the most plausible explanation for observed data (Bayesian model selection) or to account for model uncertainty (Bayesian model averaging). Both procedures rest on estimates of the model evidence, a principled trade-off between model accuracy and complexity. In DCM, the log evidence is usually approximated using variational Bayes (VB) under the Laplace approximation (VBL). Although this approach is highly efficient, it makes distributional assumptions and can be vulnerable to local extrema. An alternative to VBL is Markov Chain Monte Carlo (MCMC) sampling, which is asymptotically exact but orders of magnitude slower than VB. This has so far prevented its routine use for DCM.This paper makes four contributions. First, we introduce a powerful MCMC scheme – thermodynamic integration (TI) – to neuroimaging and present a derivation that establishes a theoretical link to VB. Second, this derivation is based on a tutorial-like introduction to concepts of free energy in physics and statistics. Third, we present an implementation of TI for DCM that rests on population MCMC. Fourth, using simulations and empirical functional magnetic resonance imaging (fMRI) data, we compare log evidence estimates obtained by TI, VBL, and other MCMC-based estimators (prior arithmetic mean and posterior harmonic mean). We find that model comparison based on VBL gives reliable results in most cases, justifying its use in standard DCM for fMRI. Furthermore, we demonstrate that for complex and/or nonlinear models, TI may provide more robust estimates of the log evidence. Importantly, accurate estimates of the model evidence can be obtained with TI in acceptable computation time. This paves the way for using DCM in scenarios where the robustness of single-subject inference and model selection becomes paramount, such as differential diagnosis in clinical applications.

Download Full-text

How to improve parameter estimates in GLM-based fMRI data analysis: cross-validated Bayesian model averaging

10.1101/095778 ◽

2016 ◽

Cited By ~ 2

Author(s):

Joram Soch ◽

Achim Pascal Meyer ◽

John-Dylan Haynes ◽

Carsten Allefeld

Keyword(s):

Data Analysis ◽

Model Selection ◽

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Bayesian Model Selection ◽

Fmri Data ◽

Parameter Estimates ◽

Fmri Data Analysis ◽

Level Analysis

AbstractIn functional magnetic resonance imaging (fMRI), model quality of general linear models (GLMs) for first-level analysis is rarely assessed. In recent work (Soch et al., 2016: “How to avoid mismodelling in GLM-based fMRI data analysis: cross-validated Bayesian model selection”, NeuroImage, vol. 141, pp. 469-489; DOI: 10.1016/j. neuroimage.2016.07.047), we have introduced cross-validated Bayesian model selection (cvBMS) to infer the best model for a group of subjects and use it to guide second-level analysis. While this is the optimal approach given that the same GLM has to be used for all subjects, there is a much more efficient procedure when model selection only addresses nuisance variables and regressors of interest are included in all candidate models. In this work, we propose cross-validated Bayesian model averaging (cvBMA) to improve parameter estimates for these regressors of interest by combining information from all models using their posterior probabilities. This is particularly useful as different models can lead to different conclusions regarding experimental effects and the most complex model is not necessarily the best choice. We find that cvBMS can prevent not detecting established effects and that cvBMA can be more sensitive to experimental effects than just using even the best model in each subject or the model which is best in a group of subjects.

Download Full-text

Spatially-dependent Bayesian model selection for disease mapping

Statistical Methods in Medical Research ◽

10.1177/0962280215627298 ◽

2016 ◽

Vol 27 (1) ◽

pp. 250-268 ◽

Cited By ~ 1

Author(s):

Rachel Carroll ◽

Andrew B Lawson ◽

Christel Faes ◽

Russell S Kirby ◽

Mehreteab Aregay ◽

...

Keyword(s):

Model Selection ◽

Bayesian Model ◽

Large Scale ◽

Bayesian Model Averaging ◽

Disease Mapping ◽

Model Averaging ◽

Bayesian Model Selection ◽

Small Scale ◽

Study Region ◽

Model Sets

In disease mapping where predictor effects are to be modeled, it is often the case that sets of predictors are fixed, and the aim is to choose between fixed model sets. Model selection methods, both Bayesian model selection and Bayesian model averaging, are approaches within the Bayesian paradigm for achieving this aim. In the spatial context, model selection could have a spatial component in the sense that some models may be more appropriate for certain areas of a study region than others. In this work, we examine the use of spatially referenced Bayesian model averaging and Bayesian model selection via a large-scale simulation study accompanied by a small-scale case study. Our results suggest that BMS performs well when a strong regression signature is found.

Download Full-text

Robust Bayesian Filtering Using Bayesian Model Averaging and Restricted Variational Bayes

2018 21st International Conference on Information Fusion (FUSION) ◽

10.23919/icif.2018.8455608 ◽

2018 ◽

Cited By ~ 2

Author(s):

S. S. Khalid ◽

N. U. Rehman ◽

Shafayat Abrar ◽

Lyudmila Mihaylova

Keyword(s):

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Variational Bayes ◽

Bayesian Filtering

Download Full-text

Bayesian Model Averaging to Account for Model Uncertainty in Estimates of a Vaccine's Effectiveness

10.1101/2021.05.12.21257126 ◽

2021 ◽

Author(s):

Carlos R Oliveira ◽

Eugene D Shapiro ◽

Daniel M Weinberger

Keyword(s):

Model Selection ◽

Model Uncertainty ◽

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Selection Methods ◽

Final Model ◽

Negative Case ◽

Confounder Selection ◽

Control Study

Vaccine effectiveness (VE) studies are often conducted after the introduction of new vaccines to ensure they provide protection in real-world settings. Although susceptible to confounding, the test-negative case-control study design is the most efficient method to assess VE post-licensure. Control of confounding is often needed during the analyses, which is most efficiently done through multivariable modeling. When a large number of potential confounders are being considered, it can be challenging to know which variables need to be included in the final model. This paper highlights the importance of considering model uncertainty by re-analyzing a Lyme VE study using several confounder selection methods. We propose an intuitive Bayesian Model Averaging (BMA) framework for this task and compare the performance of BMA to that of traditional single-best-model-selection methods. We demonstrate how BMA can be advantageous in situations when there is uncertainty about model selection by systematically considering alternative models and increasing transparency.

Download Full-text