scholarly journals Weighted approximate Bayesian computation via Sanov’s theorem

2021 ◽  
Author(s):  
Cecilia Viscardi ◽  
Michele Boreale ◽  
Fabio Corradi
Keyword(s):  
Large Deviations ◽  
Posterior Distribution ◽  
Approximate Bayesian Computation ◽  
Bayesian Computation ◽  
Information Theoretic ◽  
Discrete Random Variables ◽  
Positive Weights ◽  
Approximate Bayesian ◽  
Information Theoretic Method ◽  
Computational Resources

AbstractWe consider the problem of sample degeneracy in Approximate Bayesian Computation. It arises when proposed values of the parameters, once given as input to the generative model, rarely lead to simulations resembling the observed data and are hence discarded. Such “poor” parameter proposals do not contribute at all to the representation of the parameter’s posterior distribution. This leads to a very large number of required simulations and/or a waste of computational resources, as well as to distortions in the computed posterior distribution. To mitigate this problem, we propose an algorithm, referred to as the Large Deviations Weighted Approximate Bayesian Computation algorithm, where, via Sanov’s Theorem, strictly positive weights are computed for all proposed parameters, thus avoiding the rejection step altogether. In order to derive a computable asymptotic approximation from Sanov’s result, we adopt the information theoretic “method of types” formulation of the method of Large Deviations, thus restricting our attention to models for i.i.d. discrete random variables. Finally, we experimentally evaluate our method through a proof-of-concept implementation.

scholarly journals AKL-ABC: An Automatic Approximate Bayesian Computation Approach Based on Kernel Learning

Entropy ◽  
2019 ◽  
Vol 21 (10) ◽  
pp. 932
Author(s):  
Wilson González-Vanegas ◽  
Andrés Álvarez-Meza ◽  
José Hernández-Muriel ◽  
Álvaro Orozco-Gutiérrez
Keyword(s):  
Approximate Bayesian Computation ◽  
Kernel Learning ◽  
Bayesian Computation ◽  
Learning Stage ◽  
Vast Number ◽  
Information Theoretic ◽  
Information Theoretic Learning ◽  
Parameters Selection ◽  
Real World Datasets ◽  
Approximate Bayesian

Bayesian statistical inference under unknown or hard to asses likelihood functions is a very challenging task. Currently, approximate Bayesian computation (ABC) techniques have emerged as a widely used set of likelihood-free methods. A vast number of ABC-based approaches have appeared in the literature; however, they all share a hard dependence on free parameters selection, demanding expensive tuning procedures. In this paper, we introduce an automatic kernel learning-based ABC approach, termed AKL-ABC, to automatically compute posterior estimations from a weighting-based inference. To reach this goal, we propose a kernel learning stage to code similarities between simulation and parameter spaces using a centered kernel alignment (CKA) that is automated via an Information theoretic learning approach. Besides, a local neighborhood selection (LNS) algorithm is used to highlight local dependencies over simulations relying on graph theory. Attained results on synthetic and real-world datasets show our approach is a quite competitive method compared to other non-automatic state-of-the-art ABC techniques.

scholarly journals Component-wise Approximate Bayesian Computation via Gibbs-like steps

Biometrika ◽  
2020 ◽  
Author(s):  
Grégoire Clarté ◽  
Christian P Robert ◽  
Robin J Ryder ◽  
Julien Stoehr
Keyword(s):  
Markov Chain ◽  
Posterior Distribution ◽  
Approximate Bayesian Computation ◽  
Generative Models ◽  
Bayesian Computation ◽  
Summary Statistics ◽  
Posterior Distributions ◽  
Reduced Dimensions ◽  
Standard Solution ◽  
Approximate Bayesian

Abstract Approximate Bayesian computation methods are useful for generative models with intractable likelihoods. These methods are however sensitive to the dimension of the parameter space, requiring exponentially increasing resources as this dimension grows. To tackle this difficulty, we explore a Gibbs version of the Approximate Bayesian computation approach that runs component-wise approximate Bayesian computation steps aimed at the corresponding conditional posterior distributions, and based on summary statistics of reduced dimensions. While lacking the standard justifications for the Gibbs sampler, the resulting Markov chain is shown to converge in distribution under some partial independence conditions. The associated stationary distribution can further be shown to be close to the true posterior distribution and some hierarchical versions of the proposed mechanism enjoy a closed form limiting distribution. Experiments also demonstrate the gain in efficiency brought by the Gibbs version over the standard solution.

scholarly journals Automatic Tolerance Selection for Approximate Bayesian Computation

2021 ◽  
Author(s):  
George Karabatsos
Keyword(s):  
Posterior Distribution ◽  
Approximate Bayesian Computation ◽  
Parametric Bootstrap ◽  
Bayesian Computation ◽  
Summary Statistics ◽  
Tolerance Level ◽  
Intractable Likelihood ◽  
Node Network ◽  
Approximate Bayesian ◽  
Pseudo Data

Abstract Approximate Bayesian Computation (ABC) can provide inferences from the (approximate) posterior distribution based on intractable likelihoods. The quality of ABC inferences relies on the choice of tolerance for the distance between the observed data summary statistics, and the pseudo-data summary statistics simulated from the likelihood, used within the context of an algorithm which samples from the approximate posterior. However, the ABC literature does not provide an automatic method to select the best tolerance level for the given dataset at hand, and in ABC practice finding the best tolerance level can be time consuming. This note introduces a fast automatic estimator of the tolerance, based on the parametric bootstrap. After the tolerance estimate is calculated, it can then be input into any suitable importance sampling or MCMC algorithm to approximate from the target approximate posterior distribution. This tolerance estimator is illustrated through ABC analyses of simulated and real datasets involving several intractable likelihood models. This includes the analysis of a real 23,000-node network dataset involving stochastic search model selection.

scholarly journals Selecting Summary Statistics in Approximate Bayesian Computation for Calibrating Stochastic Models

2013 ◽  
Vol 2013 ◽  
pp. 1-10 ◽  
Author(s):  
Tom Burr ◽  
Alexei Skurikhin
Keyword(s):  
Posterior Distribution ◽  
Approximate Bayesian Computation ◽  
Measurement Data ◽  
Real Data ◽  
Model Parameters ◽  
Bayesian Computation ◽  
Summary Statistics ◽  
Full Data ◽  
User Requirement ◽  
Approximate Bayesian

Approximate Bayesian computation (ABC) is an approach for using measurement data to calibrate stochastic computer models, which are common in biology applications. ABC is becoming the “go-to” option when the data and/or parameter dimension is large because it relies on user-chosen summary statistics rather than the full data and is therefore computationally feasible. One technical challenge with ABC is that the quality of the approximation to the posterior distribution of model parameters depends on the user-chosen summary statistics. In this paper, the user requirement to choose effective summary statistics in order to accurately estimate the posterior distribution of model parameters is investigated and illustrated by example, using a model and corresponding real data of mitochondrial DNA population dynamics. We show that for some choices of summary statistics, the posterior distribution of model parameters is closely approximated and for other choices of summary statistics, the posterior distribution is not closely approximated. A strategy to choose effective summary statistics is suggested in cases where the stochastic computer model can be run at many trial parameter settings, as in the example.

Flow parameter estimation using laser absorption spectroscopy and approximate Bayesian computation

2021 ◽  
Vol 62 (2) ◽  
Author(s):  
Jason D. Christopher ◽  
Olga A. Doronina ◽  
Dan Petrykowski ◽  
Torrey R. S. Hayden ◽  
Caelan Lapointe ◽  
...  
Keyword(s):  
Parameter Estimation ◽  
Absorption Spectroscopy ◽  
Approximate Bayesian Computation ◽  
Flow Parameter ◽  
Bayesian Computation ◽  
Laser Absorption Spectroscopy ◽  
Laser Absorption ◽  
Approximate Bayesian

scholarly journals Approximate Bayesian Computation for Discrete Spaces

Entropy ◽  
2021 ◽  
Vol 23 (3) ◽  
pp. 312
Author(s):  
Ilze A. Auzina ◽  
Jakub M. Tomczak
Keyword(s):  
Approximate Bayesian Computation ◽  
Likelihood Function ◽  
Real Life ◽  
Random Variables ◽  
Bayesian Computation ◽  
Inference Problem ◽  
Markov Kernel ◽  
Inference Problems ◽  
Binary Neural Network ◽  
Approximate Bayesian

Many real-life processes are black-box problems, i.e., the internal workings are inaccessible or a closed-form mathematical expression of the likelihood function cannot be defined. For continuous random variables, likelihood-free inference problems can be solved via Approximate Bayesian Computation (ABC). However, an optimal alternative for discrete random variables is yet to be formulated. Here, we aim to fill this research gap. We propose an adjusted population-based MCMC ABC method by re-defining the standard ABC parameters to discrete ones and by introducing a novel Markov kernel that is inspired by differential evolution. We first assess the proposed Markov kernel on a likelihood-based inference problem, namely discovering the underlying diseases based on a QMR-DTnetwork and, subsequently, the entire method on three likelihood-free inference problems: (i) the QMR-DT network with the unknown likelihood function, (ii) the learning binary neural network, and (iii) neural architecture search. The obtained results indicate the high potential of the proposed framework and the superiority of the new Markov kernel.

Sign in / Sign up

Export Citation Format

Share Document