bethe free energy
Recently Published Documents


TOTAL DOCUMENTS

20
(FIVE YEARS 3)

H-INDEX

7
(FIVE YEARS 0)

Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 807
Author(s):  
İsmail Şenöz ◽  
Thijs van de Laar ◽  
Dmitry Bagaev ◽  
Bert de de Vries

Accurate evaluation of Bayesian model evidence for a given data set is a fundamental problem in model development. Since evidence evaluations are usually intractable, in practice variational free energy (VFE) minimization provides an attractive alternative, as the VFE is an upper bound on negative model log-evidence (NLE). In order to improve tractability of the VFE, it is common to manipulate the constraints in the search space for the posterior distribution of the latent variables. Unfortunately, constraint manipulation may also lead to a less accurate estimate of the NLE. Thus, constraint manipulation implies an engineering trade-off between tractability and accuracy of model evidence estimation. In this paper, we develop a unifying account of constraint manipulation for variational inference in models that can be represented by a (Forney-style) factor graph, for which we identify the Bethe Free Energy as an approximation to the VFE. We derive well-known message passing algorithms from first principles, as the result of minimizing the constrained Bethe Free Energy (BFE). The proposed method supports evaluation of the BFE in factor graphs for model scoring and development of new message passing-based inference algorithms that potentially improve evidence estimation accuracy.


Author(s):  
Yuanzhen Guo ◽  
Hao Xiong ◽  
Nicholas Ruozzi

Exact marginal inference in continuous graphical models is computationally challenging outside of a few special cases. Existing work on approximate inference has focused on approximately computing the messages as part of the loopy belief propagation algorithm either via sampling methods or moment matching relaxations. In this work, we present an alternative family of approximations that, instead of approximating the messages, approximates the beliefs in the continuous Bethe free energy using mixture distributions. We show that these types of approximations can be combined with numerical quadrature to yield algorithms with both theoretical guarantees on the quality of the approximation and significantly better practical performance in a variety of applications that are challenging for current state-of-the-art methods.


2016 ◽  
Vol 49 (4) ◽  
pp. 694-741 ◽  
Author(s):  
Victor Bapst ◽  
Amin Coja‐Oghlan

Sign in / Sign up

Export Citation Format

Share Document