scholarly journals Harnessing the Bethe free energy

2016 ◽  
Vol 49 (4) ◽  
pp. 694-741 ◽  
Author(s):  
Victor Bapst ◽  
Amin Coja‐Oghlan
Keyword(s):  
Free Energy ◽  
Bethe Free Energy

scholarly journals Variational Message Passing and Local Constraint Manipulation in Factor Graphs

Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 807
Author(s):  
İsmail Şenöz ◽  
Thijs van de Laar ◽  
Dmitry Bagaev ◽  
Bert de de Vries
Keyword(s):  
Free Energy ◽  
Latent Variables ◽  
Message Passing ◽  
Factor Graph ◽  
Estimation Accuracy ◽  
Attractive Alternative ◽  
Factor Graphs ◽  
Data Set ◽  
Bethe Free Energy ◽  
Model Evidence

Accurate evaluation of Bayesian model evidence for a given data set is a fundamental problem in model development. Since evidence evaluations are usually intractable, in practice variational free energy (VFE) minimization provides an attractive alternative, as the VFE is an upper bound on negative model log-evidence (NLE). In order to improve tractability of the VFE, it is common to manipulate the constraints in the search space for the posterior distribution of the latent variables. Unfortunately, constraint manipulation may also lead to a less accurate estimate of the NLE. Thus, constraint manipulation implies an engineering trade-off between tractability and accuracy of model evidence estimation. In this paper, we develop a unifying account of constraint manipulation for variational inference in models that can be represented by a (Forney-style) factor graph, for which we identify the Bethe Free Energy as an approximation to the VFE. We derive well-known message passing algorithms from first principles, as the result of minimizing the constrained Bethe Free Energy (BFE). The proposed method supports evaluation of the BFE in factor graphs for model scoring and development of new message passing-based inference algorithms that potentially improve evidence estimation accuracy.

Sign in / Sign up

Export Citation Format

Share Document