The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms
Abstract
Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain region-based free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a "valid" or "maxent-normal" approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the "Bethe method", the "junction graph method", the "cluster variation method", and the "region graph method". Finally, we explain how to tell whether a region-based approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 3×
“We will work out a GBP algorithm making no assumptions about the actual forms of the functions, but we note that this particular factor graph can be used to represent the probability distribution that occurs when decoding a block error-correcting code [21].”
BackgroundMethodInfluentialAn Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Cited by
Collective Classification in Network Data
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., & Eliassi‐Rad, T. (2008) · AI Magazine · doi:10.1609/aimag.v29i3.2157 · Cited by 3,293
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Deformable Model Fitting by Regularized Landmark Mean-Shift
Saragih, J. M., Lucey, S., & Cohn, J. F. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0380-4 · Cited by 787
Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation
Tu, Z., & Bai, X. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.186 · Cited by 578
The graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
Hershey, J. R., Roux, J. L., & Weninger, F. (2014) · arXiv · doi:10.48550/arxiv.1409.2574 · Cited by 412
On the quantitative analysis of deep belief networks
Salakhutdinov, R., & Murray, I. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390266 · Cited by 411
Free-Energy Minimization and the Dark-Room Problem
Friston, K., Thornton, C., & Clark, A. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00130 · Cited by 367
Message Passing Algorithms for Scalable Multitarget Tracking
Meyer, F., Kropfreiter, T., Williams, J. L., Lau, R., Hlawatsch, F., Braca, P., & Win, M. Z. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2789427 · Cited by 355
Generalised free energy and active inference
Parr, T., & Friston, K. J. (2019) · Biological Cybernetics · doi:10.1007/s00422-019-00805-w · Cited by 236
Active inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226
“…model is formulated, evidence for a plausible but distinct implementations of active inference would need to be compared, which come from various possible approximations to the free energy [84, 85, 117], each of which yields different belief updates and simulated electrophysiological responses.”
BackgroundMethodSentience and the Origins of Consciousness: From Cartesian Duality to Markovian Monism
Friston, K. J., Wiese, W., & Hobson, J. A. (2020) · Entropy · doi:10.3390/e22050516 · Cited by 186
The free energy principle made simpler but not too simple
Friston, K., Da Costa, L., Sajid, N., Heins, C., Ueltzhöffer, K., Pavliotis, G. A., & Parr, T. (2023) · Physics Reports · doi:10.1016/j.physrep.2023.07.001 · Cited by 163
“…uses a mean-field approximation, in which the joint distribution over current and future states factorises into marginal distributions at each point in time [this approximation can be finessed by conditioning on previous states, leading to a different (Bethe) variational free energy [160,161]].”
Neuronal message passing using mean-field, bethe, and marginal approximations
Parr, T., Markovic, D., Kiebel, S. J., & Friston, K. J. (2019) · Scientific Reports · doi:10.1038/s41598-018-38246-3 · Cited by 128
Training an Active Random Field for Real-Time Image Denoising
Barbu, A. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2028254 · Cited by 105
Fault-tolerant renormalization group decoder for Abelian topological codes
Duclos-Cianci, G., & Poulin, D. (2014) · Quantum Information and Computation · doi:10.26421/qic14.9-10-1 · Cited by 52
Generative models for active vision
Parr, T., Sajid, N., Da Costa, L., Mirza, M. B., & Friston, K. J. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.651432 · Cited by 46
Neural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38
Metadata
- Authors
- Yedidia, J. S., Freeman, W. T., & Weiss, Y.
- Venue
- IEEE Transactions on Information Theory
- Published
- 2005-07
- Volume
- 51
- Issue
- 7
- Pages
- 2282-2312
- ISSN
- 0018-9448 (ISSN-L)
- OpenAlex
- W2169415915
- Cited by
- 1,617
- References
- 3
- FWCI
- 104.09
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
