The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Graphical Models, Exponential Families, and Variational Inference
Abstract
The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances — including the key problems of computing marginals and modes of probability distributions — are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide variety of algorithms — among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations — can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Evolutionary trees from DNA sequences: A maximum likelihood approach.
Felsenstein, J. (1981) · Journal of Molecular Evolution · doi:10.1007/bf01734359 · Cited by 14,571
Information theory and statistical mechanics
Jaynes, E. T. (1957) · Physical Review · doi:10.1103/physrev.106.620 · Cited by 12,931
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Sampling-based approaches to calculating marginal densities
Gelfand, A. E., & Smith, A. F. M. (1990) · Journal of the American Statistical Association · doi:10.1080/01621459.1990.10476213 · Cited by 6,606
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486
Vaart, A. W. V. D. (1998) · doi:10.1017/cbo9780511802256 · Cited by 4,159
Beitrag zur Theorie des Ferromagnetismus
Ising, E. (1925) · Zeitschrift für Physik · doi:10.1007/bf02980577 · Cited by 3,871
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Self-citation
Cited by
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609
Sparse and Compositionally Robust Inference of Microbial Ecological Networks
Kurtz, Z. D., Müller, C. L., Miraldi, E. R., Littman, D. R., Blaser, M. J., & Bonneau, R. A. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004226 · Cited by 1,799
Discrete Signal Processing on Graphs
Sandryhaila, A., & Moura, J. M. F. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2238935 · Cited by 1,531
A simple new approach to variable selection in regression, with application to genetic fine mapping
Wang, G., Sarkar, A., Carbonetto, P., & Stephens, M. (2020) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12388 · Cited by 1,121
A Survey of Statistical Network Models
Goldenberg, A., Zheng, A. X., Fienberg, S. E., & Airoldi, E. M. (2010) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000005 · Cited by 819
Improved contact prediction in proteins: using pseudolikelihoods to infer Potts models
Ekeberg, M., Lövkvist, C., Lan, Y., Weigt, M., & Aurell, E. (2013) · Physical Review E · doi:10.1103/physreve.87.012707 · Cited by 661
Deep generative models of genetic variation capture the effects of mutations
Riesselman, A. J., Ingraham, J. B., & Marks, D. S. (2018) · Nature Methods · doi:10.1038/s41592-018-0138-4 · Cited by 601
Inferring Social Status and Rich Club Effects in Enterprise Communication Networks
Dong, Y., Tang, J., Chawla, N. V., Lou, T., Yang, Y., & Wang, B. (2015) · PLOS ONE · doi:10.1371/journal.pone.0119446 · Cited by 438
Truth inference in crowdsourcing: is the problem solved?
Zheng, Y., Li, G., Li, Y., Shan, C., & Cheng, R. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3055540.3055547 · Cited by 421
Message Passing Algorithms for Scalable Multitarget Tracking
Meyer, F., Kropfreiter, T., Williams, J. L., Lau, R., Hlawatsch, F., Braca, P., & Win, M. Z. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2789427 · Cited by 355
A Scalable Algorithm for Tracking an Unknown Number of Targets Using Multiple Sensors
Meyer, F., Braca, P., Willett, P., & Hlawatsch, F. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2688966 · Cited by 230
Learning from crowdsourced labeled data: a survey
Zhang, J., Wu, X., & Sheng, V. S. (2016) · Artificial Intelligence Review · doi:10.1007/s10462-016-9491-9 · Cited by 203
A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes
Mou, L., Hua, Y., & Zhu, X. X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01270 · Cited by 183
Gaussian Conditional Random Field Network for Semantic Segmentation
Vemulapalli, R., Tuzel, O., Liu, M.-Y., & Chellappa, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.351 · Cited by 150
Bayesian Recurrent Neural Networks
Fortunato, M., Blundell, C., & Vinyals, O. (2017) · arXiv · doi:10.48550/arxiv.1704.02798 · Cited by 145
Sun, S., & Xu, X. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2093575 · Cited by 137
A Theory of Usable Information Under Computational Constraints
Xu, Y., Zhao, S., Song, J., Stewart, R., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2002.10689 · Cited by 142
Neuronal message passing using mean-field, bethe, and marginal approximations
Parr, T., Markovic, D., Kiebel, S. J., & Friston, K. J. (2019) · Scientific Reports · doi:10.1038/s41598-018-38246-3 · Cited by 128
Perotte, A., Ranganath, R., Hirsch, J. S., Blei, D., & Elhadad, N. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv024 · Cited by 128
Deciphering protein evolution and fitness landscapes with latent space models
Ding, X., Zou, Z., & Brooks III, C. L. (2019) · Nature Communications · doi:10.1038/s41467-019-13633-0 · Cited by 106
Rapid identification of slow healing wounds
Jung, K., Covington, S., Sen, C. K., Januszyk, M., Kirsner, R. S., Gurtner, G. C., & Shah, N. H. (2016) · Wound Repair and Regeneration · doi:10.1111/wrr.12384 · Cited by 100
Sensitivity to pain expectations: A Bayesian model of individual differences
Hoskin, R., Berzuini, C., Acosta-Kane, D., El-Deredy, W., Guo, H., & Talmi, D. (2019) · Cognition · doi:10.1016/j.cognition.2018.08.022 · Cited by 87
Libbrecht, M. W., Ay, F., Hoffman, M. M., Gilbert, D. M., Bilmes, J. A., & Noble, W. S. (2015) · Genome Research · doi:10.1101/gr.184341.114 · Cited by 84
Bayesian mechanics for stationary processes
Da Costa, L., Friston, K., Heins, C., & Pavliotis, G. A. (2021) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0518 · Cited by 63
Types of approximation for probabilistic cognition: Sampling and variational.
Sanborn, A. N. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2015.06.008 · Cited by 45
Skill Induction and Planning with Latent Language
Sharma, P., Torralba, A., & Andreas, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.120 · Cited by 39
Neural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38
Visual Semantic Information Pursuit: A Survey
Liu, D., Bober, M., & Kittler, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2950025 · Cited by 39
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
Introducing a science interest network model to reveal country differences.
Sachisthal, M. S. M., Jansen, B. R. J., Peetsma, T. T. D., Dalege, J., van der Maas, H. L. J., & Raijmakers, M. E. J. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000327 · Cited by 32
Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022) · arXiv · doi:10.48550/arxiv.2202.09467 · Cited by 31
Item Response Theory: A Statistical Framework for Educational and Psychological Measurement
Chen, Y., Li, X., Liu, J., & Ying, Z. (2025) · Statistical Science · doi:10.1214/23-sts896 · Cited by 30
Accurate and robust protein sequence design with CarbonDesign
Ren, M., Yu, C., Bu, D., & Zhang, H. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00838-2 · Cited by 30
Tensor networks contraction and the belief propagation algorithm
Alkabetz, R., & Arad, I. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.023073 · Cited by 27
Mohammed, S., & Tashev, I. (2017) · 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN) · doi:10.1109/bsn.2017.7936037 · Cited by 29
Partially Generative Neural Networks for Gang Crime Classification with Partial Information
Seo, S., Chan, H., Brantingham, P. J., Leap, J., Vayanos, P., Tambe, M., & Liu, Y. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278758 · Cited by 17
On the minimal theory of consciousness implicit in active inference
Whyte, C. J., Corcoran, A. W., Robinson, J., Smith, R., Moran, R. J., Parr, T., Friston, K. J., Seth, A. K., & Hohwy, J. (2026) · Physics of Life Reviews · doi:10.1016/j.plrev.2025.11.002 · Cited by 15
Model Refinement Learning and an Example on Channel Estimation With Universal Noise Model
Wang, H.-M., Jiang, J.-C., & Wang, Y.-N. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036964 · Cited by 14
Towards causal algorithmic recourse
Karimi, A.-H., von Kügelgen, J., Schölkopf, B., & Valera, I. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_8 · Cited by 11
Cutting Edge: Soft Correspondences in Multimodal Scene Parsing
Namin, S. T., Najafi, M., Salzmann, M., & Petersson, L. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.141 · Cited by 9
Metadata
- Authors
- Wainwright, M. J., & Jordan, M. I.
- Venue
- Foundations and Trends® in Machine Learning
- Publisher
- Emerald
- Published
- 2008-12-18
- Volume
- 1
- Issue
- 1-2
- Pages
- 1-305
- ISSN
- 1935-8237 (ISSN-L)
- 1935-8245
- OpenAlex
- W4293052541
- Cited by
- 1,820
- References
- 13
- FWCI
- 62.01
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
