Cited by 880Gaussian Processes and Bayesian InferenceStatistical Methods and Bayesian InferenceBayesian Methods and Mixture Models

Variational free energy and the Laplace approximation

Friston, Karl, Mattout, Jérémie, Trujillo-Barreto, Nelson, Ashburner, John, Penny, Will·2007·NeuroImage

Abstract

This note derives the variational free energy under the Laplace approximation, with a focus on accounting for additional model complexity induced by increasing the number of model parameters. This is relevant when using the free energy as an approximation to the log-evidence in Bayesian model averaging and selection. By setting restricted maximum likelihood (ReML) in the larger context of variational learning and expectation maximisation (EM), we show how the ReML objective function can be adjusted to provide an approximation to the log-evidence for a particular model. This means ReML can be used for model selection, specifically to select or compare models with different covariance components. This is useful in the context of hierarchical models because it enables a principled selection of priors that, under simple hyperpriors, can be used for automatic model selection and relevance determination (ARD). Deriving the ReML objective function, from basic variational principles, discloses the simple relationships among Variational Bayes, EM and ReML. Furthermore, we show that EM is formally identical to a full variational treatment when the precisions are linear in the hyperparameters. Finally, we also consider, briefly, dynamic models and how these inform the regularisation of free energy ascent schemes, like EM and ReML.

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2006.08.035

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The ReML objective function and the variational energy are formally identical under this model.

    “It is interesting to note that the ReML objective function and the variational energy are formally identical under this model [15],[18].”

    From Hierarchical Models in the Brain (2008) · cites this paper as [15]

  • Static cases of the fixed-form scheme reduce to expectation maximisation (EM) and restricted maximum likelihood (ReML) for linear models.

    “We have discussed static cases of this fixed-form scheme previously and how it reduces to expectation maximisation (EM; [17]) and restricted maximum likelihood (ReML; [18]) for linear models [15].”

    From Hierarchical Models in the Brain (2008) · cites this paper as [15]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Friston, K., Mattout, J., Trujillo-Barreto, N., Ashburner, J., & Penny, W.
Affiliations
UCL Queen Square Institute of Neurology (GB)
Corresponding author
Friston, Karl
Venue
NeuroImage
Publisher
Elsevier BV
Published
2007-01
Volume
34
Issue
1
Pages
220-234
ISSN
1053-8119 (ISSN-L)
OpenAlex
W2113257799
Cited by
880
References
5
Citations by year
2026: 41 · 2025: 46 · 2024: 41 · 2023: 57 · 2022: 64 · 2021: 90 · 2020: 73 · 2019: 50 · 2018: 46 · 2017: 41 · 2016: 45 · 2015: 51 · 2014: 49 · 2013: 26 · 2012: 34
FWCI
27.35
Type
journal article
Language
en
Open access
closed
Locations
  • NeuroImage (publishedVersion)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • Research Explorer (The University of Manchester) (submittedVersion)
Funders
Wellcome Trust
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Methods and Mixture Models
Keywords
variational free energy, Laplace approximation, restricted maximum likelihood, variational inference, expectation-maximization algorithm, Bayesian model selection, Bayesian model averaging, hierarchical models, hyperprior, dynamic model, model complexity
MeSH terms
Bayes Theorem, Physics, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review