Cited by 1,423Statistical Methods and Bayesian InferenceBayesian Modeling and Causal InferenceFunctional Brain Connectivity Studies

Bayesian model selection for group studies

Open access
Stephan, Klaas Enno, Penny, Will D., Daunizeau, Jean, Moran, Rosalyn J., Friston, Karl J.·2009·NeuroImage

Abstract

Bayesian model selection (BMS) is a powerful method for determining the most likely among a set of competing hypotheses about the mechanisms that generated observed data. BMS has recently found widespread application in neuroimaging, particularly in the context of dynamic causal modelling (DCM). However, so far, combining BMS results from several subjects has relied on simple (fixed effects) metrics, e.g. the group Bayes factor (GBF), that do not account for group heterogeneity or outliers. In this paper, we compare the GBF with two random effects methods for BMS at the between-subject or group level. These methods provide inference on model-space using a classical and Bayesian perspective respectively. First, a classical (frequentist) approach uses the log model evidence as a subject-specific summary statistic. This enables one to use analysis of variance to test for differences in log-evidences over models, relative to inter-subject differences. We then consider the same problem in Bayesian terms and describe a novel hierarchical model, which is optimised to furnish a probability density on the models themselves. This new variational Bayes method rests on treating the model as a random variable and estimating the parameters of a Dirichlet distribution which describes the probabilities for all models considered. These probabilities then define a multinomial distribution over model space, allowing one to compute how likely it is that a specific model generated the data of a randomly chosen subject as well as the exceedance probability of one model being more likely than any other model. Using empirical and synthetic data, we show that optimising a conditional density of the model probabilities, given the log-evidences for each model over subjects, is more informative and appropriate than both the GBF and frequentist tests of the log-evidences. In particular, we found that the hierarchical Bayesian approach is considerably more robust than either of the other approaches in the presence of outliers. We expect that this new random effects method will prove useful for a wide range of group studies, not only in the context of DCM, but also for other modelling endeavours, e.g. comparing different source reconstruction methods for EEG/MEG or selecting among competing computational models of learning and decision-making.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2009.03.025

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Hidden individual differences in cognitive strategies can be detected by formulating alternative computational models and subjecting them to Bayesian model selection.

    “Such hidden individual differences can be detected by formulating alternative computational models, each reflecting different cognitive strategies, and subjecting them to Bayesian model selection (62).”
  • The GBF approach implicitly assumes that every subject uses the same model, and is therefore a Fixed Effects analysis.

    “As is made clear in [19] the GBF approach implicitly assumes that every subject uses the same model. It is therefore a Fixed Effects analysis.”

    From Comparing families of dynamic causal models (2010) · cites this paper as [19]

  • The robustness and accuracy of the variational RFX method was verified via simulations using data from synthetic populations with known frequencies of competing models.

    “The robustness and accuracy of this method was verified via simulations using data from synthetic populations with known frequencies of competing models [19].”

    From Comparing families of dynamic causal models (2010) · cites this paper as [19]

  • The use of Gibbs sampling allows relaxation of the assumption made in VB that the posterior densities over m and α factorise.

    “Additionally, the use of Gibbs sampling allows us to relax the assumption made in VB that the posterior densities over and factorise [19].”

    From Comparing families of dynamic causal models (2010) · cites this paper as [19]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J.
Affiliations
Wellcome Centre for Human Neuroimaging (GB)
Corresponding author
Stephan, Klaas Enno
Venue
NeuroImage
Publisher
Elsevier BV
Published
2009-07
Volume
46
Issue
4
Pages
1004-1017
ISSN
1053-8119 (ISSN-L)
OpenAlex
W2135173838
Cited by
1,423
References
49
Citations by year
2026: 63 · 2025: 74 · 2024: 76 · 2023: 71 · 2022: 86 · 2021: 100 · 2020: 135 · 2019: 97 · 2018: 102 · 2017: 91 · 2016: 89 · 2015: 109 · 2014: 67 · 2013: 75 · 2012: 59
FWCI
51.46
Type
journal article
Language
en
Open access
Locations
  • NeuroImage (publishedVersion)
  • PubMed (publishedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
  • PubMed Central (acceptedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • Explore Bristol Research (submittedVersion)
  • UEA Digital Repository (University of East Anglia) (submittedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
  • Zurich Open Repository and Archive (University of Zurich)
Funders
Wellcome Trust (056750, 088130)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
Keywords
Bayesian model selection, group study, dynamic causal modeling, random effects, Bayesian hierarchical model, variational inference, outliers, exceedance probability, Dirichlet distribution, model comparison
MeSH terms
Algorithms, Bayes Theorem, Brain, Cognition, Diagnostic Imaging, Humans, Models, Neurological

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review