CoChat
Sign Up
Cited by 688Functional Brain Connectivity StudiesBayesian Modeling and Causal InferenceStatistical Methods and Bayesian Inference

Comparing families of dynamic causal models

Open access
Penny, Will D., Stephan, Klaas E., Daunizeau, Jean, Rosa, Maria J., Friston, Karl J., Schofield, Thomas M., Leff, Alex P.·2010·PLoS Computational Biology
DOI 10.1371/journal.pcbi.1000709PMC PMC2837394
Open this paper in CoChat — free accountOpen PDF

Abstract

Mathematical models of scientific data can be formally compared using Bayesian model evidence. Previous applications in the biological sciences have mainly focussed on model selection in which one first selects the model with the highest evidence and then makes inferences based on the parameters of that model. This "best model" approach is very useful but can become brittle if there are a large number of models to compare, and if different subjects use different models. To overcome this shortcoming we propose the combination of two further approaches: (i) family level inference and (ii) Bayesian model averaging within families. Family level inference removes uncertainty about aspects of model structure other than the characteristic of interest. For example: What are the inputs to the system? Is processing serial or parallel? Is it linear or nonlinear? Is it mediated by a single, crucial connection? We apply Bayesian model averaging within families to provide inferences about parameters that are independent of further assumptions about model structure. We illustrate the methods using Dynamic Causal Models of brain imaging data.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes family-level Bayesian inference and Bayesian model averaging within families as alternatives to selecting a single best model, especially when many models are compared or subjects may use different models. It illustrates the approach with Dynamic Causal Models of fMRI data from an auditory word-processing study, finding evidence that auditory input enters posterior STS and that speech intelligibility modulates forward connections from P to A and P to F.

Key findings

  • Family-level inference avoids brittle best-model selection; family posteriors are invariant to the comparison set under fixed effects, and random-effects family inference is implemented via Gibbs sampling.
  • For input connectivity, the FFX posterior for region P alone receiving auditory input was 1.00; RFX expected probability was 0.44 with exceedance probability 0.78, while region A alone had RFX exceedance 0.19.
  • For modulatory families, the forward family had the highest FFX posterior (0.64) and RFX expected probability (0.52) with exceedance probability 0.66.
  • Bayesian model averaging within input family P showed that only forward connections from P to A and P to F were modulated by speech intelligibility; free-energy-based top models averaged 4.5 modulatory connections versus 2.4 in the earlier AIC/BIC-based study.

Methods

  • Dynamic Causal Models for fMRI were used with three regions: left posterior superior temporal sulcus (P), left anterior superior temporal sulcus (A), and pars orbitalis of the inferior frontal gyrus (F); all models had all-to-all endogenous connectivity, with variation in direct input regions and modulatory connections.
  • The model space crossed 64 modulatory patterns with 7 input patterns, giving 448 DCMs, fitted to data from a previously published intelligible-speech study; the subject count is not recoverable from the provided text.
  • Analysis used fixed-effects and random-effects family-level inference; RFX inference used Gibbs sampling with 10,000 discarded burn-in samples, and Bayesian model averaging was performed within families using Occam’s window.

Limitations

  • Stated: the variational Bayes implementation is not accurate for small prior counts, motivating Gibbs sampling; Gibbs sampling is preferred when the number of models exceeds the number of subjects, while VB is preferred for small model sets.
  • Stated: the relative merits of different model-evidence criteria are left to a future publication, and the automatic relevance determination approach for DCMs is currently hypothetical.
  • Evident from the text: the method is illustrated on one fMRI dataset from an auditory word-processing study, so broader biological applicability is proposed but not tested here; results also depend on the chosen family partitions and on the free-energy approximation to model evidence.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1000709

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 22 cited sources checked · 0 supported · 0 flagged

  • Findingreasoning

    For group random effects inference, the ranking of models can be critically dependent on the comparison set.

    “However, for group random effects inference the ranking of models can be critically dependent on the comparison set.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    For FFX analysis, the relative ranking of a pair of models is not dependent on the comparison set.

    “For FFX analysis the relative ranking of a pair of models is not dependent on .”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    Posterior model probabilities depend on the comparison set.

    “Second, we emphasise that posterior model probabilities depend on the comparison set.”
    Quote in the paperSubstantial claim
  • Findingnone given

    In random effects analysis, augmenting the comparison set with a single extra model can reverse the ranking of the best and second best models.

    “In random effects analysis, augmenting the comparison set with a single extra model can, for example, reverse the ranking of the best and second best models.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Both the RFX and FFX model averages within family P show that only connections from region P to region A and from region P to region F are facilitated by speech intelligibility.

    “Both the RFX and FFX model averages within family P show that only connections from P to A, and from P to F, are facilitated by speech intelligibility.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Under the fixed-effects (FFX) assumption, all subjects use the same model.
    • The authors state this as an assumption: Under the random-effects (RFX) assumption, each subject uses their own model.
  • Classified as probably not a substantial claim. 15
  • May combine two claims. 22
  • The quote only partly states this claim. 4

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Figure 7 shows the posterior densities of average modulatory connections (averaging over models and subjects), and the height of the delta functions corresponds to the total posterior probability mass of models that assume the connection is zero.

    “Figure 7 shows the posterior densities of average modulatory connections (averaging over models and subjects). The height of the delta functions in these histograms correspond to the total posterior probability mass of models which assume that the connection is zero.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Under the fixed-effects (FFX) assumption, all subjects use the same model.

  • May combine two claims.

    Findingown data

    The posterior odds of the best model relative to the worst model in Occam's window is only [value], meaning these models are not significantly better than one another.

    “The worst one is with . The posterior odds of the best relative to the worst is only (the largest it could be is ), meaning these models are not significantly better than one another.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Under the fixed-effects (FFX) assumption, all subjects use the same model.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In the RFX analysis, region P alone receiving auditory input has exceedance probability 0.78, which is much more likely than region A alone receiving auditory input (exceedance probability 0.19).

    “Out of these two possibilities it is much more likely that region P alone receives auditory input (exceedance probability ) rather than region A (exceedance probability ).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Under the random-effects (RFX) assumption, each subject uses their own model.

  • May combine two claims.

    Findingcitation

    The variational algorithm is robust for α0=1 but not accurate for α0<1.

    “It turns out that although the variational algorithm is robust for , it is not accurate for .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Text S1

      Not checked in the free run.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For RFX inference, the input was inferred to most likely enter region P alone, and in the RFX model averaging the Occam's windowing procedure was specific to each subject, with an average of [value] models in Occam's window for the input model P family.

    “For RFX inference the input was inferred to most likely enter region P alone (posterior exceedance probability, ). In the RFX model averaging the Occam's windowing procedure was specific to each subject, thus each subject can have a different number of models in Occam's window. For the input model P family there were an average of models in Occam's window and Figure 8 shows the posterior densities”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Under the random-effects (RFX) assumption, each subject uses their own model.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In the FFX analysis of modulatory families, the posterior probabilities were 0.64 for predominantly forward, 0.07 for predominantly backward, 0.29 for balanced, and 0.00 for none.

    “| Forward, | 0.64 | −0.44 | 0.52 | 0.66 |”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Under the fixed-effects (FFX) assumption, all subjects use the same model.

  • May combine two claims.

    Findingown data

    For FFX inference, restricting Bayesian model averaging to the 64 models in family P produced 16 models in Occam's window, indicating that the posterior is not sharply peaked.

    “For FFX inference the input was inferred to enter region P only. We therefore restrict the averaging to those 64 models in family P. This produces 16 models in Occam's window (itself indicating that the posterior is not sharply peaked).”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Under the fixed-effects (FFX) assumption, all subjects use the same model.

  • May combine two claims.

    Findingown data

    The model with the highest free energy does not have significantly higher evidence than the second best model, or indeed, any model in Occam's window.

    “The model with the highest free energy (see figure 6(a)) does not, however, have significantly higher evidence than the second best model, or indeed, any model in Occam's window.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The 'best model' approach can become brittle if there are a large number of models to compare, or if different subjects use different models.

    “This ‘best model’ approach is very useful but, as we shall see, can become brittle if there are a large number of models to compare, or if in the analysis of data from a group of subjects, different subjects use different models (as is the case for a random effects analysis [19]).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Family level inference removes uncertainty about aspects of model structure other than the characteristic of interest.

    “Family level inference removes uncertainty about aspects of model structure other than the characteristic of interest.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    If one groups models based on a characteristic, then the inference about the characteristic is robust.

    “If one were to group the models based on this characteristic then the inference about the characteristic would be robust.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The number of models in Occam's window is a useful indicator of how peaked the posterior is, with smaller values corresponding to peakier posteriors.

    “The number of models in the window, , is a useful indicator as smaller values correspond to peakier posteriors.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38] D Madigan, 1994, Model selection and accounting for uncertainty in graphical models using Occam&apos;s window., Journal of the American Statistical Association source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Family level inference removes uncertainty about aspects of model structure other than the characteristic one is interested in.

    “Family level inference removes uncertainty about aspects of model structure other than the characteristic one is interested in.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The results indicate that spoken words most likely stimulate a region in posterior STS, and that if the word is intelligible, connections are strengthened both to anterior STS and an inferior frontal region.

    “Our results indicate that spoken words most likely stimulate a region in posterior STS and that if the word is intelligible connections are strengthened both to anterior STS and an inferior frontal region.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The family level inference procedures described in this paper can be applied whatever method is used for estimating the model evidence.

    “The family level inference procedures described in this paper can be applied whatever method is used for estimating the model evidence.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The GBF approach implicitly assumes that every subject uses the same model, and is therefore a Fixed Effects analysis.

    “As is made clear in [19] the GBF approach implicitly assumes that every subject uses the same model. It is therefore a Fixed Effects analysis.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [19] K Stephan, 2009, Bayesian model selection for group studies., Neuroimage source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    The 'best model' approach, though useful when the number of models is small, can become brittle if there are a large number of models and if different subjects use different models.

    “We have shown that this ‘best model’ approach, though useful when the number of models is small, can become brittle if there are a large number of models, and if different subjects use different models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    It is important that the total prior count does not dominate over the number of subjects, otherwise posterior densities will be dominated by the prior rather than the data.

    “This is because it is important that the total prior count, , does not dominate over the number of subjects, otherwise posterior densities will be dominated by the prior rather than the data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The Free Energy method approached the performance of AIS and clearly outperformed BIC in comparisons based on simulation of graphical model structures.

    “In other comparisons, based on simulation of graphical model structures [6] the Free Energy method approached the performance of AIS and clearly outperformed BIC.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] M Beal, 2003, The Variational Bayesian EM algorithms for incomplete data: with application to scoring graphical model structures. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Previous work in graphical models favours the free energy approach over BIC, and work on biochemical models finds AIS to be the best of the more computationally expensive sampling methods.

    “Previous work in graphical models favours the free energy approach over BIC [6] and work on biochemical models finds AIS to be the best of the more computationally expensive sampling methods.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] M Beal, 2003, The Variational Bayesian EM algorithms for incomplete data: with application to scoring graphical model structures. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The posterior densities from Bayesian model averaging can be multimodal and are most simply evaluated by sampling.

    “This means the posteriors can be multimodal and are most simply evaluated by sampling.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The use of Bayesian model averaging is a standard approach in Bayesian statistics but has yet to be applied extensively in computational biology.

    “Before proceeding we note that the use of Bayesian model averaging is a standard approach in the field of Bayesian statistics [4], but has yet to be applied extensively in computational biology.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] J Hoeting, 1999, Bayesian Model Averaging: A Tutorial., Statistical Science

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The use of Occam's window in Bayesian model averaging is for computational expedience only.

    “We emphasise that the use of Occam's window is for computational expedience only.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The use of model families is accommodated naturally within hierarchical Bayesian models and is proposed to address model dilution.

    “The use of model families is also accomodated naturally within the framework of hierarchical Bayesian models [1] and is proposed to address the well known issue of model dilution [4].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [1] A Gelman, 1995, Bayesian Data Analysis source

      Not checked in the free run.

    • cites [4] J Hoeting, 1999, Bayesian Model Averaging: A Tutorial., Statistical Science

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Because all families have the same number of members and model priors are uniform, the maximum posterior model is also the one with highest aggregate model evidence.

    “Given that all families have the same number of members, the model priors are uniform, so the maximum posterior model is also the one with highest aggregate model evidence.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Interpretationreasoning

    From equations 13 and 22, a uniform family prior can be achieved by setting α0 = K/M.

    “From equations 13 and 22 we see that this can be achieved by setting(24)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The complexity penalty in free energy is given by the KL-divergence between the prior and approximate posterior, allowing parameters to be differentially penalised.

    “This is to be compared with the free energy approach in which the complexity penalty is given by the KL-divergence between the prior and approximate posterior [11]. This allows parameters to be differentially penalised.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] W Penny, 2004, Comparing Dynamic Causal Models., NeuroImage source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Unbiased family level inferences require uniform priors over families.

    “As we shall see in the following section, unbiased family level inferences require uniform priors over families.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    In the particular example studied, the use of Bayes factors or posterior odds ratios would be inconclusive, whereas clear conclusions can be drawn from family level inference.

    “This indicates that in the particular example we have studied the use of Bayes factors or posterior odds ratios would be inconclusive, whereas clear conclusions can be drawn from family level inference.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationnone given

    Family level inferences under RFX assumptions are more subtle and have been the main focus of this paper.

    “Family level inferences under RFX assumptions are more subtle and have been the main focus of this paper.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Bayesian Model Averaging can be used with the Discrete approach to provide estimates of the marginal density.

    “Additionally, Bayesian Model Averaging can be used with the Discrete approach to provide estimates of the marginal density .”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The issue of family versus model level inference is orthogonal to the issue of random versus fixed effects analysis.

    “The issue of family versus model level inference is orthogonal to the issue of random versus fixed effects analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The same criteria regarding fixed versus random effects apply at the family level as at the model level.

    “The same critera re. FFX versus RFX apply at the family level as at the model level.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Family level inferences under FFX assumptions are simple to implement.

    “Family level inferences under FFX assumptions are simple to implement.”
    Quote in the paperNot a substantial claim
34 of 34

  • The authors state this as an assumption.

    The Free Energy bound is tight.

    “Under the assumption that this bound is tight, model comparison can then proceed using as a surrogate for the log-model evidence.”

  • The authors state this as an assumption.

    Under the fixed-effects (FFX) assumption, all subjects use the same model.

    “We could take this to be under the FFX assumption that all subjects use the same model, or under the RFX assumption that each subject uses their own model (see equation 14).”

  • The authors state this as an assumption.

    Under the random-effects (RFX) assumption, each subject uses their own model.

    “We could take this to be under the FFX assumption that all subjects use the same model, or under the RFX assumption that each subject uses their own model (see equation 14).”

  • The paper relies on this without support.

    Dynamic causal models (DCMs) have a Gaussian form.

    “For the applications in this paper, the posterior densities from equation 27 will therefore correspond to a mixture of delta functions and Gaussians because for DCMs have a Gaussian form.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In recent years, the number of models over which people search in DCM has grown enormously, to the extent that DCM is now used to discover the best model over very large model-spaces.

    “However, in recent years, the number of models over which people search (the model-space) has grown enormously; to the extent that DCM is now used to discover the best model over very large model-spaces (e.g., Stephan et al., 2010; Penny et al., 2010).”

    From Network discovery with DCM (2011) · cites this paper as Penny et al., 2010

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • PAF model range overlaps other input families

    Minor

    The stated PAF range (285-448) overlaps PA (257-320) and PF (321-384). That contradicts the requirement that family subsets be non-overlapping.

1 of 1

Cites

Flagged first, then by citations

1 of the 46 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. J Hoeting, 1999, Bayesian Model Averaging: A Tutorial., Statistical Science, 14, 382

Cited by

Sorted by citations

Related

Authors
Penny, W. D., Stephan, K. E., Daunizeau, J., Rosa, M. J., Friston, K. J., Schofield, T. M., & Leff, A. P.
Affiliations
Wellcome Centre for Human Neuroimaging (GB)
Corresponding author
Penny, Will D.
ORCID
Venue
PLoS Computational Biology
Publisher
Public Library of Science (PLoS)
Published
2010-03-12
Volume
6
Issue
3
Pages
e1000709
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
DOI
10.1371/journal.pcbi.1000709
PMC
PMC2837394
OpenAlex
W1998151455
Semantic Scholar
2f88070d894eb8d1a96c8f92c726f0422f3bcccd
Cited by
688
References
46
Citations by year
2026: 13 · 2025: 35 · 2024: 25 · 2023: 25 · 2022: 33 · 2021: 51 · 2020: 54 · 2019: 36 · 2018: 43 · 2017: 47 · 2016: 57 · 2015: 71 · 2014: 43 · 2013: 67 · 2012: 36
FWCI
29.48
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Portal (King's College London) (publishedVersion, other-oa)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • UEA Digital Repository (University of East Anglia) (submittedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich)
Funders
  • Wellcome Trust (088130)
  • Universität Zürich
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
dynamic causal modeling, Bayesian model averaging, model comparison, marginal likelihood, neuroimaging, parameter uncertainty, model structure uncertainty
MeSH terms
Bayes Theorem, Brain, Computer Simulation, Models, Neurological, Nerve Net, Pattern Recognition, Automated, Models, Statistical, Causality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review