Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record, with a small difference in title or year; 157 of the 157 references it cites were found in a registry or an index; 7 of the 90 sources its claims cite were checked for support.

Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors

Open access

Abstract

Standard statistical practice ignores model uncertainty. Data analysts typically select a model from some class of models and then proceed as if the selected model had generated the data. This approach ignores the uncertainty in model selection, leading to over-confident inferences and decisions that are more risky than one thinks they are. Bayesian model averaging (BMA)provides a coherent mechanism for accounting for this model uncertainty. Several methods for implementing BMA have recently emerged. We discuss these methods and present a number of examples.In these examples, BMA provides improved out-of-sample predictive performance. We also provide a catalogue of currently available BMA software.

Overview

CoChat Summary

TL;DR

This tutorial argues that standard model selection ignores model uncertainty and presents Bayesian model averaging (BMA) as a coherent alternative that averages posterior distributions over models weighted by posterior model probabilities. Across examples, BMA improved out-of-sample predictive performance and produced posterior effect probabilities that often indicated weaker evidence than p-values from single selected models.

Key findings

  • BMA provides better average predictive performance than any single model under logarithmic scoring. In the primary biliary cirrhosis (PBC) example, BMA had a partial predictive score of 217.1 versus 221.6 for the top posterior-probability model, 220.7 for stepwise selection, and 222.8 for the Fleming–Harrington model; the 3.6-point difference corresponds to about 6% better prediction per event than the top model, 10% better than the Fleming–Harrington model, and 2% better than stepwise. Over 20 splits, BMA averaged 2.7 points (5% per event) better than both the top posterior-probability model and stepwise selection.
  • In the PBC example, the model with the highest approximate posterior probability was the same as the stepwise model but represented only 17% of total posterior probability; the Fleming–Harrington model placed sixth with 5%. Posterior effect probabilities were 100% for bilirubin, albumen, and age; 84% for edema; 78% for prothrombin time; 72% for urine copper; 34% for histologic stage; 22% for SGOT; and 2% for treatment DPCA.
  • In the body fat example with 13 predictors and 8,192 possible models, the top model accounted for only 14% of posterior probability and the top 10 models accounted for 57%. BMA predictive coverage was 90.8%, versus 84.4% for stepwise and minimum Mallow’s Cp and 83.5% for maximum adjusted R². Posterior effect probabilities were 100% for abdomen circumference, 97% for weight, 62% for wrist circumference, 35% for forearm, and 19% for neck.
  • BMA implementation strategies discussed include Occam’s window, Markov chain Monte Carlo model composition (MC³), leaps-and-bounds search, and BIC, Laplace, and MLE approximations, with software catalogued for linear regression, generalized linear models, survival analysis, and graphical models.

Methods

  • The paper is a tutorial/review that develops BMA theory and implementation details for linear regression, generalized linear models, survival analysis, and graphical models, then illustrates BMA in two examples.
  • PBC example: Mayo Clinic randomized trial data on 312 patients; 8 omitted for incomplete data, leaving 304 patients, of whom 123 died and 181 were censored. The analysis used 14 prognostic variables plus treatment DPCA, compared BMA via leaps-and-bounds to stepwise selection, the top posterior-probability model, and the Fleming–Harrington model using partial predictive scores and predictive discrimination.
  • Body fat example: 252 men with 13 simple body measurements; one subject with erroneous height was omitted, leaving 251. Posterior model probabilities were computed for all 2¹³ = 8,192 models under diffuse proper priors, and BMA was compared with stepwise, minimum Mallow’s Cp, and maximum adjusted R² using predictive coverage.

Limitations

  • The paper states that BMA implementation faces major difficulties: the number of terms in the model sum can be enormous, the required integrals can be hard to compute, prior model probabilities are challenging to specify, and choosing the class of models over which to average is the fundamental modeling task. It also lists open questions including prior choice, performance when the true model is not in the model class, tuning Occam’s window versus full BMA, methodology for other model classes, and more efficient computation.
  • The paper and rejoinder state that BMA is conditional on the selected model class and does not address uncertainty about model class selection; Occam’s window is heuristic and lacks formal theoretical support as an approximation to full BMA, though the authors say biases seem small in practice.
  • Discussants note additional limitations evident in the text: posterior effect probabilities can be sensitive to prior hyperparameters and the model space; Occam’s window may bias posterior effect probabilities; the PBC high-risk predictive advantage is modest when uncertainty bands are attached; and BMA in the body fat example requires all 13 measurements.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1214/ss/1009212519

Cites

157 of 157 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Occam's window: interpreting the posterior odds.
Fig. 1 Occam's window: interpreting the posterior odds.From the PDF
Fig. 2 A simple discrete graphical model.
Fig. 2 A simple discrete graphical model.From the PDF
Fig. 3 PBC example: posterior effect probabilities from BMA versus p -values from the stepwise variable selection model.
Fig. 3 PBC example: posterior effect probabilities from BMA versus p -values from the stepwise variable selection model.From the PDF
Fig. 4 Body fat example: BMA posterior distribution for β 13 commaori the coefficient for wrist circumference. The spike corresponds to P lparenOSCASB β 13 = 0 vertSHT D rparenOSCASB = 0 periodori 38 . The vertical axis on the left corresponds to the posterior distribution for β 13 and the vertical axis on the right corresponds to the posterior distribution for β 13 equal to 0. The density is scaled so that the maximum of the density is equal to P lparenOSCASB β 13 ̸= 0 vertSHT D rparenOSCASB on the right axis.
Fig. 4 Body fat example: BMA posterior distribution for β 13 commaori the coefficient for wrist circumference. The spike corresponds to P lparenOSCASB β 13 = 0 vertSHT D rparenOSCASB = 0 periodori 38 . The vertical axis on the left corresponds to the posterior distribution for β 13 and the vertical axis on the right corresponds to the posterior distribution for β 13 equal to 0. The density is scaled so that the maximum of the density is equal to P lparenOSCASB β 13 ̸= 0 vertSHT D rparenOSCASB on the right axis.From the PDF
Fig. 1 Three posterior distributions for β 13 commaori the coefficient for wrist circumference in the body fat example of Section 7 periodori 2 .
Fig. 1 Three posterior distributions for β 13 commaori the coefficient for wrist circumference in the body fat example of Section 7 periodori 2 .From the PDF

Verification

Claims the paper makes

6 Valid · 12 Not valid · 63 Not checked · 154 Own results and interpretations

96 more claims · 1 with their cited sources checked

Create a free account to see all 235 claims and every source verdict.

See all claims — free account
  • Borderline: one of the tests for a substantial claim is unclear. 1
  • It may state two separate claims as one. 63
  • Probably not a substantial claim. 93

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 148 references

Research works printed but not linked: 9

  1. Volinsky, C. T. (1997). Bayesian model averaging for censored survival models. Ph.D. dissertation, Univ. Washington, Seattle.
  2. C. (1993). Combining information: National Research Council Panel on Statistical Issues and Opportunities for Research in the Combination of Information. National Academy Press, Washington, DC. Draper, D., Hodges, J. S., Leamer, E. E., Morris, C. N. and
  3. George, E. I. (1999). Bayesian model selection. In Encyclopedia of Statistical Sciences Update 3. Wiley, New York. To appear.
  4. Hoeting, J. A. (1994). Accounting for model uncertainty in linear regression. Ph.D. dissertation, Univ. Washington, Seattle.
  5. Leamer, E. E. (1978). Specification Searches. Wiley, New York.
  6. Raftery, A. E., Madigan, D. and Volinsky, C. T. (1996). Accounting for model uncertainty in survival analysis improves predictive performance (withdiscussion). In Bayesian Statistics 5 (J. Bernardo, J. Berger, A. Dawid and A. Smith, eds.) 323-349. Oxford Univ. Press.
  7. de Finetti, B. (1974, 1975). Theory of Probability 1 and 2. (Trans. by A. F. M. Smithand A. Machi). Wiley, New York.
  8. George, E. I. (1999). Discussion of "Model averaging and model searchstrategies" by M. Clyde. In Bayesian Statistics 6 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 157-185. Oxford University Press.
  9. George, E. I. (1999). Discussion of "Model averaging and model searchby M. Clyde." In Bayesian Statistics 6 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) Oxford University Press.

Cited by

See all 4,152

Metadata

Authors
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T.
Venue
Statistical Science
Published
1999-11-01
Volume
14
Issue
4
Pages
382-417
ISSN
0883-4237 (ISSN-L)
OpenAlex
W1603903339
Cited by
4,118
References
148
Citations by year
2026: 103 · 2025: 148 · 2024: 142 · 2023: 125 · 2022: 141 · 2021: 234 · 2020: 221 · 2019: 244 · 2018: 243 · 2017: 248 · 2016: 203 · 2015: 258 · 2014: 222 · 2013: 191 · 2012: 222
FWCI
31.88
Type
journal article
Language
en
Open access
Locations
  • Statistical Science (publishedVersion)
  • Project Euclid (Cornell University) (submittedVersion)
  • Columbia Academic Commons (Columbia University)
Funders
  • National Science Foundation (9806243)
  • Office of Naval Research (N00014)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
Keywords
Bayesian model averaging, model uncertainty, model selection, out-of-sample forecasting, predictive performance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors. https://app.cochat.ai/papers/10.1214/ss/1009212519
The paper
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999). Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors. Statistical Science. https://doi.org/10.1214/ss/1009212519

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.