CoChat
Sign Up
Cited by 1,599Statistical Methods and Bayesian InferenceBayesian Modeling and Causal InferenceGaussian Processes and Bayesian Inference

Assessment and Propagation of Model Uncertainty

Open access
Draper, David·1995·Journal of the Royal Statistical Society Series B: Statistical Methodology
DOI 10.1111/j.2517-6161.1995.tb02015.x
Open this paper in CoChat — free accountOpen PDF

Abstract

SUMMARY: In most examples of inference and prediction, the expression of uncertainty about unknown quantities y on the basis of known quantities x is based on a model M that formalizes assumptions about how x and y are related. M will typically have two parts: structural assumptions S, such as the form of the link function and the choice of error distribution in a generalized linear model, and parameters θ whose meaning is specific to a given choice of S. It is common in statistical theory and practice to acknowledge parametric uncertainty about θ given a particular assumed structure S; it is less common to acknowledge structural uncertainty about S itself. A widely used approach involves enlisting the aid of x to specify a plausible single ‘best’ choice S* for S, and then proceeding as if S* were known to be correct. In general this approach fails to assess and propagate structural uncertainty fully and may lead to miscalibrated uncertainty assessments about y given x. When miscalibration occurs it will often result in understatement of inferential or predictive uncertainty about y, leading to inaccurate scientific summaries and overconfident decisions that do not incorporate sufficient hedging against uncertainty. In this paper I discuss a Bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible, by virtue of recent computational advances, and examine its implementation in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the US space shuttle.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/j.2517-6161.1995.tb02015.x

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Alternative proportional hazards models that also fit the data well but lead to substantively different estimated effect sizes, standard errors, or predictions are commonplace.

    “Models like M ∗∗ are commonplace: for striking examples see Regal and Hook (1991), Draper (1995), Madigan and York (1995), Kass and Raftery (1995), and Raftery (1996).”

    From Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors (1999) · cites this paper as Draper (1995)

    Section 6.2 (page not provided) says that other models can fit equally well or better yet have different inferential or predictive consequences. This supports the general point, but the paper does not establish that this is commonplace specifically for alternative proportional hazards models or document the claimed differences in their effect sizes and standard errors. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • The MLE approximation is useful in the BMA context.

    “Draper (1995), Raftery, Madigan and Volinsky (1996) and Volinsky et al. (1997) show its usefulness in the BMA context.”

    From Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors (1999) · cites this paper as Draper (1995)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. Adams, J. L. (1991) A computer experiment to evaluate regression strategies. Proc. Comput. Statist. Sect. Am. Statist. Ass., 55-62.
  2. Clemen, R. (1989) Combining forecasts: a review and annotated bibliography (with discussion). Int. J. Forecast., 5, 559-608.
  3. Cox, D. R. (1961) Tests of separate families of hypotheses. In Proc. 4th Berkeley Symp, Mathematical Statistics, vol. 1, pp. 105-123. Berkeley: University of California Press.
  4. --(1986) Probability forecasting. In Encyclopedia of Statistical Sciences (eds S. Kotz and N. L. Johnson), vol. 7, pp.210-218. New York: Wiley.
  5. DuMouchel, W. H. (1990) Bayesian meta-analysis. In Statistical Methodology in the Pharmaceutical Sciences (ed. D. Berry), pp. 509-529. New York: Dekker.
  6. de Finetti, B. (1972) Probability, Induction, and Statistics. New York: Wiley.
  7. Geisel, M. S. (1974) Bayesian comparisons of simple macroeconomic models. In Studies in Bayesian Econometrics and Statistics (eds S. E. Fienberg and A. Zellner), pp. 227-256. New York: North- Holland.
  8. Geisser, S. and Eddy, W. F. (1979) A predictive approach to model selection. J. Am. Statist. Ass., 74, 153-160; corrigendum, 75 (1980),765.
  9. Greenland, S. (1993) Methods for epidemiologic analyses of multiple exposures: a review and comparative study of maximum-likelihood, preliminary-testing, and empirical-Bayes regression. Statist. Med., 12, 717-736.
  10. Hoaglin, D. C., Mosteller, F. and Tukey, J. W. (1985) Exploring Data Tables, Trends, and Shapes. New York: Wiley.
  11. Kashyap, R. L. (1982) Optimal choice of AR and MA parts in autoregressive moving average models. IEEE Trans. Pattn Anal. Mach. Intell., 4, 99-104.
  12. Kass, R. E. and Raftery, A. E. (1994) Bayes factors. J. Am. Statist. Ass., 89, in the press.
  13. Leamer, E. E. (1978) Specification Searches. New York: Wiley.
  14. Lindley, D. V. (1961) The use of prior probability distributions in statistical inference. In Proc. 4th Berkeley Symp, Mathematical Statistics, vol. 1, pp. 453-468. Berkeley: University of California Press.
  15. --(1982) The Bayesian approach to statistics. In Some Recent Advances in Statistics (eds J. T. de Oliveira and B. Epstein), pp. 65-87. London: Academic Press.
  16. Mosteller, F. and Tukey, J. W. (1977) Data Analysis and Regression. Reading: Addison-Wesley.
  17. Self, M. and Cheeseman, P. (1987) Bayesian prediction for artificial intelligence. In Proc. 3rd Wkshp Uncertainty in Artificial Intelligence, Seattle, pp. 61-69.
  18. Seltzer, M. H. (1994) Sensitivity analysis for fixed effects in the hierarchical model: a Gibbs sampling approach. J. Educ. Statist., 19, in the press.
  19. Smith, A. F. M. and Spiegelhalter, D. J. (1981) Bayesian approaches to multivariate structure. In Interpreting Multivariate Data (ed. V. Barnett), pp, 335-348. New York: Wiley.
  20. Spiegelhalter, D. J. (1981) Adaptive inference using a finite mixture model. PhD Thesis. University College London, London.
  21. Tierney, L. (1990)LISP-STAT: an Object-oriented Environment for Statistical Computing and Dynamic Graphics. New York: Wiley.
  22. Weisberg, S. (1985) Applied Linear Regression, 2nd edn. New York: Wiley.
  23. West, M. and Harrison, J. (1989)BayesianForecasting and Dynamic Linear Models. New York: Springer.
  24. Zellner, A. (1971) An Introduction to Bayesian Inference in Econometrics. New York: Wiley.

Cited by

Sorted by citations

Authors
Draper, D.
Affiliations
University of Bath (GB)
Corresponding author
Draper, David
ORCID
Draper, David
Venue
Journal of the Royal Statistical Society Series B: Statistical Methodology
Publisher
Oxford University Press (OUP)
Published
1995-01-01
Volume
57
Issue
1
Pages
45-70
ISSN
  • 1369-7412 (ISSN-L)
  • 1467-9868
DOI
10.1111/j.2517-6161.1995.tb02015.x
OpenAlex
W1488022545
Cited by
1,599
References
6
Citations by year
2026: 32 · 2025: 39 · 2024: 49 · 2023: 48 · 2022: 32 · 2021: 57 · 2020: 67 · 2019: 61 · 2018: 70 · 2017: 49 · 2016: 48 · 2015: 75 · 2014: 85 · 2013: 84 · 2012: 65
FWCI
31.99
Type
journal article
Language
en
Open access
Licence
academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
Locations
  • Journal of the Royal Statistical Society Series B (Statistical Methodology) (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
Keywords
model uncertainty, structural uncertainty, error propagation, Bayesian model averaging, calibration error, forecast uncertainty, uncertainty analysis, oil price forecasting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review