The DOI matches its Crossref record; 116 of the 117 references it cites were found in a registry or an index.
Model Uncertainty, Data Mining and Statistical Inference
Abstract
This paper takes a broad, pragmatic view of statistical inference to include all aspects of model formulation. The estimation of model parameters traditionally assumes that a model has a prespecified known form and takes no account of possible uncertainty regarding the model structure. This implicitly assumes the existence of a 'true' model, which many would regard as a fiction. In practice model uncertainty is a fact of life and likely to be more serious than other sources of uncertainty which have received far more attention from statisticians. This is true whether the model is specified on subject-matter grounds or, as is increasingly the case, when a model is formulated, fitted and checked on the same data set in an iterative, interactive way. Modern computing power allows a large number of models to be considered and data-dependent specification searches have become the norm in many areas of statistics. The term data mining may be used in this context when the analyst goes to great lengths to obtain a good fit. This paper reviews the effects of model uncertainty, such as too narrow prediction intervals, and the non-trivial biases in parameter estimates which can follow data-based modelling. Ways of assessing and overcoming the effects of model uncertainty are discussed, including the use of simulation and resampling methods, a Bayesian model averaging approach and collecting additional data wherever possible. Perhaps the main aim of the paper is to ensure that statisticians are aware of the problems and start addressing the issues even if there is no simple, general theoretical fix.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“in reality, the true model is unknown, may not have a generalized linear form, or may not even exist [22].”
From A calibration hierarchy for risk models was defined: from utopia to empirical data (2016) · cites this paper as [22]
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Book Review: Time Series Analysis: Forecasting and Control
Geurts, M. D., Box, G. E. P., & Jenkins, G. M. (1977) · Journal of Marketing Research · doi:10.2307/3150485 · Cited by 19,410
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883
The Logic of Scientific Discovery
Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765
The Advanced Theory of Statistics.
Clarke, G. M., Kendall, M. G., & Stuart, A. (1978) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344782 · Cited by 6,727
A leisurely look at the bootstrap, the jackknife, and cross-validation
Efron, B., & Gong, G. (1983) · The American Statistician · doi:10.1080/00031305.1983.10483087 · Cited by 3,229
Box, G. E. P. (1976) · Journal of the American Statistical Association · doi:10.1080/01621459.1976.10480949 · Cited by 2,114
Bayesian Forecasting and Dynamic Models
West, M., & Harrison, J. (1997) · Springer eBooks · doi:10.1007/b98971 · Cited by 1,946
Cross-Validation of Regression Models
Picard, R. R., & Cook, R. D. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478083 · Cited by 1,551
Combining forecasts: A review and annotated bibliography
(1990) · Long Range Planning · doi:10.1016/0024-6301(90)90316-v · Cited by 1,508
Flagged first, then by citations
1 of the 117 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., Marquéz, J. R. G., Gruber, B., Lafourcade, B., Leitão, P. J., et al. (2013) · Ecography · doi:10.1111/j.1600-0587.2012.07348.x · Cited by 9,998
A review of methods for the assessment of prediction errors in conservation presence/absence models
FIELDING, A. H., & BELL, J. F. (1997) · Environmental Conservation · doi:10.1017/s0376892997000088 · Cited by 6,956
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118
Posada, D., & Buckley, T. R. (2004) · Systematic Biology · doi:10.1080/10635150490522304 · Cited by 3,927
Metadata
- Authors
- Chatfield, C.
- Affiliations
- University of Bath (GB)
- Corresponding author
- Chatfield, Chris
- Venue
- Journal of the Royal Statistical Society. Series A (Statistics in Society)
- Publisher
- Oxford University Press (OUP)
- Published
- 1995
- Volume
- 158
- Issue
- 3
- Pages
- 419
- ISSN
- 0964-1998 (ISSN-L)
- OpenAlex
- W2060050130
- Cited by
- 1,103
- References
- 117
- Citations by year
- 2026: 18 · 2025: 34 · 2024: 38 · 2023: 33 · 2022: 35 · 2021: 33 · 2020: 44 · 2019: 28 · 2018: 47 · 2017: 35 · 2016: 37 · 2015: 51 · 2014: 52 · 2013: 28 · 2012: 63
- FWCI
- 24.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the Royal Statistical Society Series A (Statistics in Society) (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- model uncertainty, data mining, statistical inference, model selection, prediction intervals, estimation bias, Bayesian model averaging, resampling, simulation, model misspecification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
