Model uncertainty, data mining and statistical inference
Abstract
This paper takes a broad, pragmatic view of statistical inference to include all aspects of model formulation. The estimation of model parameters traditionally assumes that a model has a prespecified known form and takes no account of possible uncertainty regarding the model structure. This implicitly assumes the existence of a 'true' model, which many would regard as a fiction. In practice model uncertainty is a fact of life and likely to be more serious than other sources of uncertainty which have received far more attention from statisticians. This is true whether the model is specified on subject-matter grounds or, as is increasingly the case, when a model is formulated, fitted and checked on the same data set in an iterative, interactive way. Modern computing power allows a large number of models to be considered and data-dependent specification searches have become the norm in many areas of statistics. The term data mining may be used in this context when the analyst goes to great lengths to obtain a good fit. This paper reviews the effects of model uncertainty, such as too narrow prediction intervals, and the non-trivial biases in parameter estimates which can follow data-based modelling. Ways of assessing and overcoming the effects of model uncertainty are discussed, including the use of simulation and resampling methods, a Bayesian model averaging approach and collecting additional data wherever possible. Perhaps the main aim of the paper is to ensure that statisticians are aware of the problems and start addressing the issues even if there is no simple, general theoretical fix.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In reality, the true model is unknown, may not have a generalized linear form, or may not even exist.
“in reality, the true model is unknown, may not have a generalized linear form, or may not even exist [22].”
From A calibration hierarchy for risk models was defined: from utopia to empirical data (2016) · cites this paper as [22]
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500 · see citing works
Book Review: Time Series Analysis: Forecasting and Control
Geurts, M. D., Box, G. E. P., & Jenkins, G. M. (1977) · Journal of Marketing Research · doi:10.2307/3150485 · Cited by 19,399 · see citing works
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883 · see citing works
The Logic of Scientific Discovery
Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765 · see citing works
The Advanced Theory of Statistics.
Clarke, G. M., Kendall, M. G., & Stuart, A. (1978) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344782 · Cited by 6,727 · see citing works
A leisurely look at the bootstrap, the jackknife, and cross-validation
Efron, B., & Gong, G. (1983) · The American Statistician · doi:10.1080/00031305.1983.10483087 · Cited by 3,229 · see citing works
Box, G. E. P. (1976) · Journal of the American Statistical Association · doi:10.1080/01621459.1976.10480949 · Cited by 2,372 · see citing works
Bayesian Forecasting and Dynamic Models
West, M., & Harrison, J. (1997) · Springer eBooks · doi:10.1007/b98971 · Cited by 1,946 · see citing works
Cross-Validation of Regression Models
Picard, R. R., & Cook, R. D. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478083 · Cited by 1,551 · see citing works
Combining forecasts: A review and annotated bibliography
(1990) · Long Range Planning · doi:10.1016/0024-6301(90)90316-v · Cited by 1,508 · see citing works
Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window
Madigan, D., & Raftery, A. E. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476894 · Cited by 1,205 · see citing works
Dangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143 · see citing works
Specification Searches: Ad Hoc Inference with Non-Experimental Data.
Richard, J., & Leamer, E. E. (1981) · Economica · doi:10.2307/2553491 · Cited by 1,129 · see citing works
Bayesian Graphical Models for Discrete Data
Madigan, D., York, J., & Allard, D. (1995) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403615 · Cited by 1,090 · see citing works
Selection of the Order of an Autoregressive Model by Akaike's Information Criterion
Shibata, R. (1976) · Biometrika · doi:10.1093/biomet/63.1.117 · Cited by 789 · see citing works
Theory of Modelling and Simulation.
Tocher, K. D., & Zeigler, B. P. (1977) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2345293 · Cited by 768 · see citing works
Approximate Bayes factors and accounting for model uncertainty in generalised linear models
Raftery, A. E. (1996) · Biometrika · doi:10.1093/biomet/83.2.251 · Cited by 634 · see citing works
A Bayesian Extension of the Minimum AIC Procedure of Autoregressive Model Fitting
Akaike, H. (1979) · Biometrika · doi:10.1093/biomet/66.2.237 · Cited by 629 · see citing works
Geisser, S. (2017) · Routledge eBooks · doi:10.1201/9780203742310 · Cited by 575 · see citing works
(1985) · American Economic Review · Cited by 568 · see citing works
Silvey, S. D. (1976) · Routledge eBooks · doi:10.1201/9780203738641 · Cited by 559 · see citing works
Subset Selection in Regression.
RAAB, G. M., & Miller, T. (1991) · Biometrics · doi:10.2307/2532418 · Cited by 547 · see citing works
The Statistical Implications of Pre-Test and Stein-Rule Estimators in Econometrics
Wilcox, R. R., Judge, G. G., & Bock, M. E. (1979) · Technometrics · doi:10.2307/1267769 · Cited by 532 · see citing works
The Design of Replicated Studies
Lindsay, R. M., & Ehrenberg, A. S. C. (1993) · The American Statistician · doi:10.1080/00031305.1993.10475983 · Cited by 491 · see citing works
Two new properties of mathematical likelihood
Fisher, R. A. (1934) · Proceedings of the Royal Society of London Series A Containing Papers of a Mathematical and Physical Character · doi:10.1098/rspa.1934.0050 · Cited by 487 · see citing works
Replications and Extensions in Marketing – Rarely Published But Quite Contrary
Hubbard, R., & Armstrong, J. (1994) · International Journal of Research in Marketing · doi:10.1016/0167-8116(94)90003-5 · Cited by 444 · see citing works
Forecasting Trends in Time Series
Gardner, E. S., & McKenzie, E. (1985) · Management Science · doi:10.1287/mnsc.31.10.1237 · Cited by 402 · see citing works
When and how to use scenario planning: A heuristic approach with illustration
Schoemaker, P. J. H. (1991) · Journal of Forecasting · doi:10.1002/for.3980100602 · Cited by 371 · see citing works
Some advances in non‐linear and adaptive modelling in time‐series
Tiao, G. C., & Tsay, R. S. (1994) · Journal of Forecasting · doi:10.1002/for.3980130206 · Cited by 362 · see citing works
Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models
Kashyap, R. L. (1982) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1982.4767213 · Cited by 329 · see citing works
[Statistical Process Monitoring and Feedback Adjustment]: Discussion
MacGregor, J. F. (1992) · Technometrics · doi:10.2307/1270030 · Cited by 319 · see citing works
STATE‐DEPENDENT MODELS: A GENERAL APPROACH TO NON‐LINEAR TIME SERIES ANALYSIS
Priestley, M. (1980) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.1980.tb00300.x · Cited by 314 · see citing works
A Consistent Test for a Unit Root
Leybourne, S. J., & McCabe, B. P. M. (1994) · Journal of Business and Economic Statistics · doi:10.1080/07350015.1994.10510004 · Cited by 304 · see citing works
The Impact of Model Selection on Inference in Linear Regression
Hurvich, C. M., & Tsai, C. (1990) · The American Statistician · doi:10.1080/00031305.1990.10475722 · Cited by 279 · see citing works
Breiman, L. (1992) · Journal of the American Statistical Association · doi:10.1080/01621459.1992.10475276 · Cited by 277 · see citing works
Computer Intensive Statistical Methods: Validation Model Selection and Bootstrap.
Buckland, S. T., & Hjorth, J. S. U. (1994) · Biometrics · doi:10.2307/2533409 · Cited by 260 · see citing works
Effects of Model Selection on Inference
Pötscher, B. M. (1991) · Econometric Theory · doi:10.1017/s0266466600004382 · Cited by 231 · see citing works
Hubbard, R., & Vetter, D. E. (1996) · Journal of Business Research · doi:10.1016/0148-2963(95)00084-4 · Cited by 230 · see citing works
The Bias of Autoregressive Coefficient Estimators
Shaman, P., & Stine, R. A. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478672 · Cited by 228 · see citing works
Choi, B. (1992) · Springer series in statistics · doi:10.1007/978-1-4613-9745-8 · Cited by 227 · see citing works
Applied Bayesian Forecasting and Times Series Analysis
Pole, A., West, M., & Harrison, J. (1994) · Medical Entomology and Zoology · Cited by 222 · see citing works
A Nearest-Centroid Technique For Evaluating The Minimum-Variance Clustering Procedure
McIntyre, R. M., & Blashfield, R. K. (1980) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1502_7 · Cited by 198 · see citing works
Replicating Cluster Analysis: Method, Consistency, and Validity
Breckenridge, J. N. (1989) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2402_1 · Cited by 188 · see citing works
Role of Models in Statistical Analysis
Cox, D. R. (1990) · Statistical Science · doi:10.1214/ss/1177012165 · Cited by 181 · see citing works
The Impact of Empirical Accuracy Studies on Time Series Analysis and Forecasting
Fildes, R., & Makridakis, S. (1995) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403481 · Cited by 172 · see citing works
Bootstrap Technology and Applications
Léger, C., Politis, D. N., Romano, J. P., & Léger, C. (1992) · Technometrics · doi:10.2307/1268938 · Cited by 167 · see citing works
Inflation of R 2 in Best Subset Regression
Rencher, A. C., & Pun, F. C. (1980) · Technometrics · doi:10.1080/00401706.1980.10486100 · Cited by 162 · see citing works
A Two-Stage Clustering Algorithm with Robust Recovery Characteristics
Milligan, G. W., & Sokol, L. M. (1980) · Educational and Psychological Measurement · doi:10.1177/001316448004000320 · Cited by 161 · see citing works
Finite sample properties of estimators for autoregressive moving average models
Ansley, C. F., & Newbold, P. (1980) · Journal of Econometrics · doi:10.1016/0304-4076(80)90013-5 · Cited by 157 · see citing works
EMPIRICAL EVIDENCE ON DICKEY‐FULLER‐TYPE TESTS
Agiakloglou, C., & Newbold, P. (1992) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.1992.tb00121.x · Cited by 154 · see citing works
Applied Statistics: Principles and Examples
Williams, D., Cox, D. R., & Snell, E. J. (1982) · Biometrics · doi:10.2307/2530479 · Cited by 150 · see citing works
Methods for Determining the Order of an Autoregressive-Moving Average Process: A Survey
Gooijer, J. G. D., Abraham, B., Gould, A., & Robinson, L. (1985) · International Statistical Review · doi:10.2307/1402894 · Cited by 145 · see citing works
Denton, F. T. (1985) · The Review of Economics and Statistics · doi:10.2307/1928442 · Cited by 131 · see citing works
REPRESENTATION DESIGN AND BRUTE-FORCE INDUCTION IN A BOEING MANUFACTURING DOMAIN
Riddle, P., Segal, R. B., & Etzioni, O. (1994) · Applied Artificial Intelligence · doi:10.1080/08839519408945435 · Cited by 127 · see citing works
Fully Bayesian analysis of ARMA time series models
Monahan, J. F. (1983) · Journal of Econometrics · doi:10.1016/0304-4076(83)90048-9 · Cited by 124 · see citing works
Prediction Error and Its Estimation for Subset-Selected Models
Roecker, E. B. (1991) · Technometrics · doi:10.2307/1269417 · Cited by 123 · see citing works
Reviewer bias against replication research.
Neuliep, J. W., & Crandall, R. (1993) · Journal of social behavior and personality · Cited by 116 · see citing works
Uncertainty, Policy Analysis and Statistics
Hodges, J. S. (1987) · Statistical Science · doi:10.1214/ss/1177013224 · Cited by 115 · see citing works
ECONOMICS AS A HARD SCIENCE: REALISTIC GOAL OR WISHFUL THINKING?
Mayer, T. F. (1980) · Economic Inquiry · doi:10.1111/j.1465-7295.1980.tb00568.x · Cited by 110 · see citing works
Neural networks: Forecasting breakthrough or passing fad?
Chatfield, C. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90043-m · Cited by 107 · see citing works
Self-citationThe Effect of Model Selection on Confidence Regions and Prediction Regions
Kabaila, P. V. (1995) · Econometric Theory · doi:10.1017/s0266466600009403 · Cited by 107 · see citing works
A comparison of the information and posterior probability criteria for model selection
Chow, G. C. (1981) · Journal of Econometrics · doi:10.1016/0304-4076(81)90073-7 · Cited by 95 · see citing works
Ahn, S. K. (1993) · Biometrika · doi:10.1093/biomet/80.4.855 · Cited by 85 · see citing works
Asymptotically efficient autoregressive model selection for multistep prediction
Bhansali, R. J. (1996) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf00050857 · Cited by 83 · see citing works
Statistics, Science and Technology
Nelder, J. A. (1986) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981525 · Cited by 83 · see citing works
Faraway, J. J. (1992) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1992.10474582 · Cited by 80 · see citing works
Greater or lesser statistics: a choice for future research
Chambers, J. M. (1993) · Statistics and Computing · doi:10.1007/bf00141776 · Cited by 79 · see citing works
Two New Tests Associated with Analysis of Variance
Azzalini, A., & Cox, D. R. (1984) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1984.tb01306.x · Cited by 79 · see citing works
Tiao, G. C., & Xu, D. (1993) · Biometrika · doi:10.1093/biomet/80.3.623 · Cited by 78 · see citing works
The Choice of Variables in Observational Studies
Cox, D. R., & Snell, E. J. (1974) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347053 · Cited by 78 · see citing works
Bhansali, R. J. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477690 · Cited by 77 · see citing works
MODELING LONG‐MEMORY PROCESSES FOR OPTIMAL LONG‐RANGE PREDICTION
Ray, B. K. (1993) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.1993.tb00161.x · Cited by 76 · see citing works
Assumptions for Statistical Inference
Hahn, G. J., & Meeker, W. Q. (1993) · The American Statistician · doi:10.1080/00031305.1993.10475924 · Cited by 75 · see citing works
Embracing the "Wider View" of Statistics
Wild, C. J. (1994) · The American Statistician · doi:10.1080/00031305.1994.10476049 · Cited by 73 · see citing works
The effects of model selection on confidence intervals for the size of a closed population
Regal, R. R., & Hook, E. B. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100506 · Cited by 71 · see citing works
Inference after variable selection in linear regression models
Zhang, P. (1992) · Biometrika · doi:10.1093/biomet/79.4.741 · Cited by 67 · see citing works
Some Progress and Problems in Meta-Analysis of Clinical Trials
Mosteller, F. C., & Chalmers, T. C. (1992) · Statistical Science · doi:10.1214/ss/1177011362 · Cited by 66 · see citing works
Likelihood and Bayesian Inference from Selectively Reported Data
Dawid, A., & Dickey, J. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10479968 · Cited by 66 · see citing works
Arnold, S. J. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600411 · Cited by 64 · see citing works
The Practice of Operational Research.
Wilson, J., & Mitchell, G. H. (1994) · Journal of the Operational Research Society · doi:10.2307/2584296 · Cited by 61 · see citing works
Some Recent Developments in Time-Series Analysis
Chatfield, C. (1977) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2345281 · Cited by 60 · see citing works
Self-citationRostron, J., & Ehrenberg, A. S. C. (1983) · Biometrics · doi:10.2307/2530842 · Cited by 52 · see citing works
York, J., Madigan, D., Heuch, I., & Lie, R. T. (1995) · Applied Statistics · doi:10.2307/2986347 · Cited by 52 · see citing works
On the Impact of Variable Selection in Fitting Regression Equations
Freedman, D. A., Navidi, W., & Peters, S. C. (1988) · Lecture Notes in Economics and Mathematical Systems · doi:10.1007/978-3-642-61564-1_1 · Cited by 51 · see citing works
Statistics and Quality Improvement
Box, G. E. P. (1994) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983359 · Cited by 51 · see citing works
On Model Uncertainty and its Statistical Implications
Dijkstra, T. K. (1988) · Lecture notes in economics and mathematical systems · doi:10.1007/978-3-642-61564-1 · Cited by 51 · see citing works
Model Selection and Forward Validation
Hjorth, U. (1982) · Cited by 47 · see citing works
Consistency of model selection and parameter estimation
Shibata, R. (1986) · Journal of Applied Probability · doi:10.1017/s0021900200117036 · Cited by 36 · see citing works
Order selection for linear time series models: a review
Bhansali, R. J. (1993) · Springer eBooks · doi:10.1007/978-1-4899-4515-0_5 · Cited by 34 · see citing works
Determining a portfolio of linear time series models
Poskitt, D. S., & Tremayne, A. R. (1987) · Biometrika · doi:10.1093/biomet/74.1.125 · Cited by 34 · see citing works
Bootstrap adaptive estimation: The trimmed-mean example
Léger, C., & Romano, J. P. (1990) · Canadian Journal of Statistics · doi:10.2307/3315837.n · Cited by 32 · see citing works
Precision, Complexity and Bayesian Model Determination
Poskitt, D. S. (1987) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1987.tb01691.x · Cited by 32 · see citing works
A Research Agenda for Assessment and Propagation of Model Uncertainty
Draper, D., Hodges, J. S., Leamer, E. E., Morris, C. N., & Rubin, D. B. (1987) · Cited by 28 · see citing works
Model Selection for Multi-Step-Ahead Forecasting
Findley, D. F. (1985) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)60699-2 · Cited by 28 · see citing works
On model selection in the computer age
Hjorth, U. (1989) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(89)90043-8 · Cited by 27 · see citing works
Use of Many Covariates in Clinical Trials
Tukey, J. W. (1991) · International Statistical Review · doi:10.2307/1403439 · Cited by 27 · see citing works
Bivariate Regression Analysis is Useless
Ehrenberg, A. S. C. (1963) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2985794 · Cited by 26 · see citing works
Quality Improvement: The New Industrial Revolution
Box, G. E. P. (1993) · International Statistical Review · doi:10.2307/1403590 · Cited by 24 · see citing works
Forecasting with Misspecified Models
Davies, N., & Newbold, P. (1980) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2346415 · Cited by 24 · see citing works
Ehrenberg, A. S. C. (1982) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981869 · Cited by 23 · see citing works
Smith, M. D. (1993) · Australian Journal of Statistics · doi:10.1111/j.1467-842x.1993.tb01335.x · Cited by 22 · see citing works
Long-term inference based on short-term forecasting models
Newbold, P., Agiakloglou, C., & Miller, J. I. (1993) · Springer eBooks · doi:10.1007/978-1-4899-4515-0_2 · Cited by 19 · see citing works
Teaching a Course in Applied Statistics
Chatfield, C. (1982) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2348001 · Cited by 19 · see citing works
Self-citationOn Model Selection Based on Validation with Applications to Pressure and Temperature Prognosis
Hjorth, U., & Holmqvist, L. (1981) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2346350 · Cited by 18 · see citing works
The effect of preliminary normality goodness of fit tests on subsequent inference.
Easterling, R. G., & Anderson, H. E. (1978) · Journal of Statistical Computation and Simulation · doi:10.1080/00949657808810243 · Cited by 17 · see citing works
Clustering Algorithms and Cluster Validation
Gordon, A. D. (1994) · Contributions to statistics · doi:10.1007/978-3-642-57991-2_29 · Cited by 17 · see citing works
Cohen, A., & Sackrowitz, H. B. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478549 · Cited by 14 · see citing works
Model Selection in Multinomial Experiments
de Leeuw, J. (1988) · Lecture notes in economics and mathematical systems · doi:10.1007/978-3-642-61564-1_7 · Cited by 13 · see citing works
The use of biased predictors in marketing research
Hill, R. C., Cartwright, P. A., & Arbaugh, J. F. (1991) · International Journal of Forecasting · doi:10.1016/0169-2070(91)90002-d · Cited by 12 · see citing works
The Shrinkage of Point Scoring Methods
Copas, J. B. (1993) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986235 · Cited by 8 · see citing works
Manpower Planning in the Face of Uncertainty
Bartholomew, D. J., Hopes, R. F. A., & Smith, A. R. (1976) · Personnel Review · doi:10.1108/eb055311 · Cited by 7 · see citing works
Regression Shrinkage Methods and Autoregressive Time Series Prediction
Copas, J. B., & Jones, M. C. (1987) · Australian Journal of Statistics · doi:10.1111/j.1467-842x.1987.tb00744.x · Cited by 5 · see citing works
Steerneman, T., & Rorijs, G. (1988) · Lecture notes in economics and mathematical systems · doi:10.1007/978-3-642-61564-1_6 · Cited by 4 · see citing works
Evaluating the impact of exploratory procedures in regression prediction
Kipnis, V. (1991) · Computational Statistics & Data Analysis · doi:10.1016/0167-9473(91)90102-8 · Cited by 4 · see citing works
Alternative Statistical Approaches to Time Series Modelling for Forecasting Purposes
Marriott, J. M., & Tremayne, A. R. (1988) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348692 · Cited by 4 · see citing works
A Sequential Approach to Testing for Structural Change in Econometric Models
Phillips, G. D. A., & McCabe, B. P. M. (1989) · Physica-Verlag eBooks · doi:10.1007/978-3-642-48412-4_8 · Cited by 3 · see citing works
Flagged first, then by citations
1 of the 117 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Dormann, C. F., Elith, J., Bacher, S., Buchmann, C. M., Carl, G., Carré, G., Marquéz, J. R. G., Gruber, B., Lafourcade, B., Leitão, P. J., et al. (2012) · Ecography · doi:10.1111/j.1600-0587.2012.07348.x · Cited by 11,116 · see citing works
A review of methods for the assessment of prediction errors in conservation presence/absence models
FIELDING, A. H., & BELL, J. F. (1997) · Environmental Conservation · doi:10.1017/s0376892997000088 · Cited by 6,956 · see citing works
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · see citing works
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118 · see citing works
Posada, D., & Buckley, T. R. (2004) · Systematic Biology · doi:10.1080/10635150490522304 · Cited by 3,927 · see citing works
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746 · see citing works
Evaluating resource selection functions
Boyce, M. S., Vernier, P. R., Nielsen, S. E., & Schmiegelow, F. K. A. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00200-4 · Cited by 2,580 · see citing works
A brief introduction to mixed effects modelling and multi-model inference in ecology
Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018) · PeerJ · doi:10.7717/peerj.4794 · Cited by 2,094 · see citing works
POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES
Gelman, A., Meng, X., & Stern, H. S. (1996) · Cited by 1,982 · see citing works
A Reality Check for Data Snooping
White, H. (2000) · Econometrica · doi:10.1111/1468-0262.00152 · Cited by 1,851 · see citing works
Assessment and Propagation of Model Uncertainty
Draper, D. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02015.x · Cited by 1,599 · see citing works
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · see citing works
Model Selection: An Integral Part of Inference
Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997) · Biometrics · doi:10.2307/2533961 · Cited by 1,717 · see citing works
Handbook of Economic Forecasting
(2013) · Handbook of economic forecasting · doi:10.1016/b978-0-444-53683-9.00023-2 · Cited by 1,290 · see citing works
A meta‐analysis of context‐dependency in plant response to inoculation with mycorrhizal fungi
Hoeksema, J. D., Chaudhary, V. B., Gehring, C. A., Johnson, N. C., Karst, J., Koide, R. T., Pringle, A., Zabinski, C. A., Bever, J. D., Moore, J. C., et al. (2010) · Ecology Letters · doi:10.1111/j.1461-0248.2009.01430.x · Cited by 1,208 · see citing works
Forstmeier, W., & Schielzeth, H. (2010) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1038-5 · Cited by 1,117 · see citing works
Using Narrative in Social Research
Elliott, J. (2005) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020246 · Cited by 1,114 · see citing works
The use of fractional polynomials to model continuous risk variables in epidemiology
Royston, P., Ambler, G., & Sauerbrei, W. (1999) · International Journal of Epidemiology · doi:10.1093/ije/28.5.964 · Cited by 1,066 · see citing works
Methods and uncertainties in bioclimatic envelope modelling under climate change
Heikkinen, R. K., Luoto, M., Araújo, M. B., Virkkala, R., Thuiller, W., & Sykes, M. T. (2006) · Progress in Physical Geography Earth and Environment · doi:10.1177/0309133306071957 · Cited by 995 · see citing works
Exponential smoothing: The state of the art—Part II
Gardner, E. S. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.03.005 · Cited by 954 · see citing works
A calibration hierarchy for risk models was defined: from utopia to empirical data
Van Calster, B., Nieboer, D., Vergouwe, Y., De Cock, B., Pencina, M. J., & Steyerberg, E. W. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.005 · Cited by 910 · see citing works
Statistical strategies for avoiding false discoveries in metabolomics and related experiments
Broadhurst, D., & Kell, D. B. (2006) · Metabolomics · doi:10.1007/s11306-006-0037-z · Cited by 819 · see citing works
Ellison, A. M. (2004) · Ecology Letters · doi:10.1111/j.1461-0248.2004.00603.x · Cited by 791 · see citing works
Strategies for comparing treatments on a binary response with multi-centre data
Agresti, A., & Hartzel, J. S. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8 · Cited by 102 · see citing works
Sensitivity and specificity of information criteria
(2019) · doi:10.1093/bib/bbz016 · Cited by 718 · see citing works
Global Change and Human Vulnerability to Vector-Borne Diseases
Sutherst, R. W. (2004) · Clinical Microbiology Reviews · doi:10.1128/cmr.17.1.136-173.2004 · Cited by 714 · see citing works
League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance
Goldstein, H., & Spiegelhalter, D. J. (1996) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983325 · Cited by 1,069 · see citing works
Linear regression and the normality assumption
Schmidt, A. F., & Finan, C. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.006 · Cited by 702 · see citing works
Frequentist Model Average Estimators
Hjort, N. L., & Claeskens, G. (2003) · Journal of the American Statistical Association · doi:10.1198/016214503000000828 · Cited by 689 · see citing works
The Multi-dimensional Analysis of Social Exclusion
Levitas, R., Pantazis, C., Fahmy, E., Gordon, D., Lloyd, E., & Patsios, D. (2007) · UEL Research Repository (University of East London) · Cited by 678 · see citing works
Marine reserves: size and age do matter
Claudet, J., Osenberg, C. W., Benedetti‐Cecchi, L., Domenici, P., García‐Charton, J. A., Pérez‐Ruzafa, Á., Badalamenti, F., Bayle-Sempere, J. T., Brito, A., Bulleri, F., et al. (2008) · Ecology Letters · doi:10.1111/j.1461-0248.2008.01166.x · Cited by 656 · see citing works
Assessing transferability of ecological models: an underappreciated aspect of statistical validation
Wenger, S. J., & Olden, J. D. (2012) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2011.00170.x · Cited by 655 · see citing works
Vergouwe, Y., Steyerberg, E. W., Eijkemans, M. J. C., & Habbema, J. D. F. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.06.017 · Cited by 636 · see citing works
Econometrics: Alchemy or Science?
Hendry, D. F. (2000) · doi:10.1093/0198293542.001.0001 · Cited by 635 · see citing works
Steyerberg, E. W., Bleeker, S. E., Moll, H. A., Grobbee, D. E., & Moons, K. G. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00047-7 · Cited by 611 · see citing works
Preoperative prediction of severe postoperative pain
Kalkman, J. C., Visser, K., Moen, J., Bonsel, J. G., Grobbee, E. D., & Moons, M. K. G. (2003) · Pain · doi:10.1016/s0304-3959(03)00252-5 · Cited by 616 · see citing works
Practical identifiability analysis of large environmental simulation models
Brun, R., Reichert, P., & Künsch, H. R. (2001) · Water Resources Research · doi:10.1029/2000wr900350 · Cited by 609 · see citing works
Measurements and analysis of end-to-end Internet dynamics
Lawrence Berkeley National Lab. (LBNL), B., & Paxson, V. (1997) · doi:10.2172/551971 · Cited by 572 · see citing works
The role of sensitivity analysis in ecological modelling
Cariboni, J., Gatelli, D., Liška, R., & Saltelli, A. (2007) · Ecological Modelling · doi:10.1016/j.ecolmodel.2005.10.045 · Cited by 564 · see citing works
Prognostic Modeling with Logistic Regression Analysis
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2001) · Medical Decision Making · doi:10.1177/0272989x0102100106 · Cited by 559 · see citing works
CLIMATE, HABITAT QUALITY, AND FITNESS IN NORTHERN SPOTTED OWL POPULATIONS IN NORTHWESTERN CALIFORNIA
Franklin, A. B., Anderson, D. R., Gutiérrez, R. J., & Burnham, K. P. (2000) · Ecological Monographs · doi:10.1890/0012-9615(2000)070[0539:chqafi]2.0.co;2 · Cited by 535 · see citing works
Palosuo, T., Kersebaum, K. C., Angulo, C., Hlavinka, P., Moriondo, M., Olesen, J. E., Patil, R. H., Ruget, F., Rumbaur, C., Takáč, J., et al. (2011) · European Journal of Agronomy · doi:10.1016/j.eja.2011.05.001 · Cited by 512 · see citing works
Stepwise Selection in Small Data Sets A Simulation Study of Bias in Logistic Regression Analysis
Steyerberg, E. W. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00103-1 · Cited by 489 · see citing works
A Bayesian network of eutrophication models for synthesis, prediction, and uncertainty analysis
Borsuk, M. E., Stow, C. A., & Reckhow, K. H. (2003) · Ecological Modelling · doi:10.1016/j.ecolmodel.2003.08.020 · Cited by 494 · see citing works
Data mining reconsidered: encompassing and the general‐to‐specific approach to specification search
Hoover, K. D., & Perez, S. J. (1999) · Econometrics Journal · doi:10.1111/1368-423x.00025 · Cited by 488 · see citing works
An Introduction to Model Selection
Zucchini, W. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1276 · Cited by 466 · see citing works
Palgrave Handbook of Econometrics
(2009) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230244405 · Cited by 448 · see citing works
Manel, S., Dias, J.-M., & Ormerod, S. J. (1999) · Ecological Modelling · doi:10.1016/s0304-3800(99)00113-1 · Cited by 435 · see citing works
Hendry, D. F., & Mizon, G. E. (2000) · doi:10.1093/0198293542.003.0007 · Cited by 430 · see citing works
Mazerolle, M. J. (2006) · Amphibia-Reptilia · doi:10.1163/156853806777239922 · Cited by 431 · see citing works
Sampling for validation of digital soil maps
Brus, D. J., Kempen, B., & Heuvelink, G. B. M. (2011) · European Journal of Soil Science · doi:10.1111/j.1365-2389.2011.01364.x · Cited by 416 · see citing works
Newbold, T. (2010) · Progress in Physical Geography Earth and Environment · doi:10.1177/0309133309355630 · Cited by 404 · see citing works
MODEL WEIGHTS AND THE FOUNDATIONS OF MULTIMODEL INFERENCE
Link, W. A., & Barker, R. (2006) · Ecology · doi:10.1890/0012-9658(2006)87[2626:mwatfo]2.0.co;2 · Cited by 400 · see citing works
Ecology of butterflies in Europe
(2010) · Choice Reviews Online · doi:10.5860/choice.47-5648 · Cited by 399 · see citing works
Goodacre, R., Timmins, É. M., Burton, R., Kaderbhai, N., Woodward, A. M., Kell, D. B., & Rooney, P. J. (1998) · Microbiology · doi:10.1099/00221287-144-5-1157 · Cited by 389 · see citing works
The Focused Information Criterion
Claeskens, G., & Hjort, N. L. (2003) · Journal of the American Statistical Association · doi:10.1198/016214503000000819 · Cited by 381 · see citing works
Novikov, D. S., Kiselev, V. G., & Jespersen, S. N. (2018) · Magnetic Resonance in Medicine · doi:10.1002/mrm.27101 · Cited by 362 · see citing works
Information theory and hypothesis testing: a call for pluralism
Stephens, P. A., Buskirk, S. W., Hayward, G. D., & del Rio, C. M. (2005) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2005.01002.x · Cited by 361 · see citing works
TEMPORAL AND SPATIAL VARIATION IN ENVIRONMENTAL IMPACT ASSESSMENT
Stewart‐Oaten, A., & Bence, J. R. (2001) · Ecological Monographs · doi:10.1890/0012-9615(2001)071[0305:tasvie]2.0.co;2 · Cited by 358 · see citing works
False Discovery Rates in PET and CT Studies with Texture Features: A Systematic Review
Chalkidou, A., O'Doherty, M. J., & Marsden, P. K. (2015) · PLoS ONE · doi:10.1371/journal.pone.0124165 · Cited by 344 · see citing works
A simulation study of artificial neural networks for nonlinear time-series forecasting
Zhang, G. P., Patuwo, B. E., & Hu, M. Y. (2001) · Computers & Operations Research · doi:10.1016/s0305-0548(99)00123-9 · Cited by 336 · see citing works
Modelling fish growth: Model selection, multi-model inference and model selection uncertainty
Katsanevakis, S. (2006) · Fisheries Research · doi:10.1016/j.fishres.2006.07.002 · Cited by 334 · see citing works
Practical Regression and Anova using R
Faraway, J. J. (2002) · Cited by 331 · see citing works
Elith, J. (2006) · Springer eBooks · doi:10.1007/0-387-22648-6_4 · Cited by 326 · see citing works
Model Averaging and Its Use in Economics
Steel, M. F. J. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191385 · Cited by 321 · see citing works
The continuing challenges of testing species distribution models
Vaughan, I., & Ormerod, S. (2005) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2005.01052.x · Cited by 341 · see citing works
Alternative methods for predicting species distribution: an illustration with Himalayan river birds
Manel, S., Dias, J.-M., Buckton, S. T., & Ormerod, S. J. (1999) · Journal of Applied Ecology · doi:10.1046/j.1365-2664.1999.00440.x · Cited by 312 · see citing works
The Use of Resampling Methods to Simplify Regression Models in Medical Statistics
Sauerbrei, W. (1999) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00155 · Cited by 310 · see citing works
Wintle, B. A., Elith, J., & Potts, J. (2005) · Austral Ecology · doi:10.1111/j.1442-9993.2005.01514.x · Cited by 306 · see citing works
Species distribution models rarely predict the biology of real populations
Lee‐Yaw, J. A., McCune, J. L., Pironon, S., & Sheth, S. N. (2021) · Ecography · doi:10.1111/ecog.05877 · Cited by 302 · see citing works
Statistical and integrative system-level analysis of DNA methylation data
Teschendorff, A. E., & Relton, C. L. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2017.86 · Cited by 292 · see citing works
Marine reserves: Fish life history and ecological traits matter
Claudet, J., Osenberg, C. W., Domenici, P., Badalamenti, F., Milazzo, M., FALCÓN, J. M., Bertocci, I., Benedetti‐Cecchi, L., García‐Charton, J. A., Goñi, R., et al. (2010) · Ecological Applications · doi:10.1890/08-2131.1 · Cited by 289 · see citing works
Heavy tail modeling and teletraffic data: special invited paper
Resnick, S. I. (1997) · The Annals of Statistics · doi:10.1214/aos/1069362376 · Cited by 288 · see citing works
Forecasting and operational research: a review
Fildes, R., Nikolopoulos, K., Crone, S. F., & Syntetos, A. A. (2008) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2602597 · Cited by 287 · see citing works
The Use of Bayesian Model Averaging to Better Represent Uncertainty in Ecological Models
Wintle, B. A., Mccarthy, M. A., Volinsky, C. T., & Kavanagh, R. P. (2003) · Conservation Biology · doi:10.1111/j.1523-1739.2003.00614.x · Cited by 286 · see citing works
Katsanevakis, S., & Maravelias, C. D. (2008) · Fish and Fisheries · doi:10.1111/j.1467-2979.2008.00279.x · Cited by 278 · see citing works
Real longitudinal data analysis for real people: Building a good enough mixed model
Cheng, J., Edwards, L. J., Maldonado‐Molina, M. M., Komro, K. A., & Muller, K. E. (2009) · Statistics in Medicine · doi:10.1002/sim.3775 · Cited by 276 · see citing works
Statistical Foundations for Model-Based Adjustments
Greenland, S., & Pearce, N. E. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031914-122559 · Cited by 274 · see citing works
Mapping epistemic uncertainties and vague concepts in predictions of species distribution
Elith, J., Burgman, M. A., & Regan, H. M. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00202-8 · Cited by 273 · see citing works
More on a Stochastic Asset Model for Actuarial Use
Wilkie, A. D. (1995) · British Actuarial Journal · doi:10.1017/s1357321700001331 · Cited by 271 · see citing works
Guidelines for Computer Modeling of Diabetes and Its Complications
American Diabetes Association Consensus Panel (2004) · Diabetes Care · doi:10.2337/diacare.27.9.2262 · Cited by 270 · see citing works
Cluster and Classification Techniques for the Biosciences
Fielding, A. H. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511607493 · Cited by 268 · see citing works
Variable selection and Bayesian model averaging in case‐control studies
Viallefont, V., Raftery, A. E., & Richardson, S. T. (2001) · Statistics in Medicine · doi:10.1002/sim.976 · Cited by 263 · see citing works
initiative, F. T. O. T. S., Sauerbrei, W., Perperoglou, A., Schmid, M. C., Abrahamowicz, M., Becher, H., Binder, H., Dunkler, D., Harrell, F. E., Royston, P. J., et al. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00074-3 · Cited by 270 · see citing works
An investigation of model selection criteria for neural network time series forecasting
Qi, M., & Zhang, G. P. (2001) · European Journal of Operational Research · doi:10.1016/s0377-2217(00)00171-5 · Cited by 258 · see citing works
Brooks, J. S., Waylen, K. A., & Mulder, M. B. (2013) · Environmental Evidence · doi:10.1186/2047-2382-2-2 · Cited by 253 · see citing works
Assessing the Value of Risk Predictions by Using Risk Stratification Tables
Janes, H., PEPE, M. S., & Gu, W. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-10-200811180-00009 · Cited by 249 · see citing works
Anderson, D. R., Burnham, K. P., & White, G. C. (1998) · Journal of Applied Statistics · doi:10.1080/02664769823250 · Cited by 311 · see citing works
Should scoring rules be based on odds ratios or regression coefficients?
Moons, K. G. M., Harrell, F. E., & Steyerberg, E. W. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00453-5 · Cited by 241 · see citing works
Statistical Issues and Assumptions of Phylogenetic Generalized Least Squares
Mundry, R. (2014) · Springer eBooks · doi:10.1007/978-3-662-43550-2_6 · Cited by 236 · see citing works
Where Mathematics meets the Internet
Willinger, W., & Paxson, V. (1998) · Cited by 232 · see citing works
de Silva, R., Nikitin, N. P., Witte, K. K., Rigby, A. S., Goode, K. M., Bhandari, S., Clark, A. L., & Cleland, J. G. F. (2005) · European Heart Journal · doi:10.1093/eurheartj/ehi696 · Cited by 227 · see citing works
Bayesian Model Averaging in Proportional Hazard Models: Assessing the Risk of a Stroke
Volinsky, C. T., Madigan, D., Raftery, A. E., & Kronmal, R. A. (1997) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00082 · Cited by 226 · see citing works
GemPy 1.0: open-source stochastic geological modeling and inversion
de la Varga, M., Schaaf, A., & Wellmann, F. (2019) · Geoscientific model development · doi:10.5194/gmd-12-1-2019 · Cited by 226 · see citing works
The significance and prediction of predation in marine fisheries
Bax, N. J. (1998) · ICES Journal of Marine Science · doi:10.1006/jmsc.1998.0350 · Cited by 225 · see citing works
Methods of Covariate Selection: Directed Acyclic Graphs and the Change-in-Estimate Procedure
Weng, H., Hsueh, Y.-H., Messam, L. L. M., & Hertz‐Picciotto, I. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp035 · Cited by 223 · see citing works
COMPONENTS OF UNCERTAINTY IN SPECIES DISTRIBUTION ANALYSIS: A CASE STUDY OF THE GREAT GREY SHRIKE
Dormann, C. F., Purschke, O., Márquez, J. R. G., Lautenbach, S., & Schröder, B. (2008) · Ecology · doi:10.1890/07-1772.1 · Cited by 222 · see citing works
Multivariable Modeling Strategies
Harrell, F. E. (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7_4 · Cited by 221 · see citing works
Models and model uncertainty in the context of risk analysis
Nilsen, T., & Aven, T. (2003) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(02)00239-9 · Cited by 221 · see citing works
COMBINING FORECASTING PROCEDURES: SOME THEORETICAL RESULTS
Yang, Y. (2004) · Econometric Theory · doi:10.1017/s0266466604201086 · Cited by 221 · see citing works
- Authors
- Chatfield, C.
- Affiliations
- University of Bath (GB)
- Corresponding author
- Chatfield, Chris
- Venue
- Journal of the Royal Statistical Society. Series A (Statistics in Society)
- Publisher
- Oxford University Press (OUP)
- Published
- 1995
- Volume
- 158
- Issue
- 3
- Pages
- 419
- ISSN
- 0964-1998 (ISSN-L)
- OpenAlex
- W2060050130
- Cited by
- 1,103
- References
- 117
- Citations by year
- 2026: 18 · 2025: 34 · 2024: 38 · 2023: 33 · 2022: 35 · 2021: 33 · 2020: 44 · 2019: 28 · 2018: 47 · 2017: 35 · 2016: 37 · 2015: 51 · 2014: 52 · 2013: 28 · 2012: 63
- FWCI
- 24.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the Royal Statistical Society Series A (Statistics in Society) (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- model uncertainty, data mining, statistical inference, model selection, prediction intervals, estimation bias, Bayesian model averaging, resampling, simulation, model misspecification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review