The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
The Estimation of Prediction Error: Covariance Penalties and Cross-Validation
Abstract
Having constructed a data-based estimation rule, perhaps a logistic regression or a classification tree, the statistician would like to know its performance as a predictor of future cases. There are two main theories concerning prediction error: (1) penalty methods such as Cp, Akaike's information criterion, and Stein's unbiased risk estimate that depend on the covariance between data points and their corresponding predictions; and (2) cross-validation and related nonparametric bootstrap techniques. This article concerns the connection between the two theories. A Rao–Blackwell type of relation is derived in which nonparametric methods such as cross-validation are seen to be randomized versions of their covariance penalty counterparts. The model-based penalty methods offer substantially better accuracy, assuming that the model is believable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
Robust Statistics: The Approach Based on Influence Functions
Ruppert, D. (1987) · Technometrics · doi:10.2307/1269782 · Cited by 3,880
Estimation of the Mean of a Multivariate Normal Distribution
Stein, C. M. (1981) · The Annals of Statistics · doi:10.1214/aos/1176345632 · Cited by 2,729
Estimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300
Self-citationMallows, C. L. (1973) · Technometrics · doi:10.2307/1267380 · Cited by 1,975
Improvements on Cross-Validation: The 632+ Bootstrap Method
Efron, B., & Tibshirani, R. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474007 · Cited by 1,636
Self-citationMallows, C. L. (1995) · Technometrics · doi:10.2307/1269729 · Cited by 1,080
Defining the Curvature of a Statistical Problem (with Applications to Second Order Efficiency)
Efron, B. (1975) · The Annals of Statistics · doi:10.1214/aos/1176343282 · Cited by 598
Self-citationHow Biased is the Apparent Error Rate of a Prediction Rule?
Efron, B. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478291 · Cited by 587
Self-citationThe Efficiency of Logistic Regression Compared to Normal Discriminant Analysis
Efron, B. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10480319 · Cited by 567
Self-citation
Cited by
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Self-citationA survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0 · Cited by 3,289
Estimation of prediction error by using K-fold cross-validation
Fushiki, T. (2011) · Statistics and Computing · doi:10.1007/s11222-009-9153-8 · Cited by 1,470
Prediction error estimation: a comparison of resampling methods
Molinaro, A. M., Simon, R., & Pfeiffer, R. M. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti499 · Cited by 1,339
Gentzkow, M., Kelly, B., & Taddy, M. (2019) · Journal of Economic Literature · doi:10.1257/jel.20181020 · Cited by 1,236
Truccolo, W., Eden, U. T., Fellows, M. R., Donoghue, J. P., & Brown, E. N. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00697.2004 · Cited by 1,049
Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models
Xu, Y. (2017) · Political Analysis · doi:10.1017/pan.2016.2 · Cited by 938
On the “degrees of freedom” of the lasso
Zou, H., Hastie, T., & Tibshirani, R. (2007) · The Annals of Statistics · doi:10.1214/009053607000000127 · Cited by 947
The composite absolute penalties family for grouped and hierarchical variable selection
Zhao, P., Rocha, G., & Yu, B. (2009) · The Annals of Statistics · doi:10.1214/07-aos584 · Cited by 631
Prediction as a Humanitarian and Pragmatic Contribution from Human Cognitive Neuroscience
Gabrieli, J. D. E., Ghosh, S. S., & Whitfield-Gabrieli, S. (2015) · Neuron · doi:10.1016/j.neuron.2014.10.047 · Cited by 535
Cross-Validation: What Does It Estimate and How Well Does It Do It?
Bates, S., Hastie, T., & Tibshirani, R. (2024) · Journal of the American Statistical Association · doi:10.1080/01621459.2023.2197686 · Cited by 442
Evaluating Prediction Rules for t -Year Survivors With Censored Regression Models
Uno, H., Cai, T., Tian, L., & Wei, L. J. (2007) · Journal of the American Statistical Association · doi:10.1198/016214507000000149 · Cited by 413
Computer Aided Diagnostic Support System for Skin Cancer: A Review of Techniques and Algorithms
Masood, A., & Ali Al-Jumaily, A. (2013) · International Journal of Biomedical Imaging · doi:10.1155/2013/323268 · Cited by 347
A robust coefficient of determination for regression
Renaud, O., & Victoria-Feser, M.-P. (2010) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2010.01.008 · Cited by 330
Li, Y., & Zhu, J. (2008) · Journal of Computational and Graphical Statistics · doi:10.1198/106186008x289155 · Cited by 316
‘All models are wrong...’: an introduction to model uncertainty
Wit, E., Heuvel, E. V. D., & Romeijn, J. (2012) · Statistica Neerlandica · doi:10.1111/j.1467-9574.2012.00530.x · Cited by 315
Machine Learning in Materials Science: Recent Progress and Emerging Applications
Mueller, T., Kusne, A. G., & Ramprasad, R. (2016) · Reviews in Computational Chemistry · doi:10.1002/9781119148739.ch4 · Cited by 295
Oracle inequalities for multi-fold cross validation
Vaart, A. W. V. D., Dudoit, S., & Laan, M. J. V. D. (2006) · Statistics & Decisions · doi:10.1524/stnd.2006.24.3.351 · Cited by 156
(1998) · doi:10.1007/3-540-26577-5_17 · Cited by 15
Metadata
- Authors
- Efron, B.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2004-09
- Volume
- 99
- Issue
- 467
- Pages
- 619-632
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W1502338185
- Cited by
- 467
- References
- 16
- FWCI
- 13.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.