The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
The Estimation of Prediction Error: Covariance Penalties and Cross-Validation
Abstract
Having constructed a data-based estimation rule, perhaps a logistic regression or a classification tree, the statistician would like to know its performance as a predictor of future cases. There are two main theories concerning prediction error: (1) penalty methods such as Cp, Akaike's information criterion, and Stein's unbiased risk estimate that depend on the covariance between data points and their corresponding predictions; and (2) cross-validation and related nonparametric bootstrap techniques. This article concerns the connection between the two theories. A Rao–Blackwell type of relation is derived in which nonparametric methods such as cross-validation are seen to be randomized versions of their covariance penalty counterparts. The model-based penalty methods offer substantially better accuracy, assuming that the model is believable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
Robust Statistics: The Approach Based on Influence Functions
Ruppert, D. (1987) · Technometrics · doi:10.2307/1269782 · Cited by 3,880
Estimation of the Mean of a Multivariate Normal Distribution
Stein, C. M. (1981) · The Annals of Statistics · doi:10.1214/aos/1176345632 · Cited by 2,729
Estimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300
Self-citationMallows, C. L. (1973) · Technometrics · doi:10.2307/1267380 · Cited by 1,975
Improvements on Cross-Validation: The 632+ Bootstrap Method
Efron, B., & Tibshirani, R. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474007 · Cited by 1,636
Self-citationMallows, C. L. (1995) · Technometrics · doi:10.2307/1269729 · Cited by 1,080
Defining the Curvature of a Statistical Problem (with Applications to Second Order Efficiency)
Efron, B. (1975) · The Annals of Statistics · doi:10.1214/aos/1176343282 · Cited by 598
Self-citationHow Biased is the Apparent Error Rate of a Prediction Rule?
Efron, B. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478291 · Cited by 587
Self-citationThe Efficiency of Logistic Regression Compared to Normal Discriminant Analysis
Efron, B. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10480319 · Cited by 567
Self-citation
Cited by
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Self-citationA survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0 · Cited by 3,289
Estimation of prediction error by using K-fold cross-validation
Fushiki, T. (2011) · Statistics and Computing · doi:10.1007/s11222-009-9153-8 · Cited by 1,470
Prediction error estimation: a comparison of resampling methods
Molinaro, A. M., Simon, R., & Pfeiffer, R. M. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti499 · Cited by 1,339
Metadata
- Authors
- Efron, B.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2004-09
- Volume
- 99
- Issue
- 467
- Pages
- 619-632
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W1502338185
- Cited by
- 467
- References
- 16
- FWCI
- 13.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Estimation of Prediction Error: Covariance Penalties and Cross-Validation. https://app.cochat.ai/papers/10.1198/016214504000000692
- The paper
- Efron, B. (2004). The Estimation of Prediction Error: Covariance Penalties and Cross-Validation. Journal of the American Statistical Association. https://doi.org/10.1198/016214504000000692
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.