The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Linear Model Selection by Cross-validation
Abstract
We consider the problem of selecting a model having the best predictive ability among a class of linear models. The popular leave-one-out cross-validation method, which is asymptotically equivalent to many other model selection methods such as the Akaike information criterion (AIC), the C p , and the bootstrap, is asymptotically inconsistent in the sense that the probability of selecting the model with the best predictive ability does not converge to 1 as the total number of observations n → ∞. We show that the inconsistency of the leave-one-out cross-validation can be rectified by using a leave-n v -out cross-validation with n v , the number of observations reserved for validation, satisfying n v /n → 1 as n → ∞. This is a somewhat shocking discovery, because nv/n → 1 is totally opposite to the popular leave-one-out recipe in cross-validation. Motivations, justifications, and discussions of some practical aspects of the use of the leave-n v -out cross-validation method are provided, and results from a simulation study are presented.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The statistical literature suggests that leave-one-out methods overestimate performance, instead recommending that larger hold-outs be used [73-75].”
From Modeling causes of death: an integrated approach using CODEm (2012) · cites this paper as [75]
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881
The Predictive Sample Reuse Method with Applications
Geisser, S. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479865 · Cited by 2,748
Estimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300
Mallows, C. L. (1995) · Technometrics · doi:10.1080/00401706.1995.10484370 · Cited by 2,083
Mallows, C. L. (1973) · Technometrics · doi:10.2307/1267380 · Cited by 1,975
Cross-Validation of Regression Models
Picard, R. R., & Cook, R. D. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478083 · Cited by 1,547
The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
Allen, D. M. (1974) · Technometrics · doi:10.1080/00401706.1974.10489157 · Cited by 1,469
An asymptotic equivalence of choice of model by cross validation and Akaike's criterion
Stone, M. (1977) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1977.tb01603.x · Cited by 1,286
BURMAN, P. (1989) · Biometrika · doi:10.1093/biomet/76.3.503 · Cited by 634
Research works printed but not linked: 3
- Gunst G. F., 1980, Regression Analysis and Its Application
- Herzberg G., 1986, Utilitas Mathematics, 29, 209
- Shorack G. R., 1986, Empirical Process with Applications to Statistics
Cited by
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
PLS-regression: a basic tool of chemometrics
Wold, S., Sjöström, M., & Eriksson, L. (2001) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(01)00155-1 · Cited by 10,084
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Cross‐validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure
Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera‐Arroita, G., Hauenstein, S., Lahoz‐Monfort, J. J., Schröder, B., Thuiller, W., et al. (2017) · Ecography · doi:10.1111/ecog.02881 · Cited by 3,689
Metadata
- Authors
- Shao, J.
- Affiliations
- University of Ottawa (CA)
- Corresponding author
- Shao, Jun
- ORCID
- Shao, Jun
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1993-06
- Volume
- 88
- Issue
- 422
- Pages
- 486-494
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2033872649
- Cited by
- 1,790
- References
- 20
- Citations by year
- 2026: 32 · 2025: 59 · 2024: 65 · 2023: 59 · 2022: 82 · 2021: 110 · 2020: 99 · 2019: 86 · 2018: 95 · 2017: 87 · 2016: 84 · 2015: 90 · 2014: 66 · 2013: 70 · 2012: 69
- FWCI
- 18.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- model selection, leave-one-out cross-validation, predictive power, linear models, Akaike information criterion, bootstrapping, asymptotic consistency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Linear Model Selection by Cross-validation. https://app.cochat.ai/papers/10.1080/01621459.1993.10476299
- The paper
- Shao, J. (1993). Linear Model Selection by Cross-validation. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1993.10476299
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.