Multiple predicting K -fold cross-validation for model selection
The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
K-fold cross-validation (CV) is widely adopted as a model selection criterion. In K-fold CV, (K−1) folds are used for model construction and the hold-out fold is allocated to model validation. This implies model construction is more emphasised than the model validation procedure. However, some studies have revealed that more emphasis on the validation procedure may result in improved model selection. Specifically, leave-m-out CV with n samples may achieve variable-selection consistency when m/n approaches to 1. In this study, a new CV method is proposed within the framework of K-fold CV. The proposed method uses (K−1) folds of the data for model validation, while the other fold is for model construction. This provides (K−1) predicted values for each observation. These values are averaged to produce a final predicted value. Then, the model selection based on the averaged predicted values can reduce variation in the assessment due to the averaging. The variable-selection consistency of the suggested method is established. Its advantage over K-fold CV with finite samples are examined under linear, non-linear, and high-dimensional models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M. L., Downing, J. R., Caligiuri, M. A., et al. (1999) · Science · doi:10.1126/science.286.5439.531 · Cited by 11,516
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Fan, J., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,900
The adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470 · cited 1×
“…Hastie, and Tibshirani (2010), Simon, Friedman, Hastie, and Tibshirani (2011), glmpath for Park and Hastie (2007), genlasso for Tibshirani and Taylor (2011), lqa for Zou (2006), elasticnet for Zou and Hastie (2005), ncvreg for Fan and Li (2001), and quantreg for Wu and Liu (2009) among many others.”
BackgroundA survey of cross-validation procedures for model selection
Arlot, S., & Célisse, A. (2010) · LillOA (Université de Lille (University Of Lille)) · Cited by 3,663
A survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522 · cited 1×
“An exhaustive literature review is provided by Arlot and Celisse (2010).”
BackgroundThe Predictive Sample Reuse Method with Applications
Geisser, S. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479865 · Cited by 2,748 · cited 1×
“To reduce computational cost of LOOCV, Geisser (1975) proposed K -fold CV.”
MethodRegularization Paths for Cox's Proportional Hazards Model via Coordinate Descent
Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2011) · Journal of Statistical Software · doi:10.18637/jss.v039.i05 · Cited by 2,499
Extended Bayesian information criteria for model selection with large model spaces
Chen, J., & Chen, Z. (2008) · Biometrika · doi:10.1093/biomet/asn034 · Cited by 2,011
Linear Model Selection by Cross-validation
Shao, J. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476299 · Cited by 1,790
Improvements on Cross-Validation: The 632+ Bootstrap Method
Efron, B., & Tibshirani, R. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474007 · Cited by 1,636
Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix
Sherman, J., & Morrison, W. J. (1950) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729893 · Cited by 1,402
Asymptotics for lasso-type estimators
Fu, W., & Knight, K. (2000) · The Annals of Statistics · doi:10.1214/aos/1015957397 · Cited by 1,276
On Model Selection Consistency of Lasso
ZhaoPeng, & Yubin, Y. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248637 · Cited by 996
L 1-Regularization Path Algorithm for Generalized Linear Models
Park, M. Y., & Hastie, T. (2007) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00607.x · Cited by 838
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
BURMAN, P. (1989) · Biometrika · doi:10.1093/biomet/76.3.503 · Cited by 634
Unifying the derivations for the Akaike and corrected Akaike information criteria
Cavanaugh, J. E. (1997) · Statistics & Probability Letters · doi:10.1016/s0167-7152(96)00128-9 · Cited by 626
The solution path of the generalized lasso
Tibshirani, R. J., & Taylor, J. (2011) · The Annals of Statistics · doi:10.1214/11-aos878 · Cited by 553
Model Selection Via Multifold Cross Validation
Zhang, P. (1993) · The Annals of Statistics · doi:10.1214/aos/1176349027 · Cited by 539 · cited 5×
“The details are in Corollary 1 of Zhang (1993).”
BackgroundMethodInfluentialThe sparsity and bias of the Lasso selection in high-dimensional linear regression
Zhang, C.-H., & Huang, J. (2008) · The Annals of Statistics · doi:10.1214/07-aos520 · Cited by 506
Shrinkage Tuning Parameter Selection with a Diverging number of Parameters
Wang, H., Li, B., & Leng, C. (2009) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00693.x · Cited by 433
Cross-validation for selecting a model selection procedure
Zhang, Y., & Yang, Y. (2015) · Journal of Econometrics · doi:10.1016/j.jeconom.2015.02.006 · Cited by 373
VARIABLE SELECTION IN QUANTILE REGRESSION
Wu, Y., & Liu, Y. (2021) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/gccn-qy67 · Cited by 358
Penalized model-based clustering with application to variable selection
Pan, W., Shen, X., & Yu, B. (2007) · Cited by 266
Selected Papers of Hirotugu Akaike
Ziegel, E. R., Akaike, H., Parzen, E., Tanabe, K., & Kitagawa, G. (1999) · Technometrics · doi:10.2307/1271382 · Cited by 213
A cross-validatory method for dependent data
BURMAN, P., CHOW, E., & NOLAN, D. (1994) · Biometrika · doi:10.1093/biomet/81.2.351 · Cited by 202
Extended BIC for small-n-large-P sparse GLM
Chen, J., & Chen, Z. (2012) · Statistica Sinica · doi:10.5705/ss.2010.216 · Cited by 198
Consistency of cross validation for comparing regression procedures
Yang, Y. (2007) · The Annals of Statistics · doi:10.1214/009053607000000514 · Cited by 166
Consistent Cross-Validated Density Estimation
Chow, Y. S., Geman, S., & Wu, L.-D. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346053 · Cited by 109
Modified Cross-Validation for Penalized High-Dimensional Linear Regression Models
Yu, Y., & Feng, Y. (2013) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2013.849200 · Cited by 51
Carmack, P. S., Schucany, W. R., Spence, J. S., Gunst, R. F., Lin, Q., & Haley, R. W. (2009) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2009.07034 · Cited by 27
Generalised correlated cross-validation
Carmack, P. S., Spence, J. S., & Schucany, W. R. (2012) · Journal of nonparametric statistics · doi:10.1080/10485252.2012.655733 · Cited by 18
Cited by
Performance of deep learning vs machine learning in plant leaf disease detection
Sujatha, R., Chatterjee, J. M., Jhanjhi, N., & Brohi, S. N. (2021) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.103615 · Cited by 653
Predicting Hard Rock Pillar Stability Using GBDT, XGBoost, and LightGBM Algorithms
Liang, W., Luo, S., Zhao, G., & Wu, H. (2020) · Mathematics · doi:10.3390/math8050765 · Cited by 496
Waheed, M., Xiong, H., Tong, K., & Lau, A. Y. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105632 · Cited by 3
Metadata
- Authors
- Jung, Y.
- Venue
- Journal of Nonparametric Statistics
- Publisher
- Informa UK Limited
- Published
- 2017-11-21
- Volume
- 30
- Issue
- 1
- Pages
- 197-215
- ISSN
- 1026-7654 (ISSN-L)
- 1048-5252
- 1029-0311
- OpenAlex
- W2768552973
- Cited by
- 499
- References
- 39
- FWCI
- 7.35
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.