Oracle inequalities for multi-fold cross validation
Open accessAbstract
We consider choosing an estimator or model from a given class by cross validation consisting of holding a nonneglible fraction of the observations out as a test set. We derive bounds that show that the risk of the resulting procedure is (up to a constant) smaller than the risk of an oracle plus an error which typically grows logarithmically with the number of estimators in the class. We extend the results to penalized cross validation in order to control unbounded loss functions. Applications include regression with squared and absolute deviation loss and classification under Tsybakov’s condition.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Cross-validation avoids overfitting by focusing on out-of-sample prediction and selects the best model from a set of candidate models.
“Cross-validation is extremely powerful-it avoids overfitting by focusing on out-of-sample prediction and selects the best model for the underlying data from a set of candidate models (this is known as the Oracle property) (van der Vaart, Dudoit, and van der Laan 2006).”
From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as van der Vaart, Dudoit, and van der Laan 2006
Cited by
Sorted by citations
A survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522 · see citing works
“…LOO Bootstrap, and .632 bootstrap for selection among minimum contrast estimators was studied in a series of papers (van der Laan and Dudoit, 2003; van der Laan et al., 2004, 2006; van der Vaart et al., 2006); these results apply in particular to least-squares regression and density estimation.”
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · see citing works
Machine Learning: An Applied Econometric Approach
Mullainathan, S., & Spiess, J. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.87 · Cited by 1,989 · see citing works
“While asymptotic results show that cross-validation tuning approximates the optimal complexity (Vaart, Dudoit, and Laan 2006), available finite-sample guidance on its implementation—such as heuristics for the number of folds (usually five to ten) or the “one standard-error rule” for tuning the…”
Backgroundvan der Laan, M. J., Polley, E., & Hubbard, A. (2007) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1309 · Cited by 1,472 · see citing works
“By a simple argument as presented in van der Laan and Dudoit (2003), Dudoit and van der Laan (2005) and van der Vaart et al. (2006), it follows that by letting the V = Vn in the V-fold cross-validation scheme converge to infinity at a slow enough rate relative to n, then either ψn = Ψ̂αn(Pn)…”
Self-citationCross-validation for selecting a model selection procedure
Zhang, Y., & Yang, Y. (2015) · Journal of Econometrics · doi:10.1016/j.jeconom.2015.02.006 · Cited by 394 · see citing works
Consistency of cross validation for comparing regression procedures
Yang, Y. (2007) · The Annals of Statistics · doi:10.1214/009053607000000514 · Cited by 168 · see citing works
Nonparametric Causal Effects Based on Longitudinal Modified Treatment Policies
D'iaz, I., Williams, N. T., Hoffman, K., & Schenck, E. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2021.1955691 · Cited by 175 · see citing works
The Highly Adaptive Lasso Estimator
Benkeser, D. C., & van der Laan, M. V. (2016) · International Conference on Data Science and Advanced Analytics · doi:10.1109/dsaa.2016.93 · Cited by 168
Self-citationSuper-Learning of an Optimal Dynamic Treatment Rule
Luedtke, A. R., & van der Laan, M. J. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0052 · Cited by 113 · see citing works
Self-citationPractical considerations for specifying a super learner
Phillips, R. V., van der Laan, M. J., Lee, H., & Gruber, S. (2023) · International Journal of Epidemiology · doi:10.1093/ije/dyad023 · Cited by 136 · see citing works
Self-citationCausal Inference for a Population of Causally Connected Units
van der Laan, M. J. (2014) · Journal of Causal Inference · doi:10.1515/jci-2013-0002 · Cited by 77 · see citing works
Self-citationA Generally Efficient Targeted Minimum Loss Based Estimator based on the Highly Adaptive Lasso
van der Laan, M. V. (2017) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0097 · Cited by 93 · see citing works
“The proof of this lemma is a simple corollary of the finite sample oracle inequality for cross-validation [11, 13, 21, 33, 34], also presented in Lemma 5 in Section A of the Appendix.”
Self-citationAssumption-lean Inference for Generalised Linear Model Parameters
Vansteelandt, S., & Dukes, O. (2022) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12504 · Cited by 62 · see citing works
Cai, W.-X., & van der Laan, M. V. (2020) · The International Journal of Biostatistics · doi:10.1515/ijb-2017-0070 · Cited by 79
Self-citationNonparametric efficient causal mediation with intermediate confounders
Diaz, I. M., Hejazi, N. S., Rudolph, K. E., & van der Laan, M. J. (2020) · Biometrika · doi:10.1093/biomet/asaa085 · Cited by 56 · see citing works
“Super learner has been shown to have important theoretical guarantees (van der Laan et al., 2006; van der Vaart et al., 2006) such as asymptotic equivalence to the oracle selector.”
BackgroundSelf-citationGruber, S., Logan, R. W., Jarrín, I., Monge, S., & Hernán, M. A. (2015) · Statistics in Medicine · doi:10.1002/sim.6322 · Cited by 64 · see citing works
“The optimal combination converges to the true data model when the prediction algorithms search over the correct portion of the solution space, and otherwise to the minimizer of a loss function-based dissimilarity measure, (⋅), with respect to target parameter ψ,E[ (?̂?) − (ψ)] [4,5,22].”
Backgroundvan der Laan, M. V., & Gruber, S. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0054 · Cited by 66
Self-citationAdaptive Kernel Methods Using the Balancing Principle
Vito, E., Pereverzyev, S., & Rosasco, L. (2010) · Foundations of Computational Mathematics · doi:10.1007/s10208-010-9064-2 · Cited by 66 · see citing works
“, hold-out and cross-validation (see [17] and more recently [9, 18, 40]) and on aggregation [39].”
BackgroundTargeted Data Adaptive Estimation of the Causal Dose–Response Curve
Diaz, I. M., & van der Laan, M. J. (2013) · Journal of Causal Inference · doi:10.1515/jci-2012-0005 · Cited by 42 · see citing works
“This fact is exploited by Wang et al. [2006] in order to define estimators of the risk as a cross-validated average of estimators of these loss functions.”
Self-citationTime-uniform central limit theory and asymptotic confidence sequences
Waudby-Smith, I., Arbour, D. T., Sinha, R., Kennedy, E. H., & Ramdas, A. (2021) · Annals of Statistics · doi:10.1214/24-aos2408 · Cited by 62
“63, 65 [60] Aad W van der Vaart, Sandrine Dudoit, and Mark J van der Laan.”
BackgroundTargeted learning in real‐world comparative effectiveness research with time‐varying interventions
Neugebauer, R. S., Schmittdiel, J. A., & van der Laan, M. J. (2014) · Statistics in Medicine · doi:10.1002/sim.6099 · Cited by 52 · see citing works
Self-citationJu, C., Combs, M., Lendle, S., Franklin, J. M., Wyss, R., Schneeweiß, S., & van der Laan, M. J. (2019) · Journal of Applied Statistics · doi:10.1080/02664763.2019.1582614 · Cited by 49 · see citing works
“It has been demonstrated that the SL performs asymptotically at least as well as the best choice among the library of candidate algorithms if the library does not contain a correctly specified parametricmodel; otherwise, it achieves the same rate of convergence as the correctly specified parametric model [6,31,32].”
Self-citationRegularization in reinforcement learning
Farahmand, A.-M. (2011) · PolyPublie (École Polytechnique de Montréal) · doi:10.7939/r34p5w · Cited by 29 · see citing works
Online cross‐validation‐based ensemble learning
Benkeser, D. C., Ju, C., Lendle, S., & van der Laan, M. J. (2017) · Statistics in Medicine · doi:10.1002/sim.7320 · Cited by 57 · see citing works
“Theoretical results guarantee that the estimator that exhibits the best estimated cross-validated performance is asymptotically equivalent with the unknown best estimator in the library (van der Laan and Dudoit, 2003; van der Vaart et al., 2006; van der Laan et al., 2006).”
Self-citationTilting methods for assessing the influence of components in a classifier
Hall, P., Titterington, D., & Xue, J.-H. (2009) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2009.00701.x · Cited by 49 · see citing works
“For recent applications of this approach in the context of classification, see, for example, Ghosh and Chaudhuri (2004), Lee et al. (2004), Bühlmann (2006) and van der Vaart et al. (2006).”
Prediction of Dead Oil Viscosity: Machine Learning vs. Classical Correlations
Hadavimoghaddam, F., Ostadhassan, M., Heidaryan, E., Sadri, M. A., Chapanova, I., Popov, E., Cheremisin, A., & Rafieepour, S. (2021) · Energies · doi:10.3390/en14040930 · Cited by 47
Semiparametric efficient empirical higher order influence function estimators
Liu, L., Mukherjee, R., Newey, W., & Robins, J. (2017) · doi:10.1920/wp.cem.2017.3017 · Cited by 46
“When b and p satisfy (2) in Theorem 5, the following estimators (cid:98) b, (cid:98) p will do so as well (van der Vaart, Dudoit and van der Laan, 2006) when the basis { z l , l ≥ 1 } are compactly supported Daubechies wavelets of sufficient regularity (at least 2 β max ): (cid:98) b ( x ) = k b l…”
hal9001: Scalable highly adaptive lasso regression in R
Hejazi, N., Coyle, J. R., & van der Laan, M. V. (2020) · Journal of Open Source Software · doi:10.21105/joss.02526 · Cited by 45
Self-citationInference for high-dimensional instrumental variables regression
Gold, D., Lederer, J., & Tao, J. (2017) · Journal of Econometrics · doi:10.1016/j.jeconom.2019.09.009 · Cited by 45
“Examples include (i) [41], who study the asymptotic properties of optimally tuned (with respect to mean square error) Lasso estimators in connection with solutions to adaptively tuned approximate message passing algorithms; (ii) AV∞ of [20], who provide `∞ estimation error guarantees; (iii) AVPr of [21], who provide g…”
BackgroundNeugebauer, R. S., Schmittdiel, J. A., Zhu, Z., Rassen, J. A., Seeger, J. D., & Schneeweiß, S. (2014) · Statistics in Medicine · doi:10.1002/sim.6377 · Cited by 43 · see citing works
“The selection of the optimal combination of the candidate learners is based on cross-validation [55, 56, 57, 58] to protect against overfitting such that the resulting learner (called ’super learner’) performs asymptotically as well (in terms of mean error) or better than any of the candidate learners considered.”
BackgroundScalable collaborative targeted learning for high-dimensional data
Ju, C., Gruber, S., Lendle, S., Chambaz, A., Franklin, J. M., Wyss, R., Schneeweiß, S., & van der Laan, M. J. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280217729845 · Cited by 40 · see citing works
“It has been proven that a SL selector can perform asymptotically as well as an oracle selector under mild assumptions (van der Laan et al. 2007; van der Laan and Dudoit 2003; van der Vaart et al. 2006).”
BackgroundSelf-citationOptimal Spatial Prediction Using Ensemble Machine Learning
Davies, M. M., & van der Laan, M. J. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2014-0060 · Cited by 36 · see citing works
“[17] (see also Appendix A for a definition within the context of fixed regression).”
BackgroundSelf-citationvan der Laan, M. J., & Starmans, R. J. C. M. (2014) · Advances in Statistics · doi:10.1155/2014/502678 · Cited by 27 · see citing works
Self-citationContinuous-time targeted minimum loss-based estimation of intervention-specific mean outcomes
Rytgaard, H. C. W., Gerds, T. A., & van der Laan, M. J. (2022) · The Annals of Statistics · doi:10.1214/21-aos2114 · Cited by 17 · see citing works
“The super learner performs asymptotically no worse than any algorithm included in its library, a property known as the oracle inequality (van der Laan and Dudoit, 2003; van der Vaart et al., 2006).”
BackgroundSelf-citationRytgaard, H. C. W., Eriksson, F., & van der Laan, M. J. (2023) · Biometrics · doi:10.1111/biom.13856 · Cited by 20 · see citing works
“…based on the log-likelihood loss can then be used to data-adaptively select the bound on the variation norm: by the oracle properties of cross-validation (van der Laan & Dudoit, 2003; van der Vaart et al., 2006), we only need at least one of the candidates to be larger than the true variation norm.”
MethodSelf-citationConsistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation
Chen, Y., Owhadi, H., & Stuart, A. (2020) · Mathematics of Computation · doi:10.1090/mcom/3649 · Cited by 36 · see citing works
Zheng, W.-J., Balzer, L., van der Laan, M. V., & Petersen, M. (2017) · Statistics in Medicine · doi:10.1002/sim.7296 · Cited by 36 · see citing works
Self-citationErtefaie, A., McKay, J. R., Oslin, D. W., & Strawderman, R. L. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2020.1753522 · Cited by 32 · see citing works
Comparing predictive abilities of longitudinal child growth models
Anderson, C., Hafen, R., Sofrygin, O., RYAN, L. M., & Community, M. O. T. H. (2018) · Statistics in Medicine · doi:10.1002/sim.7693 · Cited by 39 · see citing works
Targeted learning ensembles for optimal individualized treatment rules with time-to-event outcomes.
Díaz, I., Savenkov, O., & Ballman, K. (2017) · Biometrika · doi:10.1093/biomet/asy017 · Cited by 34
“The optimality of general cross-validation selection procedures is discussed in van der Laan & S. Dudoit & A.W. van der Vaart (2006); van der Vaart et al. (2006).”
MethodNeugebauer, R. S., Schmittdiel, J. A., & van der Laan, M. J. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0028 · Cited by 29 · see citing works
Self-citationOracle inequalities for cross-validation type procedures
Lecué, G., & Mitchell, C. (2012) · Electronic Journal of Statistics · doi:10.1214/12-ejs730 · Cited by 34 · see citing works
“Many other authors have been studying or using this method and we refer the reader to the survey of CV methods in model selection [2], the PhD thesis [10, 28] or [34] for more bibliographical references on this topic.”
MethodHejazi, N., van der Laan, M. V., Janes, H., Gilbert, P., & Benkeser, D. C. (2020) · Biometrics · doi:10.1111/biom.13375 · Cited by 31
Self-citationSuper Learner for Survival Data Prediction
Golmakani, M. K., & Polley, E. C. (2020) · The International Journal of Biostatistics · doi:10.1515/ijb-2019-0065 · Cited by 27 · see citing works
“It has been theoretically proven that a super learner performs asymptotically as well as the optimal weighted combination [14, 17, 18].”
Backgroundvan der Laan, M. V., Benkeser, D. C., & Cai, W.-X. (2019) · The International Journal of Biostatistics · doi:10.1515/ijb-2019-0092 · Cited by 29
Self-citationTargeted maximum likelihood estimation for causal inference in survival and competing risks analysis
Rytgaard, H. C. W., & van der Laan, M. J. (2022) · Lifetime Data Analysis · doi:10.1007/s10985-022-09576-2 · Cited by 17 · see citing works
“The theoretical properties of super learning are guaranteed by the general oracle inequality for cross-validation which yields that the super learner performs asymptotically as well as the best performing estimator in the library (van der Laan and Dudoit 2003; van der Vaart et al. 2006).”
Self-citationSelective machine learning of doubly robust functionals.
Cui, Y., & Tchetgen, E. T. T. (2019) · doi:10.48550/arxiv.1911.02029 · Cited by 27
“The proof of this result is based on a finite sample inequality for the excess pseudo-risk of our selector compared to the oracle’s, which requires an extension of Lemma 8.1 of van der Vaart et al. (2006) to handle second order U-statistics.”
MethodInfluentialGruber, S., Krakower, D., Menchaca, J., Hsu, K., Hawrusik, R., Maro, J. C., Cocoros, N., Kruskal, B., Wilson, I., Mayer, K., et al. (2020) · Statistics in Medicine · doi:10.1002/sim.8591 · Cited by 26 · see citing works
Díaz, I., & van der Laan, M. V. (2017) · Statistics in Medicine · doi:10.1002/sim.7389 · Cited by 26 · see citing works
“(7) The bandwidth is chosen as ĥ = n−0.1ĥopt, where ĥopt is the optimal bandwidth chosen using K-fold cross-validation (the optimality of this selector is discussed in van der Vaart et al., 2006).”
MethodSelf-citationScharfstein, D. O., Mcdermott, A., Díaz, I., Carone, M., Lunardon, N., & Turkoz, I. (2018) · Biometrics · doi:10.1111/biom.12729 · Cited by 25
Neugebauer, R. S., FIREMAN, B. H., Roy, J. A., Raebel, M. A., Nichols, G. A., & O’Connor, P. J. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.01.016 · Cited by 24 · see citing works
Nhung, H. L. T. K., Vo, H. V., Silhavy, R., Prokopova, Z., & Silhavy, P. (2022) · IEEE Access · doi:10.1109/access.2021.3139183 · Cited by 25
“The LASSO parameter λ is determined by the Leave One Out Cross-Validation (LOO-CV) method [76], [77].”
MethodStatistical inference for data‐adaptive doubly robust estimators with survival outcomes
Díaz, I. (2017) · Statistics in Medicine · doi:10.1002/sim.8156 · Cited by 24
Tran, L., Yiannoutsos, C. T., Musick, B. S., Wools‐Kaloustian, K., Siika, A. M., Kimaiyo, S. N., van der Laan, M. J., & Petersen, M. L. (2016) · Epidemiologic Methods · doi:10.1515/em-2016-0004 · Cited by 21 · see citing works
“…inequality showing the asymptotic performance also shows that the number of candidate algorithms considered in the cross-validation procedure can be polynomial in size proportional to the number of observations (van der Laan and Dudoit 2003; van der Vaart et al. 2006; van der Laan et al. 2007).”
Self-citationLocally epistatic models for genome-wide prediction and association by importance sampling
Akdemir, D., Jannink, J., & Isidro-Sánchez, J. (2017) · Genetics Selection Evolution · doi:10.1186/s12711-017-0348-8 · Cited by 20 · see citing works
“[38] Aad W Vaart, Sandrine Dudoit, and Mark J Laan.”
BackgroundZelnick, L. R., Shlipak, M. G., Soliman, E. Z., Anderson, A. H., Christenson, R. H., Lash, J. P., Deo, R., Rao, P. S., Afshinnia, F., Chen, J., et al. (2021) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.01060121 · Cited by 19 · see citing works
“Age, yr 57 (11) 57 (11) 62 (8) Men 1521 (55) 1358 (54) 163 (63) Race/ethnicity Non-Hispanic White 1203 (43) 1061 (42) 142 (55) Non-Hispanic Black 1064 (38) 985 (39) 79 (31) Hispanic 380 (14) 351 (14) 29 (11) Other 119 (4) 110 (4) 9 (3) Diabetes at baseline 1327 (48) 1181 (47) 146 (56) History of myocardial infarction…”
BackgroundNetwork Inference With the Lasso
Waldorp, L., & Haslbeck, J. M. B. (2024) · Multivariate Behavioral Research · doi:10.1080/00273171.2024.2317928 · Cited by 20 · see citing works
Cross Validation for Selection of Cortical Interaction Models From Scalp EEG or MEG
Cheung, B., Nowak, R., Lee, H. C., van Drongelen, W., & Veen, B. V. (2011) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2011.2174991 · Cited by 18 · see citing works
Ehwerhemuepha, L., Danioko, S., Verma, S., Marano, R., Feaster, W. W., Taraman, S., Moreno, T., Zheng, J., Yaghmaei, E., & Chang, A. C. (2021) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2021.100030 · Cited by 23 · see citing works
One-Step Estimation of Differentiable Hilbert-Valued Parameters
Luedtke, A., & Chung, I. (2023) · Annals of Statistics · doi:10.1214/24-aos2403 · Cited by 16
EFFICIENT AND MULTIPLY ROBUST RISK ESTIMATION UNDER GENERAL FORMS OF DATASET SHIFT
Qiu, H., Tchetgen, E. T. T., & Dobriban, E. (2023) · Annals of Statistics · doi:10.1214/24-aos2422 · Cited by 14
Efficient estimation of modified treatment policy effects based on the generalized propensity score
Hejazi, N., Benkeser, D. C., Díaz, I., & van der Laan, M. V. (2022) · doi:10.48550/arxiv.2205.05777 · Cited by 14
Self-citationMarginal Structural Models with Counterfactual Effect Modifiers
Zheng, W.-J., Luo, Z.-H., & van der Laan, M. V. (2018) · The International Journal of Biostatistics · doi:10.1515/ijb-2018-0039 · Cited by 14
Self-citationUzunangelov, V., Wong, C. K., & Stuart, J. M. (2020) · bioRxiv · doi:10.1371/journal.pcbi.1008878 · Cited by 13 · see citing works
“,∆S} and Ψ have uniformly bounded loss functions exhibits the ”oracle” property - Ψ∗ is asymptotically guaranteed to perform as well as the optimal base learner ∆s under the true data-generating distribution [80][79].”
BackgroundSelective Machine Learning of the Average Treatment Effect with an Invalid Instrumental VariablePreprint
Sun, B., Cui, Y., & Tchetgen, E. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.11882 · Cited by 5 · see citing works
“By Lemma 2.2 in Van der Vaart et al. (2006), we further have the following bounds for the first order term, P 0 [ max k∈{1,2,3},α,α′ +k ∈Ak { 2(1 + ǫ) √ n1 n1 ∑ i ( φ(α;α−k, α ′ +k)i − P1 { φ(α;α−k, α ′ +k)i }) × P1 { φ(α;α−k, α ′ +k) } − ǫ√n1[P1 { φ(α;α−k, α ′ +k) } ]2 }] ≤P0 ( 16(1 + ǫ) n 1/q−1/2…”
Influentialorigami: A Generalized Framework for Cross-Validation in R
Coyle, J. R., & Hejazi, N. (2018) · Journal of Open Source Software · doi:10.21105/joss.00512 · Cited by 12
Nhung, H. L. T. K., Vo, H. V., Silhavy, P., Prokopova, Z., & Silhavy, R. (2023) · J. Softw. Evol. Process. · doi:10.1002/smr.2611 · Cited by 10 · see citing works
“…in this study, alternative criteria are used that provide an unbiased symmetric distribution as follows: SSE (Equation [18]), MAE (Equation [19]), RMSE (Equation [20]), MBRE (Equation [21]), MIBRE (Equation [22]), MdMRE (Equation [23]), and PRED( x ) (where x = 0.25 in this study; Equation [24]).”
MethodInfluentialHuang, Y., Zhang, Y., Seaton, K. E., Rosa, S. C. D., Heptinstall, J., Carpp, L. N., Randhawa, A. K., McKinnon, L. R., McLaren, P. J., Viegas, E., et al. (2022) · EBioMedicine · doi:10.1016/j.ebiom.2022.104271 · Cited by 13 · see citing works
General oracle inequalities for model selection
Mitchell, C., & van de Geer, S. A. (2009) · doi:10.1214/08-ejs254 · Cited by 10
“A direct treatment of a cross validation scheme is to be found in van der Vaart et al. [ 24 ].”
MethodDelias, P., Mittas, N., & Florou, G. (2023) · Decision Analytics Journal · doi:10.1016/j.dajour.2023.100291 · Cited by 9 · see citing works
Personal Credit Default Discrimination Model Based on Super Learner Ensemble
Li, G., Shen, M., Li, M., & Cheng, J. (2021) · Mathematical Problems in Engineering · doi:10.1155/2021/5586120 · Cited by 8 · see citing works
Regularized DeepIV with Model Selection
Li, Z.-H., Lan, H., Syrgkanis, V., Wang, M.-D., & Uehara, M. (2024) · arXiv.org · doi:10.48550/arxiv.2403.04236 · Cited by 8
Causal Q-Aggregation for CATE Model Selection
Lan, H., & Syrgkanis, V. (2023) · International Conference on Artificial Intelligence and Statistics · doi:10.48550/arxiv.2310.16945 · Cited by 8
Cross-Validated Loss-Based Covariance Matrix Estimator Selection in High Dimensions
Boileau, P., Hejazi, N., van der Laan, M. V., & Dudoit, S. (2021) · Journal of Computational And Graphical Statistics · doi:10.1080/10618600.2022.2110883 · Cited by 8
“…is typically unknown, one turns to cross-validation for risk estimation. van der Laan and Dudoit (2003); Dudoit and van der Laan (2005); van der Vaart et al. (2006) have shown that, under general conditions on the data-generating distribution and loss function, the cross-validated estimator…”
BackgroundMethodSelf-citationHigher-Order Targeted Loss-Based Estimation
Carone, M., Díaz, I., & van der Laan, M. V. (2018) · doi:10.1007/978-3-319-65304-4_26 · Cited by 8
Self-citationHasson, H., Maddix, D., Wang, Y., Gupta, G., & Park, Y. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2305.15786 · Cited by 7
“Note that even in the case that J is finite it would not have been possible to use the oracle inequalities in Van der Vaart et al. (2006) directly to the algorithms θα(D0) ∶= Aα(Z) ○ (η1(D00), .”
BackgroundMethodInfluentialCross-validation on extreme regions
Aghbalou, A., Bertail, P., Portier, F., & Sabourin, A. (2022) · Extremes · doi:10.1007/s10687-024-00495-z · Cited by 7 · see citing works
Nonparametric Bootstrap Inference for the Targeted Highly Adaptive LASSO Estimator.
Cai, W.-X., & van der Laan, M. V. (2019) · doi:10.48550/arxiv.1905.10299 · Cited by 7
“These two universal bounds on the loss function yield the oracle inequality for the cross-validation selector among a set of candidate estimators (van der Laan and Dudoit, 2003; van der Vaart et al., 2006; van der Laan et al., 2006, 2007; Polley et al., 2011).”
BackgroundSelf-citationTargeted Learning in Healthcare Research
Gruber, S. (2015) · Big Data · doi:10.1089/big.2015.0025 · Cited by 7 · see citing works
Neugebauer, R. S., Chandra, M., Paredes, A., Graham, D. J., McCloskey, C. A., & Go, A. S. (2013) · Journal of Causal Inference · doi:10.1515/jci-2012-0003 · Cited by 8 · see citing works
Nhung, H. L. T. K., Huynh, H. T., & Vo, H. V. (2020) · Software Engineering Perspectives in Intelligent Systems · doi:10.1007/978-3-030-63322-6_64 · Cited by 7
“In this phase, we determined the appropriate k value using the Leave One Out Cross-Validation (LOOCV) technique [19, 20]; where the R-Square attains its maximum value.”
MethodRelative error model average for multiplicative models
Xia, X., Hao, M., & Li, J. (2025) · Statistics and computing · doi:10.1007/s11222-025-10773-w · Cited by 6
“Using CV to build a stronger predictor ensemble has appeared in earlier publications (Dudoit and van der Laan 2005; van der Vaart et al. 2006 and van der Laan et al. (2007)).”
Online estimation with rolling validation: Adaptive nonparametric estimation with streaming data
Zhang, T., & Lei, J. (2023) · Annals of Statistics · doi:10.1214/25-aos2561 · Cited by 6
“We also only require it to have a finite second moment given X i , which is much milder than some existing works on model selection via CV [29, 4].”
Sensitivity analysis for causality in observational studies for regulatory science
Diaz, I. M., Lee, H., Kıcıman, E., Schenck, E. J., Akacha, M., Follman, D., & Ghosh, D. (2023) · Journal of Clinical and Translational Science · doi:10.1017/cts.2023.688 · Cited by 5 · see citing works
Causal mediation analysis: selection with asymptotically valid inference
Jones, J., Ertefaie, A., & Strawderman, R. (2021) · Journal of The Royal Statistical Society Series B-statistical Methodology · doi:10.1093/jrsssb/qkae109 · Cited by 6 · see citing works
“In addition, the size of the library can grow at a polynomial rate compared with the sample size without detrimental effects to its oracle performance (Dudoit & van der Laan, 2005; van der Vaart et al., 2006).”
cvCovEst: Cross-validated covariance matrix estimator selection and evaluation in R
Boileau, P., Hejazi, N., Collica, B., van der Laan, M. V., & Dudoit, S. (2021) · Journal of Open Source Software · doi:10.21105/joss.03273 · Cited by 6
“…are guaranteed under a few non-restrictive assumptions by extending the seminal work of van der Laan & Dudoit (2003), Dudoit & van der Laan (2005), and van der Vaart et al. (2006) on data-adaptive estimator selection to high-dimensional covariance matrix estimation (Boileau et al., 2021).”
BackgroundSelf-citationKernel machines for current status data
Travis-Lumer, Y., & Goldberg, Y. (2015) · Machine-mediated learning · doi:10.1007/s10994-020-05930-3 · Cited by 6
“Oracle inequalities for penalized risk minimization with multi-fold cross validation were developed by van der Vaart et al. (2006).”
Methodvan der Laan, M. V., Luedtke, A., & Díaz, I. (2014) · Journal of Causal Inference · doi:10.1515/em-2014-0012 · Cited by 6
“Consider a substitution estimator ΨðP nÞ, such as a TMLE based on an initial super-learner-based estimator P0n (van der Laan and Dudoit, 2003; van der Vaart et al., 2006; van der Laan et al., 2006, 2007; Polley et al., 2012) that is then updated into a targeted estimator P n.”
MethodSelf-citationEnsemble learning with trees and rules: Supervised, semi-supervised, unsupervised
Akdemir, D., & Jannink, J. (2014) · Intelligent Data Analysis · doi:10.3233/ida-140672 · Cited by 6
“Aggressive use of cross-validation can be supported by the theorem in [40].”
The causal effects of modified treatment policies under network interference
Balkus, S. V., Delaney, S., & Hejazi, N. (2024) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1093/jrsssb/qkag052 · Cited by 5 · see citing works
Ensemble Prediction via Covariate-dependent Stacking
Wakayama, T., & Sugasawa, S. (2024) · Statistics and computing · doi:10.1007/s11222-025-10739-y · Cited by 5 · see citing works
Chambaz, A., Choudat, D., Huber, C., Pairon, J., & van der Laan, M. V. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt042 · Cited by 5
Self-citationThe International Journal of Biostatistics
van der Laan, M. V. (2017) · The International Journal of Biostatistics · doi:10.1515/ijb · Cited by 5 · see citing works
“…for infinite dimensional parameters of the full data distribution in CAR-censored data and causal inference models: see also van der Laan et al. (2006), van der Vaart et al. (2006), van der Laan et al. (2004), Dudoit and van der Laan (2005), Kele¸s et al. (2002), Sinisi and van der Laan (2004).”
Self-citationPseudo-observations and super learner for the estimation of the restricted mean survival time
Cwiling, A., Perduca, V., & Bouaziz, O. (2024) · Lifetime Data Analysis · doi:10.1007/s10985-025-09668-9 · Cited by 4
“Van Der Laan and Dudoit (2003), Dudoit and Van Der Laan (2005) and Van Der Vaart et al. (2006) theoretically validated the use of cross-validation to select an optimal learner among many candidates and protect against overfitting.”
Targeted estimation of state occupation probabilities for the non‐Markov illness‐death model
Munch, A., Breum, M. S., Martinussen, T., & Gerds, T. (2023) · Scandinavian Journal of Statistics · doi:10.1111/sjos.12644 · Cited by 4
“…LASSO estimators (Chetverikov et al., 2021) or general results for hyper-parameter selection using cross-validation (van der Laan & Dudoit, 2003; van der Vaart et al., 2006) it is possible to establish Assumption 2(ii) even when the number of grid points K and the partition size p increase with…”
Adversarial Monte Carlo Meta-Learning of Optimal Prediction Procedures
Luedtke, A., Chung, I., & Sofrygin, O. (2020) · Journal of machine learning research · doi:10.48550/arxiv.2002.11275 · Cited by 4 · see citing works
Finite Sample Inference for Targeted Learning
van der Laan, M. V. (2017) · doi:10.48550/arxiv.1708.09502 · Cited by 4
“…one can use our confidence intervals with the point estimate defined as a TMLE using a super-learner (van der Laan and Dudoit, 2003; van der Vaart et al., 2006; van der Laan et al., 2006, 2007; Polley et al., 2011) that includes the HAL-MLE as one of the candidate estimators in its library.”
BackgroundMethodSelf-citationMethodological and regulatory considerations for causal AI in drug development
Lee, H., Qiu, S., Haupert, S., Innes, G. K., Naumann, T., Alemayehu, D., & van der Laan, M. V. (2026) · npj Digit. Medicine · doi:10.1038/s41746-026-02477-w · Cited by 3 · see citing works
Self-citationSchuler, A., Hagemeister, A., & van der Laan, M. V. (2024) · arXiv.org · doi:10.48550/arxiv.2410.02680 · Cited by 3
“This is a direct consequence of theorem 2 and the cross-validation oracle inequality [26].”
BackgroundSelf-citationWhy Machine Learning Cannot Ignore Maximum Likelihood Estimation
van der Laan, M. V., & Rose, S. (2021) · arXiv.org · doi:10.48550/arxiv.2110.12112 · Cited by 3
Self-citation
- Authors
- Vaart, A. W. V. D., Dudoit, S., & Laan, M. J. V. D.
- Affiliations
- Vrije Universiteit Amsterdam (NL)
- Venue
- Statistics & Decisions
- Publisher
- Walter de Gruyter GmbH
- Published
- 2006-12
- Volume
- 24
- Issue
- 3
- Pages
- 351-371
- ISSN
- 0721-2631 (ISSN-L)
- OpenAlex
- W2080896439
- Cited by
- 156
- References
- 0
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 10 · 2023: 9 · 2022: 10 · 2021: 13 · 2020: 12 · 2019: 7 · 2018: 12 · 2017: 14 · 2016: 7 · 2015: 5 · 2014: 12 · 2013: 5 · 2012: 5
- FWCI
- 1.89
- Type
- journal article
- Language
- en
- Open access
- Locations
- Statistics & Risk Modeling (publishedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- VU Research Portal (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- oracle inequalities, model selection, risk bounds, regression analysis, classification, squared error loss
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
