The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates
Open accessAbstract
In binary classification problems, the area under the ROC curve (AUC) is commonly used to evaluate the performance of a prediction model. Often, it is combined with cross-validation in order to assess how the results will generalize to an independent data set. In order to evaluate the quality of an estimate for cross-validated AUC, we obtain an estimate of its variance. For massive data sets, the process of generating a single performance estimate can be computationally expensive. Additionally, when using a complex prediction method, the process of cross-validating a predictive model on even a relatively small data set can still require a large amount of computation time. Thus, in many practical settings, the bootstrap is a computationally intractable approach to variance estimation. As an alternative to the bootstrap, we demonstrate a computationally efficient influence curve based approach to obtaining a variance estimate for cross-validated AUC.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
The Predictive Sample Reuse Method with Applications
Geisser, S. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479865 · Cited by 2,748 · cited 1×
“Common types of cross-validation procedures include k-fold [3], leave-one-out [21, 7, 3], and leave-p-out [20] cross-validation.”
MethodThe Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
Allen, D. M. (1974) · Technometrics · doi:10.1080/00401706.1974.10489157 · Cited by 1,469
Research works printed but not linked: 13
- [5] Sing, T., Sander, O., Beerenwinkel, N., and Lengauer, T. (2005). ROCR: Visualizing classifier performance in R., Bioinformatics 21, 20, 3940–3941. · DOI · Paper page
- [8] Bezanson, J., Karpinski, S., Shah, V. B., and Edelman, A. (2012). Julia: A fast dynamic language for technical computing., CoRR abs/1209.5145. http://arxiv.org/abs/1209.5145. · arXiv
- [11] Efron, B. (1979). Bootstrap methods: another look at the jackknife., Ann. Statist. 7, 1, 1–26. · DOI · Paper page
- [13] Friedman, J., Hastie, T., and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent., Journal of Statistical Software 33, 1, 1–22. http://www.jstatsoft.org/v33/i01/. · DOI · Paper page
- [20] Shao, J. (1993). Linear model selection by cross-validation., J. Amer. Statist. Assoc. 88, 422, 486–494. · DOI · Paper page
- [21] Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions., J. Roy. Statist. Soc. Ser. B 36, 111–147. With discussion by G. A. Barnard, A. C. Atkinson, L. K. Chan, A. P. Dawid, F. Downton, J. Dickey, A. G. Baker, O. Barndorff-Nielsen, D. R. Cox, S. Giesser, D. Hinkley, R. R. Hocking, and A. S. Young, and with a reply by the authors. · DOI · Paper page
- [1] Ling, C., Huang, J., and Zhang, H. (2003). AUC: a statistically consistent and more discriminating measure than accuracy., Proceedings of IJCAI 2003.
- [4] Kleiner, A., Talwalkar, A., Sarkar, P., and Jordan, M. (2013). A scalable bootstrap for massive data., Journal of the Royal Statistical Society, Series B. · DOI
- [6] Venables, W. N. and Ripley, B. D. (2002)., Modern Applied Statistics with S, Fourth ed. Springer, New York. · DOI · Paper page
- [10] Bickel, P. J., Klaassen, C. A. J., Ritov, Y., and Wellner, J. A. (1993)., Efficient and adaptive estimation for semiparametric models. Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore, MD.
- [12] Efron, B. and Tibshirani, R. J. (1993)., An introduction to the bootstrap. Monographs on Statistics and Applied Probability, Vol. 57. Chapman and Hall, New York. · DOI · Paper page
- [19] Politis, D. N., Romano, J. P., and Wolf, M. (1999)., Subsampling. Springer Series in Statistics. Springer-Verlag, New York. http://dx.doi.org/ 10.1007/978-1-4612-1554-7. · DOI · Paper page
- [22] van der Vaart, A. W. and Wellner, J. A. (1996)., Weak convergence and empirical processes. Springer Series in Statistics. Springer-Verlag, New York. With applications to statistics. · DOI
Cited by
Real-time tracking of self-reported symptoms to predict potential COVID-19
Menni, C., Valdes, A. M., Freidin, M. B., Sudre, C. H., Nguyen, L. H., Drew, D. A., Ganesh, S., Varsavsky, T., Cardoso, M. J., El-Sayed Moustafa, J. S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0916-2 · Cited by 1,517
Pan-cancer integrative histology-genomic analysis via multimodal deep learning
Chen, R. J., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Noor, Z., Shaban, M., Shady, M., Williams, M., Joo, B., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.07.004 · Cited by 660
Roimi, M., Neuberger, A., Shrot, A., Paul, M., Geffen, Y., & Bar-Lavie, Y. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05876-8 · Cited by 91
Xicota, L., Ichou, F., Lejeune, F.-X., Colsch, B., Tenenhaus, A., Leroy, I., Fontaine, G., Lhomme, M., Bertin, H., Habert, M.-O., et al. (2019) · EBioMedicine · doi:10.1016/j.ebiom.2019.08.051 · Cited by 51
Machine Learning-Based Prediction of Hemoglobinopathies Using Complete Blood Count Data
Schipper, A., Rutten, M., van Gammeren, A., Harteveld, C. L., Urrechaga, E., Weerkamp, F., den Besten, G., Krabbe, J., Slomp, J., Schoonen, L., et al. (2024) · Clinical Chemistry · doi:10.1093/clinchem/hvae081 · Cited by 33
Metadata
- Authors
- LeDell, E., Petersen, M., & van der Laan, M.
- Venue
- Electronic Journal of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2015-01-01
- Volume
- 9
- Issue
- 1
- Pages
- 1583-1607
- ISSN
- 1935-7524 (ISSN-L)
- OpenAlex
- W2151699048
- Cited by
- 239
- References
- 3
- FWCI
- 9.70
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates. https://app.cochat.ai/papers/10.1214/15-ejs1035
- The paper
- LeDell, E., Petersen, M., & van der Laan, M. (2015). Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates. Electronic Journal of Statistics. https://doi.org/10.1214/15-ejs1035
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
