Cited by 409Machine Learning and AlgorithmsMachine Learning and Data ClassificationStochastic Gradient Optimization Techniques

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Algorithmic Stability and Sanity-Check Bounds for Leave-One-Out Cross-Validation

Kearns, Michael, Ron, Dana·1999·Neural Computation

DOI 10.1162/089976699300016304

Open this paper in CoChat — free account

Abstract

In this article we prove sanity-check bounds for the error of the leave-oneout cross-validation estimate of the generalization error: that is, bounds showing that the worst-case error of this estimate is not much worse than that of the training error estimate. The name sanity check refers to the fact that although we often expect the leave-one-out estimate to perform considerably better than the training error estimate, we are here only seeking assurance that its performance will not be considerably worse. Perhaps surprisingly, such assurance has been given only for limited cases in the prior literature on cross-validation. Any nontrivial bound on the error of leave-one-out must rely on some notion of algorithmic stability. Previous bounds relied on the rather strong notion of hypothesis stability, whose application was primarily limited to nearest-neighbor and other local algorithms. Here we introduce the new and weaker notion of error stability and apply it to obtain sanity-check bounds for leave-one-out for other classes of learning algorithms, including training error minimization procedures and Bayesian algorithms. We also provide lower bounds demonstrating the necessity of some form of error stability for proving bounds on the error of the leave-one-out estimate, and the fact that for training error minimization algorithms, in the worst case such bounds must still depend on the Vapnik-Chervonenkis dimension of the hypothesis class.

Integrity signals

Registry

Crossref · 10.1162/089976699300016304

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949
Untitled work
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
Estimation of Dependences Based on Empirical Data
Vapnik, V. N. (2006) · Information science and statistics · doi:10.1007/0-387-34239-7 · Cited by 2,252
Subset Selection in Regression
Miller, A. J. (2003) · Technometrics · doi:10.1198/tech.2003.s144 · Cited by 1,501
Decision theoretic generalizations of the PAC model for neural net and other learning applications
Haussler, D. H. (1992) · Information and Computation · doi:10.1016/0890-5401(92)90010-d · Cited by 845
Subset Selection in Regression.
RAAB, G. M., & Miller, T. (1991) · Biometrics · doi:10.2307/2532418 · Cited by 547
Statistical mechanics of learning from examples
Seung, H. S., Sompolinsky, H., & Tishby, N. (1992) · Physical Review A · doi:10.1103/physreva.45.6056 · Cited by 532
Toward efficient agnostic learning
Kearns, M. J., Schapire, R. E., & Sellie, L. M. (1992) · Conference on Learning Theory · doi:10.1145/130385.130424 · Cited by 353
Self-citation

Show all 18 references

Sorted by citations

Items 101–200 of 409

Machine learning for predicting levetiracetam treatment response in temporal lobe epilepsy
Croce, P., Ricci, L., Pulitano, P., Boscarino, M., Zappasodi, F., Lanzone, J., Narducci, F., Mecarelli, O., Di Lazzaro, V., Tombini, M., et al. (2021) · Clinical Neurophysiology · doi:10.1016/j.clinph.2021.08.024 · Cited by 38
Ten Challenging Problems in Federated Foundation Models
Fan, T., Gu, H., Cao, X., Chan, C. S., Chen, Q., Chen, Y., Feng, Y., Gu, Y., Geng, J., Luo, B., et al. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3555328 · Cited by 49
Bio-behavioral synchrony is a potential mechanism for mate selection in humans
Zeevi, L., Selle, N. K., Kellmann, E. L., Boiman, G., Hart, Y., & Atzil, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-08582-6 · Cited by 40
Adapting Geometric Attributes for Expression-Invariant 3D Face Recognition
Li, X., & Zhang, H. (2007) · IEEE International Conference on Shape Modeling and Applications · doi:10.1109/smi.2007.4 · Cited by 39
A visual analytics system for multi-model comparison on clinical data predictions
Li, Y., Fujiwara, T., Choi, Y. K., Kim, K. K., & Ma, K. (2020) · Visual Informatics · doi:10.1016/j.visinf.2020.04.005 · Cited by 37
Fully Empirical and Data-Dependent Stability-Based Bounds
Oneto, L., Ghio, A., Ridella, S., & Anguita, D. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2361857 · Cited by 37
EarHealth
Jin, Y., Gao, Y., Guo, X., Wen, J., Li, Z., & Jin, Z. (2022) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3498361.3538935 · Cited by 36
Are smartphones and machine learning enough to diagnose tremor?
Balachandar, A., Algarni, M., de Oliveira, L. M., Marsili, L., Merola, A., Sturchio, A., Espay, A. J., Hutchison, W. D., Balasubramaniam, A., Rudzicz, F., et al. (2022) · Journal of Neurology · doi:10.1007/s00415-022-11293-7 · Cited by 38
Stable Transductive Learning
El‐Yaniv, R., & Pechyony, D. (2006) · Lecture notes in computer science · doi:10.1007/11776420_6 · Cited by 36
Is the RdNBR a better estimator of wildfire burn severity than the dNBR? A discussion and case study in southeast China
Cai, L., & Wang, M. (2020) · Geocarto International · doi:10.1080/10106049.2020.1737973 · Cited by 36
The Development and Validation of Statistical Prediction Rules for Discriminating Between Genuine and Simulated Suicide Notes
Jones, N. J., & Bennell, C. (2007) · Archives of Suicide Research · doi:10.1080/13811110701250176 · Cited by 35
Radiomics-based machine learning differentiates “ground-glass” opacities due to COVID-19 from acute non-COVID-19 lung disease
Pizzi, A. D., Chiarelli, A. M., Chiacchiaretta, P., Valdesi, C., Croce, P., Mastrodicasa, D., Villani, M., Trebeschi, S., Serafini, F. L., Rosa, C., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-96755-0 · Cited by 34
Electroencephalography-Derived Prognosis of Functional Recovery in Acute Stroke Through Machine Learning Approaches
Chiarelli, A. M., Croce, P., Assenza, G., Merla, A., Granata, G., Giannantoni, N. M., Pizzella, V., Tecchio, F., & Zappasodi, F. (2020) · International Journal of Neural Systems · doi:10.1142/s0129065720500677 · Cited by 32
Single-cell mapper (scMappR): using scRNA-seq to infer the cell-type specificities of differentially expressed genes
Sokolowski, D., Faykoo-Martinez, M., Erdman, L., Hou, H., Chan, C., Zhu, H., Holmes, M. M., Goldenberg, A., & Wilson, M. D. (2021) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqab011 · Cited by 32
Palmprint Classification using Dual-Tree Complex Wavelets
Chen, G. Y., Bui, T. D., & Krzyżak, A. (2006) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2006.313053 · Cited by 34
A high-dimensional Wilks phenomenon
Boucheron, S., & Massart, P. (2010) · Probability Theory and Related Fields · doi:10.1007/s00440-010-0278-7 · Cited by 35
Remote Sensing Estimation of Sea Surface Salinity from GOCI Measurements in the Southern Yellow Sea
Sun, D., Su, X., Qiu, Z., Wang, S., Mao, Z., & He, Y. (2019) · Remote Sensing · doi:10.3390/rs11070775 · Cited by 33
Cross-Validation and Mean-Square Stability
Kale, S., Kumar, R., & Vassilvitskii, S. (2011) · Cited by 33
System identification of the nonlinear dynamics in the thalamocortical circuit in response to patterned thalamic microstimulationin vivo
Millard, D. C., Wang, Q., Gollnick, C., & Stanley, G. B. (2013) · Journal of Neural Engineering · doi:10.1088/1741-2560/10/6/066011 · Cited by 32
Assessing landslide susceptibility based on hybrid multilayer perceptron with ensemble learning
Hong, H. (2023) · Bulletin of Engineering Geology and the Environment · doi:10.1007/s10064-023-03409-8 · Cited by 31
Impacts of forest structure on precipitation interception and run‐off generation in a semiarid region in northern China
Liu, J., Zhang, Z., & Zhang, M. (2018) · Hydrological Processes · doi:10.1002/hyp.13156 · Cited by 31
Precision toxicity correlates of tumor spatial proximity to organs at risk in cancer patients receiving intensity-modulated radiotherapy
Wentzel, A., Hanula, P., van Dijk, L. V., Elgohari, B., Mohamed, A., Cardenas, C. E., Fuller, C. D., Vock, D. M., Canahuate, G. M., & Marai, G. E. (2020) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2020.05.023 · Cited by 31
A comparison of classifiers and features for authorship authentication of social networking messages
Li, J. S., Chen, L., Monaco, J. V., Singh, P. K., & Tappert, C. C. (2016) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.3918 · Cited by 31
An automatic method for the identification and interpretation of clustered microcalcifications in mammograms
Schmidt, F., Sorantin, E., Szepesvári, C., Graif, E., Becker, M. S., Mayer, H., & Hartwagner, K. A. (1999) · Physics in Medicine and Biology · doi:10.1088/0031-9155/44/5/011 · Cited by 31
Spatiotemporal regression Kriging to predict precipitation using time-series MODIS data
Hu, D., Shu, H., Hu, H., & Xu, J. (2017) · Cluster Computing · doi:10.1007/s10586-016-0708-0 · Cited by 30
In-depth analysis of design & development for sensor-based human activity recognition system
Choudhury, N. A., & Soni, B. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16423-5 · Cited by 29
Fast Recommendations With the M-Distance
Zheng, M., Min, F., Zhang, H., & Chen, W. (2016) · IEEE Access · doi:10.1109/access.2016.2549182 · Cited by 29
Group Chat Ecology in Enterprise Instant Messaging: How Employees Collaborate Through Multi-User Chat Channels on Slack
Wang, D., Wang, H., Yu, M., Ashktorab, Z., & Tan, M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512941 · Cited by 39
PredicTouch
Le, H. V., Schwind, V., Göttlich, P., & Henze, N. (2017) · ACM International Conference on Interactive Surfaces and Spaces (ISS) · doi:10.1145/3132272.3134138 · Cited by 31
Algorithmic Stability for Adaptive Data Analysis
Bassily, R., Kobbi Nissim, B., Smith, A., Steinke, T., Stemmer, U., & Ullman, J. (2021) · SIAM Journal on Computing · doi:10.1137/16m1103646 · Cited by 47
Model selection and error estimation without the agonizing pain
Oneto, L. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1252 · Cited by 27
The interprotein scoring noises in glide docking scores
Wang, W., Zhou, X. F., He, W., Yi, F., Chen, Y., & Chen, X. (2011) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.23173 · Cited by 28
The compressed breast during mammography and breast tomosynthesis: in vivo shape characterization and modeling
Rodríguez‐Ruiz, A., Agasthya, G. A., & Sechopoulos, I. (2017) · Physics in Medicine and Biology · doi:10.1088/1361-6560/aa7cd0 · Cited by 27
Five microRNAs in Serum Are Able to Differentiate Breast Cancer Patients From Healthy Individuals
Feliciano, A., González, L., Garcia‐Mayea, Y., Mir, C., Artola, M., Barragán-Brun, N., Álvarez, R. M., Altés, A., Castellví, J., Benavente, S., et al. (2020) · Frontiers in Oncology · doi:10.3389/fonc.2020.586268 · Cited by 27
HARU Sleep: A Deep Learning-Based Sleep Scoring System With Wearable Sheet-Type Frontal EEG Sensors
Matsumori, S., Teramoto, K., Iyori, H., Soda, T., Yoshimoto, S., & Mizutani, H. (2022) · IEEE Access · doi:10.1109/access.2022.3146337 · Cited by 27
PAC-Bayes control: learning policies that provably generalize to novel environments
Majumdar, A., Farid, A., & Sonar, A. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920959444 · Cited by 29
Optimizing Rice Near-Infrared Models Using Fractional Order Savitzky–Golay Derivation (FOSGD) Combined with Competitive Adaptive Reweighted Sampling (CARS)
Xia, Z., Yang, J., Wang, J., Wang, S., & Liu, Y. (2020) · Applied Spectroscopy · doi:10.1177/0003702819895799 · Cited by 26
Estimation of Apple Flowering Frost Loss for Fruit Yield Based on Gridded Meteorological and Remote Sensing Data in Luochuan, Shaanxi Province, China
Zhu, Y., Yang, G., Yang, H. F., Zhao, F., Han, S., Chen, R., Zhang, C., Yang, X., Liu, M., Cheng, J., et al. (2021) · Remote Sensing · doi:10.3390/rs13091630 · Cited by 26
Bias-variance tradeoffs in program analysis
Sharma, R., Nori, A. V., & Aiken, A. (2014) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/2535838.2535853 · Cited by 25
Identify a shared neural circuit linking multiple neuropsychiatric symptoms with Alzheimer’s pathology
Initiative, A. T. A. D. N., Wang, X., Ren, P., Mapstone, M. E., Conwell, Y., Porsteinsson, A. P., Foxe, J. J., Raizada, R. D. S., & Lin, F. V. (2017) · Brain Imaging and Behavior · doi:10.1007/s11682-017-9767-y · Cited by 25
Information-Theoretic Generalization Bounds for Meta-Learning and Applications
Jose, S. T., & Simeone, O. (2021) · Entropy · doi:10.3390/e23010126 · Cited by 25
Theoretical analysis of cross-validation for estimating the risk of the k-Nearest Neighbor classifierPreprint
Célisse, A., & Mary‐Huard, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04905 · Cited by 35
A Review of Stability in Topic Modeling: Metrics for Assessing and Techniques for Improving Stability
Marani, A. H., & Baumer, E. P. S. (2023) · ACM Computing Surveys · doi:10.1145/3623269 · Cited by 24
The Cross Validation Problem
Langford, J. C. (2005) · Lecture notes in computer science · doi:10.1007/11503415_47 · Cited by 24
Classification of migraine stages based on resting-state EEG power
Cao, Z., Ko, L., Lai, K., Huang, S.-B., Wang, S., & Lin, C. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280582 · Cited by 24
Multivariate analyses applied to fetal, neonatal and pediatric MRI of neurodevelopmental disorders
Levman, J. E. D., & Takahashi, E. (2015) · NeuroImage Clinical · doi:10.1016/j.nicl.2015.09.017 · Cited by 25
Statistical Learning: Stability is Sufficient for Generalization and Necessary and Sufficient for Consistency of Empirical Risk Minimization
Mukherjee, S., Niyogi, P., Poggio, T., & Rifkin, R. M. (2004) · doi:10.21236/ada459857 · Cited by 24
Remote sensing estimation of colored dissolved organic matter (CDOM) from GOCI measurements in the Bohai Sea and Yellow Sea
Ling, Z., Sun, D., Wang, S., Qiu, Z., Huan, Y., Mao, Z., & He, Y. (2019) · Environmental Science and Pollution Research · doi:10.1007/s11356-019-07435-6 · Cited by 23
Machine learning liver histology scores correlate with portal hypertension assessments in nonalcoholic steatohepatitis cirrhosis
Noureddin, M., Goodman, Z. D., Tai, D., Chng, E., Ren, Y., Boudes, P. F., Shlevin, H. H., García–Tsao, G., Harrison, S. A., & Chalasani, N. (2023) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.17363 · Cited by 23
Probabilistic machine learning for the evaluation of presurgical language dominance
Gazit, T., Andelman, F., Glikmann‐Johnston, Y., Gonen, T., Solski, A., Shapira‐Lichter, I., Ovadia, M., Kipervasser, S., Neufeld, M. Y., Fried, I., et al. (2016) · Journal of neurosurgery · doi:10.3171/2015.7.jns142568 · Cited by 22
PAC-bayesian analysis of distribution dependent priors: Tighter risk bounds and stability analysis
Oneto, L., Anguita, D., & Ridella, S. (2016) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.06.019 · Cited by 23
A Leave-One-out Cross Validation Bound for Kernel Methods with Applications in Learning
Zhang, T. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44581-1_28 · Cited by 23
Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and InterpretabilityPreprint
Dziugaite, G. K., Ben-David, S., & Roy, D. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13764 · Cited by 63
Exploratory Machine Learning Modeling of Adaptive and Maladaptive Personality Traits from Passively Sensed BehaviorPreprint
Yan, R., Ringwald, W. R., Vega, J., Kehl, M., Bae, S. W., Dey, A. K., Low, C. A., Wright, A. G. C., & Doryab, A. (2021) · PsyArXiv (OSF Preprints) · doi:10.1016/j.future.2022.02.010 · Cited by 23
A fully automatic extraction of magnetic resonance image features in glioblastoma patients
Zhang, J., Barboriak, D. P., Hobbs, H. A., & Mazurowski, M. A. (2014) · Medical Physics · doi:10.1118/1.4866218 · Cited by 22
Reproducibility in learning
Impagliazzo, R., Lei, R., Pitassi, T., & Sorrell, J. (2022) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3519935.3519973 · Cited by 23
Efficacy and Safety of a Botanical Formula Fuzheng Huayu for Hepatic Fibrosis in Patients with CHC: Results of a Phase 2 Clinical Trial
Hassanein, T., Tai, D., Liu, C., Box, T. D., Tong, M. J., Rossaro, L., Pozza, R., Glenn, J. S., Cheung, R. C., Hemaidan, A., et al. (2022) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2022/4494099 · Cited by 23
Distinguishing schizophrenia and bipolar disorder through a Multiclass Classification model based on multimodal neuroimaging data
Chen, M., Xia, X., Kang, Z., Li, Z., Dai, J., Wu, J., Chen, C., Qiu, Y., Liu, T., Liu, Y., et al. (2024) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2024.02.024 · Cited by 24
The heterogeneous feature selection with structural sparsity for multimedia annotation and hashing: a survey
Wu, F. Q., Han, Y., Liu, X., Shao, J., Zhuang, Y., & Zhang, Z. M. (2012) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-012-0001-9 · Cited by 23
Cancer Markers Selection Using Network-Based Cox Regression: A Methodological and Computational Practice
Iuliano, A., Occhipinti, A., Angelini, C., Feis, I. D., & Lió, P. (2016) · Frontiers in Physiology · doi:10.3389/fphys.2016.00208 · Cited by 22
Model selection via cross-validation in density estimation, regression, and change-points detection
Célisse, A. (2008) · HAL · Cited by 22
Scaling-Up Distributed Processing of Data Streams for Machine Learning
Nokleby, M., Raja, H., & Bajwa, W. U. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3021381 · Cited by 22
Satellite-based estimation of the impacts of summertime wildfires on PM 2.5 concentration in the United States
Xue, Z., Gupta, P., & Christopher, S. A. (2021) · Atmospheric chemistry and physics · doi:10.5194/acp-21-11243-2021 · Cited by 23
Sets of approximating functions with finite Vapnik–Chervonenkis dimension for nearest-neighbors algorithms
Klęsk, P., & Korzeń, M. (2011) · Pattern Recognition Letters · doi:10.1016/j.patrec.2011.07.012 · Cited by 21
Concentration Bounds for Unigrams Language Model
Drukh, E., & Mansour, Y. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27819-1_12 · Cited by 21
Pattern Recognition for Conditionally Independent DataPreprint
Ryabko, D. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0507040 · Cited by 21
Identification of Adulterated and Non-adulterated Norwegian Salmon Using FTIR and an Improved PLS-DA Method
Wu, T., Zhong, N., & Yang, L. (2017) · Food Analytical Methods · doi:10.1007/s12161-017-1135-4 · Cited by 20
Permeability of Fluid-Phase Phospholipid Bilayers: Assessment and Useful Correlations for Permeability Screening and Other Applications
Nitsche, J. M., & Kasting, G. B. (2013) · Journal of Pharmaceutical Sciences · doi:10.1002/jps.23471 · Cited by 19
An Active Feature Selection Strategy for DWT in Artificial Taste
Liu, T., Chen, Y., Li, D., & Wu, M. (2018) · Journal of Sensors · doi:10.1155/2018/9709505 · Cited by 20
Proposal of a new stability concept to detect changes in unsupervised data streams
Vallim, R. M. M., & de Mello, R. F. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.06.031 · Cited by 19
MRI-Based Radiomics Approach Predicts Tumor Recurrence in ER + /HER2 − Early Breast Cancer Patients
Chiacchiaretta, P., Mastrodicasa, D., Chiarelli, A. M., Luberti, R., Croce, P., Sguera, M., Torrione, C., Marinelli, C., Marchetti, C., Angelucci, D., et al. (2022) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-023-00781-5 · Cited by 21
Contributions Estimation in Federated Learning: A Comprehensive Experimental Evaluation
Chen, Y., Li, K., Li, G., & Wang, Y. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3659437.3659459 · Cited by 22
Modeling false positive error making patterns in radiology trainees for improved mammography education
Zhang, J., Silber, J. I., & Mazurowski, M. A. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.01.007 · Cited by 19
Generating highly accurate prediction hypotheses through collaborative ensemble learning
Arsov, N., Pavlovski, M., Basnarkov, L., & Kocarev, L. (2017) · Scientific Reports · doi:10.1038/srep44649 · Cited by 19
Predicting Volume and Biomass Change from Multi-Temporal Lidar Sampling and Remeasured Field Inventory Data in Panther Creek Watershed, Oregon, USA
Poudel, K. P., Flewelling, J. W., & Temesgen, H. (2018) · Forests · doi:10.3390/f9010028 · Cited by 19
CasMDN: A deep learning-based multivariate distribution modelling approach and its application in geotechnical engineering
Zhang, J., Han, S., Li, M., Li, H., Zhao, W., Wang, J., & Liang, H. (2024) · Computers and Geotechnics · doi:10.1016/j.compgeo.2024.106164 · Cited by 18
MSpectraAI: a powerful platform for deciphering proteome profiling of multi-tumor mass spectrometry data by using deep neural networks
Wang, S., Zhu, H., Zhou, H., Cheng, J., & Yang, H. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03783-0 · Cited by 18
The Application of an Unmanned Aerial System and Machine Learning Techniques for Red Clover-Grass Mixture Yield Estimation under Variety Performance Trials
Li, K.-Y., Burnside, N. G., de Lima, R. S., Villoslada, M., Sepp, K., Yang, M., Raet, J., Vain, A., Selge, A., & Sepp, K. (2021) · Remote Sensing · doi:10.3390/rs13101994 · Cited by 18
Learning to Translate Sequence and Structure to Function: Identifying DNA Binding and Membrane Binding Proteins
Langlois, R. E., Carson, M. B., Bhardwaj, N., & Lu, H. J. (2007) · Annals of Biomedical Engineering · doi:10.1007/s10439-007-9312-z · Cited by 18
Classification of Jet Fuel Properties by Near-Infrared Spectroscopy Using Fuzzy Rule-Building Expert Systems and Support Vector Machines
Xu, Z., Bunker, C. E., & Harrington, P. D. B. (2010) · Applied Spectroscopy · doi:10.1366/000370210793335115 · Cited by 18
Estimator for generalization performance of machine learning model trained by biased data collected from multiple references
Okazaki, Y., Okazaki, S., Asamoto, S., Yamaji, T., & Ishige, M. (2023) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12992 · Cited by 17
Multi-omics staging of locally advanced rectal cancer predicts treatment response: a pilot study
Cicalini, I., Chiarelli, A. M., Chiacchiaretta, P., Perpetuini, D., Rosa, C., Mastrodicasa, D., d’Annibale, M., Trebeschi, S., Serafini, F. L., Cocco, G., et al. (2024) · La radiologia medica · doi:10.1007/s11547-024-01811-0 · Cited by 17
Preventing Over-Fitting of Cross-Validation with Kernel Stability
Liu, Y., & Liao, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44851-9_19 · Cited by 17
Algorithmic stability and uniform generalization
Alabdulmohsin, I. (2015) · King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) · Cited by 19
A 3D Dual Path U-Net of Cancer Segmentation Based on MRI
He, Y., Yu, X., Liu, C., Zhang, J., Hu, K., & Zhu, H. C. (2018) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc.2018.8492781 · Cited by 17
Modeling hemodynamic responses in auditory cortex at 1.5 T using variable duration imaging acoustic noise
Hu, S., Olulade, O. A., González-Castillo, J., Santos, J., Kim, S., Tamer, G. G., Luh, W., & Talavage, T. M. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.051 · Cited by 17
Learning From An Optimization ViewpointPreprint
Sridharan, K. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1204.4145 · Cited by 17
Cross-validation Confidence Intervals for Test ErrorPreprint
Bayle, P., Bayle, A., Janson, L., & Mackey, L. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.12671 · Cited by 17
Diagnosis of T-cell-mediated kidney rejection by biopsy-based proteomic biomarkers and machine learning
Fang, F., Liu, P., Song, L., Wagner, P., Bartlett, D. L., Ma, L., Li, X., Rahimian, M. A., Tseng, G. C., Randhawa, P. S., et al. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1090373 · Cited by 16
Fluorescent electronic tongue supported with water-borne polyurethane for the discrimination of nitroaromatics in aqueous solution
Yang, T., Feng, C., Zhao, P., Wu, Y., Ding, Y., Wang, G., & Hu, A. (2020) · Journal of Materials Chemistry C · doi:10.1039/c9tc06759k · Cited by 16
Distinct effects of prematurity on MRI metrics of brain functional connectivity, activity, and structure: Univariate and multivariate analyses
Chiarelli, A. M., Sestieri, C., Navarra, R., Wise, R. G., & Caulo, M. (2021) · Human Brain Mapping · doi:10.1002/hbm.25456 · Cited by 14
A simple algorithm for learning stable machines
Andonova, S., Elisseeff, A., Evgeniou, T., & Pontil, M. (2002) · Cited by 16
An Ensemble of Component-Based and Population-Based Self-Organizing Maps for the Identification of the Degradation State of Insulated-Gate Bipolar Transistors
Rigamonti, M., Baraldi, P., Alessi, A., Zio, E., Astigarraga, D., & Galarza, A. (2018) · IEEE Transactions on Reliability · doi:10.1109/tr.2018.2834828 · Cited by 16
Student Performance Assessment Using Bayesian Network and Web Portfolios
Liu, C., Chen, G., Wang, C.-Y., & Lu, C.-F. (2002) · Journal of Educational Computing Research · doi:10.2190/el2a-tqf1-pyvm-n51v · Cited by 16
Learning to predict channel stability using biogeomorphic features
Morét, S. L., Langford, W. T., & Margineantu, D. D. (2005) · Ecological Modelling · doi:10.1016/j.ecolmodel.2005.08.011 · Cited by 16
Citizens’ opinion about investment in public transport projects in cities
Calvo, F. J., Medialdea, A., & Ferri‐García, R. (2019) · International Journal of Sustainable Transportation · doi:10.1080/15568318.2019.1630529 · Cited by 15
Near-Optimal Bounds for Cross-Validation via Loss Stability
Kumar, R., Lokshtanov, D., Vassilvitskii, S., & Vattani, A. (2013) · International Conference on Machine Learning · Cited by 15
Electronic Tongue Recognition with Feature Specificity Enhancement
Liu, T., Chen, Y., Li, D., Tao, Y., & Cao, J. (2020) · Sensors · doi:10.3390/s20030772 · Cited by 15
Bootstrapping rule induction to achieve rule stability and reduction
Waitman, L. R., Fisher, D., & King, P. H. (2006) · Journal of Intelligent Information Systems · doi:10.1007/s10844-006-1626-z · Cited by 15
Cancer classification based on multiple dimensions: SNV patterns
Li, B., Yu, L., & Gao, L. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106270 · Cited by 14

Authors: Kearns, M., & Ron, D.
Affiliations: AT&T (United States) (US)
Tel Aviv University (IL)
ORCID: Kearns, Michael
Ron, Dana
Venue: Neural Computation
Publisher: MIT Press - Journals
Published: 1999-08-01
Volume: 11
Issue: 6
Pages: 1427-1453
ISSN: 0899-7667 (ISSN-L)
1530-888X
DOI: 10.1162/089976699300016304
OpenAlex: W2138263042
Cited by: 407
References: 18
Citations by year: 2026: 4 · 2025: 7 · 2024: 11 · 2023: 18 · 2022: 22 · 2021: 26 · 2020: 35 · 2019: 46 · 2018: 23 · 2017: 22 · 2016: 27 · 2015: 18 · 2014: 17 · 2013: 24 · 2012: 14
FWCI: 4.55
Type: journal article
Language: en
Open access: closed
Locations: Neural Computation (publishedVersion)
PubMed (publishedVersion)
Funders: National Science Foundation
Office of Naval Research
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: leave-one-out cross-validation, generalization error estimation, algorithm stability, VC dimension, lower bounds
MeSH terms: Algorithms, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Algorithmic Stability and Sanity-Check Bounds for Leave-One-Out Cross-Validation

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks