Practical Considerations and Applied Examples of Cross-Validation for Model Development and Evaluation in Health Care: Tutorial
Open accessPublished version of a preprint
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Cross-validation remains a popular means of developing and validating artificial intelligence for health care. Numerous subtypes of cross-validation exist. Although tutorials on this validation strategy have been published and some with applied examples, we present here a practical tutorial comparing multiple forms of cross-validation using a widely accessible, real-world electronic health care data set: Medical Information Mart for Intensive Care-III (MIMIC-III). This tutorial explored methods such as K-fold cross-validation and nested cross-validation, highlighting their advantages and disadvantages across 2 common predictive modeling use cases: classification (mortality) and regression (length of stay). We aimed to provide readers with reproducible notebooks and best practices for modeling with electronic health care data. We also described sets of useful recommendations as we demonstrated that nested cross-validation reduces optimistic bias but comes with additional computational challenges. This tutorial might improve the community’s understanding of these important methods while catalyzing the modeling community to apply these guides directly in their work using the published code.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Stratified cross-validation can assist in establishing representative class proportions within each fold to improve model generalizability64.”
From Bias recognition and mitigation strategies in artificial intelligence healthcare applications (2025) · cites this paper as [64]
In the citing paper
“This technique is reliant on the availability of large data resources and may not fully account for minority subgroups when not sufficiently represented.”
From Bias recognition and mitigation strategies in artificial intelligence healthcare applications (2025) · cites this paper as [64]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881 · cited 1×
“…this should mitigate the source of optimistic bias introduced when cross-validation is used to tune model parameters on the same data used for model performance evaluation (ie, observed performance can be spuriously high owing to randomness in the data and the learning algorithm) [28,29].”
MethodAssessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Antlanger, M., Aschauer, S., Kopecky, C., Hecking, M., Kovarik, J. J., Werzowa, J., Mascherbauer, J., Genser, B., Säemann, M. D., & Bonderman, D. (2017) · Kidney and Blood Pressure Research · doi:10.1159/000473868 · Cited by 2,808
Bias in error estimation when using cross-validation for model selection
Varma, S., & Simon, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-91 · Cited by 2,485 · cited 1×
“…this should mitigate the source of optimistic bias introduced when cross-validation is used to tune model parameters on the same data used for model performance evaluation (ie, observed performance can be spuriously high owing to randomness in the data and the learning algorithm) [28,29].”
MethodEstimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300 · cited 1×
“Traditional bootstrapping is referred to as out-of-bag bootstrapping, whereas further improvements include the “0.632” and “0.632+” methods that apply additional forms of bias adjustment [23,24].”
MethodInfluentialTowards better clinical prediction models: seven steps for development and an ABCD for validation
Steyerberg, E. W., & Vergouwe, Y. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu207 · Cited by 1,928
Improvements on Cross-Validation: The 632+ Bootstrap Method
Efron, B., & Tibshirani, R. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474007 · Cited by 1,636 · cited 1×
“Traditional bootstrapping is referred to as out-of-bag bootstrapping, whereas further improvements include the “0.632” and “0.632+” methods that apply additional forms of bias adjustment [23,24].”
MethodInfluentialSteyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491 · cited 1×
“Clinical usefulness based in decision analysis is the third major area of evaluation [22].”
BackgroundSelection bias in gene extraction on the basis of microarray gene-expression data
Ambroise, C., & McLachlan, G. J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.102102699 · Cited by 1,480 · cited 1×
“Cross-validation surveys with practical examples, such as those involving microarray and neurologic data, have been published [12,13].”
Background
Research works printed but not linked: 3
- Mosteller F Tukey JW Lindzey G Aronson E Data analysis, including statistics Handbook of Social Psychology, Volume 2 1968 Boston, MA Addison-Wesley 1 26
- Kohavi R A study of cross-validation and bootstrap for accuracy estimation and model selection Proceedings of the 14th International Joint Conference on Artificial Intelligence - Volume 2 1995 IJCAI'95 August 20-25, 1995 Montreal, Quebec
- Cawley GC Talbot NL On over-fitting in model selection and subsequent selection bias in performance evaluation J Mach Learn Res 2010 11 2079 107
Cited by
Items 101–186 of 186
Barros, T., Figueiredo, T., Gamboa, P. V., & Santos, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5476112
Barros, T., Figueiredo, T., Gamboa, P., & Santos, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5817138
MAWC-Net: A Multi-Scale Attention Wavelet Convolutional Neural Network for Soil pH Prediction
Cheng, X., Liu, Z., Kang, Y., Xie, X., Deng, Y., Lu, Q., Tang, J., Shi, Y., & Zhao, J. (2025) · Applied Sciences · doi:10.3390/app16010054
Taleghani, E., Lyu, R., Shackleford, T. L., Rex, J., Hale, I. C., Talavage, T. M., & Florczynski, M. M. (2026) · Journal of the American Academy of Orthopaedic Surgeons · doi:10.5435/jaaos-d-25-01150
Buianova, I., Silvestrin, M., Deng, J. D., & Pat, N. (2026) · eLife · doi:10.7554/elife.108109.2
Sheikh, I. R., Sun, X., Chen, X., Dong, Z., Song, H., & Wang, F. (2026) · Results in Engineering · doi:10.2139/ssrn.6156347
Colorectal Cancer Patient Survival Prediction Using TabNet Model
Damayanti, R. Y., Wisesty, U. N., & Rismala, R. (2025) · International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA) · doi:10.1109/icicyta68677.2025.11362595
“To ensure robustness and mitigate the effects of variance in the data split, the model was initially trained using a 5-fold cross-validation strategy [16].”
Lin, W., Xiao, C., Xiao, L., Fang, J., Xu, X., & Fang, Y. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1733312
Jiang, Y., Wang, X., Hu, W., Zou, S., Peng, L., Wang, Z., Hou, S., Li, J., & Tai, J. (2026) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2026.1740863
“In this research, we used patient-wise aggregation and splits with Stratified 10-fold CV to prevent leakage across multiple clips from the same child and to match deployment to new patients (Bradshaw et al., 2023; Wilimitis and Walsh, 2023).”
Yu, T., Yuan, J., Xiang, Y., & Li, M. (2026) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-026-00843-0
Acceleration of detailed VLSI routing using machine learning methods
Stempkovsky, A. L., Telpuhkov, D. V., Solovyev, R. A., Mkrtychan, I. A., & Shafeev, I. I. (2026) · INFORMACIONNYE TEHNOLOGII · doi:10.17587/it.32.67-76
Choi, S., Nah, S., Lee, J. W., Kim, M., & Han, S. (2026) · Pediatric Emergency Care · doi:10.1097/pec.0000000000003587
Anejjar, K., Amazal, F. A., & Idri, A. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17915-9_11
Rijab, S., Khorrami, M. K., & Mohammadi, M. (2026) · Journal of the Iranian Chemical Society · doi:10.1007/s13738-026-03351-y
Automated Fabric Defect Detection Using RTMDet: Application in Denim Manufacturing
Liu, M., & Kato, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7072-0_44
Zhou, F., Lokker, C., Parrish, R., Haynes, R. B., Iorio, A., Saha, A., & Afzal, M. (2026) · JMIR AI · doi:10.2196/77311
Ko, C., Jeon, D., Song, Y., & Lee, Y. (2026) · Applied Sciences · doi:10.3390/app16062979
Sheikh, I. R., Sun, X., Chen, X., Dong, Z., Song, H., & Wang, F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6478732
Recent advances in machine learning-enhanced extracellular vesicle omics for oncology
Wang, Z., Lu, W. M., Guo, Z., Liu, H., Yin, Z., Ma, S., Wang, Z., Huang, Z., Sun, X. W., Xu, Y., et al. (2026) · Journal of Nanobiotechnology · doi:10.1186/s12951-026-04342-0 · Cited by 1
LiDAR-Enhanced 3D VR and consumer decision-making: a new era for virtual tourism
Rashidin, M. S., Taheri, B., & Chiappa, G. D. (2026) · Information Technology & Tourism · doi:10.1007/s40558-026-00369-z · Cited by 1
“The physiological cross-validation thus enhances construct validity and supports the development of more predictive models for immersive tourism experience design (Wilimitis and Walsh 2023).”
Machine Learning for Secure Cardiovascular Risk Assessment in Distributed Healthcare Environments
Tariq, A., Arikhad, M., & Sultana, A. (2026) · International Journal of Innovative Research in Computer Science & Technology · doi:10.55524/ijircst.2026.14.1.19 · Cited by 1
Construction of a classification model for dementia among Brazilian adults aged 50 and over
Menezes, F., Barretto, M. C. F. G., Garcia, E. Q. C., Ferreira, T. A. E., & Alves, J. G. B. (2026) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2026.1789012 · Cited by 1
Zheng, D., Wang, J., Liu, J., Yuan, H., Xue, J., Zeng, H., Zhang, W., Li, X., Cai, J., & Tian, X. (2026) · Applied Radiation and Isotopes · doi:10.1016/j.apradiso.2026.112635
Ramineni, S. K. A., Song, Z., Garg, A., & Kamchoom, V. (2026) · Indian geotechnical journal · doi:10.1007/s40098-026-01539-w
A Robust Deep Learning Approach for COPD Automated Detection
Xu, S., Deo, R. C., Prasad, S. S., Barua, P. D., Soar, J., & Acharya, R. (2026) · Sensors · doi:10.3390/s26092713
“Importantly, Grad-CAM visualization provides clini-https cally meaningful evidence, showing that the model focuses on physiologically relevant respiratory signatures [50].”
Dermawan, D., Chtita, S., & Alotaiq, N. (2026) · Pharmaceutics · doi:10.3390/pharmaceutics18050542 · Cited by 1
Moassefi, M., Decker, P. A., Conte, G. M., Kosel, M., Molinaro, A. M., Niederschweiberer, M. A., Nikanpour, Y., Ruff, M. W., Burns, T. C., Farooq, U., et al. (2026) · Cancer Research Communications · doi:10.1158/2767-9764.crc-25-0710
“Nested cross-validation could be incorporated into the study design of the ensemble approach to get an unbiased estimate of prediction performance and overfitting (36); however, this requires further splitting apart the model development cohort into smaller training and held-out validation sets.”
Минимизация ошибок нейросетевых моделей в библиотечно-издательской сфере: подходы и их эффективность
Рзянкин, И. С. (2025) · Sovremennye informacionnye tehnologii i IT-obrazovanie. · doi:10.25559/sitito.021.202502.272-286
Hossain, R., Ali, L. E., Ripon, K. S. N., Bulbul, M. F., & Tonmoy, M. M. H. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491498 · Cited by 1
“For model selection and threshold tuning, unbiased performance estimates call for rigorous procedures such as nested cross-validation, although the associated computational cost must be managed carefully [24], [25].”
Tan, W. Y., Lee, T. Y., Tan, K. B., Koh, M. S., Abisheganaden, J. A., Lam, S. S. W., Chotirmall, S. H., Yadav, C. P., Yii, A. C. A., Tiew, P. Y., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-116086
Polpitiya, A. D., Cox, C., Butler, H., Badsha, M. B., Sommerville, L. J., Boniface, J. J., Saade, G. R., & Kearney, P. E. (2026) · Diagnostics · doi:10.3390/diagnostics16101487 · Cited by 1
“…coefficients were validated and demonstrating acceptable performance measurements, they were updated using the validated training pipeline (excluding feature selection steps) with the full cohort dataset to ensure that the best possible final Model would be deployed for clinical testing [39,40].”
Exploratory PET/CT Radiomics for Predicting Early Progression in Locally Advanced Pancreatic Cancer
Fiore, M., Cordelli, E., Petrianni, G. M., D’Ercole, G., Trecca, P., Taralli, S., Vaccara, V. L., Caputo, D. D., Ippolito, E., Calcagni, M. L., et al. (2026) · Diagnostics · doi:10.3390/diagnostics16101499
“Previous methodological studies have shown that nested or repeated cross-validation schemes are preferable in small-sample settings, as they better account for model-selection uncertainty and reduce overfitting, although some degree of optimism may still remain [20,21].”
Design and Characterization of the Synthetic Microbial Genome
Twala, N., Takundwa, M. M., Prinsloo, E., & Raj, D. B. T. G. (2026) · Springer eBooks · doi:10.1007/978-981-95-3957-4_10
Arun, N., Weston, A., Islam, F., Bolan, C. W., Toskich, B., Gil-López, F., & Galeotti, J. (2026) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi61048.2026.11515550
“Following established practice for limited-sample settings [33], [34], predictions from all outer test folds are pooled to compute overall AUROC, providing a conservative yet robust estimate of out-of-sample performance while using each sample exactly once for evaluation.”
Prediction of cooking-induced food transformations: A survey and outlook towards machine learning
Jmaa, A. B. B., Hanon, C., Parfait, J.-Y., & Debaste, F. (2026) · Journal of Food Engineering · doi:10.1016/j.jfoodeng.2026.113175
Sanker, V., Mahesh, D., Li, Z., Thaller, A., Heesen, P., Liverani, L., Wang, D., Cavagnaro, M. J., Medikonda, R. T., Prolo, L. M., et al. (2026) · Journal of Imaging · doi:10.3390/jimaging12060251
“This limits the ability to assess variability in model performance and may result in optimistic or unstable estimates [27].”
Foodeh, T., Korani, M. A., Teymourzadeh, M., Hassanpour-Fard-Khor, M., Abdollahi, Z., Hatami, B., Payandeh, A., Shojaeshafiei, F., Ebrahimi, S. M. S., Fayazbakhsh, F., et al. (2026) · Neurosurgical Review · doi:10.1007/s10143-026-04362-2
Kulkarni, M., Joshi, D., & Patel, S. (2026) · Journal of Pharmaceutical Innovation · doi:10.1007/s12247-026-10795-5
“In addition to overs - ampling, stratified K-Fold cross-validation (with 5, 7, and 10 folds) was employed to address class imbalance during the model training phase effectively [ 41, 42].”
Alluri, D. S., & Pabon‐Rodriguez, F. M. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-106819
“41 42 K-fold cross-validation mitigates the risk of overfitting, provides more reliable estimates of the model’s performance, and enhances the reliability and generalis-ability of the results, making it a robust validation strategy for ML applications.”
Szponar, M., Gmaj, B., Orłowski, J., & Kamiński, J. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110779
Ibrahim, A. S. I., & Sönmez, Ö. (2026) · Chemical Engineering and Processing - Process Intensification · doi:10.2139/ssrn.6945149
Acoustic Correlates of Phases of Understanding in a Naturalistic Literary Task
Lazic, M., & Woodruff, E. (2026) · Electronics · doi:10.3390/electronics15122700
Artificial intelligence reshaping the gout diagnosis and treatment paradigm
Zhang, Y., Niu, B., Xu, W., Hua, Y., & Xia, D. (2026) · Journal of Pharmaceutical Analysis · doi:10.1016/j.jpha.2026.101702
Ventura, V. D. G. J., de Andrade, C. M. V., Almeida, J. M., Pessoa, B. P., Polanczyk, C. A., Fonseca, G., Boersma, E. H., Vianna, H. R., Farah, K. D. P., da Rocha, L. C. D., et al. (2026) · JMIR Formative Research · doi:10.2196/86379
“After selecting the optimal hyperparameter configuration, the model was retrained on the full training data (training + validation) following standard practice and prior work [39,40], as well as the default behavior of widely used libraries such as scikit-learn (GridSearchCV with refit=True).”
InfluentialKhan, A., Ali, A., Hanan, F., & Mohmand, M. I. (2026) · Healthcare · doi:10.3390/healthcare14131881
Shah, N. J., Ramshekar, A., Asare‐Bediako, B., Tankersley, M. P., Huang, H.-C., Beri, S., Kunz, E., Lee, A., & Hartnett, M. E. (2026) · Translational Vision Science & Technology · doi:10.1167/tvst.15.6.41
Naturalistic driving data can identify elevated driving risk in adults with Type 1 diabetes
Joshi, A., Rizzo, M., & Sharma, A. (2026) · Traffic Injury Prevention · doi:10.1080/15389588.2026.2685782
A local outpatient practice-level prediction model for short-term psychiatric emergency presentation
Havlík, J., Tyrrell, B. D., Polaschek, J. X., Bell, N., & Arzubi, E. (2026) · JAMIA Open · doi:10.64898/2026.06.29.26356785
Shi, G., Guan, Z., Li, H., & Tian, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7049460
Lee, P., Wang, H.-S., Hsu, S., & Liu, C.-H. (2026) · Diagnostics · doi:10.3390/diagnostics16162585
Akhpanbayeva, A., & Kumalakov, B. (2026) · IEEE International Conference on Smart Information Systems and Technologies (SIST) · doi:10.1109/sist61674.2026.11596223
“The system implements random split without replacement method which stops feature values from transferring between different sets according to standards that maintain objective accuracy evaluation according to reference [12] and reference [19].”
Zhu, Z., Xiang, Y., & Tian, X. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1878438
Luo, M., Harsini, S., Rahmim, A., & Salmanpour, M. R. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02093-w
Hussein, H. M., Shawky, B. T., Barry, H. M. A., & Sayed, I. E. T. E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7207439
Shrinkage Regression: Ridge Regression, LASSO, and Elastic Net Regression
Kovač, N., Ratković, K., Simeunović, M., & Farahani, H. (2026) · Springer eBooks · doi:10.1007/978-3-032-18312-5_15
An interpretable machine learning framework for pediatric appendicitis and peritonitis diagnosis
Dyuthishree, A., Shenoy, B., Sampathila, N., Ujwal, P., Prabhu, O., & Chadaga, K. (2026) · Healthcare Analytics · doi:10.1016/j.health.2026.100483
Özmen, B. B., & Schwarz, G. S. (2026) · Journal of Plastic Reconstructive & Aesthetic Surgery · doi:10.1016/j.bjps.2026.08.006
Reliable device state detection from RF telemetry using optimized machine learning classifiers
Zhang, Y., Chen, J., Jing, R., & Zheng, M. (2026) · Computer Networks · doi:10.1016/j.comnet.2026.112682
Haekal, M., Khotimah, S. N., Suwandi, G. R. F., & Haryanto, F. (2026) · Physiological Measurement · doi:10.1088/1361-6579/ae99aa
“Model development was performed using record-wise 10-fold cross-validation to prevent data leakage between training and validation sets (Wilimitis and Walsh 2023).”
Kao, P.-C., Wu, C.-C., & Yeh, J. (2026) · Diagnostics · doi:10.3390/diagnostics16162557 · Cited by 2
“It also enabled the selection of the optimal model for further performance evaluation on the validation dataset [28].”
Yang, Z.-R., Fu, Y., Mou, Y., Tian, F., & Qian-min, Y. (2026) · Natural Gas Industry B · doi:10.1016/j.ngib.2026.07.006
Prediction of Side Effects from Breast Radiation Therapy—Integration of Clinical and Genomic Data
Janapy, A. A., Webb, A. J., Jandu, H. K., Aguado-Barrera, M. E., Aguado-Flor, E., Azria, D., Bourgier, C., Bultijnck, R., Choudhury, A., Ruysscher, D. D., et al. (2026) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2026.08.026
Metabolic and Arterial Stiffness Correlates of Aerobic Capacity in Male Occupational Personnel
Liu, Y., Conner, M. J., Dawes, J. J., Martin, S. E., Gonzalez, D. E., & Kreider, R. B. (2026) · International Journal of Sports Medicine · doi:10.1055/a-2940-6578
He, S., Mehboob, S., Afghan, N., & Hung, N. M. (2026) · Journal of Food Composition and Analysis · doi:10.1016/j.jfca.2026.109474 · Cited by 1
Tariq, M. U. N., Akram, R., Sabir, I., Masud, M. I., Dmour, H. A., & Kashif, M. (2026) · Journal of Optics · doi:10.1007/s12596-026-03272-2
Learn to Build Basic Machine Learning Models from Health Data: A Hands-On Guide for Non-Coders
Mondal, H., Sarangi, P. K., & Mondal, S. (2026) · Indian journal of radiology and imaging - new series/Indian journal of radiology and imaging/Indian Journal of Radiology & Imaging · doi:10.1055/s-0046-1827802
Baleba, S. B. S., Omondi, V. O., Aigbedion-Atalor, P. O., Peter, E., Diallo, S., & Agboka, K. M. (2026) · Journal of Insect Science · doi:10.1093/jisesa/ieag087 · Cited by 1
Havlík, J., Murphy, M. J., Rhee, T. G., Humphreys, K. N., & Rosenheck, R. A. (2026) · Nature Mental Health · doi:10.1038/s44220-026-00717-2
“We tuned hyperparameters on and subsequently tested these classifiers’ performance on our training data in a systematic manner using a nested 5 × 5 stratified cross-validation framework 72,73 with a dependent variable of past-year AUD, OUD or CUD.”
Automated Platelet Classification Under Different Solvent Conditions Through Deep Learning
Daniel, V. O., Coupland, C. A., Sander, N., Mboli, J. S., Calaminus, S. D. J., & Sousa, E. (2026) · Journal of Imaging · doi:10.3390/jimaging12100519
“Cross-validation provided robust performance assessment across multiple data splits [55,56].”
Xu, T., Li, D., Wang, F., Sun, Y., & Sun, H. (2026) · Frontiers in Forests and Global Change · doi:10.3389/ffgc.2026.1894535
Early Sound-Based Meal Onset Detection Using Hidden Markov Models (HMMs)
Lema-Pérez, L., Erntsen, M. G., Stavdahl, Ø., & Fougner, A. L. (2026) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2026.3732514
“Because feature selection and hyperparameter tuning were performed prior to this analysis, the LOSO CV results should be interpreted as an assessment of cross-subject robustness rather than as a fully independent estimate of generalisation performance [53].”
Scalable AutoML Frameworks for End-to-End Optimization of Data Science Workflows
Udariansyah, D., Alshar’e, M., Yadav, P. K., & Tomar, A. (2026) · Journal of Data Sciences · doi:10.61453/jods.v20260208
Yılmaz, L. Z. B., Küçükakçalı, Z., & Akbulut, S. (2026) · Viruses · doi:10.3390/v18091009
“In small-sample omics studies, data-dependent operations—including scaling, feature selection, and tuning—should therefore be restricted to the training partition and repeated within each resampling iteration to reduce preprocessing leakage and optimistic bias in performance estimation [24,25,28].”
Khamehchi, M., Cornelli, U., Luque, R., & Casella, C. (2026) · Microplastics · doi:10.3390/microplastics5030184
Smartphone Passive Digital Phenotyping in Frontotemporal Lobar DegenerationPreprint
Paolillo, E. W., Dhanam, S., Taylor, J. C., Sanderson‐Cimino, M. E., Fregly, R., Saloner, R., Casaletto, K. B., Kramer, J. H., Miller, B. L., Seeley, W. W., et al. (2026) · medRxiv · doi:10.64898/2026.09.18.26363439
Seyhan, S., Uğur, L., & Erdoğan, M. M. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15197469
Kembayeva, Z., Dosymbetova, G., & Nurgaliyev, M. (2026) · EKTU Journal of Information and Communication Sciences · doi:10.51885/3134-8025_jics_2026_3_4
Khan, I., Khan, I., Noor, U., Irfan, M., & Zhiqiang, L. (2026) · Archives of Microbiology · doi:10.1007/s00203-026-05209-6
Machine learning in placental research: a review and guide for future studies
Smith, K. W., Solmonson, A., Lemaire, M., & Wilson, S. L. (2026) · Placenta · doi:10.1016/j.placenta.2026.10.001
Krämer, M., & Kramer, T. (2026) · JMIR Medical Informatics · doi:10.2196/97372
“First, the inclusion of 6 different ML models with cross-validation and independent test evaluation strengthens internal validity [50,51].”
Industrial Validation of Generative Domain Adaptation for Cross-Fabric Defect Detection
Liu, M., & Kato, S. (2026) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa68377.2026.11724973
“…takes a few hours per defect type, and fine-tuning the detector takes about an hour, so a new fabric can typically be supported within roughly one GPU-day without interrupting production. e) Evaluation Methodology: To obtain robust results, we used a 5-fold cross-validation procedure [22].”
Farid, T. B., Islam, M. A., & Banik, S. C. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.113610
Seilheimer, R. L., Vardalakis, N., Choi, W. B., Qiu, L., Campos, G., Nho, Y., Chen, H., Shuvaev, S., Noecker, A. M., Chang, A., et al. (2026) · medRxiv · doi:10.64898/2026.10.19.26365556
Huang, B., Shen, L., Hu, F., Zhou, J., Song, X., Dong, Z., Wu, J., Fu, H., Feng, J., Yang, Z., et al. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1968970
Development of standard weight equation and standard length categories for Emerald Bowfin
Nepal, V. (2026) · North American Journal of Fisheries Management · doi:10.1093/najfmt/vqag104
Huang, Y. Q., Hothi, H., Weiss, S., Hoang, P., Tan, C., Straus, S. E., Li, Z., Guan, J., Beaton, D., Liu, B., et al. (2026) · Brain Injury · doi:10.1080/02699052.2026.2752517
Metadata
- Authors
- Wilimitis, D., & Walsh, C. G.
- Affiliations
- Vanderbilt University (US)
- Corresponding author
- Walsh, Colin G
- Venue
- JMIR AI
- Publisher
- JMIR Publications Inc.
- Published
- 2023-12-18
- Volume
- 2
- Pages
- e49023
- ISSN
- 2817-1705 (ISSN-L)
- PMC
- PMC11041453
- OpenAlex
- W4389883512
- Cited by
- 185
- References
- 33
- Citations by year
- 2026: 113 · 2025: 63 · 2024: 9
- FWCI
- 17.44
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- JMIR AI (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- National Institute of Mental Health (R01MH120122, R01MH118233, R01MH121455, R01MH116269)
- National Human Genome Research Institute (RM1HG009034, U54HG012510)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- cross-validation, nested cross-validation, k-fold cross-validation, model validation, artificial intelligence in healthcare, MIMIC-III, mortality prediction, length of stay prediction, electronic health records, optimism bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.