The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Recommendations for Reporting Machine Learning Analyses in Clinical Research
Open accessAbstract
Use of machine learning (ML) in clinical research is growing steadily given the increasing availability of complex clinical data sets. ML presents important advantages in terms of predictive performance and identifying undiscovered subpopulations of patients with specific physiology and prognoses. Despite this popularity, many clinicians and researchers are not yet familiar with evaluating and interpreting ML analyses. Consequently, readers and peer-reviewers alike may either overestimate or underestimate the validity and credibility of an ML-based model. Conversely, ML experts without clinical experience may present details of the analysis that are too granular for a clinical readership to assess. Overwhelming evidence has shown poor reproducibility and reporting of ML models in clinical research suggesting the need for ML analyses to be presented in a clear, concise, and comprehensible manner to facilitate understanding and critical evaluation. We present a recommendation for transparent and structured reporting of ML analysis results specifically directed at clinical researchers. Furthermore, we provide a list of key reporting elements with examples that can be used as a template when preparing and submitting ML-based manuscripts for the same audience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similar to internal validation of the AIPM, external AIPM model validation can be based on discrimination (area under the receiver operating characteristic curve, sensitivity, specificity, positive and negative predictive values), calibration (calibration plot)5,6,13,17,49,56,78, and Decision Curve Analysis5,6,17,49,68,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Code for the complete model building pipeline should be published in well-documented scripts with computer environment requirements when possible7,8,13,14,16,19,20,24,25,27,33,50,62,64,65, including statements about any restrictions to access or re-use.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“To ensure the generalizability of the AIPM to the intended healthcare setting, developers are advised to extensively validate the model for representative data from that setting6–8,10,13–15,24,27,56,64,66,68,72,77,79,87,88 (see phase 1, Representativeness).”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
In the citing paper
“and is best informed - a priori - by medical expert knowledge or existing literature13,17,76.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
In the citing paper
“It has been widely recommended to report any measures taken to prevent overfitting3,7,8,13,15.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
In the citing paper
“One commonly referred strategy is feature selection13,15,27,47,76, for which it is explicitly recommended that selection should work independently of model training (unlike in methods as forward and backward selection) and is best informed - a priori - by medical expert knowledge or existing literature13,17,76.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
In the citing paper
“for which it is explicitly recommended that selection should work independently of model training (unlike in methods as forward and backward selection)”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
In the citing paper
“To facilitate transparency and reproducibility of the developmental process, the used computational architecture, high-performance techniques, software packages, and versioning (data, model, configurations and training scripts) should be reported13,14,16,50,64–66.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
Collins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,319
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Deep learning for healthcare: review, opportunities and challenges
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2014) · International Journal of Surgery · doi:10.1016/j.ijsu.2014.07.014 · Cited by 2,969
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710
Phenomapping for Novel Classification of Heart Failure With Preserved Ejection Fraction
Shah, S. J., Katz, D. H., Selvaraj, S., Burke, M. A., Yancy, C. W., Gheorghiade, M., Bonow, R. O., Huang, C.-C., & Deo, R. C. (2015) · Circulation · doi:10.1161/circulationaha.114.010637 · Cited by 1,157
Self-citation
Cited by
Sorted by citations
Items 201–269 of 269
Hurley, N. C., Desai, N., Dhruva, S. S., Khera, R., Schulz, W. L., Huang, C., Curtis, J. P., Masoudi, F. A., Rumsfeld, J. S., Negahban, S. N., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000906 · Cited by 1
Self-citationThe Case for Including Data and Code with ML Publications in Laboratory Medicine
Master, S. R. (2022) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfac088 · Cited by 1
Machine Learning in Vascular Surgery: A Systematic Review and Critical AppraisalPreprint
Li, B., Feridooni, T., Cuen-Ojeda, C., Kishibe, T., de Mestral, C., Mamdani, M., & Al-Omran, M. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3867678 · Cited by 1
Huang, A. W., Haslberger, M., Coulibaly, N., Galárraga, O., Oganisian, A., Belbasis, L., & Panagiotou, O. A. (2021) · Research Square · doi:10.21203/rs.3.rs-1064149/v1 · Cited by 1
Cardiovascular waveforms - can we extract more from routine signals?
Nandi, M., Anton, M., & Lyle, J. V. (2022) · JRSM Cardiovascular Disease · doi:10.1177/20480040221121438 · Cited by 2
Bopche, R., & Damås, J. K. (2024) · medRxiv · doi:10.1101/2024.04.15.24305877 · Cited by 2
Helland, G. B., Beyer, M. K., Enriquez, B. A. B., Ihle‐Hansen, H., Ihle‐Hansen, H., Andersson, S., Leonardsen, E. H., Stangeland, H., Ùjhelyi, B., Hagberg, G., et al. (2024) · medRxiv · doi:10.1101/2024.04.16.24305935 · Cited by 1
Zhu, G., Yang, N., Yi, Q., Xu, R., Zheng, L., Zhu, Y., Li, J., Che, J., Chen, C., Lu, Z., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4358372/v1 · Cited by 1
Weiss, A., & Karamlou, T. (2021) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2021.04.101
Huber, M., Luedi, M. M., & Andereggen, L. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.878091
Using machine learning methods to predict the lactate trend of sepsis patients in the ICUPreprint
Arslantaş, M. K., Aşuroğlu, T., Arslantaş, R., Pashazade, E., Dinçer, P. Ç., Altun, G. T., & Kararmaz, A. (2022) · Research Square · doi:10.21203/rs.3.rs-1855422/v2 · Cited by 1
Artificial intelligence methodology in clinical research
Prabhakaran, S., & Hong, H. J. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-90300-4.00081-1
Agrawal, A., Ramu, S. K., Shekhar, S., Isogai, T., Bansal, A., Yun, J. J., Reed, G. W., Puri, R., Krishnaswamy, A., & Kapadia, S. R. (2023) · medRxiv · doi:10.1101/2023.04.14.23288606
A Robust Deep Learning Techniques for No-Show Prediction in Hospital Appointments
Nguyen, P. T., Dang, D. T., & Nguyen, V. D. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-43247-7_1 · Cited by 3
Zrubka, Z., Kovács, L., Motahari-Nezhad, H., Czere, J. T., Gulàcsi, L., & Péntek, M. (2023) · Research Square · doi:10.21203/rs.3.rs-3430896/v1
Image quality assessment for MRI: Is it up to the task?
Seghier, M. L. (2023) · European Workshop on Visual Information Processing · doi:10.1109/euvip58404.2023.10323060
Otokiti, A. U., Shih, H.-J., Ozoude, M. M., Siguachi, I., Warsame, L., Williams, K. S., & Akinloye, S. J. (2023) · doi:10.2196/preprints.55618
A process for evaluating Machine Learning models for Healthcare Applications
de Souza, C. M. P., Barreto, C. A. D. S., de Brito, B. A. O., Targino, V. V., Malaquias, R. S., Filho, I. D. M. B., & Pereira, A. G. D. O. (2024) · Seven Editora eBooks · doi:10.56238/sevened2024.001-049
Big Data in Pediatric Oncology
Wyatt, K. D., & Volchenboum, S. L. (2024) · Hematology/Oncology Clinics of North America · doi:10.1016/j.hoc.2026.02.004
Human intelligence for authors, reviewers and editors using artificial intelligence
Palacios, M. (2024) · Interdisciplinary Journal of Epidemiology and Public Health · doi:10.18041/2665-427x/ijeph.1.11268
Artificial Intelligence in Orthopedic Sports Medicine
Myers, T. G., & Mannava, S. (2024) · Springer eBooks · doi:10.1007/978-3-030-65430-6_191-1
Machine Learning-Generated Clinical Data as Collateral Research: A Global Neuroethical Analysis
Afolabi, M., & Sodeke, S. O. (2024) · The American Journal of Bioethics · doi:10.1080/15265161.2024.2388724
Dasa, O., Bai, C., Sajdeya, R., Kimmel, S. E., Pepine, C. J., Gurka, M. J., Laubenbacher, R., Pearson, T. A., & Mardini, M. (2024) · JMIR Public Health and Surveillance · doi:10.2196/54421
Helland, G. B., Beyer, M. K., Enriquez, B. A. B., Ihle‐Hansen, H., Ihle‐Hansen, H., Andersson, S., Leonardsen, E. H., Stangeland, H., Ùjhelyi, B., Hagberg, G., et al. (2024) · Stroke Vascular and Interventional Neurology · doi:10.1161/svin.124.001494
Enhancing hospital protection measures reduces frontline medical workers’ stress during the pandemic
Xiaoxia, Z., Yan, F., Junwei, W., Bingyao, Z., & Fei, X. (2024) · BMC Psychology · doi:10.1186/s40359-024-02185-8
Prediction of mortality in sepsis patients using stacked ensemble machine learning algorithm
Babu, M., Sappani, M., Joy, M., Chandiraseharan, V. K., Jeyaseelan, L., & Sudarsanam, T. (2024) · Journal of Postgraduate Medicine · doi:10.4103/jpgm.jpgm_357_24
A novel approach to identify the fingerprint of stroke gait using deep unsupervised learningPreprint
David, S., Georgievska, S., Geng, C., Liu, Y., & Punt, M. (2024) · medRxiv · doi:10.1101/2024.12.19.24319338
Li, H., Chen, R., Xia, J., He, F., Zhang, Y., Tang, S., & Ni, C. (2025) · Journal of Traditional Chinese Medical Sciences · doi:10.1016/j.jtcms.2025.03.010
Hunter, D. D., Hunter, L., Kerner, D., Mullan, A. F., Vanmeter, D. E., Khapov, I., Finch, A. S., Hevesi, S. A., Porter, I. E., West, C. P., et al. (2026) · JMIR Medical Education · doi:10.2196/81052
Li, J., Zhang, B., Liao, Y., Wei, L., Huang, Q., Lin, L., Chen, J., & Chen, H. (2025) · Frontiers in Genetics · doi:10.3389/fgene.2025.1636065
Saputro, S. A., Pattanaprateep, O., Pattanateepapon, A., Karmacharya, S., & Thakkinstian, A. (2021) · Figshare · doi:10.6084/m9.figshare.16916743
Saputro, S. A., Pattanaprateep, O., Pattanateepapon, A., Karmacharya, S., & Thakkinstian, A. (2021) · Figshare · doi:10.6084/m9.figshare.16916746
Saputro, S. A., Pattanaprateep, O., Pattanateepapon, A., Karmacharya, S., & Thakkinstian, A. (2021) · Figshare · doi:10.6084/m9.figshare.16916740
Hossain, M. Z., Daskalaki, E., Brüstle, A., Desborough, J., Lueck, C. J., & Suominen, H. (2022) · Figshare · doi:10.6084/m9.figshare.21130079
Hossain, M. Z., Daskalaki, E., Brüstle, A., Desborough, J., Lueck, C. J., & Suominen, H. (2022) · Figshare · doi:10.6084/m9.figshare.21130078
Hossain, M. Z., Daskalaki, E., Brüstle, A., Desborough, J., Lueck, C. J., & Suominen, H. (2022) · Figshare · doi:10.6084/m9.figshare.21130076
Hossain, M. Z., Daskalaki, E., Brüstle, A., Desborough, J., Lueck, C. J., & Suominen, H. (2022) · Figshare · doi:10.6084/m9.figshare.21130075
Saputro, S. A., Pattanaprateep, O., Pattanateepapon, A., Karmacharya, S., & Thakkinstian, A. (2021) · Figshare · doi:10.6084/m9.figshare.16916737
AI-Powered Personalization in Blockchain-Based Service Ecosystems
Matchanova, B., Ashirova, A. I., Eshchanov, T., & Ahmad, M. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-4387-7.ch007
Chang, W.-D., & Kim, M. S. (2026) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000001414
PUMAA : Establishing a protocol for utilizing machine learning in forensic anthropological analyses
Faisal, E., & Rogers, T. L. (2026) · Journal of Forensic Sciences · doi:10.1111/1556-4029.70280
Hurley, N. C., Mortazavi, B. J., Dhruva, S. S., Ross, J. S., Ngufor, C. G., Curtis, J. P., Krumholz, H. M., & Desai, N. R. (2026) · Medical Devices Evidence and Research · doi:10.2147/mder.s585943
Self-citationSıddıkoğlu, D. (2026) · World Journal of Urology · doi:10.1007/s00345-026-06327-y
Shen, T., Homme, J. L., Sheele, J. M., & Jones, D. D. (2026) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2026.02.036
Li, W., Xiang, Y., Han, G., SHEN, Y., Gu, S., Ma, D., Chi, J., Li, Q., Wang, W., Dai, S., et al. (2026) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000002334
Huang, N., Wang, W., Yang, R., Liao, Z., Yang, H., & Chen, X. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1739432
Aitken, M., Andrade, B. F., Krasinkiewicz, C. E., Chan, S. W. S., Gandhi, S. P., Pun, J. W., Hamilton-Wright, A., Wang, W., Sack, L., Weisz, J. R., et al. (2026) · Trials · doi:10.1186/s13063-026-09739-8
Machine learning prediction models for myocardial injury after non-cardiac surgery: A scoping review
Nadesan, P., Gandhi, B., Schwalm, J.-D., Petch, J., Sibbald, M., & Khemani, E. (2026) · Trends in Anaesthesia and Critical Care · doi:10.1016/j.tacc.2026.101662
Krebs, Č., Mihevc, M., & Šter, M. P. (2026) · BMC Public Health · doi:10.1186/s12889-026-27784-5
Zou, M., & Ju, S. (2026) · Frontiers in Endocrinology · doi:10.3389/fendo.2026.1821550
Islam, M. M., & Okere, A. N. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106504
Publishing Prediction Models in Precision Oncology: A Summary of Guidelines and Recommendations
DelRocco, N. J., Jee, J., & Lü, Y. (2026) · JCO Precision Oncology · doi:10.1200/po-26-00356
Good things come in threes: evaluating clinical utility of machine learning-derived clusters
Lisik, D., de Kok, J. W. T. M., Barón, N. B., Vanfleteren, L. E. G. W., Nwaru, B. I., & Basna, R. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag098 · Cited by 1
Szponar, M., Gmaj, B., Orłowski, J., & Kamiński, J. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110779
Fraser, A. G. (2026) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztag095
Applying machine learning to pharmacovigilance data: A proof‐of‐concept study
Barus, R., Schirò, P., Faillie, J., Freppel, R., Béné, J., de Canecaude, C., & Montastruc, F. (2026) · British Journal of Clinical Pharmacology · doi:10.1002/bcp.70655
Effectiveness of Artificial Intelligence in Cardiovascular Disease Prevention: A Systematic Review
Shirvankar, C. M., Puterman‐Salzman, L., Iglesies‐Grau, J., Zomahoun, H. T. V., & Rahimi, S. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30813-9_42
Yao, Q., Zhang, H., Jiang, J., & Yuan, F. (2026) · Annals of Human Biology · doi:10.1080/03014460.2026.2669070
Blanchet, R., Doan, N., Nejatinamini, S., Valderrama, C. E., Hammond, D. G., Vanderlee, L., & Olstad, D. L. (2026) · Journal of Nutrition · doi:10.1016/j.tjnut.2026.101743
Dashti, M., Khosraviani, F., Ghadimi, N., Kalantari, S., Amirzade-Iranaq, M. H., Hemmati, G., Arzani, S., Pirayesh, Z., & Motie, P. (2026) · Imaging Science in Dentistry · doi:10.5624/isd.20250325
Lu, S. L., Fei, M., Elsissy, J. G., & Schneiderman, B. A. (2026) · Orthopedics · doi:10.3928/01477447-20260720-01
Noviyani, R., Nguyen, H. N., Duong, K. L. T., Ngo, T. V., Ngo, N. T. N., Wijayakusuma, I. G. N. L., Widhiartini, I. A. A., & Duong, K. N. C. (2026) · Journal of Applied Pharmaceutical Science · doi:10.1177/22313354261466853
Missing Data in Clinical Machine Learning: Modeling Decisions, Pitfalls, and Reporting Standards
Papadopoulos, G., & Georgiou, E. (2023) · Journal of Health Informatics and Digital Systems · doi:10.68159/o473138174
Elgazzar, F. M., Sohsah, G. N., & Eldayem, Y. B. A. (2026) · Annales de Toxicologie Analytique · doi:10.1016/j.toxac.2026.08.009
Washton, J., & Toma, M. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15187237
Wang, Z., & Borah, B. J. (2026) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2026.2746821
AI Model Reporting Standards in Health Care: Systematic Review and Consolidation
Akhmad, E. S., Slob, D., Gomes, A. L., Mingels, S., Dekker, A. L. A. J., & van Soest, J. (2026) · JMIR AI · doi:10.2196/92329
Bonanno, M., Calderone, A., Lonia, G., Migale, S., Ravì, D., Quartarone, A., Milardi, D., Celesti, A., & Calabrò, R. S. (2026) · Digital Health · doi:10.1177/20552076261487745
Metadata
- Authors
- Stevens, L. M., Mortazavi, B. J., Deo, R. C., Curtis, L., & Kao, D. P.
- Affiliations
- American Heart Association (US)
- Texas A&M University (US)
- Brigham and Women's Hospital (US)
- Duke University (US)
- University of Colorado Anschutz (US)
- Venue
- Circulation: Cardiovascular Quality and Outcomes
- Published
- 2020-10
- Volume
- 13
- Issue
- 10
- Pages
- e006556-e006556
- ISSN
- 1941-7705 (ISSN-L)
- 1941-7713
- PMC
- PMC8320533
- OpenAlex
- W3094171951
- Cited by
- 268
- References
- 34
- Citations by year
- 2026: 39 · 2025: 43 · 2024: 72 · 2023: 46 · 2022: 43 · 2021: 24 · 2020: 1
- FWCI
- 11.20
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Circulation Cardiovascular Quality and Outcomes (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Heart, Lung, and Blood Institute (K08HL125725, L30HL110124)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- machine learning, reporting guidelines, clinical research, reproducibility, transparency, peer review, manuscript template, model validation, cardiovascular research
- MeSH terms
- Data Accuracy, Machine Learning, Data Interpretation, Statistical, Humans, Periodicals as Topic, Research Design, Editorial Policies, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
