Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessThe DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Abstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 901–1,000 of 1,005
Burghoorn, D. G., Booij, S. H., Schoevers, R. A., & Riese, H. (2024) · BJPsych Open · doi:10.1192/bjo.2024.789 · Cited by 10
“18 Following the Cochrane Handbook, potential effect modifiers may include study-level characteristics, such as population type, type of intervention or treatment, length of follow-up or methodology (i.e. design and quality).”
Naemi, A., Tashk, A., Azar, A. S., Samimi, T., Tavassoli, G., Mohasefi, A. B., Khanshan, E. N., Najafabad, M. H., Tarighi, V., Wiil, U. K., et al. (2025) · Cancers · doi:10.3390/cancers17030558 · Cited by 10
Risk Prediction Models for Sentinel Node Positivity in Melanoma
Ma, B., Gandhi, M., Czyz, S., Jia, J., Rankin, B. D., Osman, S., Jonsson, E. L., Robertson, L. H., Parsons, L. M., & Temple‐Oberle, C. F. (2025) · JAMA Dermatology · doi:10.1001/jamadermatol.2025.0113 · Cited by 8
Lu, W., Wang, N., Fang, X., Yang, H., He, H., Qin, D., & Hua, F. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105686 · Cited by 9
Sawesi, S., Jadhav, A., & Rashrash, B. M.-E. (2025) · JMIR Medical Informatics · doi:10.2196/67859 · Cited by 12
“The articles selected for inclusion were then subjected to a quality assessment using the Prediction model Risk of Bias Assessment Tool (PROBAST) tool [13], which categorizes bias as low, medium, or high (see Multimedia Appendix 3).”
Rigorous validation of machine learning in laboratory medicine: guidance toward quality improvement
Miller, H. A., & Valdes, R. (2025) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2025.2488842 · Cited by 15
Zhang, Z., Wang, S., Xu, Z., Sun, Y., Zhou, X., Zhou, R., Li, Q., & Wang, G. (2025) · Renal Failure · doi:10.1080/0886022x.2025.2500663 · Cited by 8
“…extract relevant data based on the critical appraisal and Data extraction for Systematic Reviews of Prediction modeling Studies checklists (cHaRmS) [18]. in case of disagreements regarding study selection, a discussion involving three authors was held to reach a consensus. the extracted data…”
Diagnostic Prediction Models for Sarcopenia: A Systematic Review and Meta-Analysis
Zhang, X., Lian, R., Tang, H., Luo, S., Chen, X., Lü, J., & Yang, M. (2025) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2025.105698 · Cited by 10
Predictive accuracy of ophthalmic artery Doppler for pre-eclampsia: a systematic review
Arkorful, J., Browne, J. L., Adu‐Bonsaffoh, K., Ali, S., Srofenyoh, E., & Bloemenkamp, K. W. M. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-094348 · Cited by 13
Risk prediction models for ovarian hyperstimulation syndrome: a systematic review and meta-analysis
Liu, J., Liu, F., Xu, W., Zhang, L., Tian, Q., Du, L., Li, S., & Zhang, M. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07971-9 · Cited by 10
“A standardized data extraction form was developed based on the critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) checklist [11] and implemented in Excel 2019.”
Predictive models for chemotherapy-induced oral mucositis: a systematic review
Tao, Y., Zeng, X. G., & Mao, H. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1608505 · Cited by 11
Selmer, N. W., Berthelsen, C. B., Grootven, B. V., Meyer, G., & Loft, M. I. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105188 · Cited by 14
Li, X., Hu, X. L., Xu, H., Yu, P., & Ju, H. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1680180 · Cited by 11
Deep learning models for ICU readmission prediction: a systematic review and meta-analysis
Koumantakis, E., Remoundou, K., Colombi, N., Fava, C., Roussaki, I. G., Visconti, A., & Berchialla, P. (2025) · Critical Care · doi:10.1186/s13054-025-05642-x · Cited by 11
“We used the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) to create the data extraction form [22].”
Yu, C. W. G., Wongwitwichote, K., Mansfield, M., Deane, J. A., Devecchi, V., & Falla, D. (2025) · European Journal of Pain · doi:10.1002/ejp.70168 · Cited by 13
“A data extraction table was constructed with reference to the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (Moons et al. 2014).”
Piccolo, R., Laino, A., Vitale, A. P., Franzone, A., Giacoppo, D., Bossone, E., Capodanno, D., & Esposito, G. (2025) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2025.120574 · Cited by 9
Ou, Y., Jiang, D., Li, P., Zhang, W., Zhou, Y., Chen, Y., & Yin, X. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1667792 · Cited by 14
Xu, S., Wu, Y., Li, J., Pan, X., Zhang, X., Liu, Y., Zhang, F., & Tong, N. (2020) · Postgraduate Medicine · doi:10.1080/00325481.2020.1821234 · Cited by 8
Harris, J., Purssell, E., Ream, E., Jones, A., Armes, J., & Cornelius, V. R. (2020) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2020.151089 · Cited by 9
Henderson, I., Rimmer, M. P., Keay, S. D., Sutcliffe, P. A., Khan, K. S., Yasmin, E., & Wattar, B. H. A. (2020) · F&S Reviews · doi:10.1016/j.xfnr.2020.11.002 · Cited by 12
Roberts, D. J., Nagpal, S., Stelfox, H. T., Brandys, T., Corrales‐Medina, V., Dubois, L. A., & McIsaac, D. I. (2021) · JMIR Research Protocols · doi:10.2196/28759 · Cited by 9
“We formulated the study-focused clinical question using the PICOTS (Population, Index Prognostic Factor, Comparison of Prognostic Factors, Outcome, Timing, and Setting) framework for posing clinical questions for systematic reviews of prognostic factor studies [34,36].”
MethodLuijken, K., Song, J. T., & Groenwold, R. H. H. (2022) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00121-1 · Cited by 11
“The quantity that a clinical prediction model targets is defined by specifying the outcome, (candidate) predictors, population, setting, time of prediction, and prediction horizon as specifically as possible [6].”
BackgroundLiu, J., Liu, P., Lei, M.-R., Zhang, H., You, A.-L., & Luan, X. (2022) · Iranian Journal of Public Health · doi:10.18502/ijph.v51i7.10082 · Cited by 11
“Quality assessment The included studies were evaluated according to the bias risk assessment tool of the CHARMS checklist (16).”
MethodMoons, K. G. M., Altman, D. G., Reitsma, J. B., loannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2022) · Digital Diagnostics · doi:10.17816/dd110794 · Cited by 10
Self-citationChoon-Quinones, M., Zelei, T., Németh, B., Tóth, M., Jia, X. Y., Barnett, M., Keown, P. A., Durie, B. G. M., Harousseau, J., Hose, D., et al. (2022) · Journal of Medical Economics · doi:10.1080/13696998.2022.2144056 · Cited by 8
“Disease progression models were evaluated using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) 12 , as applicable.”
MethodChen, R., Zheng, J., Li, C., Chen, Q., Zeng, Z., Li, L., Chen, M., & Zhang, S. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1215116 · Cited by 10
Daines, L., Bonnett, L. J., Tibble, H., Boyd, A., Thomas, R., Price, D. B., Turner, S., Lewis, S. C., Sheikh, A., & Pinnock, H. (2023) · Wellcome Open Research · doi:10.12688/wellcomeopenres.19078.1 · Cited by 9
Overton, M., Swain, N., Falling, C., Gwynne‐Jones, D., Fillingim, R. B., & Mani, R. (2024) · Osteoarthritis and Cartilage Open · doi:10.1016/j.ocarto.2024.100439 · Cited by 9
“This study was developedin consultation with the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS), the Checklist for Reporting Ecological Momentary Assessment Studies (CREMAS) and EMA literature[19,20].”
MethodPisavadia, K., Spencer, L. H., Tuersley, L., Coates, R., Ayers, S., & Edwards, R. T. (2024) · BMJ Open · doi:10.1136/bmjopen-2022-068941 · Cited by 12
Asres, G. D., & Gessesse, Y. K. (2024) · Human Factors in Healthcare · doi:10.1016/j.hfh.2024.100078 · Cited by 11
Karlsen, A. P. H., Sunde, P. B., Olsen, M. H., Laigaard, J., Folkersen, C., Tran, T. X. M., Rasmussen, I., Kjartansdóttir, S. P., Saito, A., Andersen, M. A., et al. (2024) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14508 · Cited by 11
“26 Data extraction : We will use the CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) 27 and PROBAST 26 Excel template for data extraction.”
Kunath, F., Spek, A., Jensen, K., Zengerling, F., & Schmidt, S. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0167-3 · Cited by 8
“and adapted for our purpose [19].”
Elias, S. G., Kok, L., Witteman, B. J. M., Goedhard, J. G., Romberg-Camps, M. J. L., Muris, J. W. M., de Wit, N. J., & Moons, K. G. M. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.09.014 · Cited by 8
Self-citationMonte Carlo decision curve analysis using aggregate data
Hozo, I., Tsalatsanis, A., & Djulbegović, B. (2017) · European Journal of Clinical Investigation · doi:10.1111/eci.12723 · Cited by 9
Schlüssel, M. M., Keene, D. J., Collins, G. S., Bostock, J., Byrne, C., Goodacre, S. W., Gwilym, S. E., Hagan, D. A., Haywood, K. L., Thompson, J., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-022802 · Cited by 10
Palazón‐Bru, A., Herencia, J. A. C., Rueda, M. S., Artigao-Rodenas, L. M., Divisón, J. A., Escribano, F. M., Ponce-García, I., & on behalf of GEVA (Group of Vascular Diseases From Albacete) (2018) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0000000000000534 · Cited by 7
Clinical prognostic models for severe dengue: a systematic review protocol
Dao, T. P., Long, K. Q., Khanh, L. V. D., Ong, P. T., Sy, H. L., Ngoc, T. L., & Lam, P. K. (2019) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15033.2 · Cited by 7
Vazquez-Montes, M., Debray, T. P. A., Taylor, K., Speich, B., Jones, N. R., Collins, G. S., Hobbs, F. D. R. R., Magriplis, E., Maruri-Aguilar, H., Moons, K. G. M., et al. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00081-4 · Cited by 7
“In pairs, we will independently extract data according to a piloted form that will combine adapted versions of the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling studies (CHARMS) checklist [38] to assess the methodological quality conduct of the included prognostic models and the…”
MethodSelf-citationDong, W., Wan, E. Y. F., Bedford, L. E., Wu, T., Wong, C. K. H., Tang, E. H. M., & Lam, C. L. K. (2020) · Public Health · doi:10.1016/j.puhe.2020.06.020 · Cited by 9
Restrepo‐Escobar, M., Granda, P. A., Aguirre–Acevedo, D. C., Zapata, J. H., Vásquez, G. M., & Jaimes, F. A. (2021) · Lupus · doi:10.1177/0961203320983462 · Cited by 7
“An information extraction form was created based on the CHARMS checklist.(25) Information was extracted by one of the reviewers, and the information extracted from each of the studies included was assessed by the second reviewer.”
MethodLing, J. E. H., Coughlan, T., Polkinghorne, K. R., & Kanellis, J. (2021) · BMC Gastroenterology · doi:10.1186/s12876-021-01655-2 · Cited by 9
“Model predictors, effect estimates (hazard ratios), missing data, events-per-variable (EPV) rate were extracted to assess study quality [15, 26].”
MethodSystematic Reviews of Prognostic Factor Studies
Riley, R. D., Moons, K. G. M., Altman, D. G., Collins, G. S., & Debray, T. P. A. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch17 · Cited by 9
Self-citationPredictive models for the incidence of Parkinson’s disease: systematic review and critical appraisal
Chen, Y., Gao, Y., Sun, X., Liu, Z., Zhang, Z., Qin, L., Song, J., Wang, H., & Wu, X. (2022) · Reviews in the Neurosciences · doi:10.1515/revneuro-2022-0012 · Cited by 10
Real-World Practice of Gastric Cancer Prevention and Screening Calls for Practical Prediction Models
He, S., Sun, D., Li, H., Cao, M., Yu, X., Lei, L., Peng, J., Li, J., Li, N., & Chen, W. (2022) · Clinical and Translational Gastroenterology · doi:10.14309/ctg.0000000000000546 · Cited by 11
“This systematic review was prospectively registered at the International Prospective Register of Systematic Reviews (registration number: CRD42021203804) andwas conducted following the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (20).”
MethodInfluentialTanaka, M. D., Geubels, B. M., Grotenhuis, B. A., Marijnen, C. A. M., Peters, F. P., van der Mierden, S., Maas, M., & Couwenberg, A. M. (2023) · Cancers · doi:10.3390/cancers15153945 · Cited by 8
“The data extraction form was based on the TRIPOD statement [39], the 11 domains of the “Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies” (CHARMS) checklist [41], and the “Prediction model Risk Of Bias Assessment Tool” (PROBAST) [42].”
MethodWilson, J., Chowdhury, F., Hassan, S., Harriss, E., Alves, F. P., Dahal, P., Stepniewska, K., & Guerin, P. J. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-075597 · Cited by 9
Alalawi, A., Gallina, A., Sterling, M. M., & Falla, D. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-033298 · Cited by 9
Bhuia, M. R., Islam, M. A., Nwaru, B. I., Weir, C. J., & Sheikh, A. (2020) · Journal of Global Health · doi:10.7189/jogh.10.020409 · Cited by 12
Mudrik, A., Tsur, A., Nadkarni, G. N., Efros, O., Glicksberg, B. S., Soffer, S., & Klang, E. (2024) · medRxiv · doi:10.1101/2024.08.08.24311699 · Cited by 11
Clinical predictive models of invasive Candida infection: A systematic literature review
Rauseo, A. M., Aljorayid, A., Olsen, M. A. A., Larson, L., Lipsey, K. L., Powderly, W. G., & Spec, A. (2021) · Medical Mycology · doi:10.1093/mmy/myab043 · Cited by 9
Roberts, D. J., Sypes, E. E., Nagpal, S., Niven, D. J., Mamas, M. A., McIsaac, D. I., van Walraven, C., Shorr, R., Graham, I. D., Stelfox, H. T., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-053920 · Cited by 7
Chen, J., Lu, G., Wang, Z., Zhang, J., Ding, J., Zeng, Q., Chai, L., Zhao, L., Yu, H., & Li, Y. (2023) · The Laryngoscope · doi:10.1002/lary.30931 · Cited by 11
“24,25 The inclusion criteria for this study were as follows: (1) studies that developed or validated a prediction model for patients who underwent dysphagia after receiving mechanical ventilation in any ICU; (2) a prediction rule was de fi ned as the combination of three or more risk factors.”
MethodAutomatic transparency evaluation for open knowledge extraction systems
Basereh, M., Caputo, A., & Brennan, R. (2023) · Journal of Biomedical Semantics · doi:10.1186/s13326-023-00293-9 · Cited by 15
“The number of participants, the eligibility criteria, the follow-up times, if there has been participants [63, 64] 6.”
BackgroundLifestyle predictors of colorectal cancer in European populations: a systematic review
Mertens, E., Keuchkarian, M., Vasquez, M. S., Vandevijvere, S., & Peñalvo, J. L. (2024) · BMJ Nutrition Prevention & Health · doi:10.1136/bmjnph-2022-000554 · Cited by 10
Xu, Z., Hu, Y., Shao, X., Shi, T., Yang, J., Wan, Q., & Liu, Y. (2024) · Cardiology · doi:10.1159/000538639 · Cited by 15
“Three researchers independently extracted the following data from the included studies [19]: authors, publication year, country of the study, sample sizes, the numbers of predictive variables, data source, endpoint outcome, time interval, types of ML models, AUC, validation methods, data set…”
Giraldo‐Roldán, D., Araújo, A. L. D., Moraes, M. C., da Silva, V. M., Ribeiro, E. C. C., Cerqueira, M., Saldivia‐Siracusa, C., Sousa‐Neto, S. S., Pérez‐de‐Oliveira, M. E., Lopes, M. A., et al. (2024) · Journal of Oral Pathology and Medicine · doi:10.1111/jop.13548 · Cited by 9
Walsh, M. E., Kristensen, P. K., Hjelholt, T. J., Hurson, C. J., Walsh, C., Ferris, H., Crozier‐Shaw, G., Keohane, D., Geary, E., O’Halloran, A., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111439 · Cited by 9
Nejad, M. C., Matthiesen, R. V., Dukovska‐Popovska, I., Jakobsen, T. L., & Johansen, J. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105631 · Cited by 11
Hancox, Z., Pang, A., Conaghan, P. G., Kingsbury, S. R., Clegg, A. P., & Relton, S. D. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102999 · Cited by 11
Zhang, B., Guo, B., Kong, H., Yang, L., Yan, H., Liu, J., Zhou, Y., An, R., & Wang, F. (2024) · Journal of Inflammation Research · doi:10.2147/jir.s498740 · Cited by 8
Gao, Y., Yuan, B., Fan, P., Li, M., & Chen, J. (2024) · Medicine · doi:10.1097/md.0000000000040588 · Cited by 7
Xu, J., Zhao, X., Li, F., Xiao, Y., & Li, K. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17577 · Cited by 12
Zhu, X., Ventura, E. F., Bansal, S., Wijeyesekera, A., & Vimaleswaran, K. S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109661 · Cited by 8
Ahluwalia, N., Hussain, A., Providencia, R. A., & SCHILLING, R. J. (2025) · Arrhythmia & Electrophysiology Review · doi:10.15420/aer.2024.24 · Cited by 7
“10,11 The systematic review was prospectively registered (PROSPERO registration CRD42024504756).”
Risk prediction models for patients with recurrent diabetic foot ulcers: A systematic review
Zhou, Z., Yu, J., Yan, H., Xu, J., Wang, S., & Wen, J. (2025) · Public Health · doi:10.1016/j.puhe.2025.105744 · Cited by 9
Gebretsadik, G. G., Biratu, A. K., Gessessew, A., Lassi, Z. S., Kahsay, A. B., & Mulugeta, A. (2025) · Journal of Health Population and Nutrition · doi:10.1186/s41043-025-00888-1 · Cited by 12
“The two reviewers (GG and AM) extracted data using a modified CHARMS-PF checklist [17].”
Diagnosis models to predict peripheral arterial disease: a systematic review and meta analysis
Quan, X., Xiong, H., Liu, X., Song, P., Wang, D., Chen, Q., Hu, X., & Shi, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-10459-3 · Cited by 9
“…was guided by the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (the CHARMS checklist) 8 , and included the following components: basic information of studies(title, first author, publication year, study type, whether multicenter,…”
Hmido, S. B., Rahim, H. A., Ingwersen, E. W., Burchell, G. L., Hompes, R., van der Peet, D. L., Sonneveld, E., Geert Kazemier, M., & Daams, F. D. (2025) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2025.110367 · Cited by 10
Verhagen, C. A. M., Gholamiankhah, F., Buijsman, E. C. M., Broersen, A., van Erp, G. C. M., van der Velden, A. L., Rahmani, H., van der Leij, C., Brecheisen, R., Lanocita, R., et al. (2025) · Cancers · doi:10.3390/cancers17162656 · Cited by 7
“Data were extracted using the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [13].”
AI radiomics predicts spatial glioma recurrence on preoperative MRI: a systematic review
Bijlsma, S., Maal, T., Rubbert, C., Mannil, M., Meijer, F. J. A., van der Kolk, A. G., de Jong, G., & Henssen, D. J. H. A. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.112412 · Cited by 10
Jian, R., Zhang, J., Zeng, Y., Zhou, T., Wu, Y., Wu, L., Yu, Y., & Xi, C. (2025) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2025.1659184 · Cited by 7
Machine Learning for Predicting Stroke Risk Stratification Using Multiomics Data: Systematic Review
Yoo, H. Y., Shin, H., Kim, E. E., & Son, Y.-J. (2026) · Journal of Medical Internet Research · doi:10.2196/85654 · Cited by 8
Tan, J., Qi, Y., Liu, C., Xiong, Y., He, Q., Zhang, G., Chen, M., He, G., Wang, W., Liu, X., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.07.009 · Cited by 7
Aparcana-Granda, D. J., Ascencio, E. J., & Carrillo-Larco, R. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058921 · Cited by 6
Chen, Z., Jiang, H., He, W., Li, D., Lin, M., Wang, M., Shang, M., & Zhang, W. (2022) · Frontiers in Nutrition · doi:10.3389/fnut.2022.849034 · Cited by 22
A systematic review of rodent control as part of infectious disease control programs
Zeppelini, C. G., Callefe, J. L. R., Coelho, R. H., Martinez-Silveira, M. S., Khalil, H., Belmain, S. R., Bertherat, E., Begon, M. E., & Costa, F. (2022) · International Journal of Pest Management · doi:10.1080/09670874.2022.2094491 · Cited by 7
“We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA statement and checklist) guidelines (Moher et al. 2009 ) and the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS guidelines) (Moons et al. 2014 ).”
Wang, X., Wang, L., Wang, Z., Sun, Y. E., Liu, X., Li, F., & Zheng, Y. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.979160 · Cited by 7
Risk Prediction Models for Long-Term Survival after Cardiac Surgery: A Systematic Review
Argus, L., Taylor, M., Ouzounian, M., Venkateswaran, R., & Grant, S. W. (2023) · The Thoracic and Cardiovascular Surgeon · doi:10.1055/s-0043-1760747 · Cited by 7
“11,12 This includes information about participants, outcomes, candidate predictors, missing data, model development, model performance, model evaluation, results, and discussion and is summarized in ► Supplementary Table S2 (available online only).”
Pommerich, U. M., Stubbs, P. W., Eggertsen, P. P., Fabricius, J., & Nielsen, J. F. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.02.009 · Cited by 8
Yoo, J. H., Chong, B., Barber, P. A., Stinear, C. M., & Wang, A. (2023) · Neurorehabilitation and neural repair · doi:10.1177/15459683231173668 · Cited by 8
Rebelo, F. M., Carrilho, T. R. B., Canuto, R., Schlüssel, M. M., Farias, D. R., Ohuma, E. O., & Kac, G. (2023) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.1080/14767058.2023.2230510 · Cited by 9
“The lack of blindness in the outcome assessment may overestimate the method ’ s predictive ability, especially when the outcome requires subjective interpretation (e.g. Apgar score) [18, 39].”
BackgroundMethodPrognostic models for predicting insomnia treatment outcomes: A systematic review
Holler, E., Du, Y., Barboi, C., & Owora, A. H. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.12.017 · Cited by 10
Risk Prediction Models for Sarcopenia in Dialysis Patients: A Systematic Review
Leng, Y.-J., Wang, G., Xie, R.-N., Jiang, X., Li, C.-X., Nie, Z., & Li, T. (2024) · Journal of Renal Nutrition · doi:10.1053/j.jrn.2024.05.009 · Cited by 8
He, Q., You, Z., Dong, Q., Guo, J., & Zhang, Z. (2024) · Frontiers in Microbiology · doi:10.3389/fmicb.2024.1458670 · Cited by 7
den Eynde, R. V., Vrancken, A., Foubert, R., Tuand, K., Vandendriessche, T., Schrijvers, A., Verbrugghe, P., Devos, T., CALSTER, B. V., & Rex, S. (2024) · Transfusion · doi:10.1111/trf.18108 · Cited by 8
Ghith, N., Frølich, A., & Merlo, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0189050 · Cited by 9
E/e' in relation to outcomes in ST‐elevation myocardial infarction
Tai, S. B., Lau, W. R., Gao, F., Hamid, N. B., Amanullah, M. R., Fam, J. M., Yap, J. J. L., Ewe, S. H., Chan, M. Y., Yeo, K. K., et al. (2020) · Echocardiography · doi:10.1111/echo.14652 · Cited by 7
Predictive Performance of the FIF Screening Tool in 2 Cohorts of Community-Living Older Adults
Frisendahl, N., Ek, S., Rosendahl, E., Boström, A., Fagerström, C., Elmståhl, S., & Welmer, A. (2020) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2020.04.037 · Cited by 7
Kimani, L., Howitt, S. H., Tennyson, C., Templeton, R., McCollum, C. N., & Grant, S. W. (2021) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2021.02.056 · Cited by 7
Zhong, J., Si, L., Zhang, G., Huo, J., Xing, Y., Hu, Y., Zhang, H., & Yao, W. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01683-9 · Cited by 11
Alalawi, A., Mazaheri, M., Gallina, A., Luque-Suárez, A., Sterling, M. M., & Falla, D. (2021) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000001015 · Cited by 13
“69) 71 Participants were allowed to pursue any form of treatment Several type of treatments and medications were reported including physiotherapy, chiropractic, acupuncture, simple analgesics, NSAIDs, codeine, antidepressants, steroids, and opioids Atherton et al45 7 UK Emergency department (4) Median 8 d 765 Median (…”
BackgroundExternal validation of the European Society of Thoracic Surgeons morbidity and mortality risk models
de Antonio, D. G., Carrasco, S. C., Román, A. R., Royuela, A., Barturen, M. G., Obiols, C., Call, S., Royo, Í., Díaz, J. L. R., Cabañero-Sánchez, A., et al. (2022) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezac170 · Cited by 10
Previous therapy and the recurrence rate of basal cell carcinoma after Mohs surgery: a meta-analysis
Sooksamran, A., Pichai, P., Suphannaphong, M., & Singthong, S. (2023) · Archives of Dermatological Research · doi:10.1007/s00403-023-02558-w · Cited by 8
“in 2014 reported that previously treated BCC has a higher risk of recurrence following MMS than BCC without prior treatment [7].”
BackgroundMethodInfluentialLuo, Q., Luo, Y., Cui, T., & Li, T. (2023) · Archives of Sexual Behavior · doi:10.1007/s10508-023-02574-x · Cited by 7
“We conducted our systematic review based on the steps outlined in the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) (Moons et al., 2014), as well as the guidance paper by Debray et al.”
MethodInfluentialHryciw, B. N., Hryciw, N., Tran, A., Fernando, S. M., Rochwerg, B. N., Burns, K. E. A., & SEELY, A. J. E. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000005865 · Cited by 7
Kleuskens, D. G., van Veen, C. M. C., Groenendaal, F., Ganzevoort, W., Gordijn, S. J., van Rijn, B. B., Lely, A. T., Schuit, E., & Kooiman, J. (2023) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.26245 · Cited by 8
“CHARMS was used to extract information on study design, population characteristics, predictors and outcomes assessed by the prediction models, and statistical analysis, including handling of missing data, model-building strategies and model-performance measurements 9,10 .”
BackgroundMethodKillingmo, R. M., Tveter, A. T., Pripp, A. H., Tingulstad, A., Maas, E. T., Rysstad, T. L., & Grotle, M. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080567 · Cited by 9
Childhood prediction models for hypertension later in life: a systematic review
Hamoen, M., de Kroon, M. L. A., Welten, M., Raat, H., Twisk, J. W. R., Heymans, M. W., & Vergouwe, Y. (2018) · Journal of Hypertension · doi:10.1097/hjh.0000000000001970 · Cited by 8
“Data were extracted from the articles using a standardized data extraction form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [15].”
MethodInfluentialSelf-citationXu, H.-L., Liu, F.-H., Chen, H., Xiao, Q., Sun, H., Shi, Y., Gao, S., Zhao, Y., Gong, T., & Wu, Q. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4124049 · Cited by 7
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.