The DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessAbstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 701–800 of 1,004
Gemma, L., Pecoraro, A., Sebastianelli, A., Spatafora, P., Sessa, F., Nicoletti, R., Gravas, S., Campi, R., Serni, S., & Gacci, M. (2024) · Prostate Cancer and Prostatic Diseases · doi:10.1038/s41391-024-00795-2 · Cited by 14
“Data were extracted by two members of the team (L.G. and A.S.) according to the established checklist [9].”
Jin, M. X., Qin, P. P., Xia, A. W. L., Kan, R. L. D., Zhang, B. B. B., Tang, A. H. P., Li, A. S. M., Lin, T. T. Z., Giron, C. G., Pei, J. J., et al. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105695 · Cited by 16
Zandavi, S. M., Kim, C., Goodwin, T., Thilakanathan, C., Bostanara, M., Akon, A. C., Mouiee, D. A., Barisic, S., Majeed, A., Kemp, W. W., et al. (2024) · Liver International · doi:10.1111/liv.16050 · Cited by 14
“17 Diverging from k - fold cross-validation, where the test cohort closely mirrors the training cohort's underlying distribution, our approach introduces distribution differences between the test and train datasets while incorporating sufficient heterogeneity in terms of the aetiology of underlying…”
InfluentialRisk prediction models for disability in older adults: a systematic review and critical appraisal
Zhou, J., Xu, Y., Yang, D., Zhou, Q., Ding, S., & Pan, H. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05409-z · Cited by 18
“According to the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Model-ling Studies (CHARMS) [30], two reviewers (JZ and YX) independently performed data extraction, and any disa-greements were resolved by discussion or consultation with a third reviewer…”
MethodPost-Cardiac arrest outcome prediction using machine learning: A systematic review and meta-analysis
Zobeiri, A., Rezaee, A., Hajati, F., Argha, A., & Alinejad‐Rokny, H. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105659 · Cited by 23
Vidal, R., Grotle, M., Johnsen, M. B., Yvernay, L., Hartvigsen, J., Ostelo, R., Kjønø, L. G., Enstad, C. L., Killingmo, R. M., Halsnes, E. H., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111593 · Cited by 19
Moretto, C., Kopper, P. M. P., Münchow, E. A., & Scarparo, R. K. (2025) · International Endodontic Journal · doi:10.1111/iej.14223 · Cited by 20
TUG1, SPRY4-IT1, and HULC as valuable prognostic biomarkers of survival in cancer
Zhong, Y., Chen, Z., Guo, S., Liao, X., Xie, H., Zheng, Y., Cai, B., Huang, P., Liu, Y., Zhou, Q. Y., et al. (2017) · Medicine · doi:10.1097/md.0000000000008583 · Cited by 14
Delli, K., Villa, A., Farah, C. S., Celentano, A., Ojeda, D., Peterson, D. E., Jensen, S. B., Glurich, I., & Vissink, A. (2019) · Oral Diseases · doi:10.1111/odi.13041 · Cited by 14
“Modelling Studies’ (CHARMS) was used to facilitate data extraction (Moons et al, 2014).”
MethodLópez-Jiménez, N., Sánchez, F. G., Pailos, R. H., Rodrigo-Álvaro, V., Pascual-Pedreño, A. I., Moreno‐Cid, M., Delgado‐Rodríguez, M., & Hernández‐Martínez, A. (2021) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16947 · Cited by 14
Artificial intelligence for decision support in surgical oncology - a systematic review
Wagner, M., Schulze, A., Haselbeck-Köbler, M., Probst, P., Brandenburg, J. M., Kalkum, E., Majlesara, A., Ramouz, A., Klotz, R., Nickel, F., et al. (2022) · Artificial Intelligence Surgery · doi:10.20517/ais.2022.21 · Cited by 19
Risk Prediction Models in Perioperative Medicine: Methodological Considerations
Collins, G. S., Ma, J., Gerry, S., Ohuma, E. O., Odondi, L., Trivella, M., de Beyer, J. A., & Vazquez-Montes, M. (2016) · Current anesthesiology reports · doi:10.1007/s40140-016-0171-8 · Cited by 15
Predicción de mortalidad en recién nacidos prematuros. Revisión sistemática actualizada
del Río, R., Thió, M., Bosio, M., Figueras, J., & Iriondo, M. (2020) · Anales de Pediatría · doi:10.1016/j.anpedi.2019.11.003 · Cited by 17
Sattler, L. N., Hing, W. A., Rathbone, E. N., & Vertullo, C. J. (2020) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-020-03499-5 · Cited by 14
Cooray, S. D., Boyle, J. A., Soldatos, G., Zamora, J., Fernandez-Felix, B. M., Allotey, J., Thangaratinam, S., & Teede, H. J. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038845 · Cited by 14
Ahmeidat, A., Kotts, W. J., Wong, J., McLernon, D. J., & Black, M. E. (2021) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2021.05.011 · Cited by 14
Beumer, B. R., Takagi, K., Vervoort, B., Buettner, S., Umeda, Y., Yagi, T., Fujiwara, T., Steyerberg, E. W., & IJzermans, J. N. M. (2021) · Annals of Surgical Oncology · doi:10.1245/s10434-021-10235-3 · Cited by 16
“This retrospective cohort study is reported according to the critical appraisal and data extraction for systematic reviews of the prediction modelling studies (CHARMS) checklist and the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines (Electronic suppl…”
MethodKillingmo, R. M., Chiarotto, A., van der Windt, D., Storheim, K., Bierma‐Zeinstra, S., Småstuen, M. C., Zolic-Karlsson, Z., Vigdal, Ø. N., Koes, B. W., & Grotle, M. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08180-2 · Cited by 14
Latt, P. M., Soe, N. N., Fairley, C. K., Chow, E. P. F., Johnson, C., Shah, P., Maatouk, I., Zhang, L., & Ong, J. J. (2025) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2025.107922 · Cited by 16
O’Toole, C. C., Boakye, N. F., Hannigan, A., & Jalali, A. (2025) · Prostate Cancer and Prostatic Diseases · doi:10.1038/s41391-025-01014-2 · Cited by 15
Machine learning methods for predicting adverse drug events: A systematic review
Chalabianloo, N., Ahmadi, F., Omrani, M. A., Abdullah, S. S., Rostamzadeh, N., Jafari, A., Izzedin, L., Sedig, K., & Muanda, F. T. (2025) · British Journal of Clinical Pharmacology · doi:10.1002/bcp.70377 · Cited by 17
Prediction models of diabetes complications: a scoping review
Ndjaboué, R., Ngueta, G., Rochefort-Brihay, C., Delorme, S., Guay, D., Ivers, N. M., Shah, B. R., Straus, S. E., Yu, C. H. Y., Comeau, S., et al. (2022) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2021-217793 · Cited by 16
Zhang, S., Zhang, K., Chen, W. Y., & Wu, C. (2023) · Health Science Reports · doi:10.1002/hsr2.1309 · Cited by 13
“(CHARMS checklist), which listed relevant items to extract from individual studies in a systematic review of prediction models.(7) The following information was extracted: author name, publication year, study design, study population, geographical location, predicted outcome, prediction horizon (i.”
Tiberio, P., Gaudio, M., Belloni, S., Pindilli, S., Benvenuti, C., Jacobs, F., Saltalamacchia, G., Zambelli, A., Santoro, A., & Sanctis, R. D. (2023) · Cancers · doi:10.3390/cancers15133424 · Cited by 15
“The CHARMS checklist (the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) guided the study selection and data extraction processes [37] (Supplementary Figure S1).”
MethodByrne, J. F., Mongan, D., Murphy, J., Healy, C., Föcking, M., Cannon, M. D., & Cotter, D. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02623-y · Cited by 14
“Risk of bias assessments were carried out using the prediction model risk of bias assessment tool (PROBAST) [30, 31], which assesses risk of bias in the selection of participants, measurement of predictors or outcomes and in the analysis.”
MethodMultivariable prediction models of caries increment: a systematic review and critical appraisal
Havsed, K., Petersson, G. H., Isberg, P., Pigg, M., Svensäter, G., & Rohlin, M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02298-y · Cited by 17
A meta‐analysis of diabetes risk prediction models applied to prediabetes screening
Liu, Y., Yu, S., Feng, W., Mo, H., Hua, Y., Zhang, M., Zhu, Z., Zhang, X., Wu, Z., Zheng, L., et al. (2024) · Diabetes Obesity and Metabolism · doi:10.1111/dom.15457 · Cited by 15
“We followed the Cochrane Guidelines for Systematic Review Evaluation and Data Extraction for Predictive Modelling Studies (CHARMS) checklist 22 and performed a systematic evaluation based on the Pre-ferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.”
MethodJia, Y., Cui, N., Jia, T., & Song, J. P. (2024) · ESC Heart Failure · doi:10.1002/ehf2.14696 · Cited by 14
Fu, Y., Feller, D., Koes, B. W., & Chiarotto, A. (2024) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2024.12081 · Cited by 17
Oakley, W., Tandle, S., Perkins, Z., & Marsden, M. E. R. (2024) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000004385 · Cited by 17
Li, J., Dan, K., & Ai, J. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1281940 · Cited by 21
Clinical prognostic factors for older people: A systematic review and meta-analysis
Veronese, N., Fazzari, A., Armata, M. G., Parisi, A., Parrinello, A., Petralia, V., Saccaro, C., Vesco, M., Tagliaferri, F., Fittipaldo, V. A., et al. (2024) · Ageing Research Reviews · doi:10.1016/j.arr.2024.102345 · Cited by 12
“(Moons et al., 2014) The eight investigators (AF, MA, AP, AP, VP, CS, MV, FT) extracted data independently which included author of study, year of study, type of study (retrospective or prospective), total sample size, mean age with the standard deviation (SD), percentage of females, setting,…”
MethodGalimzhanov, A. M., Tun, H. N., Sabitov, Y., Perone, F., Kursat, T. M., Tenekecioğlu, E., & Mamas, M. A. (2024) · European Journal of Clinical Investigation · doi:10.1111/eci.14295 · Cited by 16
“13,17 The extracted data included information on study characteristics (publication year, country, study design, follow-up period), study population (inclusion criteria, sample size, mean age, gender, prevalence of risk factors, provided laboratory parameters) and effect estimates.”
Huang, J., Chen, J., Yang, J., Han, M., Xue, Z., Wang, Y., Xu, M., Qi, H., & Wang, Y. (2024) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2024.103808 · Cited by 17
Prediction models for postpartum stress urinary incontinence: A systematic review
Wang, L., Zhang, M., Sha, K., Qiao, Y., & Dong, Q. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37988 · Cited by 14
Battle, C. E., Cole, E. M., Carter, K., & Baker, E. J. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-01107-6 · Cited by 16
“Appli-cability refers to the extent to which the primary study matches the review question, and thus is applicable for the intended use of the reviewed prediction model(s) in the target population” (Moons et al., 2014).”
BackgroundMethodShin, H. A., Kang, H., & Choi, M. (2025) · Healthcare Informatics Research · doi:10.4258/hir.2025.31.1.23 · Cited by 20
“The data extraction form was based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [15].”
Rehman, A. U., Neyra, J. A., Chen, J., & Ghazi, L. (2025) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2025.2497843 · Cited by 21
“…in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Extension for Scoping Reviews) 2018 statement and the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) 2018 guidelines [22,23].”
Tierney, J. F., Vale, C. L., Parelukar, W. R., Rydzewska, L. H. M., & Halabi, S. (2019) · European Urology Focus · doi:10.1016/j.euf.2019.01.005 · Cited by 12
“validation to ensure that rigorous tools will be produced(38-40).”
BackgroundGeeson, C., Wei, L., & Franklin, B. D. (2019) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14119 · Cited by 13
Zhang, F., Wong, C. J. J., Chiu, Y., Ensor, J., Mohamed, M., Peat, G., & Mamas, M. A. (2021) · International Journal of Clinical Practice · doi:10.1111/ijcp.14345 · Cited by 16
Systematic Review of Prediction Models for Preterm Birth Using CHARMS
Kim, J., & Lee, J. Y. (2021) · Biological Research For Nursing · doi:10.1177/10998004211025641 · Cited by 14
“One study developed prediction models for single and multiple fetuses, respectively (Tan et al., 2007), and three studies developed models for primipara and multipara (Mercer et al., 1996; Morken et al., 2014).”
BackgroundMethodResultInfluentialHodel, J., Stucki, G., & Prodinger, B. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.07.015 · Cited by 15
Kappes, J. R., Huber, D., Kirchebner, J., Sonnweber, M., Günther, M. P., & Lau, S. (2021) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x211062139 · Cited by 13
“To avoid overfitting, it is advisable to include imputation, filtering, balancing and model building in a cross-validation process kept separate from model-testing (Moons et al., 2014; Studerus et al., 2017).”
MethodBoden, I., Bernabéu, M. O., Dhillon, B., Dorward, D. A., MacCormick, I. J. C., Megaw, R. D., & Tochel, C. (2022) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2022.109869 · Cited by 15
“Eligibility criteria for included studies were based on the CHARMS checklist [15]:”
Black, N., Henderson, I., Wattar, B. H. A., & Quenby, S. M. (2022) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000004940 · Cited by 15
Prediction Model Risk-of-Bias Assessment Tool for coronary artery lesions in Kawasaki disease
Huang, H., Dong, J., Wang, S., Shen, Y., Zheng, Y., Jiang, J., Zeng, B., Li, X., Yang, F., Ma, S., et al. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1014067 · Cited by 13
“Then, the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist was used for data extraction (11).”
BackgroundMethodInfluentialJeon, D., Chavda, S., Rennert‐May, E., & Leal, J. R. (2023) · Journal of Hospital Infection · doi:10.1016/j.jhin.2023.01.003 · Cited by 14
Karter, A. J., Parker, M. M., Moffet, H. H., Lipska, K. J., Laiteerapong, N., Grant, R. W., Lee, C., & Huang, E. S. (2023) · Journal of General Internal Medicine · doi:10.1007/s11606-023-08219-y · Cited by 13
“LEAD had similar performance by race/ethnicity, gender, and type of diabetes 39,46,50 and generalized well when assessed in other time periods (i.e., the 2010 Cohort with up to 10 years of follow-up and the 2019 Cohort with 2 years of follow-up which overlapped with the beginning of the COVID-19…”
MethodGao, S., Albu, E., Tuand, K., Cossey, V., Rademakers, F. E., CALSTER, B. V., & Wynants, L. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.07.019 · Cited by 16
Hilkens, N. A., Algra, A., & Greving, J. P. (2015) · Journal of Thrombosis and Haemostasis · doi:10.1111/jth.13196 · Cited by 13
Tariciotti, L., Palmisciano, P., GIORDANO, M., REMOLI, G., LACORTE, E., Bertani, G., Locatelli, M., DIMECO, F., Caccavella, V. M., & PRADA, F. (2021) · Journal of Neurosurgical Sciences · doi:10.23736/s0390-5616.21.05483-7 · Cited by 19
Hunik, L., Chaabouni, A., van Laarhoven, T., Hartman, T. O., Leijenaar, R. T. H., Cals, J., Uijen, A. A., & Schers, H. (2025) · JMIR Medical Informatics · doi:10.2196/62862 · Cited by 21
Yu, M.-Y., Yoo, H. Y., Han, G. I., Kim, E. E., & Son, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/76215 · Cited by 15
“This systematic review adhered to the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [11] and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [12] guidelines.”
Dhiman, P., Ma, J., Gibbs, V. N., Rampotas, A., Kamal, H., Arshad, S., Kirtley, S., Doree, C. J., Murphy, M. F., Collins, G. S., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.002 · Cited by 17
Gokhale, S., Taylor, D., Gill, J., Hu, Y., Zeps, N., Lequertier, V., Teede, H. J., & Enticott, J. (2023) · Digital Health · doi:10.1177/20552076231177497 · Cited by 16
Wainstein, M., Flanagan, E., Johnson, D. W., & Shrapnel, S. (2023) · Frontiers in Nephrology · doi:10.3389/fneph.2023.1220214 · Cited by 16
“This review was designed and conducted according to Preferred Reporting Items and Systematic Review and Meta-Analyses Frontiers in Nephrology 02 (PRISMA) Protocol guideline (7) and the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (8).”
MethodInfluentialAlves, V. R., Cardoso, J. S., & da Gama, J. M. P. (2023) · Nuclear Medicine and Molecular Imaging · doi:10.1007/s13139-023-00821-6 · Cited by 15
“Predictive modelling for diagnosis purposes follows the same principles as the diagnosis tests regarding obtaining an unbiased reference standard [77].”
BackgroundMethodYoong, S. Q., Bhowmik, P., Kapparath, S., & Porock, D. C. (2024) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djae036 · Cited by 14
Sample size determination for prediction models via learning‐type curves
Dayimu, A., Simidjievski, N., Demiris, N., & Abraham, J. (2024) · Statistics in Medicine · doi:10.1002/sim.10121 · Cited by 20
“5 Whilst simple, such criteria do not consider the predictors’ type, magnitude and possible values (eg, categorical variables may require more events)—often leading to poorly fitted models that fail to generalize well to out-of-sample data.”
BackgroundLi, J., Li, G., Liu, Z., Yang, X., & Yang, Q. (2024) · American Journal of Infection Control · doi:10.1016/j.ajic.2024.07.006 · Cited by 13
Diagnostic Performance of Machine Learning-based Models in Neonatal Sepsis: A Systematic Review
Kainth, D., Prakash, S., & Sankar, M. J. (2024) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000004409 · Cited by 12
Luo, Y., Cui, X., Zhou, J., Zhuang, Y., Zheng, C., Su, Q., Gan, Y., Li, Z., & Zeng, H. (2024) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000002983 · Cited by 12
“Based on an empirical study, a rule of thumb for at least 10 EPV was proposed and is widely accepted as a means of avoiding over fi tting (11).”
Predictive Modeling of Early Neurological Deterioration in Patients with Acute Ischemic Stroke
Zhang, J., Luo, Z., & Zeng, Y. (2024) · World Neurosurgery · doi:10.1016/j.wneu.2024.08.017 · Cited by 12
Walker, H., Day, S., Grant, C. H., Jones, C., Ker, R., Sullivan, M. K., Jani, B., Gallacher, K., & Mark, P. B. (2024) · BMC Medicine · doi:10.1186/s12916-024-03649-9 · Cited by 13
“Only 70% of studies reporting calibration ( n = 50) did so graphically, which is the preferred form of reporting calibration [28].”
BackgroundMethodBoldrini, L., Charles-Davies, D., Romano, A., Mancino, M., Nacci, I., Tran, H. E., Bono, F., Boccia, E., Gambacorta, M. A., & Chiloiro, G. (2024) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2024.109463 · Cited by 13
Abdaltawab, A., Chang, L., Mansour, M., & Koubeissi, M. Z. (2024) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2024.110212 · Cited by 15
Prediction models for treatment response in migraine: a systematic review and meta-analysis
Chen, Q., Zhang, J., Cao, B., Hu, Y., Kong, Y., Li, B., & Liu, L. (2025) · The Journal of Headache and Pain · doi:10.1186/s10194-025-01972-x · Cited by 15
“For the systematic review, we adopted the populations, interventions, comparators, outcomes, timing, types of studies and settings (PICOTS) system, which was recommended by the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [26].”
Roheger, M., Liebermann‐Jordanidis, H., Krohm, F., Adams, A., & Kalbe, E. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.636355 · Cited by 14
“Data Extraction Two review authors (MR and HLJ) independently extracted the data according to the Critical appraisal and data extraction for systematic reviews of prediction modeling studies_ prognostic factors (CHARMS_PF) checklist (Moons et al., 2014) to investigate the reporting of prognostic factors.”
MethodEvidence summary of human milk fortifier in preterm infants
Gu, X., Shi, X., Zhang, L., Zhou, Y., Cai, Y., Jiang, W., & Zhou, Q. (2021) · Translational Pediatrics · doi:10.21037/tp-21-476 · Cited by 15
“The risk prediction model adopted the bias risk assessment tool Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) list (18) of the clinical prediction model to evaluate the quality of the included literatures.”
MethodTran, A. (2021) · OSF Preprints (OSF Preprints) · Cited by 12
Majithia-Beet, G., Naemi, R., & Issitt, R. W. (2022) · Perfusion · doi:10.1177/02676591221115267 · Cited by 12
Gehringer, C. K., Martin, G. P., Hyrich, K. L., Verstappen, S. M. M., & Sergeant, J. C. (2022) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2022.152076 · Cited by 13
Rademaker, M. M., Meijers, S. M., Smit, A. L., & Stegeman, I. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12020695 · Cited by 13
“This was based on the CHARMS checklist and previous research projects [9,12,13].”
MethodInfluentialPrediction of new onset postoperative atrial fibrillation using a simple Nomogram
Zhu, S., Che, H., Fan, Y., & Jiang, S. (2023) · Journal of Cardiothoracic Surgery · doi:10.1186/s13019-023-02198-1 · Cited by 12
“The definition of ML and its differences from statistical models have not yet been defined [24].”
BackgroundArshi, B., Wynants, L., Rijnhart, E., Reeve, K. A., Cowley, L., & Smits, L. J. M. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073174 · Cited by 16
Li, S., Zhang, J., Li, N., Wang, D., & Zhao, X. (2023) · CNS Neuroscience & Therapeutics · doi:10.1111/cns.14288 · Cited by 12
“15 A total of seven variables were finally included in the multivariable logistics regression analysis to predict poor functional outcome.”
MethodLangeslag, J. F., Berendse, K., Daams, J. G., Onland, W., Leeflang, M., van Kaam, A. H., & de Haan, T. R. (2023) · Neonatology · doi:10.1159/000530411 · Cited by 12
“This systematic review was performed according to the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [10] and reported following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines [11].”
MethodGeersing, G., Kraaijpoel, N., Buller, H. R., van Doorn, S., van Es, N., Gal, G. L., Huisman, M. V., Kearon, C., Kline, J. A., Moons, K. G. M., et al. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0032-7 · Cited by 15
“in these particular studies as well [20].”
Self-citationDeliu, N., Cottone, F., Collins, P. G. S., Anota, A., & Efficace, F. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-025054 · Cited by 11
“Methodological quality assessment Critical appraisal of methodological quality and risk of bias of included papers will be undertaken independently by two or more reviewers according to an adapted version of the CHARMS checklist for critical appraisal of prognostic model studies.(31) Following these guidelines, the re…”
MethodPrediction of mortality in premature neonates. An updated systematic review
del Río, R., Thió, M., Bosio, M., Figueras, J., & Iriondo, M. (2020) · Anales de Pediatría (English Edition) · doi:10.1016/j.anpede.2019.11.005 · Cited by 12
Simon-Pimmel, J., Foucher, Y., Léger, M., Feuillet, F., Bodet-Contentin, L., Cinotti, R., Frasca, D., & Dantan, É. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047279 · Cited by 13
Lai, N. M., Gerard, J. P., Ngim, C. F., Kamar, A. A., & Chen, K. (2021) · Neonatology · doi:10.1159/000519497 · Cited by 14
Günther, M. P., Kirchebner, J., Schulze, J. B., Känel, R., & Euler, S. (2022) · European Journal of Cancer Care · doi:10.1111/ecc.13555 · Cited by 11
External Validation of AI Models for Skin Diseases: A Systematic Review
Fernandes, T. R. S., Teles, A. S., Fernandes, J. R. N., Lima, L. D. B., Sousa, D. L., Soares, R. D. C., & Teixeira, S. S. (2025) · IEEE Access · doi:10.1109/access.2025.3584904 · Cited by 12
“The development of this checklist was based on key domains and methodological guidance from established tools such as PROBAST [97] and CHARMS [67].”
Zhang, H., Wu, X.-Y., Ni, W., Wu, J., Zhou, S., Li, J., Jin, M., Zhao, S., Jiang, Z., Wu, C., et al. (2025) · Journal of Advanced Research · doi:10.1016/j.jare.2025.08.042 · Cited by 23
“Data extraction followed the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [21] ( Table S1).”
Machine Learning in the Prediction of Venous Thromboembolism: Systematic Review and Meta-Analysis
Ma, R., Yu, W., Tian, J., Tang, Y., Fang, H., Ming, X., & Liu, H. (2025) · Journal of Medical Internet Research · doi:10.2196/77339 · Cited by 11
“We extracted data from the included studies using a standardized evidence table based on the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) checklist [9].”
Reudink, M., Slooter, C. D., Janssen, L., Lieverse, A. G., Roumen, R. M. H., & Slooter, G. D. (2021) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2021.102997 · Cited by 14
The Magnitude of NCD Risk Factors in Ethiopia: Meta-Analysis and Systematic Review of Evidence
Tesfay, F. H., Backholer, K., Zorbas, C., Bowe, S. J., Alston, L. V., & Bennett, C. M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095316 · Cited by 13
“Data were extracted using a data extraction proforma adapted from the Joanna Briggs Institute (JBI) [43] and other literature [44].”
MethodAly, F., Hansen, C. R., Mouiee, D. A., Sundaresan, P., Haidar, A., Vinod, S. K., & Holloway, L. C. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2023.109629 · Cited by 11
Performance of a Prediabetes Risk Prediction Model: A Systematic Review
Liu, Y., Feng, W., Lou, J., Qiu, W., Shen, J., Zhu, Z., Hua, Y., Zhang, M., & Billong, L. F. (2022) · Heliyon · doi:10.1016/j.heliyon.2023.e15529 · Cited by 17
Duo, Y., Song, S., Qiao, X., Zhang, Y., Xu, J., Zhang, J., Peng, Z., Chen, Y., Nie, X., Sun, Q., et al. (2023) · Diabetes Therapy · doi:10.1007/s13300-023-01480-8 · Cited by 13
“In according to the CHARMS recommendations [30], we extracted coefficients from the multi-variate logistic regression and used these to calculate the GDM risk score.”
MethodZhang, W., Cheng, W., Fujiwara, K., Evans, R. D., & Zhu, C. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3349353 · Cited by 16
Issanov, A., Aravindakshan, A., Puil, L. J., Tammemägi, M. C., Lam, S. C., & Dummer, T. (2024) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00166-4 · Cited by 14
“The search strategy has been built around the three main concepts according to the PICOTS format, which is a specific format to systematic reviews of prediction models [45]: population — nonsmokers, outcome — lung cancer, and index models — prediction models.”
MethodInfluentialParsaei, M., Arvin, A., Taebi, M., Seyedmirzaei, H., Cattarinussi, G., Sambataro, F., Pigoni, A., Brambilla, P., & Delvecchio, G. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1384828 · Cited by 17
Rayner, D. G., Kim, B. Y., & Foroutan, F. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111360 · Cited by 15
Yue, Z., Sun, Y., Pan, Y., Dai, Y., Xiao, Y., & Yu, Y. (2024) · Australian Critical Care · doi:10.1016/j.aucc.2024.05.003 · Cited by 19
Risk Prediction Models as an Emerging Trend for Managing Cancer‐Related Fatigue: A Systematic Review
Zhang, Y., Li, L., Li, X., Zhang, S., Zhou, L., Chen, X., & Hu, X. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16680 · Cited by 12
Locquet, M., Jacob, S., Geets, X., & Beaudart, C. (2024) · BMC Cancer · doi:10.1186/s12885-024-13281-8 · Cited by 11
Mortality Risk Prediction Models for People With Kidney Failure: A Systematic Review
Jarrar, F., Pasternak, M., Harrison, T. G., James, M. T., Quinn, R. R., Lam, N. N., Donald, M., Elliott, M. J., Lorenzetti, D., Strippoli, G. F., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.53190 · Cited by 18
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.