The DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessAbstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 201–300 of 1,004
Klement, W., & Emam, K. E. (2023) · Journal of Medical Internet Research · doi:10.2196/48763 · Cited by 72
Loymans, R. J. B., Debray, T. P. A., Honkoop, P., Termeer, E. H., Snoeck-Stroband, J. B., Schermer, T. R., Assendelft, W. J., Timp, M. L., Chung, K. F., Sousa, A. R., et al. (2018) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2018.02.004 · Cited by 68
Cho, S. M., Austin, P. C., Ross, H. J., Abdel-Qadir, H. M., Chicco, D., TOMLINSON, G. A., Taheri, C., Foroutan, F., Lawler, P. R., Billia, F., et al. (2021) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.02.020 · Cited by 74
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · European Journal of Clinical Investigation · doi:10.1111/eci.12376 · Cited by 66
Self-citationde Winter, M. A., van Es, N., Buller, H. R., Visseren, F. L. J., & Nijkeuter, M. (2021) · Thrombosis Research · doi:10.1016/j.thromres.2020.12.031 · Cited by 65
A Prediction Rule to Stratify Mortality Risk of Patients with Pulmonary Tuberculosis
Bastos, H. N., Osório, N. S., Castro, A. G., Ramos, A., Carvalho, T., Meira, L., Araújo, D., Almeida, L. M., Boaventura, R. D. P., Fragata, P., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0162797 · Cited by 60
“The reporting of this study conforms to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement [17] and to the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [18].”
MethodTewarie, I. A., Senders, J. T., Kremer, S. W. F., Devi, S., Gormley, W. B., Arnaout, O. M., Smith, T. R., & Broekman, M. L. D. (2020) · Neurosurgical Review · doi:10.1007/s10143-020-01430-z · Cited by 64
Schultebraucks, K., Sijbrandij, M., Galatzer‐Levy, I. R., Mouthaan, J., Olff, M., & van Zuiden, M. (2021) · Neurobiology of Stress · doi:10.1016/j.ynstr.2021.100297 · Cited by 72
“Result reporting is based on the CHARMS-checklist (Moons et al., 2014) for prognostic models.”
MethodVieira, B. H., Pamplona, G. S. P., Fachinello, K., Silva, A. K., Foss, M. P., & Salmón, C. E. G. (2022) · Intelligence · doi:10.1016/j.intell.2022.101654 · Cited by 69
Tran, A., Rochwerg, B., Fan, E., Bělohlávek, J., Suverein, M. M., van de Poll, M. C. G., Lorusso, R., Price, S., Yannopoulos, D., MacLaren, G., et al. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2023.110004 · Cited by 66
Kraaijenga, V. J. C., Smit, A. L., Stegeman, I., Smilde, J. J. M., van Zanten, G. A., & Grolman, W. (2015) · Clinical Otolaryngology · doi:10.1111/coa.12571 · Cited by 59
Halligan, S., Menu, Y., & Mallett, S. (2021) · European Radiology · doi:10.1007/s00330-021-07971-1 · Cited by 62
“PROBAST should be used to appraise component studies [48, 56].”
MethodSelf-citationKabore, E., MacDonald, C., Kaboré, A., Didier, R. J., Arveux, P., Méda, N., Boutron‐Ruault, M., & Guénancia, C. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.0569 · Cited by 69
van den Boorn, H. G., Engelhardt, E. G., van Kleef, J. J., Sprangers, M. A. G., van Oijen, M. G. H., Abu–Hanna, A., Zwinderman, A. H., Coupé, V. M. H., & van Laarhoven, H. W. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0192310 · Cited by 58
Halladay, J. E., BENNETT, K. J., Weist, M. D., Boyle, M., Manion, I. G., Campo, M., & Georgiades, K. (2020) · Journal of School Psychology · doi:10.1016/j.jsp.2020.05.003 · Cited by 80
Kolk, M. Z. H., Deb, B., Ruipérez-Campillo, S., Bhatia, N. K., Clopton, P., Wilde, A. A. M., Narayan, S. M., Knops, R. E., & Tjong, F. V. Y. (2023) · EBioMedicine · doi:10.1016/j.ebiom.2023.104462 · Cited by 67
“19,20 Population Subjects from whom electrophysiological signals were obtained for the purpose of predicting the occurrence of the outcome(s) of interest were included.”
BackgroundVernooij, J. E. M., Koning, N. J., Geurts, J. W., Holewijn, S., Preckel, B., Kalkman, C. J., & Vernooij, L. M. (2023) · Anaesthesia · doi:10.1111/anae.15988 · Cited by 64
“According to recommendations in the CHARMS checklist, one author (JV) extracted the data following a preconstructed data extraction form [20].”
MethodRicke, E., Dijkstra, A., & Bakker, E. W. P. (2023) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2023.1035023 · Cited by 69
Chowdhury, M., Yeasmin, F., Rabi, D. M., Ronksley, P. E., & Turin, T. C. (2018) · Journal of Diabetes and its Complications · doi:10.1016/j.jdiacomp.2018.10.010 · Cited by 62
Impact analysis studies of clinical prediction rules relevant to primary care: a systematic review
Wallace, E., Uijen, M. J. M., Clyne, B., Zarabzadeh, A., Keogh, C., Galvin, R., Smith, S. M., & Fahey, T. P. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009957 · Cited by 56
“6 41 The issue of methodological quality has recently been addressed with the publication of two standardised reporting guidelines for CPR derivation and validation studies, and systematic reviews of CPRs.(42) 43 These publications will have an important role to play in standardising CPR research and in promoting robu…”
InfluentialCardiovascular risk prediction models for women in the general population: A systematic review
Baart, S. J., Dam, V., Scheres, L. J. J., Damen, J. A., Spijker, R., Schuit, E., Debray, T. P. A., Fauser, B. C. J. M., Boersma, E. H., Moons, K. G. M., et al. (2019) · PLoS ONE · doi:10.1371/journal.pone.0210329 · Cited by 63
Self-citationBallanti, S., Campagnini, S., Liuzzi, P., Hakiki, B., Scarpino, M., Macchi, C., Oddo, C. M., Carrozza, M. C., Grippo, A., & Mannini, A. (2022) · Clinical Neurophysiology · doi:10.1016/j.clinph.2022.09.017 · Cited by 64
“For the data collection, the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) was used (Moons et al., 2014).”
Tufts PACE Clinical Predictive Model Registry: update 1990 through 2015
Wessler, B. S., Paulus, J. K., Lundquist, C. M., Ajlan, M., Natto, Z. S., Janes, W., Jethmalani, N., Raman, G., Lutz, J. S., & Kent, D. M. (2017) · Diagnostic and Prognostic Research · doi:10.1186/s41512-017-0021-2 · Cited by 62
“All data extraction done after 2015 has been aligned with the recently published Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement [8] and Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) Checklist [10…”
MethodStrijker, M., Chen, J., Mungroop, T. H., Jamieson, N. B., van Eijck, C. H. J., Steyerberg, E. W., Wilmink, J. W., Koerkamp, B. G., van Laarhoven, H. W. M., & Besselink, M. G. (2019) · British journal of surgery · doi:10.1002/bjs.11111 · Cited by 61
McGranaghan, P., Saxena, A., Rubens, M., Radenkovic, J., Bach, D., Schleußner, L., Pieske, B. M., Edelmann, F., & Trippel, T. D. (2020) · Biomarkers · doi:10.1080/1354750x.2020.1716073 · Cited by 59
Scores for preoperative risk evaluation of postoperative mortality
Kivrak, S., & Haller, G. (2020) · Best practice & research. Clinical anaesthesiology · doi:10.1016/j.bpa.2020.12.005 · Cited by 62
The Combined Effects of Ocean Acidification and Heavy Metals on Marine Organisms: A Meta-Analysis
Jin, P., Zhang, J., Wan, J., Overmans, S., Gao, G., Ye, M., Dai, X., Zhao, J., Xiao, M., & Xia, J. (2021) · Frontiers in Marine Science · doi:10.3389/fmars.2021.801889 · Cited by 63
A systematic review of microbial markers for risk prediction of colorectal neoplasia
Yu, L., Zhao, G., Wang, L., Zhou, X., Sun, J., Li, X., He, Y., He, Y., Kofonikolas, K., Bogaert, D., et al. (2022) · British Journal of Cancer · doi:10.1038/s41416-022-01740-7 · Cited by 55
“For risk prediction studies, we appraised each model using the checklist for critical appraisal and data extraction of systematic reviews of prediction modelling studies (CHARMS) [20].”
MethodSun, Z., Dong, W., Shi, H., Ma, H., Cheng, L., & Huang, Z. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.812276 · Cited by 67
“We conducted this analysis according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement (20) and the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) (21).”
MethodHealth impact of urban green spaces: a systematic review of heat-related morbidity and mortality
Nazish, A., Abbas, K., & Sattar, E. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-081632 · Cited by 74
Prediction of Walking and Arm Recovery after Stroke: A Critical Review
Kwah, L. K., & Herbert, R. (2016) · Brain Sciences · doi:10.3390/brainsci6040053 · Cited by 56
Ziauddeen, N., Roderick, P., Macklon, N., & Alwan, N. A. (2017) · Obesity Reviews · doi:10.1111/obr.12640 · Cited by 69
“The list of data extraction was based on the CHARMS checklist published by the Cochrane Prognosis Methods Group (26).”
Prediction models for the incidence and progression of periodontitis: A systematic review
Du, M., Bo, T., Kapellas, K., & Peres, M. A. (2018) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13037 · Cited by 58
“The CHARMS (Moons et al., 2014) (Supporting information Appendix S2) was used for data extraction.”
MethodInfluentialKhalifa, M., Magrabi, F., & Gallego, B. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0940-7 · Cited by 61
Beumer, B. R., Buettner, S., Galjart, B., van Vugt, J. L. A., de Man, R. A., IJzermans, J. N. M., & Koerkamp, B. G. (2021) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2021.09.012 · Cited by 58
Perry, L. A., Liu, Z., Loth, J., Penny‐Dimri, J. C., Plummer, M. P., Segal, R., & Smith, J. A. (2021) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2021.07.001 · Cited by 57
van der Endt, V. H. W., Milders, J., de Vries, B. B. L. P., Trines, S. A. I. P., Groenwold, R. H. H., Dekkers, O. M., Trevisan, M., Carrero, J. J., van Diepen, M., Dekker, F. W., et al. (2022) · EP Europace · doi:10.1093/europace/euac096 · Cited by 61
Machine Learning–Based Prediction Models for Delirium: A Systematic Review and Meta-Analysis
Xie, Q., Wang, X., Pei, J., Wu, Y., Guo, Q., Su, Y., Yan, H., Nan, R., Chen, H., & Dou, X. (2022) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2022.06.020 · Cited by 58
Lu, J., Callahan, A., Patel, B., Morse, K., Dash, D., Pfeffer, M. A., & Shah, N. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.27779 · Cited by 61
Application of machine learning in predicting the risk of postpartum depression: A systematic review
Zhong, M., Zhang, H., Yu, C., Jiang, J., & Duan, X. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.08.070 · Cited by 59
Aldin, A., Umlauff, L., Estcourt, L. J., Collins, P. G. S., Moons, K. G. M., Engert, A., Kobe, C., von Tresckow, B., Haque, M., Foroutan, F., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012643.pub3 · Cited by 59
Self-citationYi, C. Y., Childs, C., Peng, C., & Robinson, D. (2021) · Building and Environment · doi:10.1016/j.buildenv.2021.108550 · Cited by 59
Prediction of Risk of Death for Patients Starting Dialysis: A Systematic Review and Meta-Analysis
Anderson, R. T., Cleek, H., Pajouhi, A., Bellolio, F., Mayukha, A., Hart, A., Hickson, L. J., Feely, M. A., Wilson, M. E., Connolly, R. M. G., et al. (2019) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.00050119 · Cited by 54
ICU Delirium-Prediction Models: A Systematic Review
Ruppert, M. M., Lipori, J., Patel, S. H., Ingersent, E., Cupka, J., Ozrazgat‐Baslanti, T., Loftus, T. J., Rashidi, P., & Bihorac, A. (2020) · Critical Care Explorations · doi:10.1097/cce.0000000000000296 · Cited by 76
Prediction models for clinical outcome after cochlear implantation: a systematic review
Velde, H. M., Rademaker, M. M., Damen, J. A., Smit, A. L., & Stegeman, I. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.04.005 · Cited by 52
Mapping Landslide Susceptibility Over Large Regions With Limited Data
Woodard, J., Mirus, B. B., Crawford, M. M., Or, D., Leshchinsky, B., Allstadt, K. E., & Wood, N. (2023) · Journal of Geophysical Research Earth Surface · doi:10.1029/2022jf006810 · Cited by 64
“to estimate the predictors' coefficients within a prediction model (Moons et al., 2014; Pavlou et al., 2016; Peduzzi et al., 1996).”
BackgroundObesity as an independent risk factor for COVID-19 severity and mortality
Najafabadi, B. T., Rayner, D. G., Shokraee, K., Shokraie, K., Panahi, P., Rastgou, P., Seirafianpour, F., Landi, F. M., Alinia, P., Parnianfard, N., et al. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015201 · Cited by 69
Fu, H., Hou, D., Xu, R., You, Q., Li, H., Yang, Q., Wang, H., Gao, J., & Bai, D. (2023) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2023.104623 · Cited by 50
Carrick, R. T., Park, J. G., McGinnes, H. L., Lundquist, C. M., Brown, K., Janes, W., Wessler, B. S., & Kent, D. M. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.119.017625 · Cited by 50
McAleenan, A., Kelly, C., Spiga, F., Kernohan, A., Cheng, H., Dawson, S., Schmidt, L., Robinson, T., Brandner, S., Faulkner, C., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013316.pub2 · Cited by 55
“…status of the same patients using at least two techniques, we will extract data on the following items relevant to prognostic factor studies, derived from the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) (Moons 2014).”
MethodInfluentialUsman, T. M., Saheed, Y. K., Nsang, A. S., Ajibesin, A., & Rakshit, S. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102617 · Cited by 62
Chiasakul, T., Lam, B. D., McNichol, M. C., Robertson, W. E., Rosovsky, R. P. G., Lake, L. L., Vlachos, I. S., Adamski, A., Reyes, N., Abe, K., et al. (2023) · European Journal Of Haematology · doi:10.1111/ejh.14110 · Cited by 57
Huang, X., Ribeiro, J. D., & Franklin, J. C. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00239 · Cited by 54
Kreuzberger, N., Damen, J. A., Trivella, M., Estcourt, L. J., Aldin, A., Umlauff, L., Vazquez-Montes, M., Wolff, R. F., Moons, K. G. M., Monsef, I., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012022.pub2 · Cited by 52
Self-citationPrediction modelling in the early detection of neonatal sepsis
Sahu, P., Stanly, E. A. R., Lewis, L., Prabhu, K., Rao, M., & Kunhikatta, V. (2022) · World Journal of Pediatrics · doi:10.1007/s12519-021-00505-1 · Cited by 56
“Data extraction and quality assessment Data extraction was carried out by all the reviewers based on Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [19, 20].”
MethodManothummetha, K., Mongkolkaew, T., Tovichayathamrong, P., Boonyawairote, R., Meejun, T., Srisurapanont, K., Phongkhun, K., Sanguankeo, A., Torvorapanit, P., Moonla, C., et al. (2023) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2023.04.021 · Cited by 50
Critical Assessment of Clinical Prognostic Tools in Melanoma
Mahar, A. L., Compton, C. C., Halabi, S., Hess, K. R., Gershenwald, J. E., Scolyer, R. A., & Groome, P. A. (2016) · Annals of Surgical Oncology · doi:10.1245/s10434-016-5212-5 · Cited by 51
Systematic review of prediction models in relapsing remitting multiple sclerosis
Brown, F. S., Glasmacher, S. A., Kearns, P. K. A., MacDougall, N., Hunt, D. P. J., Connick, P., & Chandran, S. (2020) · PLoS ONE · doi:10.1371/journal.pone.0233575 · Cited by 54
“Data extraction (following CHARMS [20]) was performed by one reviewer and quality assessment by two reviewers.”
MethodInfluentialSu, N., Lagerweij, M. D., & van der Heijden, G. J. M. G. (2021) · Journal of Dentistry · doi:10.1016/j.jdent.2021.103664 · Cited by 55
Romijn, M., Dhiman, P., Finken, M. J. J., van Kaam, A. H., Katz, T. A., Rotteveel, J., Schuit, E., Collins, P. G. S., Onland, W., & Torchin, H. (2023) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.01.024 · Cited by 53
Frondelius, T., Atkova, I., Miettunen, J., Rello, J., Vesty, G. M., Chew, H. S. J., & Jansson, M. M. (2023) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2023.11.009 · Cited by 58
Smith, T., Muller, D. C., Moons, K. G. M., Cross, A. J., Johansson, M., Ferrari, P., Fagherazzi, G., Peeters, P. H. M., Severi, G., Hüsing, A., et al. (2018) · Gut · doi:10.1136/gutjnl-2017-315730 · Cited by 50
Self-citationWang, J., Zhang, Z., Li, Y., Xu, Y., Wan, K., & Chen, Y. (2019) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2019.05.004 · Cited by 54
Stubbs, D. J., Grimes, L. A., & Ercole, A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0226480 · Cited by 48
“Logistic regression models and CHARMS checklist Seven studies generated a multivariable model for one of our pre-specified outcomes using, or including, CPET derived variables (Table 3).”
MethodInfluentialAsamani, J. A., Christmals, C. D., & Reitsma, G. (2021) · Health Policy and Planning · doi:10.1093/heapol/czab022 · Cited by 47
Wadey, C. A., Weston, M. E., Dorobantu, D. M., Pieles, G. E., Stuart, G., BARKER, A. R., Taylor, R. S., & Williams, C. A. (2021) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwab125 · Cited by 48
A scoping review of artificial intelligence applications in clinical trial risk assessment
Teodoro, D., Naderi, N., Yazdani, A., Zhang, B., & Bornet, A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01886-7 · Cited by 70
“Each researcher read the full-texts separately and used a standard spreadsheet to extract item information based on the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) checklist 183 .”
Systematic review of risk prediction models for falls after stroke
Walsh, M. E., Horgan, F., Walsh, C., & Galvin, R. (2016) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2015-206475 · Cited by 49
van Doorn, S., Debray, T. P. A., Kaasenbrood, F., Hoes, A. W., Rutten, F. H., Moons, K. G. M., & Geersing, G. (2017) · Journal of Thrombosis and Haemostasis · doi:10.1111/jth.13690 · Cited by 50
Self-citationSurvival prediction tools for esophageal and gastroesophageal junction cancer: A systematic review
Gupta, V., Coburn, N. G., Kidane, B., Hess, K. R., Compton, C. C., Ringash, J. G., Darling, G. E., & Mahar, A. L. (2018) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2018.03.146 · Cited by 48
Beynon, A. M., Hébert, J. J., Hodgetts, C. J., Boulos, L., & Walker, B. F. (2020) · European Spine Journal · doi:10.1007/s00586-019-06278-6 · Cited by 57
“Data from included studies were extracted by two review authors (AB and CH) using a data collection form based on the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) guideline [19].”
MethodMedina‐Lara, A., Grigore, B., Lewis, R. A., Peters, J., Price, S., Landa, P., Robinson, S., Neal, R. D., Hamilton, W. T., & Spencer, A. E. (2020) · Health Technology Assessment · doi:10.3310/hta24660 · Cited by 56
Contributions and limitations of using machine learning to predict noise-induced hearing loss
Chen, F., Cao, Z., Grais, E. M., & Zhao, F. (2021) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-020-01648-w · Cited by 52
Damen, J. A., Debray, T. P. A., Pajouheshnia, R., Reitsma, J. B., Scholten, R. J. P. M., Moons, K. G. M., & Hooft, L. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026160 · Cited by 50
“There is a need for more guidance on how to score items of critical appraisal checklists for prediction model studies, such as the CHARMS checklist.(15) Several options exist to gain more empirical insight in design-related bias in prediction model studies.”
BackgroundSelf-citationOwusuaa, C., Dijkland, S. A., Nieboer, D., van der Rijt, C. C. D., & van der Heide, A. (2022) · BMC Pulmonary Medicine · doi:10.1186/s12890-022-01911-5 · Cited by 52
“Two researchers independently scored all studies on risk-of-bias with a customized appraisal checklist that consisted of the Quality in Prognosis Studies tool, complemented with items from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist (Additional file 1: e-…”
MethodExternally validated prediction models for pre‐eclampsia: systematic review and meta‐analysis
Tiruneh, S. A., Vu, T. T. T., Moran, L., Callander, E. J., Allotey, J., Thangaratinam, S., Rolnik, D. L., Teede, H. J., Wang, R., & Enticott, J. (2023) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.27490 · Cited by 62
“Data for included articles were extracted using the CHecklist for critical Appraisal and data extraction in systematic Reviews of clinical prediction Modelling Studies (CHARMS) 21 .”
MethodDsouza, V. S., Leyens, L., Kurian, J. R., Brand, A. M., & Brand, H. (2025) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2025.02.008 · Cited by 56
van der Heijden, A. A. W. A., Nijpels, G., Badloe, F. M. S., Lovejoy, H. L., Peelen, L. M., Feenstra, T., Moons, K. G. M., Slieker, R. C., Herings, R. M. C., Elders, P. J. M., et al. (2020) · Diabetologia · doi:10.1007/s00125-020-05134-3 · Cited by 56
“CHARMS Checklist for critical appraisal and data extraction for systemic reviews of prediction modelling studies DCS Diabetes Care System O/E Ratio of the observed and expected retinopathy events PROBAST Prediction model risk of bias assessment tool TRIPOD Transparent reporting of a multivariable prediction model for…”
MethodSelf-citationPecoraro, A., Campi, R., Bertolo, R. G., Mir, M. C., Marchioni, M., Serni, S., Joniau, S., Poppel, H. V., Albersen, M., & Roussel, E. (2023) · European Urology Oncology · doi:10.1016/j.euo.2022.11.007 · Cited by 49
ESMO basic requirements for AI-based biomarkers in oncology (EBAI)
Aldea, M. D., Salto‐Tellez, M., Marra, A., Umeton, R., Stenzinger, A., Koopman, M., Prelaj, A., Kehl, K. L., Gilbert, S., Leßmann, M., et al. (2025) · Annals of Oncology · doi:10.1016/j.annonc.2025.11.009 · Cited by 68
Chen, Z., Liu, X., Yang, Q., Wang, Y.-J., Miao, K., Gong, Z., Yu, Y., Leonov, A., Liu, C., Feng, Z., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.1671 · Cited by 43
Prediction Models for Intravenous Immunoglobulin Resistance in Kawasaki Disease: A Meta-analysis
Kuniyoshi, Y., Tsujimoto, Y., Banno, M., Taito, S., Ariie, T., Takahashi, N., Tokutake, H., & Takada, T. (2023) · PEDIATRICS · doi:10.1542/peds.2022-059175 · Cited by 46
Seath, P., Macedo-Orrego, L. E., & Velayudhan, L. (2023) · International Psychogeriatrics · doi:10.1017/s1041610223000509 · Cited by 51
“A modi fi ed version of the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies, Adapted for Prognostic Factor Studies (CHARMS-PF) was used to guide information extraction (Moons et al ., 2014; Riley et al ., 2019) (Table 1).”
Undernutrition as a risk factor for tuberculosis disease
Franco, J. V. A., Bongaerts, B. W. C., Metzendorf, M., Risso, A., Guo, Y., Silva, L. P., Boeckmann, M., Schlesinger, S., Damen, J. A., Richter, B., et al. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015890.pub2 · Cited by 97
Diagnosing malignant melanoma in ambulatory care: a systematic review of clinical prediction rules
Harrington, E., Clyne, B., Wesseling, N., Sandhu, H., Armstrong, L., Bennett, H., & Fahey, T. P. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014096 · Cited by 45
de Jong, Y., Ramspek, C. L., van der Endt, V. H. W., Rookmaaker, M. B., Blankestijn, P. J., Vernooij, R. W. M., Verhaar, M. C., Bos, W. J. W., Dekker, F. W., Ocak, G., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.03.015 · Cited by 46
Protease activity as a prognostic factor for wound healing in venous leg ulcers
Westby, M. J., Dumville, J. C., Stubbs, N. C., Norman, G., Wong, J. K. F., Cullum, N. A., & Riley, R. D. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012841.pub2 · Cited by 48
Barreñada, L., Ledger, A., Dhiman, P., Collins, G., Wynants, L., Verbakel, J. Y., Timmerman, D., Valentin, L., & Van Calster, B. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2023-000817 · Cited by 42
Ghaderzadeh, M., Aria, M., & Asadi, F. (2021) · BioMed Research International · doi:10.1155/2021/9942873 · Cited by 44
“For predictive models, the data were extracted from the CHARMS checklist modified for the purpose of this study, which also includes an assessment of the risk of deviation [14].”
MethodMethod for Data Quality Assessment of Synthetic Industrial Data
Iantovics, L. B., & Enăchescu, C. (2022) · Sensors · doi:10.3390/s22041608 · Cited by 52
“This criterion has a relatively general acceptance [24,67,68].”
BackgroundWingbermühle, R. W., van Trijffel, E., Nelissen, P. M., Koes, B. W., & Verhagen, A. P. (2017) · Journal of physiotherapy · doi:10.1016/j.jphys.2017.11.013 · Cited by 43
“In that case, the model could obtain idiosyncratic features that are specific to the derivation data itself, resulting in a model that predicts accurately in a derivation sample but performs poorly when applied to other individuals.(38) Too many predictors and categorical variables were often selected in derivation st…”
MethodGregório, T., Pipa, S., Cavaleiro, P., Atanásio, G., Albuquerque, I., Castro‐Chaves, P., & Azevedo, L. F. (2018) · Neurocritical Care · doi:10.1007/s12028-018-0633-6 · Cited by 42
Predictors of dropout in interdisciplinary chronic pain management programmes: A systematic review
Oosterhaven, J., Wittink, H. M., Mollema, J., Kruitwagen, C. L. J. J., & Devillé, W. L. J. M. (2018) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-2502 · Cited by 47
Developing and Validating Risk Assessment Models of Clinical Outcomes in Modern Oncology
Halabi, S., Li, C., & Luo, S. (2019) · JCO Precision Oncology · doi:10.1200/po.19.00068 · Cited by 52
“632+, are considered appropriate approaches to model validation.(32,53,91,92)”
BackgroundMethodMowbray, F., Manlongat, D., Correia, R. H., Strum, R. P., Fernando, S. M., McIsaac, D. I., de Wit, K., Worster, A., Costa, A. P., Griffith, L. E., et al. (2021) · Resuscitation · doi:10.1016/j.resuscitation.2021.06.009 · Cited by 47
Considerations in the reliability and fairness audits of predictive models for advance care planning
Lu, J., Sattler, A. L., Wang, S. X. Y., Khaki, A. R., Callahan, A., Fleming, S. L., Fong, R., Ehlert, B., Li, R., Shieh, L. Y., et al. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.943768 · Cited by 45
“Thus, a prerequisite for reporting summary statistics and model subgroup performance, as recommended by many model reporting guidelines (9,11–13,15 , 17 Lastly, it is important to verify the validity of source data in detail i.e., via manual inspection of the raw data, summary statistics, and…”
MethodGates, M., Pillay, J., Nuspl, M., Wingert, A., Vandermeer, B., & Hartling, L. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02181-w · Cited by 48
“There are at least 12 published fracture risk prediction tools available [36, 37]; however, not all tools are easily accessible to clinicians nor have all tools been calibrated for Canada or validated in populations outside of their derivation cohort, limiting their use [38].”
BackgroundSingh, P., Adderley, N. J., Hazlehurst, J., Price, M. J., Tahrani, A. A., Nirantharakumar, K., & Bellary, S. (2021) · Diabetes Care · doi:10.2337/dc21-0166 · Cited by 50
Forecasting future global food demand: A systematic review and meta-analysis of model complexity
Flies, E. J., Brook, B. W., Blomqvist, L., & Buettel, J. C. (2018) · Environment International · doi:10.1016/j.envint.2018.07.019 · Cited by 51
Campagnini, S., Liuzzi, P., Mannini, A., Riener, R., & Carrozza, M. C. (2022) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-022-01031-5 · Cited by 39
“Data collection For the data collection, the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) was used [24].”
Method
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.