The DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessAbstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 801–900 of 1,004
Gupta, A. K., Mustafiz, C., Mutahar, D., Zaka, A., Parvez, R., Mridha, N., Stretton, B., Kovoor, J. G., Bacchi, S., Ramponi, F., et al. (2025) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2025.01.037 · Cited by 15
Migiddorj, B., Batterham, M. J., & Win, K. T. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105914 · Cited by 14
Risk Prediction Models for Perioperative Hypothermia: A Systematic Review
Liu, J., Liu, F., Xu, W., Du, L., Li, Y., Liang, A., Li, B., & Zhang, M. (2025) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s538891 · Cited by 16
Adolescent experiences and exposures: a systematic review of life course outcomes
Banati, P., Vedechkina, M., Rubeis, V. D., Kalai, S., Massaquoi, L., Sadana, R., Baltag, V., & Baird, S. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103453 · Cited by 16
Kovács, G. G., Devercelli, G., Zelei, T., Hirji, I., Vokó, Z., & Keown, P. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231646 · Cited by 11
“per the relevant PICOTS (patient, intervention, comparator, outcome, timing, and setting) elements as defined in the CHARMS (checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies) methodology [28].”
MethodA general presentation on how to carry out a CHARMS analysis for prognostic multivariate models
Palazón‐Bru, A., Martín‐Pérez, F., Mares‐García, E., Beneyto‐Ripoll, C., Gil-Guillén, V. F., Pérez-Sempere, Á., & Carbonell‐Torregrosa, M. Á. (2020) · Statistics in Medicine · doi:10.1002/sim.8660 · Cited by 12
Pattern of Uveitis in Iran: A Systematic Review
Bagheri, M., Ahoor, M., Jafari, A., Hashemi, H. S., & Mohammadkhani, M. (2021) · Journal of Ophthalmic and Vision Research · doi:10.18502/jovr.v16i1.8255 · Cited by 11
“for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS).[24] Then, the data were extracted.”
MethodDeveloping scenarios that evoke shoot/don't-shoot errors
Biggs, A. T. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103397 · Cited by 12
Papadomanolakis‐Pakis, N., Haroutounian, S., Christiansen, C. F., & Nikolajsen, L. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053618 · Cited by 13
Sonnweber, M., Lau, S., & Kirchebner, J. (2022) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x221102799 · Cited by 15
“To reduce overfitting and to avoid final model selection to be influenced by data processing, nested resampling (Moons et al., 2014; Studerus et al., 2017) was employed for initial modeling: In an inner loop data processing and model training are performed imbedded in fivefold-cross-validation and…”
MethodHuang, Q., Hu, Y.-C., Wang, C.-K., Huang, J., Shen, M., & Ren, L. (2022) · Biological Research For Nursing · doi:10.1177/10998004221131993 · Cited by 17
“In this systematic review and meta-analysis, we formulated inclusion and exclusion criteria based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) guidance (Moons et al., 2014), and the detailed information follows.”
MethodJiang, L., Wu, M., Li, H., Liang, J., Chen, J., & Liu, L. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11216299 · Cited by 16
“Based on the Kendall rule and events per variable (EPV) theory [25,26], the theoretical sample size is as least 90.”
MethodKunonga, P., Kenny, R. P. W., Astin, M., Bryant, A., Kontogiannis, V., Coughlan, D., Richmond, C., Eastaugh, C., Beyer, F. R., Pearson, F., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073306 · Cited by 11
Eissa, A., Elsherbiny, A., Zoeir, A., Sandri, M. F., Pirola, G. M., Puliatti, S., Prete, C. D., Sighinolfi, M. C., Micali, S., Rocco, B., et al. (2019) · Minerva Urologica e Nefrologica · doi:10.23736/s0393-2249.19.03427-1 · Cited by 12
Gates, M., Pillay, J., Thériault, G., Limburg, H., Grad, R. M., Klarenbach, S. W., Korownyk, C. S., Reynolds, D. L., Riva, J. J., Thombs, B. D., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1094-5 · Cited by 13
“We will report our findings in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement [116] or the Checklist for the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies [58], as applicable to the research question.”
MethodDiscussing the Score of Cardioembolic Ischemic Stroke in Chagas Disease
Mendes, F. D. S. N. S., Mediano, M. F. F., da Silva, R. S., Xavier, S. S., do Brasil, P. E. A. A., Saraiva, R. M., Hasslocher‐Moreno, A. M., & de Sousa, A. S. (2020) · Tropical Medicine and Infectious Disease · doi:10.3390/tropicalmed5020082 · Cited by 11
“There are increasingly acceptable standards in the literature regarding development, validation and update of prediction/decision models [20].”
BackgroundXu, W., Huang, J., Yu, Q., Yu, H., Pu, Y., & Shi, Q. (2021) · Neurological Sciences · doi:10.1007/s10072-021-05219-w · Cited by 12
Systematic review of prediction models for postacute care destination decision-making
Kennedy, E. E., Bowles, K. H., & Aryal, S. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab197 · Cited by 13
Liu, Z., Perry, L. A., Penny-Dimri, J. C., Handscombe, M., Overmars, I., Plummer, M. P., Segal, R., & Smith, J. A. (2021) · Transplantation Direct · doi:10.1097/txd.0000000000001261 · Cited by 11
Sickness presenteeism explained by balancing perceived positive and negative effects
Lohaus, D., Habermann, W., & Nachreiner, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.963560 · Cited by 15
“Because several researchers suggest a ratio of 10 cases per variable (Moons et al., 2014; Pavlou et al., 2015), we applied forward and backward inclusion of variables to reduce the high number of variables (45 effects).”
MethodKo, S. Q., Valsdottir, L. R., Strom, J. B., Cheng, Y., Hirayama, A., Liu, P., Yanagisawa, N., Yen, H., Shen, C., & Yeh, R. W. (2018) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2018.08.025 · Cited by 12
Reporting Standards and Critical Appraisal of Prediction Models
Wee, L., van Kuijk, S. M. J., Dankers, F. J. W. M., Traverso, A., Welch, M. L., & Dekker, A. L. A. J. (2018) · Springer eBooks · doi:10.1007/978-3-319-99713-1_10 · Cited by 14
Clinical Prediction Rules: Challenges, Barriers, and Promise
Wallace, E., & Johansen, M. E. (2018) · The Annals of Family Medicine · doi:10.1370/afm.2303 · Cited by 11
Watson, P., Clarkin, J., & Lomax, C. (2021) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2021.100681 · Cited by 14
Wang, Y., Zheng, R., Wu, Y., Liu, T., Hao, L., Liu, J., Shi, L., & Guo, Q. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105094 · Cited by 16
Xu, S., Coleman, R. L., Wan, Q., Gu, Y., Meng, G., Song, K., Shi, Z., Qian, X., Tuomilehto, J. O., Holman, R. R., et al. (2022) · Cardiovascular Diabetology · doi:10.1186/s12933-022-01622-5 · Cited by 12
“The process of defining the review question, study eligibility criteria, and data extraction was performed following the applicable guidance from a checklist for critical appraisal and data extraction for systemic review of prediction modelling studies (CHARMS) [7].”
MethodInfluentialFarhart, P. J., Beakley, D., Diwan, A. D., Duffield, R., Rodriguez, E., Chamoli, U., & Watsford, M. L. (2023) · BMC Sports Science Medicine and Rehabilitation · doi:10.1186/s13102-023-00732-1 · Cited by 15
“Data from included studies was extracted by two reviewers (PF and DB) using a modified template based on the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [36].”
MethodGuo, X., Xu, H., Zhang, J., Hao, B., & Yang, T. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e22226 · Cited by 10
Jiang, H., Chen, R., Li, Y., Hao, C., Song, G., Hua, Z., Li, J., Wang, Y., & Wei, W. (2023) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000002629 · Cited by 12
Seow‐En, I., Koh, Y. X., Zhao, Y. J., Ang, B. H., Tan, I. E., Chok, A. Y., Tan, E., & Au, M. K. H. (2023) · Annals of Hepato-Biliary-Pancreatic Surgery · doi:10.14701/ahbps.23-078 · Cited by 13
“The data collection was structured as per the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) [19].”
Sun, Y., Liu, Y., Zhu, Y., Luo, R., Luo, Y., Wang, S., & Feng, Z. (2024) · Current Medical Research and Opinion · doi:10.1080/03007995.2024.2307346 · Cited by 11
“The compilation of extracted variables adhered to the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies (CHARMS) 22 .”
MethodZhang, L., Jin, S., Wang, Y., Zhang, Z., Jia, H., Li, D., & Lü, Q. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110339 · Cited by 10
Risk prediction model for postoperative pneumonia in esophageal cancer patients: A systematic review
Jiang, Y., Li, Z., Jiang, W., Wei, T., & Chen, B. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1419633 · Cited by 11
Han, M., Zhao, Q., Zhao, J., Xue, X., & Wu, H. (2024) · World Journal of Surgery · doi:10.1002/wjs.12335 · Cited by 10
Sharif, A. F., Kasemy, Z. A. A., Alshalawi, K. S., & Sobh, Z. K. (2025) · Critical Reviews in Toxicology · doi:10.1080/10408444.2024.2445021 · Cited by 11
“Based on the obtained results, the model adjustment in case of modest performance is conducted (Moons et al. 2014).”
Yismaw, M. B., Tafere, C., Tefera, B. B., Demsie, D. G., Feyisa, K., Addisu, Z. D., Zeleke, T. K., Siraj, E. A., Worku, M. C., & Berihun, F. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105858 · Cited by 11
Hesami, Z., Sabzehali, F., Khorsand, B., Alipour, S., Sadeghi, A., Asri, N., Pazienza, V., & Houri, H. (2025) · iScience · doi:10.1016/j.isci.2025.112314 · Cited by 19
Fathy, W., Émériaud, G., & Farida Cheriet, A. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103126 · Cited by 17
Smeaton, T., McElwaine, P., Cullen, J., Santos-Martínez, M. J., Deasy, E., Widdowson, M., & Grimes, T. (2020) · Drugs & Therapy Perspectives · doi:10.1007/s40267-019-00700-1 · Cited by 10
Gao, Y., Chen, Y., Hu, M., Gan, T., Sun, X., Zhang, Z., He, W., & Wu, X. (2022) · Journal of Applied Gerontology · doi:10.1177/07334648221097084 · Cited by 11
“…standard data extraction form by referring to the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (Moons et al., 2014) and other well-conducted systematic reviews of prediction models (Falconer et al., 2018; Wynants et al., 2020).”
MethodSarrió‐Sanz, P., Martínez-Cayuelas, L., Lumbreras, B., Sánchez‐Caballero, L., Palazón‐Bru, A., Gil-Guillén, V. F., & Gómez‐Pérez, L. (2022) · European Journal of Clinical Investigation · doi:10.1111/eci.13822 · Cited by 10
Lyu, X., Zhang, D., Pan, H., Zhu, H., Chen, S., & Lü, L. (2022) · Endocrine · doi:10.1007/s12020-022-03081-0 · Cited by 10
Mkhwanazi, Z. A., Nyambuya, T. M., Mfusi, S. A., & Nkambule, B. B. (2022) · BMC Cancer · doi:10.1186/s12885-022-10223-0 · Cited by 11
“Data extraction Two reviewers (ZAM and BBN) independently extracted data items from the included studies defined in the critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies for Prognostic factors CHARMS-PF checklist [34].”
Methoddos Santos, M. A., Alexandre, V., Risk, E. N., de Oliveira, W. A., Peres, R. S., & de Oliveira‐Cardoso, É. A. (2023) · Psico-USF · doi:10.1590/1413-82712023280312 · Cited by 10
Zhou, B., Ai, W., & Cao, H. (2023) · International Journal of Orthopaedic and Trauma Nursing · doi:10.1016/j.ijotn.2023.101077 · Cited by 12
Factors associated with moderate neonatal hyperthyrotropinemia
Cortés‐Castell, E., Juste, M., Palazón‐Bru, A., Goicoechea, M., Gil-Guillén, V. F., & Baeza, M. R. (2019) · PLoS ONE · doi:10.1371/journal.pone.0220040 · Cited by 10
A systematic review of predictive models for recurrence and mortality in patients with tongue cancer
Palazón‐Bru, A., Mares‐García, E., López‐Bru, D., Mares‐Arambul, E., Gil-Guillén, V. F., & Carbonell‐Torregrosa, M. Á. (2019) · European Journal of Cancer Care · doi:10.1111/ecc.13157 · Cited by 10
Bonnett, L. J., Varese, F., Smith, C. T., Flores, A., & Yung, A. R. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0066-5 · Cited by 10
Lai, X., Liu, J., Zhang, T., Feng, L., Jiang, P., Kang, L., Liu, Q., & Gao, Y. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-039813 · Cited by 10
Prognosis of adults and children following a first unprovoked seizure
Neligan, A., Adan, G., Nevitt, S. J., Pullen, A., Sander, J. W. A. S., & Marson, A. G. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013847 · Cited by 10
Geethadevi, G. M., Quinn, T., George, J., Anstey, K. J., Bell, J. S., & Cross, A. J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014885 · Cited by 12
Wang, Y., Chen, W., Zhu, M., & Xian, L. (2021) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2021.692530 · Cited by 11
“Prognostic prediction models involve the prediction of the probability or risk of a particular outcome or event in the future, such as prediction of recurrence or death after diagnosis of cancer or mortality after surgery (Moons et al., 2014).”
BackgroundSilva, K. D., Enticott, J., Barton, C., Forbes, A. B., Saha, S. K., & Nikam, R. (2021) · Digital Health · doi:10.1177/20552076211047390 · Cited by 9
Ventricular function in congenital diaphragmatic hernia: a systematic review and meta-analysis
Prasad, R., Saha, B., & Kumar, A. (2021) · European Journal of Pediatrics · doi:10.1007/s00431-021-04303-9 · Cited by 10
Germini, F., Noronha, N., Philip, B. A., Olasupo, O. O., Pete, D., Navarro, T., Keepanasseril, A., Matino, D., de Wit, K., Parpia, S., et al. (2022) · Journal of Thrombosis and Haemostasis · doi:10.1111/jth.15723 · Cited by 10
Baseline platelet count in percutaneous coronary intervention: a dose–response meta-analysis
Galimzhanov, A. M., Sabitov, Y., Tenekecioğlu, E., Tun, H. N., Alasnag, M. A., & Mamas, M. A. (2022) · Heart · doi:10.1136/heartjnl-2022-320910 · Cited by 12
Tohidinezhad, F., Willems, Y., Berbée, M., Limbergen, E. V. V., Verhaegen, F., Dekker, A., & Traverso, A. (2022) · Journal of Contemporary Brachytherapy · doi:10.5114/jcb.2022.119427 · Cited by 7
“A data extraction form was developed to collect all relevant information based on recommendations in CHARMS checklist (see Supplementary Material S2 for the data extraction form) [17].”
MethodPredicting the outcome of plantar heel pain in adults: a systematic review of prognostic factors
Gülle, H., Morrissey, D., Tan, X. L., Cotchett, M., Miller, S., Jeffrey, A. B., & Prior, T. D. (2023) · Journal of Foot and Ankle Research · doi:10.1186/s13047-023-00626-y · Cited by 12
“Data extraction and data synthesis Data were extracted from studies on September 2020 according to the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [18].”
MethodInfluentialA one-year risk score to predict all-cause mortality in hypertensive inpatients
Martínez‐Díaz, A. M., Palazón‐Bru, A., la Rosa, D. M. F.-D., Ramírez-Prado, D., Navarro-Juan, M., Pérez-Ramírez, N., & Gil-Guillén, V. F. (2018) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2018.07.010 · Cited by 10
Safiya Parvin, A., & Saleena, B. (2023) · ICT Express · doi:10.1016/j.icte.2023.07.007 · Cited by 12
Stanley, C., Lennon, D. T., Moran, C. J., Vasireddy, A., & Rowan, F. E. (2023) · Injury · doi:10.1016/j.injury.2023.111017 · Cited by 9
May, S., Giordano, T. P., & Gottlieb, A. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad217 · Cited by 10
Suliman, A., Masud, M. M., Serhani, M. A., Abdullahi, A. S., & Oulhaj, A. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-082654 · Cited by 10
“22 The results of this review will be reported according to the updated PRISMA 2020 checklist.”
BackgroundWu, C.-C., Poly, T. N., Weng, Y.-C., Lin, M., & Islam, M. M. (2024) · Diagnostics · doi:10.3390/diagnostics14151594 · Cited by 11
“All data items were collected according to the Cochrane guidance for data collection and the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [24].”
MethodRisk prediction model for in-stent restenosis following PCI: a systematic review
Xiang, Q., Xiong, X., Liu, S., Zhang, M., Li, Y., Wang, H., Wu, R., & Chen, L. (2024) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2024.1445076 · Cited by 9
Chandramohan, D., Garapati, H. N., Nangia, U., Simhadri, P. K., Lapsiwala, B. J., Jena, N. K., & Singh, P. K. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1447057 · Cited by 16
Prognostic prediction models for adverse birth outcomes: A systematic review
Muche, A. A., Baruda, L. L., Pons‐Duran, C., Fite, R. O., Gelaye, K. A., Worku, A., Taddesse, L., Bekele, D., Tolera, G., Chan, G. J., et al. (2024) · Journal of Global Health · doi:10.7189/jogh.14.04214 · Cited by 12
“The Checklist for Critical Appraisal and Data Extraction For Systematic Reviews of Prediction Modelling Studies (CHARMS) was used [24].”
Hesseling, B., Prinsze, N., van Etten‐Jamaludin, F. S., Perry, S. I. B., Eygendaal, D. E., Mathijssen, N. M. C., & Snoeker, B. A. M. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02694-y · Cited by 9
“A medical librarian (FJ) designed and performed a sensitive search strategy guided by the PICOTS format [4, 5] (Table 1).”
Xia, Z., Cao, S., Li, T., Yuan, Q., & Zhong, Y. (2024) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s489819 · Cited by 9
Exploring the Role of Large Language Models in Melanoma: A Systematic Review
Zarfati, M., Nadkarni, G. N., Glicksberg, B. S., Harats, M., Greenberger, S., Klang, E., & Soffer, S. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13237480 · Cited by 14
Xu, Y., Zhao, H., Wu, S., Wang, J., Zhou, J., Ding, S., Li, W., Wu, W., Yang, Z., Xu, H., et al. (2025) · Advances in Wound Care · doi:10.1089/wound.2024.0238 · Cited by 9
“14 This led to the creation of Table 1 (overview of basic data of the included studies), which presents information such as country, study design, participants, location, data source, timing of IAPI measurement, and IAPI cases/total sample (%).”
Murugesu, S., Linton‐Reid, K., Braun, E., Barcroft, J. F., Cooper, N. C., Pikovsky, M., Novak, A., Parker, N., Stalder, C. M., Al‐Memar, M., et al. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07283-y · Cited by 11
“Selection of variables were determined by those with a p-value of below 0.157 as per the Akaike Information Criterion [24].”
InfluentialMalik, S., Naqvi, A., Tenorio, B. G., Farrukh, F., Tariq, R., & Adler, D. G. (2025) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000002172 · Cited by 11
Risk prediction models for diabetic retinopathy: a systematic review
Huang, H., Wu, Y., Ye, H., Li, J., Chen, L., & Huang, X. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1556049 · Cited by 12
Aznar-Tortonda, V., Palazón‐Bru, A., & Gil-Guillén, V. F. (2020) · PeerJ · doi:10.7717/peerj.10380 · Cited by 9
“This is important as, when applying mathematical models, no information that is available should be omitted, with quantitative variables to study their functional form and with qualitative variables not to create groups to dichotomize (Moons et al., 2014).”
Lee, H. J., Lee, A. Y. M., Yang, X., Law, Y. M., Huang, H. H., Lau, W. K., Lee, L. S., Ho, H. S., Cheng, C. W. S., Yuen, J. S. P., et al. (2021) · Urologic Oncology Seminars and Original Investigations · doi:10.1016/j.urolonc.2021.03.003 · Cited by 10
van Beek, D., Fredriksson, S., Haegele, S., Raffaelli, M., Riss, P., & Almquist, M. (2021) · BJS Open · doi:10.1093/bjsopen/zrab151 · Cited by 9
Day, E., Eldred‐Evans, D., Prevost, A. T., Ahmed, H. U., & Fiorentino, F. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01481-w · Cited by 10
Liu, J., Liu, M., Li, J., & Rong, J. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062648 · Cited by 9
Risk Prediction Models for Hospital Mortality in General Medical Patients: A Systematic Review
Hydoub, Y. M., Walker, A. P., Kirchoff, R. W., Alzu’bi, H., Chipi, P. Y., Gerberi, D., Burton, M. C., Murad, M. H., & Dugani, S. B. (2023) · American Journal of Medicine Open · doi:10.1016/j.ajmo.2023.100044 · Cited by 9
Diagnostic accuracy and predictive value in differentiating the severity of dengue infection
Low, G. K. K., Kagize, J., Faull, K. J., & Azahar, A. (2019) · Tropical Medicine & International Health · doi:10.1111/tmi.13294 · Cited by 8
“Moons KGM, de Groot JAH, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist.”
MethodInfluentialClephas, P. R. D., Hoeks, S. E., Trivella, M., Guay, C. S., Singh, P. M., Klimek, M., & Heesen, M. A. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051554 · Cited by 9
Chen, Y., Xi, M., Johnson, A. E. W., TOMLINSON, G. A., Campigotto, A. J., Chen, L., & Sung, L. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104706 · Cited by 10
Nonlinear Machine Learning in Warfarin Dose Prediction: Insights from Contemporary Modelling Studies
Zhang, F., Liu, Y., Ma, W., Zhao, S., Chen, J., & Gu, Z. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12050717 · Cited by 9
“These were adapted from the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [16] and the Prediction model Risk Of Bias Assessment Tool (PROBAST) [17].”
MethodJover-Esplá, A. G., Palazón‐Bru, A., la Rosa, D. M. F.-D., Ferrandiz, G. S., Méstre, M. S., de Juan-Herrero, J., & Gil-Guillén, V. F. (2018) · Oral Oncology · doi:10.1016/j.oraloncology.2018.03.021 · Cited by 9
“However, this model assessed age in a linear way, when according to the literature on prediction models the functional form of continuous predictors should be analyzed [25].”
InfluentialWhat does need to know about developing clinical prediction models?
Abbasi, M., & Naderi, M. (2018) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2018.12.006 · Cited by 9
Villalaín, C., Moon‐Grady, A. J., Herberg, U., Strainic, J. P., Cohen, J. L., Shah, A. M., Levi, D. S., Gómez‐Montes, E., Herraı̀z, I., & Galindo, A. (2023) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.26176 · Cited by 8
Kaizu, Y., Tamura, S., Saito, H., Hayashi, S., Iwamoto, H., & Miyata, K. (2023) · The Journals of Gerontology Series A · doi:10.1093/gerona/glad205 · Cited by 8
Yan, C., Yang, Q., Li, R., Yang, A., Fu, Y., Wang, J., Li, Y., Cheng, Q., & Hu, S. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e20099 · Cited by 13
“Each group independently extracted data from eligible articles based on a critical appraisal and data extraction checklist for systematic reviews of prediction modelling studies (CHARMS) checklist [14].”
MethodA systematic review of tools for predicting complications in patients with influenza-like illness
Marx, T., Khelifi, N., Xu, I., Ouellet, L., Poirier, A., Huard, B., Mallet, M., Bergeron, F., Boissinot, M., Bergeron, M. G., et al. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23227 · Cited by 9
“Two independent reviewers (N.K. and I.X.) extracted data from the selected articles using a pretested standardized form based on the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies, aka CHARMS [9].”
MethodJelicic, J., Larsen, T. S., Andjelic, B., Juul‐Jensen, K., & Bukumirić, Z. (2024) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2024.104293 · Cited by 8
Zeng, Q., Li, Z., Gui, S., Wu, J., Liu, C., Wang, T., Peng, D., & Zeng, G. (2024) · Journal of orthopaedic surgery · doi:10.1177/10225536241249591 · Cited by 11
“23”
Yang, T.-R., Ji, P., Deng, X., Feng, X., He, M., Wang, R., & Li, X. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1378462 · Cited by 12
Karl, A., Klimas, R., Katsimpoura, M., Sgodzai, M., Theile-Ochel, S., Poser, P. L., Gisevius, B., Faissner, S., Salmen, A., Nastos, I., et al. (2024) · Journal of Neurology · doi:10.1007/s00415-024-12581-0 · Cited by 9
“Due to the small number of cases, an analysis of relapse events and risk factors was here not conducted [14].”
BackgroundLazzereschi, L., Birks, J., & Colling, R. T. (2024) · Histopathology · doi:10.1111/his.15292 · Cited by 10
“11 For each study, the following were extracted: authors, year of publication, study design, country of origin, institution, eligibility, recruitment period, inclusion and exclusion criteria, study dates, and participant and tumour demographics/characteristics (if available).”
Askar, M., Tafavvoghi, M., Småbrekke, L., Bongo, L. A., & Svendsen, K. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309175 · Cited by 10
de Mattos, B. P., Mattjie, C., Ravazio, R., Barros, R. C., & Grassi‐Oliveira, R. (2024) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-024-01403-z · Cited by 9
“…and risk of bias was conducted independently by the three main reviewers at both the study and outcome levels, using two distinct instruments recommended by the EQUATOR network (EQUATOR Network, 2024): the MI-CLAIM checklist (Norgeot et al., 2020) and the CHARMS checklist (Moons et al., 2014).”
The concept of perioperative risk: a narrative review
Zabolotskikh, I. B. (2024) · Annals of critical care · doi:10.21320/1818-474x-2024-4-40-57 · Cited by 9
Travlos, C. K., Chirgwin-Dasgupta, A., Trinh, E., Sniderman, A. D., Alam, A., & Mavrakanas, T. A. (2024) · Clinical Kidney Journal · doi:10.1093/ckj/sfae347 · Cited by 9
“This review was registered on PROSPERO ( CRD42024543041) and was conducted according to the PRISMA statement and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies ( CHARMS; online Supplementary data) [14 , 15 ].”
Crawford, J., Kelder, J., & Wilson, G. (2024) · Quality & Quantity · doi:10.1007/s11135-024-02003-8 · Cited by 12
“Critical appraisal skills are crucial to scholars’ work in their disciplines (Heller et al. 2008 ); Moons et al. 2014 ).”
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.