The DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessAbstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 301–400 of 1,004
Deimazar, G., & Sheikhtaheri, A. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105246 · Cited by 47
Tiruneh, S. A., Vu, T. T. T., Rolnik, D. L., Teede, H. J., & Enticott, J. (2024) · Current Hypertension Reports · doi:10.1007/s11906-024-01297-1 · Cited by 52
Refining Prognosis in Lung Cancer
Mahar, A. L., Compton, C. C., McShane, L. M., Halabi, S., Asamura, H., Rami–Porta, R., & Groome, P. A. (2015) · Journal of Thoracic Oncology · doi:10.1097/jto.0000000000000652 · Cited by 41
Hansen, P. R., Feineis, M., & Abdulla, J. (2019) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2019.02.018 · Cited by 45
Miller, H. A., Yin, X., Smith, S. A., Hu, X., Zhang, X., Yan, J., Miller, D. M., van Berkel, V. H., & Frieboes, H. B. (2021) · Lung Cancer · doi:10.1016/j.lungcan.2021.04.012 · Cited by 45
Multivariable regression: understanding one of medicine’s most fundamental statistical tools
Varady, N. H., Pareek, A., Eckhardt, C. M., Williams, R. J., Madjarova, S. J., Ollivier, M. P., Martin, R. K., KARLSSON, J. L., & Nwachukwu, B. U. (2022) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-022-07215-9 · Cited by 39
Shi, C., Dumville, J. C., & Cullum, N. A. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2018.08.005 · Cited by 42
“prognostic topics using the CHARMS checklist (Moons et al., 2014), and considered only longitudinal studies as Steyerberg et al.”
BackgroundMethodInfluentialValidation of childhood asthma predictive tools: A systematic review
Colicino, S., Munblit, D., Minelli, C., Čustović, A., & Cullinan, P. (2019) · Clinical & Experimental Allergy · doi:10.1111/cea.13336 · Cited by 38
“Data extraction was performed in duplicate by two reviewers (SC and DM) using a form developed with the CHARMS checklist.(32) The reviewers resolved any discrepancies through discussion with two other authors (CM and PC), until a consensus was reached.”
MethodBuettner, S., Galjart, B., Beumer, B. R., van Vugt, J. L. A., van Eijck, C. H. J., Polak, W. G., de Jonge, J., Homs, M. Y. V., van Driel, L. M. J. W., Pawlik, T. M., et al. (2020) · HPB · doi:10.1016/j.hpb.2020.07.007 · Cited by 40
Barboi, C., Tzavelis, A., & Muhammad, L. N. (2022) · JMIR Medical Informatics · doi:10.2196/35293 · Cited by 44
“Data extraction was performed by CB, reviewed by AT, and guided by the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) checklist [22] specifically designed for systematic reviews of prognostic prediction models.”
MethodGhith, N., Wagner, P., Frølich, A., & Merlo, J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0148187 · Cited by 39
Permpalung, N., Kittipibul, V., Mekraksakit, P., Rattanawong, P., Nematollahi, S., Zhang, S. X., & Steinke, S. M. (2020) · Transplantation · doi:10.1097/tp.0000000000003576 · Cited by 44
“Data Extraction and Quality Assessment The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies was used to guide comprehensive data extraction from included studies.(27) A standardized data collection form was used to obtain the following information from each st…”
MethodThe impact of machine learning in predicting risk of violence: A systematic review
Parmigiani, G., Barchielli, B., Casale, S., Mancini, T., & Ferracuti, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.1015914 · Cited by 46
Natural language processing for clinical notes in dentistry: A systematic review
Pethani, F., & Dunn, A. G. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104282 · Cited by 54
Geeson, C., Wei, L., & Franklin, B. D. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008335 · Cited by 44
Primary cam morphology; bump, burden or bog-standard? A concept analysis
Dijkstra, H. P., Ardern, C. L., Serner, A., Mosler, A., Weir, A., Roberts, N. W., Auliffe, S. M., Oke, J. L., Khan, K. M., Clarke, M. J., et al. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-103308 · Cited by 43
“…(domains adapted from the CHecklist for critical Appraisal and data extraction for Systematic Reviews of prediction Modelling Studies (CHARMS 18 ), quality and risk of bias assessment (combining the Quality in Prognosis Studies tool 19 20 and Risk of Bias tool for Non-randomised Studies 21…”
MethodRusso, D., Mariani, P., Caponio, V. C. A., Russo, L. L., Fiorillo, L., Zhurakivska, K., Muzio, L. L., Laino, L., & Troiano, G. (2021) · Cancers · doi:10.3390/cancers13225755 · Cited by 38
“) for patient diagnosed with OSCC undergoing surgery with or without adjuvant therapy; (ii) at least one between with Overall Survival (OS) and Disease-Free Survival (DFS) had to be reported as outcome; (iii) studies had to follow TRIPOD and CHARMS checklist [13,19]; (iv) the prognostic model had to be internally vali…”
MethodHendriks, P. M., van de Groep, L. D., Veen, K. M., van Thor, M. C. J., Gunput, S. T. G., Boersma, E. H., Boomars, K. A., Post, M. C., & van den Bosch, A. E. (2022) · American Heart Journal · doi:10.1016/j.ahj.2022.05.006 · Cited by 51
Araújo, A. L. D., Moraes, M. C., Pérez‐de‐Oliveira, M. E., da Silva, V. M., Saldivia‐Siracusa, C., Pedroso, C. M., Lopes, M. A., Vargas, P. A., Kochanny, S. E., Pearson, A. T., et al. (2023) · Oral Oncology · doi:10.1016/j.oraloncology.2023.106386 · Cited by 37
Reeve, K. A., Ön, B. I., Havla, J. B., Burns, J., Gosteli, M., Alabsawi, A., Alayash, Z., Götschi, A., Seibold, H., Mansmann, U., et al. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013606.pub2 · Cited by 59
Factors associated with handgrip strength across the life course: A systematic review
Ribeiro, L. W., Berndt, S., Mielke, G. I., Doust, J. A., & Mishra, G. D. (2024) · Journal of Cachexia Sarcopenia and Muscle · doi:10.1002/jcsm.13586 · Cited by 47
“16 The following data for study characteristics were extracted from each included study: study design, participants (e.g., eligibility and recruitment method; age and gender distribution; duration of follow-up), sample size, outcome (e.g., de fi nition and method for HGS measurements), factors…”
Tsegaye, B., Snell, K. I. E., Archer, L., Kirtley, S., Riley, R. D., Sperrin, M., CALSTER, B. V., Collins, G. S., & Dhiman, P. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111675 · Cited by 55
Prediction models for postoperative pulmonary complications: a systematic review and meta-analysis
Huang, Z., Han, Y., Zhuang, H., Jiang, J., Zhou, C., & Yu, H. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2025.04.025 · Cited by 48
Noma, H., Furukawa, T. A., Maruo, K., Imai, H., Shinohara, K., Tanaka, S., Ikeda, K., Yamawaki, S., & Cipriani, A. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.03.031 · Cited by 36
He, Y., Ong, A. Y., Li, X., Din, F. V., Brown, E. R., Timofeeva, M. N., Wang, Z., Farrington, S. M., Campbell, H., Dunlop, M. G., et al. (2019) · Surgical Oncology · doi:10.1016/j.suronc.2019.05.014 · Cited by 37
Systematic review of predictive models of microbial water quality at freshwater recreational beaches
Heasley, C., Sánchez, J., Tustin, J., & Young, I. (2021) · PLoS ONE · doi:10.1371/journal.pone.0256785 · Cited by 35
“CHARMS does not score studies based on bias, therefore, we did not determine an overall risk-of-bias score or rating for each study.”
MethodInfluentialChallenges in translational machine learning
Couckuyt, A., Seurinck, R., Emmaneel, A., Quintelier, K. L. A., Novák, D., Gassen, S. V., & Saeys, Y. (2022) · Human Genetics · doi:10.1007/s00439-022-02439-8 · Cited by 42
Guo, Z., Tang, J., Haiying, H., Yuping, Z., Yu, Q., & Jingfen, J. (2022) · BMC Public Health · doi:10.1186/s12889-022-13995-z · Cited by 45
“A standardized electronic form followed by the checklist for appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS checklist) [22] was constructed to facilitate the data extraction process (Table S2).”
MethodChen, Z., Hu, B., Liu, X., Becker, B., Eickhoff, S. B., Miao, K., Gu, X., Tang, Y., Dai, X., Li, C., et al. (2023) · BMC Medicine · doi:10.1186/s12916-023-02941-4 · Cited by 35
“Data extraction and coding To ensure transparency and reproducibility, we extracted and coded data by referring to guidelines, including PRISMA [36], CHARMS checklist [37] (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies), and TRIPOD [38] (Transparent Reportin…”
MethodMachine learning for the life-time risk prediction of Alzheimer’s disease: a systematic review
Rowe, T. W., Katzourou, I. K., Stevenson‐Hoare, J. O., Bracher‐Smith, M., Ivanov, D. K., & Escott‐Price, V. (2021) · Brain Communications · doi:10.1093/braincomms/fcab246 · Cited by 44
Rahimi, A. K., Canfell, O. J., Chan, W., Sly, B. P., Pole, J. D., Sullivan, C. M., & Shrapnel, S. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104758 · Cited by 41
Mishra, A. K., Chong, B., Arunachalam, S. P., Oberg, A. L., & Majumder, S. (2024) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000002870 · Cited by 42
Dean, J., Welsh, L. C., Wong, K. H., Aleksic, A., Dunne, E. M., Islam, M. R., Patel, A., Patel, P. H., Petkar, I., Phillips, I., et al. (2017) · Clinical Oncology · doi:10.1016/j.clon.2016.12.001 · Cited by 36
“Randomly splitting datasets into a development and independent validation sample, although often carried out, provides little additional information over internal validation and increases the risk of bias in small datasets [33,34].”
BackgroundGuidelines for conducting and publishing systematic reviews in Psychology
Carvalho, L. D. F., Pianowski, G., & dos Santos, M. A. (2019) · Estudos de Psicologia (Campinas) · doi:10.1590/1982-0275201936e180144 · Cited by 55
“Checklists for bias risk verification (Moons et al., 2014).”
Prediction models for prostate cancer to be used in the primary care setting: a systematic review
Aladwani, M., Lophatananon, A., Ollier, W., & Muir, K. R. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034661 · Cited by 40
Prognostic models for amyotrophic lateral sclerosis: a systematic review
Xu, L., He, B., Zhang, Y., Chen, L., Fan, D., Zhan, S., & Wang, S. (2021) · Journal of Neurology · doi:10.1007/s00415-021-10508-7 · Cited by 35
Lee, J. W., Philteos, J., Levin, M., Namavarian, A., Propst, E. J., & Wolter, N. E. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2021.1548 · Cited by 42
Amacher, S. A., Blatter, R., Briel, M., Appenzeller‐Herzog, C., Bohren, C., Becker, C., Beck, K., Groß, S., Tisljar, K., Sutter, R., et al. (2022) · Critical Care · doi:10.1186/s13054-022-04263-y · Cited by 36
“Data collection and reporting for this systematic review and meta-analysis followed the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), and the Meta-analysis of Observa…”
MethodInfluentialKom, I. Y., Dongelmans, D. A., de Keizer, N. F., Jager, K. J., Schut, M. C., Abu–Hanna, A., & Klopotowska, J. E. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad014 · Cited by 35
“7,8 The implementation of electronic health record (EHR) systems opened the opportunity to reuse data in these systems for ADE diagnosis and prognosis.”
BackgroundMethodArcher, R. J., Everson-Hock, E., Hamilton, J., Stevens, J., Essat, M., Poku, E. N., Clowes, M., Pandor, A., & Stevenson, M. (2018) · Health Technology Assessment · doi:10.3310/hta22660 · Cited by 35
Cardiovascular Disease Prognostic Models in Latin America and the Caribbean: A Systematic Review
Carrillo-Larco, R. M., Altez-Fernandez, C., Pacheco-Barrios, N. V., Claudia Bambs, S., Irazola, V. E., Miranda, J. J., Danaei, G., & Perel, P. (2019) · Global Heart · doi:10.1016/j.gheart.2019.03.001 · Cited by 42
“The research question and search strategy were de fi ned following international guidelines for systematic reviews of prognostic models [13,14].”
MethodDevelopment of frailty measurement tools using administrative health data: A systematic review
Nghiem, S. H., Sajeewani, D., Henderson, K., Afoakwah, C., Byrnes, J., Moyle, W., & Scuffham, P. (2020) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2020.104102 · Cited by 38
Modern Machine-Learning Predictive Models for Diagnosing Infectious Diseases
Alqaissi, E., Alotaibi, F. S., & Ramzan, M. S. (2022) · Computational and Mathematical Methods in Medicine · doi:10.1155/2022/6902321 · Cited by 48
Lijftogt, N., Luijnenburg, T. W. F., Vahl, A. C., Wilschut, E. D., Leijdekkers, V. J., Fiocco, M., Wouters, M. W. J. M., & Hamming, J. F. (2017) · British journal of surgery · doi:10.1002/bjs.10571 · Cited by 34
Chaboyer, W. P., Coyer, F. M., Harbeck, E., Thalib, L., Latimer, S. L., Wan, C. S., Tobiano, G., Griffin, B. R., Campbell, J., Walker, R., et al. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104189 · Cited by 33
Packet, B., Page, A. ‐., Cattani, L., Bosteels, J., Deprest, J. A., & Richter, J. (2023) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.26292 · Cited by 39
Prognostic factors for falls in Parkinson’s disease: a systematic review
Murueta‐Goyena, A., Muiño, O., & Gómez‐Esteban, J. C. (2023) · Acta Neurologica Belgica · doi:10.1007/s13760-023-02428-2 · Cited by 39
Ramis, M., Chang, A., & Nissen, L. M. (2017) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12268 · Cited by 39
“Specifically, the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) tool (Moons et al., 2014) was used to identify appropriate study designs and the Quality in Prognosis Studies (QUIPS) tool (Hayden, van der Windt, Cartwright, Côté, &…”
MethodPrediction models for the development of COPD: a systematic review
Matheson, M. C., Bowatte, G., Perret, J. L., Lowe, A. J., Senaratna, C. V., Hall, G. L., de Klerk, N. H., Keogh, L. A., McDonald, C. F., Waidyatillake, N. T., et al. (2018) · International Journal of COPD · doi:10.2147/copd.s155675 · Cited by 33
“Critical appraisal Risk of bias and applicability concerns in the selected studies were assessed using the CHARMS checklist (Table 4 and the details of CHARMS checklist [Supplementary material]).(15) Of the four model development studies, three used longterm follow-up or long-term data collected in electronic database…”
Barker, K. L., Room, J., Knight, R., Dutton, S., Toyé, F., Leal, J., Kent, S., Kenealy, N., Schussel, M. M., Collins, P. G. S., et al. (2020) · Health Technology Assessment · doi:10.3310/hta24650 · Cited by 34
Polygenic risk prediction models for colorectal cancer: a systematic review
Sassano, M., Mariani, M., Quaranta, G., Pastorino, R., & Boccia, S. (2022) · BMC Cancer · doi:10.1186/s12885-021-09143-2 · Cited by 33
“), for articles deemed relevant, using an in-depth piloted data extraction form and following an adapted version of the “CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies” (CHARMS) checklist [14].”
MethodTran, A., Fernando, S. M., Brochard, L. J., Fan, E., Inaba, K., Ferguson, N. D., Calfee, C. S., Burns, K. E. A., Brodie, D., McCREDIE, V. A., et al. (2021) · European Respiratory Journal · doi:10.1183/13993003.00857-2021 · Cited by 38
Foley, K., Riddell, Z. C., Coles, B., Roberts, S. A., & Willis, B. H. (2022) · British Journal of Radiology · doi:10.1259/bjr.20220152 · Cited by 32
Exum, A. C., Sutton, C. A., Bellitti, J. S., Yi, R., & Fazzino, T. L. (2023) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2023.209037 · Cited by 39
Halicka, M., Wilby, M. J., Duarte, R. V., & Brown, C. A. (2023) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-023-06446-2 · Cited by 47
“, achieving MCID) per predictor for dichotomous outcomes [37].”
BackgroundMethodMachine learning‐based prediction models for pressure injury: A systematic review and meta‐analysis
Pei, J., Guo, X., Tao, H., Wei, Y., Zhang, H., Ma, Y., & Han, L. (2023) · International Wound Journal · doi:10.1111/iwj.14280 · Cited by 41
“Based on the CHARMS checklist 17 and the QUADAS 18 risk of bias tool, two reviewers independently abstracted quantitative and descriptive information from the eligible articles.”
Abdulazeem, H. M., Whitelaw, S., Schauberger, G., & Klug, S. J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0274276 · Cited by 46
Systematic review of clinical prediction models to support the diagnosis of asthma in primary care
Daines, L., McLean, S., Buelo, A., Lewis, S. C., Sheikh, A., & Pinnock, H. (2019) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-019-0132-z · Cited by 38
“Randomised controlled trials, cohort studies (prospective or retrospective), cross-sectional, nested case–control and case–cohort studies were eligible for inclusion.(39) Exclusion criteria Studies were excluded if: 1.”
BackgroundMethodInfluentialPredicting treatment effects in unipolar depression: A meta-review
Gillett, G., Tomlinson, A., Efthimiou, O., & Cipriani, A. (2020) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2020.107557 · Cited by 36
Koçak, B., Kaya, O. K., Erdim, Ç., Kuş, E. A., & Kılıçkesmez, Ö. (2020) · American Journal of Roentgenology · doi:10.2214/ajr.20.22847 · Cited by 36
Ogero, M. O., Sarguta, R. J., Malla, L., Aluvaala, J., Agweyu, A., English, M., Onyango, N. O., & Akech, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035045 · Cited by 37
Prognostic Models Predicting Mortality in Preterm Infants: Systematic Review and Meta-analysis
van Beek, P. E., Andriessen, P., Onland, W., & Schuit, E. (2021) · PEDIATRICS · doi:10.1542/peds.2020-020461 · Cited by 34
Alhmoud, B., Bonnici, T., Patel, R. S., Melley, D. D., Williams, B., & Banerjee, A. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045849 · Cited by 37
Predictors of Mortality in Patients with Advanced Cancer—A Systematic Review and Meta-Analysis
Owusuaa, C., Dijkland, S. A., Nieboer, D., van der Heide, A., & van der Rijt, C. C. D. (2022) · Cancers · doi:10.3390/cancers14020328 · Cited by 32
“also independently extracted data to assess the risk-of-bias of individual studies using the Quality in Prognosis Studies with adapted items from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist [10,11].”
MethodHalabi, S., Yang, Q., Roy, A., Luo, B., Araujo, J. C., Logothetis, C., Sternberg, C. N., Armstrong, A. J., Carducci, M. A., Chi, K. N., et al. (2023) · Journal of Clinical Oncology · doi:10.1200/jco.22.02661 · Cited by 30
“In validating thismodel, we followed the recent guidelines for reporting of predictionmodel development and validation.(24-26) We used external validation, which is the most rigorous approach for estimating the prediction error, where the coefficients from our OS model were applied to the external data sets.”
MethodChuleerarux, N., Manothummetha, K., Moonla, C., Sanguankeo, A., Kates, O. S., Hirankarn, N., Phongkhun, K., Thanakitcharu, J., Leksuwankun, S., Meejun, T., et al. (2022) · Blood Advances · doi:10.1182/bloodadvances.2022008530 · Cited by 52
“27 DECEMBER 2022 • VOLUME 6, NUMBER 24 employed to guide comprehensive data extraction from included studies.(17) We extracted study design, country, center, study year, study period, type of SARS-CoV-2 vaccine, immunogenicity tests, study limitations, and other important comments.”
Fazel, S., Vazquez-Montes, M., Molero, Y., Runeson, B., D'Onofrio, B. M., Larsson, H., Lichtenstein, P., Walker, J., Sharpe, M., & Fanshawe, T. R. (2023) · BMJ Mental Health · doi:10.1136/bmjment-2023-300673 · Cited by 34
Graphics and statistics for cardiology: clinical prediction rules
Woodward, M., Tunstall‐Pedoe, H., & Peters, S. A. E. (2017) · Heart · doi:10.1136/heartjnl-2016-310210 · Cited by 36
Pinart, M., Kunath, F., Lieb, V., Tsaur, I., Wullich, B., & Schmidt, S. (2018) · World Journal of Urology · doi:10.1007/s00345-018-2574-2 · Cited by 36
Prognostic prediction models for diabetic retinopathy progression: a systematic review
Haider, S., Sadiq, S. N., Moore, D., Price, M. J., & Nirantharakumar, K. (2019) · Eye · doi:10.1038/s41433-018-0322-x · Cited by 35
Chowdhury, M., Yeasmin, F., Rabi, D. M., Ronksley, P. E., & Turin, T. C. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025579 · Cited by 33
“Study quality was assessed using the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies) Checklist.(17) The following items were evaluated for each study: Was inclusion/exclusion criteria for study participants specified?; Was there non-biased selection of study parti…”
Pandor, A., Horner, D. E., Davis, S., Goodacre, S. W., Stevens, J., Clowes, M., Hunt, B. J., Nokes, T., Keenan, J., & de Wit, K. (2019) · Health Technology Assessment · doi:10.3310/hta23630 · Cited by 38
Al‐Ameri, A. A. M., Wei, X., Wen, X., Wei, Q., Guo, H., Zheng, S., & Xu, X. (2020) · Transplant International · doi:10.1111/tri.13585 · Cited by 31
“This study was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [4], the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [5], and the Transparent Reporting of a Multivariable Predi…”
MethodGünther, M. P., Kirchebner, J., & Lau, S. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00415 · Cited by 34
Prediction Models for Osteoporotic Fractures Risk: A Systematic Review and Critical Appraisal
Sun, X., Chen, Y., Gao, Y., Zhang, Z., Qin, L., Song, J., Wang, H., & Wu, X. (2022) · Aging and Disease · doi:10.14336/ad.2021.1206 · Cited by 35
“Two reviewers (XS and YC) independently extracted the data with a pre-developed data extraction form, which was developed by following the guidance of the critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) checklist [19].”
MethodYang, Q., Fan, X., Cao, X. Q., Hao, W., Lu, J., Wei, J., Tian, J., Yin, M., & Ge, L. (2022) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.14475 · Cited by 35
Gao, Y., Li, S., Jin, Y., Zhou, L., Sun, S., Xu, X., Li, S., Yang, H., Zhang, Q., & Wang, Y. (2022) · JMIR Public Health and Surveillance · doi:10.2196/35750 · Cited by 30
“…was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) statement [22], the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies, and the prediction model performance guidelines [23,24].”
Protease activity as a prognostic factor for wound healing in venous leg ulcers
Westby, M. J., Dumville, J. C., Stubbs, N. C., Norman, G., Wong, J. K. F., Cullum, N. A., & Riley, R. D. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012841 · Cited by 35
Tangel, V. E., Krul, S. D., Stolker, R. J., Bramer, W. M., de Graaff, J. C., & Hoeks, S. E. (2022) · Anesthesiology · doi:10.1097/aln.0000000000004369 · Cited by 32
“) extracted information from the included studies in accordance with elements of the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist.(7) Data elements that were extracted were the following: authors, year of publication, shorthand name of models (if applicable),…”
MethodTaran, S., Angeloni, N. A., Pinto, R. L., Lee, S., McCREDIE, V. A., Schultz, M. J., Robba, C., Taccone, F. S., & Adhikari, N. K. J. (2022) · Critical Care Medicine · doi:10.1097/ccm.0000000000005769 · Cited by 36
“We developed the data extraction spreadsheet using the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies, adapted for prognostic factors (CHARMS-PF) (15, 20).”
MethodHryciw, B. N., Ghossein, J., Rochwerg, B. N., Meggison, H., Fernando, S. M., Kyeremanteng, K., Tran, A., & SEELY, A. J. E. (2024) · Critical Care Explorations · doi:10.1097/cce.0000000000001025 · Cited by 50
Li, G., Zhang, Y., Li, K., Liu, X., Lu, Y., Zhang, Z., Liu, Z., Wu, Y., Liu, F., Huang, H. D., et al. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2024.101666 · Cited by 43
Xu, Z., Fan, L., Xin, Y., Wang, Y., & Wang, Y. (2024) · Respiratory Research · doi:10.1186/s12931-024-03033-4 · Cited by 31
Predictive medicine in multiple sclerosis: A systematic review
Havas, J., Leray, E., Rollot, F., Casey, R., Michel, L., Lejeune, F., Wiertlewski, S., Laplaud, D. A., & Foucher, Y. (2020) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2020.101928 · Cited by 30
“We extracted the data using the CHeck list for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (11).”
MethodValidation of Risk Prediction Models to Detect Asymptomatic Carotid Stenosis
Poorthuis, M. H. F., Halliday, A. W., Massa, M. S., Sherliker, P., Clack, R. M., Morris, D., Clarke, R., de Borst, G. J., Bulbulia, R. A., & Lewington, S. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.119.014766 · Cited by 30
Ophey, A., Roheger, M., Folkerts, A., Skoetz, N., & Kalbe, E. (2020) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2020.575804 · Cited by 40
“Relevant data considering general study characteristics (e.g., participants’ demographics, WMT features) and prognostic factor and/or model analyses were independently extracted by two reviewers (AO andMR) according to the CHARMS checklist (Moons et al., 2014).”
MethodInfluentialSystematic review and meta-analysis of mortality risk prediction models in adult cardiac surgery
Sinha, S., Dimagli, A., Dixon, L. K., Gaudino, M. F. L., Caputo, M., Vohra, H. A., Angelini, G. D., & Benedetto, U. (2021) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivab151 · Cited by 34
“The data extraction items were based on the CHARMS checklist [14] and the risk of bias was assessed using the PROBAST tool [15, 16] (Prospero ID: CRD42020220983).”
MethodGeethadevi, G. M., Quinn, T. J., George, J., Anstey, K. J., Bell, J. S., Sarwar, M. R., & Cross, A. J. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014885.pub2 · Cited by 37
Gokhale, S., Taylor, D., Gill, J., Hu, Y., Zeps, N., Lequertier, V., Prado, L., Teede, H. J., & Enticott, J. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1192969 · Cited by 48
“Analytical pipeline The median number of predictors used was 12 (5-1001).”
BackgroundMethodDeng, W., Wang, D., Wan, Y.-D., Lai, S., Ding, Y., & Wang, X. (2024) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1287434 · Cited by 30
Bahreiny, S. S., Ahangarpour, A., & Aghaei, M. (2024) · Reproductive and Developmental Medicine · doi:10.1097/rd9.0000000000000089 · Cited by 33
“Data were extracted using a checklist that included crucial elements [26,27] .”
MethodZhang, M., Zheng, Y., Maidaiti, X., Liang, B., Wei, Y., & Sun, F. (2024) · Health Data Science · doi:10.34133/hds.0165 · Cited by 38
“The prognostic prediction model centers on the probability of a specific outcome occurring in the future, based on an individual’s current health status [5,6].”
BackgroundXu, W., Zhou, Y., Jiang, Q., Fang, Y., & Yang, Q. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1407348 · Cited by 34
Diabetes as a risk factor for tuberculosis disease
Franco, J. V. A., Bongaerts, B. W. C., Metzendorf, M., Risso, A., Guo, Y., Silva, L. P., Boeckmann, M., Schlesinger, S., Damen, J. A., Richter, B., et al. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016013.pub2 · Cited by 71
Viljoen, J. L., Goossens, I., Monjazeb, S., Cochrane, D. M., Vargen, L. M., Jonnson, M. R., Blanchard, A. J. E., Li, S. M. Y., & Jackson, J. R. (2024) · Behavioral Sciences & the Law · doi:10.1002/bsl.2698 · Cited by 39
Meta-analysis of prediction models for predicting lymph node metastasis in thyroid cancer
Liu, F., Han, F., Lu, L., Chen, Y., Guo, Z., & Yao, J. (2024) · World Journal of Surgical Oncology · doi:10.1186/s12957-024-03566-4 · Cited by 29
“The studies’ risk of bias was assessed using a comprehensive evaluation tool called PROBAST, which covers four domains: participants, predictive variables, results, and statistical analysis [36, 37].”
MethodImplementation and Updating of Clinical Prediction Models: A Systematic Review
Saelmans, A., Seinen, T. M., Pera, V., Markus, A. F., Fridgeirsson, E. A., John, L. H., Schiphof‐Godart, L., Rijnbeek, P. R., Reps, J., & Williams, R. D. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100228 · Cited by 53
Gomes, C., Geels, J., Debray, T. P. A., Malekzadeh, A., Asselbergs, F. W., & Linschoten, M. (2025) · BMJ · doi:10.1136/bmj-2025-084062 · Cited by 40
“We used the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to extract data from selected studies, 22 supplemented by items extracted during a previous systematic review on cardiovascular risk prediction models.”
Evidence synthesis in prognosis research
Debray, T. P. A., de Jong, V. M. T., Moons, K. G. M., & Riley, R. D. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0059-4 · Cited by 33
“Recently, much guidance has been written on how to define a review question [13], define the PICOTS (Patients, Index prognostic factor or model, Comparator factor or model, Outcomes, Timing of prognostication, Setting of prognostication), define the search strategy, design the data extraction list [14], and do risk of…”
BackgroundSelf-citationOphey, A., Giehl, K., Rehberg, S., Eggers, C., Reker, P., van Eimeren, T., & Kalbe, E. (2020) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2020.02.002 · Cited by 33
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.