Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessThe DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Abstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 601–700 of 1,005
Haller, M. C., Aschauer, C., Wallisch, C., Leffondré, K., van Smeden, M., Oberbauer, R. W., & Heinze, G. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.01.025 · Cited by 21
Tohidinezhad, F., Perri, D. D., Zegers, C. M. L., Dijkstra, J. B., Anten, M. H. M. E., Dekker, A. L. A. J., van Elmpt, W., Eekers, D. B. P., & Traverso, A. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.853472 · Cited by 18
“Data Extraction One reviewer extracted the data using a standard form designed according to the recommendation in the CHARMS statement (Moons et al., 2014).”
MethodRiley, S. M., Zhang, Q., Tse, W.-Y., Connor, A., & Wei, Y. (2022) · Transplant International · doi:10.3389/ti.2022.10397 · Cited by 20
Pandor, A., Daru, J., Hunt, B. J., Rooney, G., Hamilton, J., Clowes, M., Goodacre, S. W., Nelson‐Piercy, C., & Davis, S. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-065892 · Cited by 21
Taylor, B., Barboi, C., & Boustani, M. A. (2023) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18426 · Cited by 18
“Reviews of Prediction Modeling Studies (CHARMS) checklist.(19) The extracted data included (a) study location and setting, (b) sample size, (c) age range, (d) study design, (e) statistical methods, (f) criteria used for patient selection, (g) model development, validation, and performance, (h) data sources, (i) PDM ch…”
MethodSun, C., Hon, C. K. H., Way, K. A., Jimmieson, N. L., & Xia, B. (2023) · Journal of Management in Engineering · doi:10.1061/jmenea.meeng-5594 · Cited by 18
Zhang, X., Wang, X., Xu, L., Liu, J., Ren, P., & Wu, H. (2023) · European journal of medical research · doi:10.1186/s40001-023-01027-4 · Cited by 23
“Two reviewers (X–XZ and XW) constructed standardized forms based on the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [15].”
MethodHagag, A. M., Kaye, R., Hoang, V., Riedl, S., Anders, P., Stuart, B. L., Traber, G. L., Appenzeller‐Herzog, C., Schmidt-Erfurth, U. M., Bogunović, H., et al. (2023) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2023.10.010 · Cited by 22
“26,29 The risk of bias in each of the included studies was assessed using the quality in prognostic factor studies (QUIPS) tool.”
Yang, H., Lu, S., & Yang, L. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02449-9 · Cited by 25
“A standardized form was used to tabulate the included articles and retrieved related information in accordance with the CHARMS checklist [20].”
MethodBraun, C., Hanchard, N. C. A., Batterham, A. M., Handoll, H. H., & Betthäuser, A. (2015) · Physical Therapy · doi:10.2522/ptj.20150475 · Cited by 24
Donald, S., McIntyre, W. F., Dingwall, O., Hiebert, B. M., Ponnampalam, A., & Seifer, C. M. (2019) · Transfusion · doi:10.1111/trf.15442 · Cited by 23
PIWI family proteins as prognostic markers in cancer: a systematic review and meta-analysis
Mentis, A. A., Dardiotis, E., Romas, N. A., & Papavassiliou, A. G. (2019) · Cellular and Molecular Life Sciences · doi:10.1007/s00018-019-03403-y · Cited by 20
Stojanov, T., Modler, L., Müller, A. M., Aghlmandi, S., Appenzeller‐Herzog, C., Loucas, R., Loucas, M., & Audigé, L. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05030-4 · Cited by 20
“Two review authors (either LM, TS, ML, or RL) independently extracted data from selected studies following an adapted version of the CHARMS-PF [14].”
MethodInfluentialPredicting the need for supportive services after discharged from hospital: a systematic review
Kobewka, D. M., Mulpuru, S., Chassé, M., Thavorn, K., Lavallée, L. T., English, S. W., Neilipovitz, B., Neilipovitz, J., Forster, A. J., & McIsaac, D. I. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-4972-6 · Cited by 22
Ewington, L. J., Black, N., Leeson, C., Wattar, B. H. A., & Quenby, S. M. (2024) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17802 · Cited by 21
“21,25 The first limitation arose from the quality of the data obtained.”
Janoudi, G., Uzun, M., Fell, D. B., Ray, J. G., Foster, A. M., Giffen, R., Clifford, T., & Walker, M. C. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000515 · Cited by 20
Chen, K., Zhang, Y., Zhang, L., Zhang, W., & Chen, Y. (2024) · Journal of Thrombosis and Haemostasis · doi:10.1016/j.jtha.2024.11.001 · Cited by 20
Hillier, B., Scandrett, K., Coombe, A., Hernandez‐Boussard, T., Steyerberg, E. W., Takwoingi, Y., Veličković, V. M., & Dinnes, J. (2025) · PLoS Medicine · doi:10.1371/journal.pmed.1004518 · Cited by 20
Number of Publications on New Clinical Prediction Models: A Bibliometric Review
Arshi, B., Wynants, L., Rijnhart, E., Reeve, K. A., Cowley, L., & Smits, L. J. M. (2025) · JMIR Medical Informatics · doi:10.2196/62710 · Cited by 28
Prognostic rules for predicting cognitive syndromes following stroke: A systematic review
Drozdowska, B. A., McGill, K., McKay, M., Bartlam, R., Langhorne, P., & Quinn, T. J. (2021) · European Stroke Journal · doi:10.1177/2396987321997045 · Cited by 21
Dudareva, M., Hotchen, A., McNally, M., Hartmann‐Boyce, J., Scarborough, M., & Collins, P. G. S. (2021) · Journal of Bone and Joint Infection · doi:10.5194/jbji-6-257-2021 · Cited by 17
“Guidance is available for publication and assessing reporting quality (Hayden et al., 2013; Moons et al., 2014, 2015; Collins et al., 2015; Grant et al., 2018; Wolff et al., 2019).”
BackgroundMethodAparicio, R. E. A., Ball, J., Yiallourou, S. R., Venkataraman, P., Marwick, T. H., & Carrington, M. J. (2021) · Preventive Medicine · doi:10.1016/j.ypmed.2021.106819 · Cited by 21
Accuracy of Machine Learning Models to Predict In-hospital Cardiac Arrest: A Systematic Review
Moffat, L., & Xu, D. (2021) · Clinical Nurse Specialist · doi:10.1097/nur.0000000000000644 · Cited by 17
Helmrich, I. R., Mikolić, A., Kent, D. M., Lingsma, H. F., Wynants, L., Steyerberg, E. W., & Klaveren, D. V. (2022) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00122-0 · Cited by 21
“For the development studies, the data extraction form was based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist [8], and included the source of data, participants, outcome, sample size, predictors, missing data, model development, performance measu…”
MethodRisk Prediction Models of Natural Menopause Onset: A Systematic Review
Raeisi‐Dehkordi, H., Kummer, S., Raguindin, P. F., Dejanovic, G. M., Taneri, P. E., Cardona, I., Kastrati, L., Minder, B., Voortman, T., Marques-Vidal, P. M., et al. (2022) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgac461 · Cited by 19
Prediction models of vaginal birth after cesarean delivery: A systematic review
Deng, B., Li, Y., Chen, J., Guo, J., Tan, J., Yang, Y., & Liu, N. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104359 · Cited by 20
The risk assessment tool for intensive care unit readmission: A systematic review and meta-analysis
Long, J., Wang, M., Li, W., Cheng, J., Yuan, M., Zhong, M., Zhang, Z., & Zhang, C. (2023) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2022.103378 · Cited by 20
Chen, F., Kantagowit, P., Nopsopon, T., Chuklin, A., & Pongpirul, K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278729 · Cited by 29
“The Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) has been used to formulate review questions and data extraction [28].”
MethodVenn, M. L., Hooper, R., Pampiglione, T., Morton, D. G., Nepogodiev, D., & Knowles, C. H. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073085 · Cited by 23
Clinical Decision Support System Used in Spinal Disorders: Scoping Review
Toh, Z. A., Berg, B., Han, Q., Hey, H. W. D., Pikkarainen, M. A., Grotle, M., & He, H. G. (2024) · Journal of Medical Internet Research · doi:10.2196/53951 · Cited by 21
“…models using validated tools, such as the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies to extract prognostic model studies and the Prediction Model Study Risk of Bias Assessment Tool to assess the quality of these models [76,77].”
Stroke risk prediction models: A systematic review and meta-analysis
Asowata, O. J., Okekunle, A. P., Olaiya, M. T., Akinyemi, J. O., Owolabi, M. O., & Akpa, O. M. (2024) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2024.122997 · Cited by 24
Xin, X., Wu, S., Xu, H.-L., Ma, Y., Bao, N., Gao, M., Han, X., Gao, S., Zhang, S., Zhao, X., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102897 · Cited by 25
Lagerlund, M., Merlo, J., Vicente, R. P., & Zackrisson, S. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140244 · Cited by 20
“For this purpose a formal risk score equation that could be generalized to other contexts and time periods could be developed [31].”
BackgroundKeene, D. J., Schlüssel, M. M., Thompson, J., Hagan, D. A., Williams, M. A., Byrne, C., Goodacre, S. W., Cooke, M. W., Gwilym, S. E., Hormbrey, P., et al. (2018) · Health Technology Assessment · doi:10.3310/hta22640 · Cited by 20
Feng, Q., May, M., Ingle, S. M., Lu, M. S., Yang, Z., & Tang, J.-L. (2019) · BioMed Research International · doi:10.1155/2019/5634598 · Cited by 16
“e data extraction form was designed according to the CHARMS checklist [14], supplemented with other items obtained from methodological guidance studies and previous systematic reviews [15, 16].”
MethodPalazón‐Bru, A., Mares‐García, E., López‐Bru, D., Mares‐Arambul, E., la Rosa, D. M. F.-D., Carbonell‐Torregrosa, M. D. L. Á., & Gil-Guillén, V. F. (2019) · Head & Neck · doi:10.1002/hed.26025 · Cited by 19
Kyriakopoulou, A., Serghiou, S., Dimopoulou, D., Arista, I. A., Ψαλτοπούλου, Θ., Dinopoulos, A., & Papaevangelou, V. (2020) · Journal of Infection · doi:10.1016/j.jinf.2020.02.012 · Cited by 19
Mortensen, J., Kjeldsen, S. S., Honoré, H., & Pedersen, A. R. (2020) · Respiratory Care · doi:10.4187/respcare.07497 · Cited by 20
“Therefore, patients and next of kin may often approach clinicians with questions about the timeframe for decannulation.(1) Being able to present an evidence-based prognosis for decannulation is a valuable supplement for health care professionals working with tracheostomized patients, both in clinical decision-making a…”
Bryant, A., Hiu, S. K. W., Kunonga, P., Gajjar, K. B., Craig, D., Vale, L. D., Winter-Roach, B. A., Elattar, A., & Naik, R. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015048 · Cited by 19
Pratt, M., Forbes, J. D., Knox, N., Domselaar, G. V., & Bernstein, C. N. (2022) · Gastroenterology · doi:10.1053/j.gastro.2021.12.287 · Cited by 19
Zhang, H., Shao, J., Chen, D., Zou, P., Cui, N., Tang, L., Wang, D., & Ye, Z. (2020) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s283949 · Cited by 19
“Data Extraction and Critical Appraisal A standardized electronic form based on the CHARMS checklist was constructed to facilitate the data extraction process.(15) Information about objective, source of data, participants, outcome(s) to be predicted, candidate predictors, sample size, missing data, model development, m…”
MethodRibeiro, C. D. S., Uenishi, R. H., Domingues, A. D. S., Nakano, E. Y., Botelho, R. B. A., Raposo, A., & Zandonadi, R. P. (2024) · Nutrients · doi:10.3390/nu16152428 · Cited by 20
“To collect data from the primary studies that were included in Phase 2 selection within this study, the CHARMS checklist (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies) was used [22].”
MethodMortality prediction models for community-dwelling older adults: A systematic review
Exmann, C. J., Kooijmans, E. C. M., Joling, K. J., Burchell, G. L., Hoogendijk, E. O., & van Hout, H. P. J. (2024) · Ageing Research Reviews · doi:10.1016/j.arr.2024.102525 · Cited by 18
“A predefined extraction form (Supplementary material S2), was created based on the PRISMA recommendations and the CHARMS checklist (Moons et al., 2014).”
MethodWang, X., Zhang, P., Lu, H., Luo, D., Yang, D., Li, K., Qiu, S., Zeng, H., & Zeng, X. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-088253 · Cited by 26
Pham, C. T., Nguyen, H. T., Le, H. H. T. C., Tran, N. Q. L., Quoc, D. K., Bui, V., Phung, H. N., Phung, D. T., & Chu, C. M.-Y. (2025) · Tropical Medicine & International Health · doi:10.1111/tmi.70045 · Cited by 27
“The quality of studies was also assessed based on the relevant items from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [23].”
Ophey, A., Rehberg, S., Giehl, K., Eggers, C., Reker, P., van Eimeren, T., & Kalbe, E. (2021) · Neurorehabilitation and neural repair · doi:10.1177/1545968320981956 · Cited by 17
ESVM‐SWRF : Ensemble SVM ‐based sample weighted random forests for liver disease classification
Padmakala, S., Subasini, C. A., Karuppiah, S. P., & Sheeba, A. (2021) · International Journal for Numerical Methods in Biomedical Engineering · doi:10.1002/cnm.3525 · Cited by 17
“47 After the total data mining procedures, analyze each model performance depending upon ELTA.”
MethodLi, L., Li, X., Li, W., Ding, X., Zhang, Y., Chen, J., & Li, W. (2022) · BMC Cancer · doi:10.1186/s12885-022-09841-5 · Cited by 16
“Data extraction We constructed a form according to the CHARMS checklist [11], and standardized extraction of data for each article.”
BackgroundMethodInfluentialYu, J., Liu, X., Zhu, Z., Yang, Z., He, J., Zhang, L., & Lu, H. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1138234 · Cited by 18
“In the subgroup analysis, the Abbreviations CVD, cardiovascular disease; PLWH, people living with HIV; PROBAST, Prediction model Risk Of Bias ASsessment Tool; D:A:D, the Data collection on Adverse Effects of Anti-HIV Drugs Study risk equation; FRS, Framingham Risk Score; CHD, coronary heart disease; PCE, the pooled co…”
MethodWang, S., Huang, H., Hou, M., Xu, Q., Qian, W., Tang, Y., Li, X., Qian, G., Ma, J., Zheng, Y., et al. (2023) · Pediatric Research · doi:10.1038/s41390-023-02558-6 · Cited by 18
Neumann, C., Schneider, F., Hilfenhaus, G., Vecchione, L., Felsenstein, M., Ihlow, J., Geisel, D., Sander, S., Pratschke, J., Stintzing, S., et al. (2023) · Cancers · doi:10.3390/cancers15082367 · Cited by 21
“The data were extracted from the hospital’s patient records strictly following the CHARMS-PF (checklist for critical appraisal and data extraction for systemic reviews of prediction modelling studies) [9].”
MethodJalleh, R. J., Umapathysivam, M. M., Plummer, M. P., Deane, A. M., Jones, K. L., & Horowitz, M. (2023) · Reviews in Endocrine and Metabolic Disorders · doi:10.1007/s11154-023-09823-3 · Cited by 22
“This systematic review and meta-analysis of cohort and case-control studies was designed in accordance with the latest methodological guidance [ 15 , 16 ], and was reported in compliance with the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) guidelines [17].”
MethodTran, A., Fernando, S. M., Rochwerg, B. N., Hawes, H. G., Hameed, M. S., Dawe, P., Garraway, N., Evans, D. C., Kim, D. S., Biffl, W. L., et al. (2024) · Injury · doi:10.1016/j.injury.2024.111319 · Cited by 21
Musto, L., Smith, A. J., Pepper, C., Bujkiewicz, S., & Bown, M. J. (2024) · British journal of surgery · doi:10.1093/bjs/znae239 · Cited by 19
“In healthcare, these models often use information about a patient’s demographics and associated co-morbidities to calculate the person’s risk/prognosis 3 .”
BackgroundMethodAraújo, A. L. D., Sperandio, M., Calabrese, G., Faria, S. S., Cárdenas, D. A. C., Martins, M. D., Saldivia‐Siracusa, C., Giraldo‐Roldán, D., Pedroso, C. M., Vargas, P. A., et al. (2025) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2025.01.004 · Cited by 16
López‐Alcalde, J., Antequera, A., Stallings, E., Muriel, A., Fernandez-Felix, B. M., Solà, I., del Campo, R., Alonso, M. P., Vidal, F. G., Fidalgo, P., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035927 · Cited by 16
Risk Factors for Readmission After Knee Arthroplasty Based on Predictive Models: A Systematic Review
Mahajan, S. M., Nguyen, C., Bui, J., Kunde, E., Abbott, B. T., & Mahajan, A. S. (2020) · Arthroplasty Today · doi:10.1016/j.artd.2020.04.017 · Cited by 16
Cerebral Small Vessel Disease and Functional Outcome Prediction After Intracerebral Hemorrhage
Hostettler, I. C., Schwarz, G., Ambler, G., Wilson, D., Banerjee, G., Seiffge, D. J., Shakeshaft, C., Lunawat, S., Cohen, H., Yousry, T. A., et al. (2021) · Neurology · doi:10.1212/wnl.0000000000011746 · Cited by 17
“Calibration refers to the ability of a score to accurately predict the percentage of patients with the outcome of interest.(18) We assessed calibration using the Hosmer-Lemeshow (HL) goodness-of-fit statistic; a value of p < 0.”
MethodHalligan, S., Boone, D. J., Archer, L., Ahmad, T., Bloom, S. L., Rodriguez‐Justo, M., Taylor, S. A., & Mallett, S. (2021) · Health Technology Assessment · doi:10.3310/hta25450 · Cited by 19
Self-citationNaye, F., Décary, S., Houle, C., LeBlanc, A., Cook, C. E., Dugas, M., Skidmore, B., & Tousignant‐Laflamme, Y. (2023) · Physical Therapy · doi:10.1093/ptj/pzad021 · Cited by 20
Predicting 1-year disability and mortality of injured older adults
Jeffery, A. D., Dietrich, M. S., & Maxwell, C. A. (2018) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2018.01.003 · Cited by 16
Damen, J. A., & Hooft, L. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0049-6 · Cited by 17
Delirium risk in non-surgical patients: systematic review of predictive tools
Flaws, D., Byrne, G. J., & Mudge, A. (2019) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2019.05.013 · Cited by 20
Su, K., Ko, H., Chou, K., Hsiao, Y., Su, V. Y., Perng, D., & Kou, Y. R. (2019) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-019-0135-9 · Cited by 16
Hoffman, H., Sims, J. J., Inoa‐Acosta, V., Hoit, D. A., Arthur, A. S., Draytsel, D. Y., Kim, Y. S., & Goyal, N. (2024) · Journal of NeuroInterventional Surgery · doi:10.1136/jnis-2024-021759 · Cited by 15
Wang, Y., Zhang, Z., Zhang, Z., Chen, X., Liu, J., & Liu, M. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02771-w · Cited by 20
“…a predefined form based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [11], which included information about study characteristics, predictors, and model development and validation, as follows: (1) Study characteristics:…”
Fung, M. Y., Wong, Y. L., Cheung, K. M., Bao, K. H. K., & Sung, W. W. Y. (2025) · BMC Palliative Care · doi:10.1186/s12904-025-01696-4 · Cited by 19
Kim, Y., Kim, M., Kim, Y., & Choi, M. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105133 · Cited by 23
Eder, M., Kainz, A., Omić, H., Aigner, C., Copic, D., Kotton, C. N., Kamar, N., Wojciechowski, D., Hirsch, H. H., & Oberbauer, R. W. (2025) · Kidney International · doi:10.1016/j.kint.2025.06.014 · Cited by 15
Hallet, J. I., Pessaux, P., Beyfuss, K. A., Jayaraman, S., Serrano, P. E., Martel, G., Coburn, N. G., Piardi, T., & Mahar, A. L. (2018) · Surgical Endoscopy · doi:10.1007/s00464-018-6479-3 · Cited by 15
Haider, I. T., Simonian, N., Schnitzer, T. J., & Edwards, W. B. (2020) · Annals of Biomedical Engineering · doi:10.1007/s10439-020-02606-w · Cited by 15
Jia, H., Lu, B., Zhao, Z. Y., Yang, Y., Wang, F., Zhou, M., & Sun, X. (2022) · Eye and Vision · doi:10.1186/s40662-022-00287-1 · Cited by 20
“The sample size was mainly determined based on the number of covariates in the logistic model [26, 27].”
Zhang, H., Tian, W., & Sun, Y. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03087-3 · Cited by 16
“Clinical trials have strict inclusion and exclusion criteria, which lead to selection bias, so their model does not apply to the general population [26].”
BackgroundBai, A., Carty, C. P., & Dai, S. H. (2022) · Saudi Journal of Ophthalmology · doi:10.4103/sjopt.sjopt_219_21 · Cited by 19
“This systematic review was completed following the recommendations of the Preferred Reporting Items for Systematic reviews and Meta-Analyses [19] statement and the research question was formulated according to the CHARMS [20] checklist for systematic reviews of prediction models.”
MethodKim, M., Park, S., Kim, C., & Choi, M. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104411 · Cited by 19
O’Rielly, C. M., Harrison, T. G., Andruchow, J. E., Ronksley, P. E., Sajobi, T. T., Robertson, H. L., Lorenzetti, D., & McRae, A. D. (2023) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2022.12.028 · Cited by 20
The accuracy of statistical shape models in predicting bone shape: A systematic review
Patil, A., Kulkarni, K., Xie, S., Bull, A. M. J., & Jones, G. G. (2023) · International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.2503 · Cited by 18
“The same reviewers utilised a two‐part standardised form to extract data based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) tool.(13) The first form identified general study characteristics such as the publication year, study authors and anatomical area…”
MethodMarzano, L. A., Kazory, A., Husain‐Syed, F., & Ronco, C. (2023) · Clinical Endocrinology · doi:10.1111/cen.14918 · Cited by 16
“4 | Search strategy This systematic review was completed in accordance with the CHARMS and Preferred Reporting Items for Systematic Reviews and Meta‐analyses (PRISMA) reporting guidelines.(24,26) A systematic search of the MEDLINE, Ovid Embase and Web of Science databases was completed to identify all studies that des…”
Methodvan den Brink, D. A., de Vries, I. S. A., Datema, M., Perot, L., Sommers, R., Daams, J. G., Calis, J. C. J., Brals, D., & Voskuijl, W. P. (2022) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.113448 · Cited by 19
Hu, Y., Lu, H. S., Ren, L., Yang, M., Shen, M., Huang, J., Huang, Q., & Fu, L. (2023) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2023.104546 · Cited by 17
Sarraj, M., Hache, P., Foroutan, F., Oitment, C. T. A., Marion, T. E., Guha, D., & Pahuta, M. A. (2023) · The Spine Journal · doi:10.1016/j.spinee.2023.07.020 · Cited by 15
Merlo, J., Öberg, J., Khalaf, K., Vicente, R. P., & Leckie, G. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-063117 · Cited by 16
Zhao, Y., Wang, Y., & Xu, J. (2023) · Academic Radiology · doi:10.1016/j.acra.2023.10.050 · Cited by 17
Farimani, R. M., Karim, H., Atashi, A., Tohidinezhad, F., Bahaadini, K., Abu–Hanna, A., & Eslami, S. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-00965-4 · Cited by 29
“A strength of our study is that we systematically assessed the studies found by a framework published by Moons et al. (CHARMS) [14] extended with additional items from other studies that developed a prediction model [12, 56, 57] to assess the studies and models on reporting and methodological…”
BackgroundMethodInfluentialPrediction models for amputation after diabetic foot: systematic review and critical appraisal
Huang, J., Yang, J., Qi, H., Xu, M., Xu, X. S., & Zhu, Y. (2024) · Diabetology & Metabolic Syndrome · doi:10.1186/s13098-024-01360-6 · Cited by 20
“The Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist was used to guide the data extraction [25].”
MethodHilkens, N. A., Li, L., Rothwell, P. M., Algra, A., & Greving, J. P. (2018) · Stroke · doi:10.1161/strokeaha.117.019259 · Cited by 16
“As a consequence, absolute risks may be underestimated in a real-world setting and associations between predictors and outcome may differ.(14,15) External validation of a risk score in observational data could, therefore, provide valuable External Validation of Risk Scores for Major Bleeding in a Population-Based Coho…”
Mahajan, S. M., Mahajan, A. S., Nguyen, C., Bui, J., Abbott, B. T., & Osborne, T. F. (2020) · Journal of Orthopaedics · doi:10.1016/j.jor.2020.03.045 · Cited by 19
Choi, H. J., Kim, I., Lee, H., Oh, H. J., Ahn, M. K., Baek, W. I., Kim, Y. E., Oh, S. H., Lee, B. S., Namgoong, J., et al. (2021) · EBioMedicine · doi:10.1016/j.ebiom.2022.103890 · Cited by 17
Schmitz, R., Wilthagen, E. A., van Duijnhoven, F. H., van Oirsouw, M., Verschuur, E., Lynch, T., Punglia, R. S., Hwang, E., Wesseling, J., Schmidt, M. K., et al. (2022) · Cancers · doi:10.3390/cancers14133259 · Cited by 15
“The methodological quality of the development and validation of the DCIS prediction models was assessed using the widely endorsed CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies) checklist [37,38].”
MethodGeeson, C., Wei, L., & Franklin, B. D. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017509 · Cited by 18
“► have minimal measurement error (as this may dilute their prognostic value).(49)”
BackgroundMethodInfluentialChen, H., Han, J. H., Li, J., Xiong, J., Wang, D., Han, M., Shen, Y., & Lu, W. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2024.1522911 · Cited by 18
Return to work after major trauma: a systematic review
Neubert, A., Hempe, S., Bieler, D., Schulz, D., Jaekel, C., Bernhard, M., & Windolf, J. (2025) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-025-01351-0 · Cited by 16
“An adapted version of the data extraction sheet by the Cochrane Methods Prognosis Group was used guided by the data extractions items described in Moons (2014) [16, 20].”
Dong, Y., Zeng, H., Yang, L., Song, H., & Li, Q. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00093-7 · Cited by 19
“According to the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [10], two investigators (Y. D. and H. Z.) respectively collect data from the included studies.”
Kirchebner, J., Lau, S., & Sonnweber, M. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03117-1 · Cited by 17
“To avoid overfitting, it is advisable to include imputation, filtering, balancing and model building in a cross validation process kept separate from model-testing [61, 62].”
MethodAtmoko, W., Raharja, P. A. R., Birowo, P., Hamid, A. R. A. H., Taher, A., & Rasyid, N. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251235 · Cited by 16
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling studies (CHARMS) was used to facilitate the data extraction process [14].”
MethodGalimzhanov, A. M., Tenekecioğlu, E., Rustamova, F., Tun, H. N., & Mamas, M. A. (2022) · Angiology · doi:10.1177/00033197211070908 · Cited by 14
Diagnostic models predicting paediatric viral acute respiratory infections: a systematic review
Rankin, D. A., Peetluk, L. S., Deppen, S. A., Slaughter, J. C., Katz, S. E., Halasa, N., & Khankari, N. K. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-067878 · Cited by 15
Belsti, Y., Moran, L., Handiso, D. W., Versace, V. L., Goldstein, R. F., Mousa, A., Teede, H. J., & Enticott, J. (2023) · Current Diabetes Reports · doi:10.1007/s11892-023-01516-0 · Cited by 17
Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review update
Parekh, A., Satish, S., Dulhanty, L., Berzuini, C., & Patel, H. C. (2023) · Journal of NeuroInterventional Surgery · doi:10.1136/jnis-2023-021107 · Cited by 15
Daele, E. V., Vanommeslaeghe, H., Peirsman, L., Nieuwenhove, Y. V., Ceelen, W. P., & Pattyn, P. (2024) · Journal of Gastrointestinal Surgery · doi:10.1016/j.gassur.2024.02.003 · Cited by 16
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.