The DOI matches its Crossref record; 156 of the 157 references it cites were found in a registry or an index.
Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist
Open accessAbstract
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was employed to define the objectives of this systematic review and meta-analysis (29).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“We employed the CHARMS checklist and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines to develop standardized forms for data extraction (29, 32).”
From Machine learning-based risk predictive models for diabetic kidney disease in type 2 diabetes mellitus patients: a systematic review and meta-analysis (2025) · cites this paper as [29]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citation2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 1×
“Numerous recommendations for reporting missing data in medical research have been proposed [103,104,114,115].”
BackgroundA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“The number of events-pervariable (EPV) is commonly used to calculate the sample size, where attaining a sample size with an EPV of ten or more is frequently recommended to avoid overfitting [97–101].”
Method
Cited by
Items 401–500 of 1,004
Jackson, S. M., Perry, L. A., Caruana, C. B., Ramson, D. M., Campbell, R. J., Liu, Z., Nguyen, J. K., Douglas, N. W. R., Kok, J., & Penny‐Dimri, J. C. (2020) · Vascular and Endovascular Surgery · doi:10.1177/1538574420951315 · Cited by 29
Townsend, R., Manji, A., Allotey, J., Heazell, A. E. P., Jorgensen, L., Magee, L. A., Mol, B. W., Snell, K. I. E., Riley, R. D., Sandall, J., et al. (2020) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16487 · Cited by 32
“(4,5) (6) In 2016, a systematic review reported on over 100 prognostic models developed for use in obstetrics.”
BackgroundMethodAsgari, S., Khalili, D., Hosseinpanah, F., & Hadaegh, F. (2021) · Intern J Endocrinol Metab · doi:10.5812/ijem.109206 · Cited by 34
Nadarajah, R., Alsaeed, E. S., Hurdus, B., Aktaa, S., Hogg, D., Bates, M. G. D., Cowan, C., Wu, J., & Gale, C. P. (2021) · Heart · doi:10.1136/heartjnl-2021-320036 · Cited by 31
Predicting inhospital admission at the emergency department: a systematic review
Brink, A., Alsma, J., van Attekum, L. A., Bramer, W. M., Zietse, R., Lingsma, H. F., & Schuit, S. C. E. (2021) · Emergency Medicine Journal · doi:10.1136/emermed-2020-210902 · Cited by 36
“15 This checklist is applicable for studies deriving a prediction model.”
Jia, K. M., Eilerts-Spinelli, H., Edun, O., Lam, K., Howes, A., Thomas, M. L., & Eaton, J. W. (2022) · Journal of the International AIDS Society · doi:10.1002/jia2.25861 · Cited by 29
Duong, H. P., Garcia, A., Hilfiker, R., Léger, B., & Lüthi, F. (2022) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2021.791351 · Cited by 35
Predicting postpartum haemorrhage: A systematic review of prognostic models
Carr, B. L., Jahangirifar, M., Nicholson, A. E., Li, W., Mol, B. W., & Licqurish, S. (2022) · Australian and New Zealand Journal of Obstetrics and Gynaecology · doi:10.1111/ajo.13599 · Cited by 35
Prediction Models for Heart Failure in the Community: A Systematic Review and Meta-Analysis
Nadarajah, R., Younsi, T., Romer, E., Raveendra, K., Nakao, Y. M., Nakao, K., Shuweihdi, F., Hogg, D., Arbel, R., Zahger, D., et al. (2023) · European Journal of Heart Failure · doi:10.1002/ejhf.2970 · Cited by 31
“5,6 However, there is substantial heterogeneity of risk within HF stage A/B, and specific groups are not included in current HF staging schema but are nonetheless at increased risk for symptomatic HF. 7 To match the intensity of prevention efforts with the absolute risk of the individual, an…”
Backgroundvan Boekel, A. M., van der Meijden, S. L., Arbous, M. S., Nelissen, R. G. H. H., Veldkamp, K. E., Nieswaag, E. B., Jochems, K. F. T., Holtz, J., Veenstra, A. V. I., Reijman, J., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0312968 · Cited by 36
The development of CHAMP: a checklist for the appraisal of moderators and predictors
van Hoorn, R., Tummers, M., Booth, A., Gerhardus, A., Rehfuess, E. A., Hind, D., Bossuyt, P. M., Welch, V. A., Debray, T. P. A., Underwood, M. R., et al. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0451-0 · Cited by 31
Aznar-Tortonda, V., Palazón‐Bru, A., la Rosa, D. M. F.-D., Espínola-Morel, V., Pérez-Pérez, B. F., León-Ruiz, A. B., & Gil-Guillén, V. F. (2019) · British Journal of General Practice · doi:10.3399/bjgp19x706577 · Cited by 30
“7 The study presented here was based on the statistical methodology recommended by leading experts in the development of predictive models in health sciences, 25 and the power of the performance of the tool was good.”
MethodBrand, E. C., Elias, S. G., Minderhoud, I. M., van der Veen, J. J., Baert, F. J., Laharie, D., Bossuyt, P. J., Bouhnik, Y., Buisson, A., Lambrecht, G. L. Y., et al. (2019) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2019.12.014 · Cited by 30
Mortality Risk Models for Persons with Dementia: A Systematic Review
Smith, E. E., & Ismail, Z. (2021) · Journal of Alzheimer s Disease · doi:10.3233/jad-201364 · Cited by 30
“elling Studies (CHARMS) [4] was used to guide”
MethodUnderstanding the Decision-Making Process Between Presenteeism and Absenteeism
Lohaus, D., & Habermann, W. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.716925 · Cited by 31
Aldin, A., Umlauff, L., Estcourt, L. J., Collins, P. G. S., Moons, K. G. M., Engert, A., Kobe, C., von Tresckow, B., Haque, M., Foroutan, F., et al. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012643.pub2 · Cited by 33
Self-citationSantos, R. J., Coleman, H. G., Cairnduff, V., & Kunzmann, A. T. (2022) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000002022 · Cited by 33
“Quality assessment The CHARMS checklist was used to guide the extraction of relevant data from the included studies and guide the risk-of-bias assessment (22).”
MethodMachine Learning in Colorectal Cancer Risk Prediction from Routinely Collected Data: A Review
Burnett, B., Zhou, S., Brophy, S. T., Davies, P., Ellis, P., Kennedy, J., Bandyopadhyay, A., Parker, M., & Lyons, R. A. (2023) · Diagnostics · doi:10.3390/diagnostics13020301 · Cited by 33
“Data items were extracted from the fourteen selected articles using the CHARMS checklist [37] for assessment of applicability and the risk of bias by a single author (BB).”
MethodPrognosis of adults and children following a first unprovoked seizure
Neligan, A., Adan, G., Nevitt, S. J., Pullen, A., Sander, J. W. A. S., Bonnett, L. J., & Marson, A. G. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013847.pub2 · Cited by 40
Clinical prediction models using machine learning in oncology: challenges and recommendations
Collins, G. S., Chester-Jones, M., Gerry, S., Ma, J., Matos, J., Sehjal, J., Tsegaye, B., & Dhiman, P. (2025) · BMJ Oncology · doi:10.1136/bmjonc-2025-000914 · Cited by 27
Self-citationZu, W., Huang, X., Xu, T., Du, L., Wang, Y., Wang, L., & Nie, W. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287308 · Cited by 38
“Table 1 shows the aim of this review according to the CHARMS guidelines.”
BackgroundMethodInfluentialBahreiny, S. S., Bastani, M., Dabbagh, M. R., Ghorbani, H., Aghaei, M., Zahedian, M., & Fard, R. M. (2024) · Middle East Fertility Society Journal · doi:10.1186/s43043-023-00162-6 · Cited by 34
Didier, A. J., Nigro, A., Noori, Z., Omballi, M., Pappada, S. M., & Hamouda, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1365777 · Cited by 32
Zaka, A., Mutahar, D., Gorcilov, J., Gupta, A. K., Kovoor, J. G., Stretton, B., Mridha, N., Sivagangabalan, G., Thiagalingam, A., Chow, C. K., et al. (2024) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae074 · Cited by 32
Prognosis research ideally should measure time-varying predictors at their intended moment of use
Whittle, R., Royle, K., Jordan, K. P., Riley, R. D., Mallen, C. D., & Peat, G. (2017) · Diagnostic and Prognostic Research · doi:10.1186/s41512-016-0006-6 · Cited by 27
“Predictions from these models should have optimal performance at the time that they are practically implemented—the “intended moment of using the model” [5].”
BackgroundMethodCarrillo-Larco, R. M., Aparcana-Granda, D. J., Mejia, J. R., Barengo, N. C., & Bernabé‐Ortiz, A. (2019) · Diabetic Medicine · doi:10.1111/dme.14114 · Cited by 29
“These processes led to the selection of reports for inclusion in the qualitative summary, from which key information was extracted onto a data collection form developed by the authors based on international guidelines for systematic reviews of prognosis models (CHARMS checklist) [18,19].”
InfluentialSouwer, E. T. D., Bastiaannet, E., Steyerberg, E. W., Dekker, J. T., van den Bos, F., & Portielje, J. E. A. (2020) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2020.04.006 · Cited by 27
Holden, S., Kasza, J., Winters, M., van Middelkoop, M., Adolescent Knee Health Group, & Rathleff, M. S. (2021) · Pain · doi:10.1097/j.pain.0000000000002184 · Cited by 29
“critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS).(19) The data that were extracted included publication details, eligibility criteria, population, outcomes, potential prognostic variables, interventions (if applicable), sample size, response rates, and missing data.”
MethodBeulens, J. W. J., Yauw, J. S., Elders, P. J. M., Feenstra, T., Herings, R. M. C., Slieker, R. C., Moons, K. G. M., Nijpels, G., & van der Heijden, A. A. W. A. (2021) · Diabetologia · doi:10.1007/s00125-021-05448-w · Cited by 35
“CHARMS Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies DCS Diabetes Care System PODUS Prediction of diabetic foot ulcerations”
InfluentialSelf-citationFernandez-Felix, B. M., Barca, L. V., García‐Esquinas, E., Correa-Pérez, A., Fernández‐Hidalgo, N., Muriel, A., López‐Alcalde, J., Díaz, N. Á., Pijoán, J. I., Ribera, A., et al. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2021.05.051 · Cited by 29
Structured reporting to improve transparency of analyses in prognostic marker studies
Sauerbrei, W., Haeussler, T., Balmford, J., & Huebner, M. (2022) · BMC Medicine · doi:10.1186/s12916-022-02304-5 · Cited by 31
“For a systematic review of prediction models, the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) was developed [70].”
MethodPrognostic Models in Severe Traumatic Brain Injury: A Systematic Review and Meta-analysis
Vieira, R. D. C. A., Silveira, J. C. P., Paiva, W. S., de Oliveira, D. V., de Souza, C. P. E., Santos, E. S., & de Sousa, R. M. C. (2022) · Neurocritical Care · doi:10.1007/s12028-022-01547-7 · Cited by 34
Wheatley, J., Liu, Z., Loth, J., Plummer, M. P., Penny‐Dimri, J. C., Segal, R., Smith, J. A., & Perry, L. A. (2022) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14170 · Cited by 31
Parsaei, M., Taghavizanjani, F., Cattarinussi, G., Moghaddam, H. S., Camillo, F. D., Akhondzadeh, S., Sambataro, F., Brambilla, P., & Delvecchio, G. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.08.034 · Cited by 32
Avula, V. R., Wu, K. C., & Carrick, R. T. (2023) · JACC Advances · doi:10.1016/j.jacadv.2023.100686 · Cited by 39
Tran, A., Matar, M. M., Steyerberg, E. W., Lampron, J., Taljaard, M., & Vaillancourt, C. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0480-0 · Cited by 27
“If any suitable multivariable prognostic models for our composite outcome of interest are identified, the quality of each identified multivariable model will be described based on items identified in the CHARMS checklist [17], including the appropriateness of the outcomes and predictors, techniques used for variable s…”
BackgroundMethodInfluentialOthman, R., Jayakaran, P., Swain, N., Dassanayake, S., Tumilty, S., & Mani, R. (2020) · Pain Practice · doi:10.1111/papr.12943 · Cited by 32
Prediction models for breast cancer-related lymphedema: a systematic review and critical appraisal
Qiu, L., Yang, T., Jin, Y., & Die, Y. M. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02084-2 · Cited by 30
“This systematic review was conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [25, 26] and registered on the International Prospective Register of Systematic Reviews PROSPERO (CRD42021258832), and the difference with registration is we added the Embase d…”
MethodLi, J., Zhu, M., & Li, Y. (2024) · Renal Failure · doi:10.1080/0886022x.2024.2380748 · Cited by 35
“…developed a list of information to be extracted from the included literature based on the CHARMS checklist for data extraction in systematic reviews of prediction modeling studies [29] and Appendix 11.1 of the JBI Manual for Evidence Synthesis [28] (Supplementary Appendix IV: Data extraction list).”
MethodInfluentialMalhotra, A. K., Shakil, H., Smith, C. W., Huang, Y. Q., Kwong, J. C. C., Thorpe, K. E., Witiw, C. D., Kulkarni, A. V., Wilson, J. R., & Nathens, A. B. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01714-y · Cited by 34
Kausch, S. L., Moorman, J. R., Lake, D. E., & Keim‐Malpass, J. (2021) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2021.103035 · Cited by 31
Risk Factors for Mobility Decline in Community-Dwelling Older Adults: A Systematic Literature Review
Nicolson, P. J. A., Sanchez-Santos, M. T., Bruce, J., Kirtley, S., Ward, L., Williamson, E. M., & Lamb, S. E. (2021) · Journal of Aging and Physical Activity · doi:10.1123/japa.2020-0482 · Cited by 26
“The review is reported according to the Preferred Reporting Items for Systematic Review and Meta-Analyses guidelines (Moher, Liberati, Tetzlaff, & Altman, 2009), and the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (Moons et al., 2014).”
MethodTran, A., Fernando, S. M., Rochwerg, B. N., Inaba, K., Bertens, K. A., Engels, P. T., Balaa, F. K., Kubelik, D., Matar, M. M., Lenet, T., et al. (2021) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000003502 · Cited by 28
“We performed extraction using a modified Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies checklist for prognostic factors.(16) Using the Quality in Prognosis Studies tool, two reviewers (A.”
MethodPrediction models in first-episode psychosis: systematic review and critical appraisal
Lee, R., Leighton, S., Thomas, L., Gkoutos, G. V., Wood, S. J., Fenton, S., Deligianni, F., Cavanagh, J., & Mallikarjun, P. K. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.219 · Cited by 32
Razaghizad, A., Oulousian, E., Randhawa, V. K., Ferreira, J. P., Brophy, J. M., Greene, S. J., Guida, J., Felker, G. M., Fudim, M., Tsoukas, M. A., et al. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.121.024833 · Cited by 30
Stroke mortality prediction using machine learning: systematic review
Schwartz, L., Anteby, R., Klang, E., & Soffer, S. (2022) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2022.120529 · Cited by 32
Diagnosis of Engaging Bipolar Bone Defects in the Shoulder Using 2-Dimensional Computed Tomography
Burns, D. M., Chahal, J., Shahrokhi, S., Henry, P. J., Wasserstein, D. N., Whyne, C. M., Theodoropoulos, J. S., Ogilvie‐Harris, D. J., & Dwyer, T. (2016) · The American Journal of Sports Medicine · doi:10.1177/0363546516655797 · Cited by 26
“A sample size of 45 defect combinations was considered adequate to achieve a minimum of 10 outcome events (positive and negative) per predictor variable in the logistic model with contingency.(12) Classification accuracy as well as regression coefficients with standard errors and significance are reported.”
MethodWhittle, R., Peat, G., Belcher, J., Collins, P. G. S., & Riley, R. D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.05.008 · Cited by 26
“Therefore, the predictions from these 64 models should have optimal performance when being practically implemented at the “intended 65 moment of using the model” [5].”
BackgroundMethodSong, Z., Yang, Z., Hou, M., & Shi, X. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.951881 · Cited by 34
Hampton, J. S., Kenny, R. P. W., Rees, C. J., Hamilton, W. T., Eastaugh, C., Richmond, C., & Sharp, L. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102204 · Cited by 30
Dormosh, N., van de Loo, B. P. A., Heymans, M. W., Schut, M. C., Medlock, S. K., van Schoor, N. M., van der Velde, N., & Abu–Hanna, A. (2024) · Age and Ageing · doi:10.1093/ageing/afae131 · Cited by 33
“Data extraction was performed in accordance with the criteria of the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) checklist [17] (see Supplementary Appendix 3).”
MethodMao, A., Su, J., Ren, M., Chen, S., & Zhang, H. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05688-0 · Cited by 34
van Es, N., Wells, P. S., & Carrier, M. (2017) · Thrombosis Research · doi:10.1016/j.thromres.2017.02.016 · Cited by 31
“The CHARMS checklist was used for the critical appraisal of the clinical prediction scores, rules, and models derived in a VTE population (Supplementary Table 1) [36].”
MethodPrognostic models for predicting the severity and mortality in people with acute pancreatitis
Gurusamy, K. S., Debray, T. P. A., & Rompianesi, G. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013026 · Cited by 31
Gregório, T., Pipa, S., Cavaleiro, P., Atanásio, G., Albuquerque, I., Castro‐Chaves, P., & Azevedo, L. F. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003744 · Cited by 25
Othman, R., Dassanayake, S., Jayakaran, P., Tumilty, S., Swain, N., & Mani, R. (2019) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000000775 · Cited by 29
Prediction modelling studies for medical usage rates in mass gatherings: A systematic review
Remoortel, H. V., Scheers, H., Buck, E. D. J. D., Haenen, W., & Vandekerckhove, P. G. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234977 · Cited by 28
“big data analysis), external model validation studies or studies that applied observations from few mass gatherings to another prediction model, were included according to the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [9].”
MethodDelirium risk prediction models for intensive care unit patients: A systematic review
Chen, J., Yu, J., & Zhang, A. (2020) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2020.102880 · Cited by 33
Prognostic models for chronic kidney disease: a systematic review and external validation
van Rijn, M. H. C., van de Luijtgaarden, M. W. M., van Zuilen, A. D., Blankestijn, P. J., Wetzels, J. F. M., Debray, T. P. A., & van den Brand, J. A. J. G. (2020) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfaa155 · Cited by 31
Frondelius, T., Atkova, I., Miettunen, J., Rello, J., & Jansson, M. M. (2021) · Journal of Critical Care · doi:10.1016/j.jcrc.2021.10.001 · Cited by 31
Prediction Models for Bronchopulmonary Dysplasia in Preterm Infants: A Systematic Review
Peng, H., Zhan, Y.-L., Chen, Y., Jin, Z.-C., Liu, F., Wang, B., & Yu, Z. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.856159 · Cited by 29
“The reviewers used a standardized data extraction form based on the CHARMS checklist (10).”
MethodA Systematic Review and Meta-Analysis of Prognostic Nomograms After UTUC Surgery
Pallauf, M., König, F., D’Andrea, D., Laukhtina, E., Mostafaei, H., Motlagh, R. S., Quhal, F., Aydh, A., Yanagisawa, T., Kawada, T., et al. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.907975 · Cited by 25
Risk prediction models for breast cancer: a systematic review
Zheng, Y., Li, J., Wu, Z. L., Li, H., Cao, M., Li, N., & He, J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055398 · Cited by 30
Reynolds, A., Vranic-Peters, M. J., Lai, A., Grayden, D. B., Cook, M. J., & Peterson, A. D. H. (2023) · Epilepsia · doi:10.1111/epi.17548 · Cited by 27
AI/ML advances in non-small cell lung cancer biomarker discovery
Çalışkan, M., & Tazaki, K. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1260374 · Cited by 29
Dretzke, J., Ensor, J., Bayliss, S. E., Hodgkinson, J., Lordkipanidzé, M., Riley, R. D., Fitzmaurice, D., & Moore, D. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-140 · Cited by 27
“Guidelines on data extraction and critical appraisal of risk prediction model studies have recently been published [7], and it is likely that some issues may overlap with prognostic studies.”
Pathological personality traits assessment using Facebook: Systematic review and meta-analyses
Carvalho, L. D. F., & Pianowski, G. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.01.061 · Cited by 29
“…(TRIPOD; (Collins et al., 2015)), and a reference article (Mihura et al., 2012); for bias risk: Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies (CHARMS; (Moons et al., 2014)), and a reference article (McGilloway, Hall, Lee, & Bhui, 2010).”
MethodDecision support systems for incurable non-small cell lung cancer: a systematic review
Révész, D., Engelhardt, E. G., Tamminga, J. J., Schramel, F. M. N. H., Onwuteaka-Philipsen, B. D., van de Garde, E. M. W., Steyerberg, E. W., Jansma, E. P., de Vet, H. C. W., & Coupé, V. M. H. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0542-1 · Cited by 25
“Also, we have not used the complete CHARMS checklist [15] to assess methodological quality of the included DSS, as many items were not reported.”
MethodLim, S. M. S., Wong, P.-L., Sulaiman, H., Atiya, N., Hisham, R., & Liew, S. M. (2019) · Journal of Hospital Infection · doi:10.1016/j.jhin.2019.01.012 · Cited by 25
Cardiovascular risk in patients with type 2 diabetes: A systematic review of prediction models
Galbete, A., Tamayo, I., Librero, J., Enguita-Germán, M., Cambra, K., & Ibáñez-Beroiz, B. (2022) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2021.109089 · Cited by 32
Prognostic clinical prediction models for acute post‐surgical pain in adults: a systematic review
Papadomanolakis‐Pakis, N., Munch, P. V., Carlé, N., Uhrbrand, C. G., Haroutounian, S., & Nikolajsen, L. (2024) · Anaesthesia · doi:10.1111/anae.16429 · Cited by 36
“Study eligibility was guided by the CHARMS checklist [22] and included prediction model development with or without internal validation, prediction model development with external validation and external validation studies alone (with or without model updating).”
Liu, L., Blake, V. C., Barman, M., Luxan, B. G., Churches, T., Kennedy, G., Ooi, S., Delaney, G. P., & Jorm, L. R. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf176 · Cited by 57
“43 The data fields in the form included basic study information, data source characteristics, a description of the clinical registry, data elements extracted from clinical text, NLP method definitions, NLP algorithms used, model performance and evaluation, availability of research outcomes, and…”
Botelho, S. F., Pantuzza, L. L. N., Marinho, C. P., & Reis, A. M. M. (2020) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2020.08.002 · Cited by 26
Development and validation of the Adverse Inpatient Medication Event model (AIME)
Falconer, N., Barras, M. A., Abdel‐Hafez, A., Radburn, S., & Cottrell, N. (2020) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14560 · Cited by 25
“10.(28) Using the above alpha level, significant variables from univariable analysis were included in the multivariable analysis.”
MethodZhao, Y., Liu, C., Zeng, Y.-Q., Zhou, A., Duan, J., Cheng, W., Sun, T., Li, X., Ma, L., Liu, Q.-M., et al. (2020) · Therapeutic Advances in Respiratory Disease · doi:10.1177/1753466620977376 · Cited by 24
Farshidfar, S. S., Cadman, J., Deng, D., Appleyard, R. C., & Dabirrahmani, D. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262684 · Cited by 24
“A customised checklist consisting of 16 appraisal questions was developed based on previous reviews in computational modelling of the human motion for prediction modelling studies with clinical/translational outcomes [11] and human motion analysis [12, 13], assessing the quality of the selected studies.”
MethodDixon, L. K., Akberali, U. K., Tommaso, E. D., George, S. J., Johnson, T., & Bruno, V. D. (2022) · International Journal of Cardiology · doi:10.1016/j.ijcard.2022.04.030 · Cited by 29
Predictive Modeling for Readmission to Intensive Care: A Systematic Review
Ruppert, M. M., Loftus, T. J., Small, C., Li, H., Ozrazgat-Baslanti, T., Balch, J., Holmes, R., Tighe, P. J., Upchurch, G. R., Efron, P. A., et al. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000848 · Cited by 24
“Study information was extracted from each of the relevant articles using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist; bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) (12, 13).”
MethodQi, W., Wang, Y., Li, C., He, K., Wang, Y., Huang, S., Li, C., Guo, Q., & Hu, J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.04.026 · Cited by 32
Melson, E., Davitadze, M., Malhotra, K., PCOS SEva working group, Mousa, A., Teede, H. J., Boivin, J., Thondan, M., Tay, C. T., & Kempegowda, P. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2023.1217468 · Cited by 34
Mbuya-Bienge, C., Pashayan, N., Kazemali, C. D., Lapointe, J., Simard, J., & Nabi, H. (2023) · Cancers · doi:10.3390/cancers15225380 · Cited by 26
“Data extraction was undertaken independently by the same two researchers (C. Mbuya-Bienge and C.D. Kazemali) using a grid based on the CHARMS checklist of relevant items to extract from individual studies of prediction tools [29].”
MethodOsteochondritis Dissecans of the Knee Associated With Mechanical Overload
van der Weiden, G. S., van Cruchten, S., van Egmond, N., Mastbergen, S. C., Husen, M., Saris, D. B. F., & Custers, R. J. H. (2024) · The American Journal of Sports Medicine · doi:10.1177/03635465231211497 · Cited by 24
Bahreiny, S. S., Aghaei, M., Dabbagh, M. R., Ghorbani, H., Javidan, M., & Fard, R. M. (2024) · Asian Pacific Journal of Reproduction · doi:10.4103/apjr.apjr_95_23 · Cited by 27
Harris, J., Cornelius, V. R., Ream, E., Cheevers, K., & Armes, J. (2017) · Supportive Care in Cancer · doi:10.1007/s00520-017-3688-6 · Cited by 31
“A standardised data extraction form designed specifically for systematic reviews of prediction models, the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [28], was adapted for this review.”
MethodThurtle, D. R., Rossi, S. H., Berry, B., Pharoah, P. D. P., & Gnanapragasam, V. J. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029149 · Cited by 24
“The review aim, search strategy and study inclusion and exclusion criteria were framed using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS).(16) The search strategy was informed by previous similar studies, including publications which tested an…”
MethodMoghaddam, N. M., Fathi, M., & Ramezankhani, A. (2019) · Australian Critical Care · doi:10.1016/j.aucc.2019.05.005 · Cited by 24
A systematic review of risk prediction models for perioperative mortality after thoracic surgery
Taylor, M., Hashmi, S. F., Martin, G. P., Shackcloth, M. J., Shah, R. D., Booton, R., & Grant, S. W. (2020) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivaa273 · Cited by 24
“Where a clinical prediction model was identified, the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies [9] was used to extract data from the included study.”
Chandna, A., Tan, R., Carter, M. J., Bruel, A. V. D., Verbakel, J. Y. J., Koshiaris, C., Salim, N., Lubell, Y., Turner, P., & Keitel, K. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2020-003451 · Cited by 22
Lozanovski, V. J., Probst, P., Arefidoust, A., Ramouz, A., Aminizadeh, E., Nikdad, M., Khajeh, E., Ghamarnejad, O., Shafiei, S., Ali‐Hasan‐Al‐Saegh, S., et al. (2021) · Transplant International · doi:10.1111/tri.13861 · Cited by 26
“Study data were extracted using the CHARMS checklist (checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies) [15].”
MethodSafari, F., Kehelpannala, C., Safarchi, A., Batarseh, A. M., & Vafaee, F. (2023) · Cancers · doi:10.3390/cancers15102780 · Cited by 30
“In order to progress towards implementation and technology readiness, extensive external validation is required, wherein the model’s predictive performance is quantified using data collected from participant cohorts external, temporally and/or geographically, to the development dataset [189].”
MethodTran, A., Fernando, S. M., Rochwerg, B. N., Hameed, M. S., Dawe, P., Hawes, H. G., Haut, E. R., Inaba, K., Engels, P. T., Zarychanski, R., et al. (2024) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000004326 · Cited by 33
“15 We contacted the corresponding author for completeness where these values could not be obtained from the published report.”
Systematic Review of Prognosis Models in Predicting Tooth Loss in Periodontitis
Chow, D. Y., Tay, J. R. H., & Nascimento, G. G. (2024) · Journal of Dental Research · doi:10.1177/00220345241237448 · Cited by 26
“Independent data extraction was done with standardized forms based on the CHARMS checklist (Checklist for critical Appraisal and data extraction for systemic Reviews of prediction Modelling Studies) (Moons et al. 2014).”
MethodPinto, C. M., Sánchez-Tecuatl, M., Berra‐Romani, R., Castro, I. D. M., Etchegaray‐Morales, I., Munguía‐Realpozo, P., Cárdenas-García, M., Arellano-Avendaño, F. J., & García‐Carrasco, M. (2024) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2024.152501 · Cited by 24
Critelli, B., Hassan, A., Lahooti, I., Noh, L., Park, J. S., Tong, K., Lahooti, A., Matzko, N. E., Adams, J. N., Liß, L., et al. (2025) · PLoS Medicine · doi:10.1371/journal.pmed.1004432 · Cited by 29
Cama-Olivares, A., Braun, C. G., Takeuchi, T., O’Hagan, E. C., Kaiser, K. A., Ghazi, L., Chen, J., Forni, L. G., Kane‐Gill, S. L., Ostermann, M., et al. (2025) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000702 · Cited by 31
Vermeulen, R. J., Andersson, V., Banken, J., Hannink, G., Govers, T. M., Rovers, M. M., & Rikkert, M. O. (2025) · Alzheimer s & Dementia · doi:10.1002/alz.70069 · Cited by 28
Karnuta, J. M., Featherall, J. T., Lawrenz, J. M., Gordon, J., Golubovsky, J. L., Thomas, J., Ramanathan, D., Simpfendorfer, C. S., Nystrom, L. M., Babić, M. S., et al. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001187 · Cited by 24
“It is a commonly accepted practice to use 10 events per variable for modeling a given outcome [26, 27].”
BackgroundKornblith, A. E., Graf, J., Addo, N., Newton, C. R., Callcut, R. A., Grupp‐Phelan, J., & Jaffe, D. M. (2020) · Academic Emergency Medicine · doi:10.1111/acem.13959 · Cited by 25
“to be included in our multivariate analysis.(25) Univariate regression was used to evaluate each variable’s association with identification of IAI in the present study.”
MethodAkyea, R. K., Leonardi‐Bee, J., Asselbergs, F. W., Patel, R. S., Durrington, P. N., Wierzbicki, A. S., Ibiwoye, O. H., Kai, J., Qureshi, N., & Weng, S. F. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034564 · Cited by 27
Grigore, B., Lewis, R. A., Peters, J., Robinson, S., & Hyde, C. (2020) · BMC Cancer · doi:10.1186/s12885-020-07572-z · Cited by 22
Metadata
- Authors
- Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S.
- Affiliations
- University Medical Center Utrecht (NL)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G. M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-14
- Volume
- 11
- Issue
- 10
- Pages
- e1001744
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4196729
- OpenAlex
- W2077663753
- Semantic Scholar
- c79bfa9a67c8745d7620fcfe8cf582b9d0407073
- Cited by
- 2,018
- References
- 157
- Citations by year
- 2026: 319 · 2025: 348 · 2024: 253 · 2023: 229 · 2022: 214 · 2021: 219 · 2020: 144 · 2019: 93 · 2018: 74 · 2017: 57 · 2016: 35 · 2015: 28 · 2014: 1
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Cancer Research UK (16895)
- National Institute for Health and Care Research (NF-SI-0513-10131)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (project 918.10.615, 918.10.615)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- critical appraisal, data extraction, predictive modeling, prognostic model, diagnostic model, systematic reviews, reporting quality
- MeSH terms
- Decision Support Techniques, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.