Cited by 657Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyClinical practice guidelines implementation

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

A guide to systematic review and meta-analysis of prediction model performance

Open access

Debray, Thomas P A, Damen, Johanna A A G, Snell, Kym I E, Ensor, Joie, Hooft, Lotty, Reitsma, Johannes B, Riley, Richard D, Moons, Karel G M·2017·BMJ

DOI 10.1136/bmj.i6460

Open this paper in CoChat — free account Open PDF

Abstract

Validation of prediction models is highly recommended and increasingly common in the literature. A systematic review of validation studies is therefore helpful, with meta-analysis needed to summarise the predictive performance of the model being validated across different settings and populations. This article provides guidance for researchers systematically reviewing and meta-analysing the existing evidence on a specific prediction model, discusses good practice when quantitatively summarising the predictive performance of the model across studies, and provides recommendations for interpreting meta-analysis estimates of model performance. We present key steps of the meta-analysis and illustrate each step in an example review, by summarising the discrimination and calibration performance of the EuroSCORE for predicting operative mortality in patients undergoing coronary artery bypass grafting.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1136/bmj.i6460

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal
Ndiaye, M., Lespez, L., Tribolo, C., Rassé, M., Hadjas, I., Davidoux, S., Huysecom, É., & Douze, K. (2024) · Monash University Research Portal (Monash University) · doi:10.3929/ethz-b-000667478 · Cited by 24,214
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Estimating the mean and variance from the median, range, and the size of a sample
Hozo, S. P., Djulbegovic, B., & Hozo, I. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-13 · Cited by 9,062
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citation
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citation
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citation
Chapter 9: Analysing Data and Undertaking Meta-Analyses
Deeks, J. J., Higgins, J. P. T., & Altman, D. G. (2008) · Cited by 2,782
The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419
Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement
Stewart, L. A., Clarke, M., Rovers, M., Riley, R. D., Simmonds, M., Stewart, G., & Tierney, J. F. (2015) · JAMA · doi:10.1001/jama.2015.3656 · Cited by 2,134
Self-citation
Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Self-citation

Show all 48 references

Sorted by citations

Items 601–648 of 648

Risk prediction models for cognitive impairment in patients with chronic kidney disease: a systematic review
Dong, L., Chen, Y., Lin, X., Shi, J., Liu, H., & Zhong, Y. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1851336
Systematic review and meta-analysis of risk prediction models for anastomotic leak after gastric cancer surgery
Sun, Y., Lu, R., Xu, Y., Jia, P., & Liu, Z. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1855091
Prediction models for mortality in patients with acute on chronic liver failure: systematic review and critical appraisal
Song, T., Zhao, J., Jiang, B., Wang, X. H., Li, Z., & 庞秋云 (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1829188
Development and validation of a simplified cumulative complication burden score (sccbs) for preterm birth prediction at hospital admission
Lu, J., Xu, H., & Xu, Q. (2026) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-026-10159-4
Performance of Survival Prediction Tools in Patients With Gastrointestinal Stromal Tumors: A Systematic Review and Meta‐Analysis
Zhu, A., Wong, C., Adhikari, N. K. J., Ip, K. Y., Mahar, A. L., Hsu, A. T., Hallet, J. I., & Coburn, N. G. (2026) · Journal of Surgical Oncology · doi:10.1002/jso.70303
Machine learning based prediction models for first stroke in community primary care: a systematic review and meta-analysis
Zhang, Z., Zhang, Y., Li, Z., Tian, C., Liang, J., Xie, J., & Lei, J. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03636-5
AI for Prognosis Among People Living With HIV: Protocol for a Systematic Review and Meta-Analysis
Yehadji, D., Hofmann, M., Isaakidis, P., Alidou, S., Ale, O. A. J., Yena, K. M., GUIDIGAN, K., Gray, G., Pindra, N., & Diallo, G. (2026) · JMIR Research Protocols · doi:10.2196/91989
Prognostic models for predicting intensive care unit admission or mortality in critically ill adults not yet been admitted to the intensive care unit
Vieira, A. G. D. S., Pinto, A. C. P. N., Garcia, A. A. A., Nawa, R. K., Mól, C. G., Duenas, J. D., Atallah, Á. N., Shrestha, G. S., Schultz, M. J., Saconato, H., et al. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016115
Prognostic models for diabetic kidney disease outcomes: A systematic review
Yang, D., Fan, Y., Xu, Y., Jin, K., Ma, Y., & Wang, R. (2026) · Diabetic Medicine · doi:10.1111/dme.70391
Externally validated risk prediction models for gestational diabetes mellitus: A systematic review and meta‐analysis
Seifu, B. L., Belsti, Y., Yimer, N. B., Goldstein, R. F., Enticott, J., & Teede, H. J. (2026) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.70300
Evaluation of risk prediction models for coronary artery disease: a systematic review, meta- analysis, and machine learning validationPreprint
Ouyang, T., Qi, D., Miao, X., Xie, X., Deng, X., Wang, F., Cao, L., Zhao, C., Zhu, R., Sheng, S., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-9670632/v1
Molecular‐Based Risk Prediction Models for Recurrence After Curative Treatment of Early‐Stage Lung Cancer: A Systematic Review and Meta‐Analysis
Ling, A., Samuel, E., Mahendran, R., Zalcberg, J. R., & Stirling, R. G. (2026) · Thoracic Cancer · doi:10.1111/1759-7714.70345
Machine learning and data-driven models for predicting post-stroke dysphagia: a systematic review and meta-analysisPreprint
Yazdi, S. M., Motevaselian, M., Khatami, S., Radfar, N., Jourahmad, Z., & Perez, H. A. (2026) · medRxiv · doi:10.64898/2026.07.15.26358113
Non-invasive Tests for the Prediction of Liver-related Events in Patients with MASLD: A Systematic Review and Meta-analysis
Wang, X. W., Dong, X., 钱洛丹, Yang, Z., Yang, D., & Niu, Y. (2026) · Academic Radiology · doi:10.1016/j.acra.2026.06.068
Neuroimaging biomarkers for predicting acupuncture treatment response in chronic pain: a systematic review and meta-analysis protocol
Wu, J., Yu, Q., Xu, Z., Wang, B., Chen, Z., Li, Q., & Xu, T. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-120106
Risk prediction models for venous thromboembolism in the intensive care unit: a systematic review protocol assessing model performance, risk of bias, and clinical applicability
Zhu, F., Chen, J., Tong, W., Qiu, P., Mu, J., Xu, X., & Zhang, Q. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03259-x
The Federated Estimand: Healthcare Federated Learning as Evidence Synthesis
Veeraragavan, N. R., Lambert, P. C., Aagnes, B., & Nygård, J. F. (2026) · International Conference on Federated Learning and Intelligent Computing Systems (FLICS) · doi:10.1109/flics70075.2026.11621899
Predicting postoperative infections: prediction models and their evaluation
de Mul, N., Scheffer-Wesdorp, K. M., Verlaan, D., Mondelli, G., Schellekens, W. M., Bonten, M. J. M., Derde, L. P. G., Cremer, O. L., & Vernooij, L. M. (2026) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000002470
Risk prediction models for inadequate bowel preparation in colonoscopy: a systematic review and meta-analysis
Zhang, Y., Qian, J., Peng, S., Shen, Y., Sun, J., Yuqun, G., 左新燕, Miao, Y., Jiang, Y., Huang, J., et al. (2026) · Scandinavian Journal of Gastroenterology · doi:10.1080/00365521.2026.2710815
Machine-Learning Versus Traditional Scores for Predicting Outcomes After Coronary Artery Bypass Graft Surgery: A Systematic Review and Meta-Analysis
Gupta, A. K., Zaka, A., Tyagi, D., Mutahar, D., Parvez, R., Muston, B. T., Farag, M., Lombardo, A., Eranki, A., Wilson-Smith, A., et al. (2026) · Surgical Innovation · doi:10.1177/15533506261468192
Non-Invasive Sociodemographic Cues for Type 2 Diabetes Risk Stratification in South Africa
Fahimi, R., Alrubayyi, A., Kamal, H. M., Ja’afar, I. K., Alhothify, M., Barber, T. M., & Uthman, O. A. (2026) · Diabetology · doi:10.3390/diabetology7080151
Identification of cognitive impairment in patients with hypertension: a systematic review and critical appraisal of existing models
Sun, Y., Wang, J., Sun, F., Shi, M., Tian, X., Shang, H., Wang, W., Chang, T., Zhao, J., Huang, Y., et al. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1858790
Artificial intelligence for predicting surgical difficulty in laparoscopic cholecystectomy: a systematic review and meta-analysis
Liu, H., Wu, X., & Luo, F. (2026) · Surgical Endoscopy · doi:10.1007/s00464-026-13257-8
Machine learning-based prediction model for postpartum stress urinary incontinence risk: a systematic review and meta-analysis
Zhong, X., Wang, Y. F., Chai, W., Lin, Y., Zheng, N., & Huang, J. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1875678
Dataset-structured evidence synthesis for machine-learning prediction models: a methodological framework and worked examplePreprint
Abdelhay, O., Abdel-Hay, D., Qasem, A. F., & Altamimi, T. (2026) · Research Square · doi:10.21203/rs.3.rs-10424152/v1
AI Models for Predicting Acute Kidney Injury (AKI) and Post-AKI Mortality: Systematic Review and Meta-Analysis
Xiao, H., Liang, J., Li, G., Yang, Y., Li, T., Chen, X., Ao, Q., Wu, Y., & Song, Q. (2026) · Journal of Medical Internet Research · doi:10.2196/104269
Machine learning for metabolic syndrome prediction: a systematic review of model performance, risk of bias, and diagnostic circularity
Etoom, M., & Ahmad, M. M. (2026) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-026-00869-2
Development and Internal Validation of a Clinical–Infectious Prediction Model Incorporating the Serum Pepsinogen I/II Ratio to Identify Histologically Confirmed Gastric Atrophy
Kerimkulov, N., & Nurgaliyeva, B. (2026) · Caucasian Medical Journal · doi:10.67076/cmj.2026.v4n2.02
CT-Based Peritumoral and Perirenal Fat Radiomics in Renal Cell Carcinoma: A Systematic Review and Meta-Analysis of Grade, Stage, and Adherent Perinephric Fat Prediction
Mopti, A. N. M. A., Alshehri, A. H. D., & Alqahtani, A. M. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15176720
Relapse prediction and individualized treatment-effect modeling in relapsing multiple sclerosis: a systematic review
Budil, M., Rožánek, M., & Hospodková, P. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1940704
Imaging‑based models for predicting cerebrovascular complications of carotid stenosis
Li, B., Du, X., Liu, Y., Zhang, H., Liu, S., Wang, S., Zhao, X., Sheng, F., Lu, M., & Cai, J. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016391
Systematic review and meta-analysis of risk prediction models for anti-tuberculosis drug-induced liver injury in East Asian populations
Li, H., Feng, X., Zheng, X., & Zhong, X. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1958154
External validation reveals poor calibration despite preserved discrimination for a vasopressin treatment-signal prediction model across ICU databasesPreprint
Sun, M., Li, Z., & Hu, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-10559528/v1
Reporting quality and risk of bias in systematic reviews of clinical prediction models: a cross-sectional overview of 1004 studies
Zhang, M., Xia, Y., Mo, H., Zhu, Y., Fang, L., Xiang, Q., Shen, J., Wang, L., & Lou, J. (2026) · Epidemiologic Reviews · doi:10.1093/epirev/mxag013
Deep learning-based radiomics and pathomics for decoding tumor microenvironment and predicting immunotherapy outcomes in gastric cancer
Shen, X., Liu, Z., Yu, W., Li, F., Lu, Y., Li, M., Sun, P., & Liu, A. (2026) · Frontiers in Immunology · doi:10.3389/fimmu.2026.1970256
Application of type 2 inflammation-related biomarkers in risk prediction models for acute exacerbations in adult asthma: a systematic review
Li, M., Zihan, Y., & Wang, M. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1900214
Prognostic Models for Incident Hypertension: Time to Move from Development to Implementation—An Umbrella Review
Vera-Ponce, V. J., & Ballena-Caicedo, J. (2026) · Diagnostics · doi:10.3390/diagnostics16193069
Traditional Chinese and Western medicine-integrated risk prediction models for coronary heart disease: a systematic review and meta-analysis
Xie, C., Zhuang, H., Zhengquan, D., Zhou, Y., Liang, H., & Wang, B. (2026) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2026.1921708
Risk Prediction Models and Predictors of Low Anterior Resection Syndrome in Patients with Rectal Cancer: A Systematic Review and Meta-Analysis
Zhou, L., Wang, C., Yang, R., Zhang, Z., Wang, L., & Zhang, J. (2026) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2026.152395
Machine Learning for Clinical Outcome Prediction After Transcatheter Aortic Valve Replacement: A Systematic Review and Meta-Analysis of Predictive Performance, Validation, and Clinical Readiness
Alizadehasl, A., Bastami, E. A., Jebelli, S. F. H., Davani, D. N., Bahramnejad, A., Aliabadi, A. Y., Moussavi, S. M., Ghaffari, A., Amani‐Beni, R., Darouei, B., et al. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15197692
Benchmarking machine learning for cardiovascular disease classification across 46 countries with geographic and temporal transportability assessmentPreprint
Ng’ambi, W. F., Zyambo, C., & Kazembe, L. N. (2026) · Research Square · doi:10.21203/rs.3.rs-11021830/v1
Mental Health Factors in Cardiothoracic Surgery Risk Prediction Models: A Systematic Review
Rios-Monterrosa, J. L., Artunduaga, M. C., Rasheed, N., Aguiar, C., Castro, S. J., Jaime, M., Lang, E., Woodward, A. L., & Sun, L. Y. (2026) · Journal of the American Heart Association · doi:10.1161/jaha.126.052512
A Novel Framework for Meta-analysis of Linear Regression ModelsPreprint
Sarma, I. D., & Saikia, D. (2026) · Research Square · doi:10.21203/rs.3.rs-10958900/v1
Prediction Models for Immune Checkpoint Inhibitor Efficacy in Gastric Cancer: A Systematic Review and Meta-analysis (Preprint)Preprint
Guo, J., Zheng, L., Ni, X., Zhang, K., Wang, C., Fan, W., Huang, Z., Du, Z., Jiang, T., & Zhang, G. (2026) · doi:10.2196/preprints.114743
Risk prediction models for postoperative pneumonia in lung cancer patients: a systematic review and meta-analysis
Li, C., Jiang, Y., & Wang, Y. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1967523
Diagnostic efficacy of AI-driven radiomics in predicting lymph node metastasis in cholangiocarcinoma: a systematic meta-analysis
Xu, H., Wu, Y., Fang, H., & Gu, Z. (2026) · International Journal of Surgery · doi:10.1097/js9.0000000000005583
EFFECTIVENESS OF PREDICTIVE ANALYTICS ALGORITHMS FOR EARLY SEPSIS DETECTION IN THE INTENSIVE CARE UNIT ON TIME TO INTERVENTION AND MORTALITY : A SYSTEMATIC REVIEW AND META-ANALYSIS OF RANDOMIZED CONTROLLED TRIALS AND PRIMARY STUDIES
Viantoro, Y., & Yuniarmi, F. (2026) · The International Journal of Medical Science and Health Research · doi:10.70070/sehv7w52
Predictive modelling approaches for childhood diarrhoeal disease: a systematic review (2000–2025)
Qi, S., Nobel, S. M. N., Ramadas, A., Gu, H., & Quek, K. F. (2026) · Infectious Diseases of Poverty · doi:10.1186/s40249-026-01508-8

Authors: Debray, T. P. A., Damen, J. A. A. G., Snell, K. I. E., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. M.
Affiliations: University Medical Center Utrecht (NL)
Keele University (GB)
Corresponding author: Debray, Thomas P A
ORCID: Debray, Thomas P A
Damen, Johanna A A G
Snell, Kym I E
Ensor, Joie
Hooft, Lotty
Reitsma, Johannes B
Riley, Richard D
Moons, Karel G M
Venue: BMJ
Publisher: BMJ
Published: 2017-01-05
Volume: 356
Pages: i6460
ISSN: 1756-1833 (ISSN-L)
DOI: 10.1136/bmj.i6460
OpenAlex: W2570878489
Cited by: 657
References: 48
Citations by year: 2026: 109 · 2025: 98 · 2024: 94 · 2023: 81 · 2022: 64 · 2021: 82 · 2020: 56 · 2019: 33 · 2018: 27 · 2017: 11
FWCI: 8.62
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.bmj.org/licenses/tdm/1.0/terms-and-conditions.html (tdm)
Locations: BMJ (publishedVersion)
PubMed (publishedVersion)
Pure Amsterdam UMC (publishedVersion)
Keele Research Repository (Keele University) (submittedVersion, cc-by)
Data Archiving and Networked Services (DANS) (submittedVersion)
Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
HighWire Press Open Archive (submittedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
Funders: Medical Research Council (MR/L004933/1, G0902393)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords: prediction model validation, systematic reviews, meta-analysis, predictive performance, discrimination (statistics), calibration, EuroSCORE, postoperative mortality, coronary artery bypass grafting, risk prediction model
MeSH terms: Systematic Reviews as Topic, Coronary Artery Bypass, Decision Support Techniques, Humans, Meta-Analysis as Topic, Risk Assessment, Validation Studies as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A guide to systematic review and meta-analysis of prediction model performance

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks