Cited by 371Statistical Methods in EpidemiologyClinical Reasoning and Diagnostic SkillsMeta-analysis and systematic reviews

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.

Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance

Open access

Van Calster, Ben, Vickers, Andrew J.·2015·Medical Decision Making

DOI 10.1177/0272989x14547233

Open this paper in CoChat — free account Open PDF

Abstract

Decision-analytic measures to assess clinical utility of prediction models and diagnostic tests incorporate the relative clinical consequences of true and false positives without the need for external information such as monetary costs. Net Benefit is a commonly used metric that weights the relative consequences in terms of the risk threshold at which a patient would opt for treatment. Theoretical results demonstrate that clinical utility is affected by a model';s calibration, the extent to which estimated risks correspond to observed event rates. We analyzed the effects of different types of miscalibration on Net Benefit and investigated whether and under what circumstances miscalibration can make a model clinically harmful. Clinical harm is defined as a lower Net Benefit compared with classifying all patients as positive or negative by default. We used simulated data to investigate the effect of overestimation, underestimation, overfitting (estimated risks too extreme), and underfitting (estimated risks too close to baseline risk) on Net Benefit for different choices of the risk threshold. In accordance with theory, we observed that miscalibration always reduced Net Benefit. Harm was sometimes observed when models underestimated risk at a threshold below the event rate (as in underestimation and overfitting) or overestimated risk at a threshold above event rate (as in overestimation and overfitting). Underfitting never resulted in a harmful model. The impact of miscalibration decreased with increasing discrimination. Net Benefit was less sensitive to miscalibration for risk thresholds close to the event rate than for other thresholds. We illustrate these findings with examples from the literature and with a case study on testicular cancer diagnosis. Our findings strengthen the importance of obtaining calibrated risk models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1177/0272989x14547233

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers
Pencina, M. J., D’Agostino, R. B., & Steyerberg, E. W. (2010) · Statistics in Medicine · doi:10.1002/sim.4085 · Cited by 2,483
Clinical Prediction Models
Steyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491
Criteria for Evaluation of Novel Markers of Cardiovascular Risk
Hlatky, M. A., Greenland, P., Arnett, D. K., Ballantyne, C. M., Criqui, M. H., Elkind, M. S. V., Go, A. S., Harrell, F. E., Hong, Y., Howard, B. V., et al. (2009) · Circulation · doi:10.1161/circulationaha.109.192278 · Cited by 1,146
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600
Two further applications of a model for binary regression
COX, D. R. (1958) · Biometrika · doi:10.1093/biomet/45.3-4.562 · Cited by 556
Therapeutic decision making: a cost-benefit analysis
Pauker, S. G., & Kassirer, J. P. (1975) · New England Journal of Medicine · doi:10.1056/nejm197507312930505 · Cited by 507
Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers
Austin, P. C., & Steyerberg, E. W. (2014) · Statistics in Medicine · doi:10.1002/sim.5941 · Cited by 362

Show all 29 references

Research works printed but not linked: 1

Vickers AJ, 2010, Semin Oncol, 76, 1298

Sorted by citations

Items 201–300 of 371

Recalibrating prognostic models to improve predictions of in‐hospital child mortality in resource‐limited settings
Ogero, M. O., Ndiritu, J., Sarguta, R. J., Tuti, T., Aluvaala, J., & Akech, S. (2023) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12948 · Cited by 7
A nomogram based on a gene signature for predicting the prognosis of patients with head and neck squamous cell carcinoma
Yang, B., Fu, L., Xu, S., Xiao, J., Li, Z., & Liu, Y. (2019) · The International Journal of Biological Markers · doi:10.1177/1724600819865745 · Cited by 7
Development and evaluation of predictive models for pregnancy risk in UK dairy cows
Barden, M., Hyde, R., Green, M. J., Bradley, A., Can, E., Clifton, R., Lewis, K., Manning, A., & O’Grady, L. (2024) · Journal of Dairy Science · doi:10.3168/jds.2023-24623 · Cited by 7
Machine learning and radiomics for ventricular tachyarrhythmia prediction in hypertrophic cardiomyopathy: insights from an MRI-based analysis
Durmaz, E. Ş., Karabacak, M., Ozkara, B. B., Kargın, O. A., Demir, B., Raimoglou, D., Aygun, A. A., Adaletli, İ., Baş, A., & Durmaz, E. (2024) · Acta Radiologica · doi:10.1177/02841851241283041 · Cited by 7
How the Intended Use of Polygenic Risk Scores Guides the Design and Evaluation of Prediction Studies
Martens, F. K., & Janssens, A. C. J. W. (2019) · Current Epidemiology Reports · doi:10.1007/s40471-019-00203-7 · Cited by 6
A prognostic nomogram for pancreatic ductal adenocarcinoma patients’ all-cause survival in a Surveillance, Epidemiology, and End Results analysis
Li, C., Ren, Z., Han, D., Wang, H., Xu, F., Zheng, S., & Lyu, J. (2020) · Translational Cancer Research · doi:10.21037/tcr-19-2962 · Cited by 6
RETRACTED ARTICLE: The diagnostic nomogram of platelet-based score models for hepatic alveolar echinococcosis and atypical liver cancer
Du, Q., Wang, Y., Guan, S., Hu, C., Li, M., Zhou, L., Zhang, M., Chen, Y., Mei, X., Sun, J., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-55563-3 · Cited by 6
Retracted
Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity
Meid, A. D., González‐González, A. I., Dinh, T. S., Blom, J. W., van den Akker, M., Elders, P. J. M., Thiem, U., de Gaudry, D. K., Swart, K. M. A., Rudolf, H., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045572 · Cited by 6
“…being eligible if they were aged 60 years or older, had been diagnosed with at least one chronic condition defined using the O'Halloran list, 39 and had at least one chronic prescription at study baseline ( ≤ 2 weeks duration in PRIMUM, ≤ 2 months in ISCOPE and ≤ 3 months in Opti-Med and RIME).”
BackgroundMethod
Nomogram for accurate and quantitative prediction of the risk of psoriatic arthritis in Chinese adult patients with moderate and severe plaque psoriasis
Chen, Z., Wang, Y., Lan, X., Yang, M., Ding, L., Li, G., Hong, P., Bai, Y., Liu, Y., & Li, W. (2021) · European Journal of Dermatology · doi:10.1684/ejd.2021.4095 · Cited by 6
Calculating Probability in Sex Offender Risk Assessment
Elwood, R. W. (2016) · International journal of offender therapy and comparative criminology · doi:10.1177/0306624x16677784 · Cited by 8
External Validation of Carbapenem-Resistant Enterobacteriaceae Acquisition Risk Prediction Model in a Medium Sized Hospital
Seo, S. M., & Jeong, I. S. (2020) · Journal of Korean Academy of Nursing · doi:10.4040/jkan.20137 · Cited by 6
Development and validation of a simplified nomogram predicting individual critical illness of risk in COVID‐19: A retrospective study
Xu, R., Cui, J., Liu, H., Wang, Y., Wang, T., Ye, D., Lv, Y., & Liu, Q. (2020) · Journal of Medical Virology · doi:10.1002/jmv.26551 · Cited by 6
Incidence and Prognosis Nomogram of Small Solitary Lung Cancer (≤2 cm) With Extra-Thoracic Metastasis at Initial Diagnosis:A Population-Based Study
Hu, A., Chen, Z., Liu, C., Gao, Y., Deng, C., & Liu, X. (2022) · Cancer Control · doi:10.1177/10732748221141560 · Cited by 6
Non‐parametric inference on calibration of predicted risks
Sadatsafavi, M., & Petkau, J. (2024) · Statistics in Medicine · doi:10.1002/sim.10138 · Cited by 5
Hypothesis: Net benefit as an objective function during development of machine learning algorithms for medical applications
Vickers, A. J., Hollingsworth, A., Bozzo, A., Chatterjee, A., & Chatterjee, S. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105844 · Cited by 5
Prediction models developed using artificial intelligence: similar predictive performances with highly varying predictions for individuals – an illustration in deep vein thrombosis
Yusufujiang, M., Navarro, C. L. A., Damen, J. A., Takada, T., Geersing, G., Hooft, L., Schuit, E., Moons, K. G. M., de Jong, V. M. T., & van Smeden, M. (2026) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00216-5 · Cited by 5
Inferring FDG-PET-positivity of lymph node metastases in proven lung cancer from contrast-enhanced CT using radiomics and machine learning
Gorodetski, B., Becker, P., Baur, A., Hartenstein, A., Rogasch, J., Furth, C., Amthauer, H., Hamm, B., Makowski, M., & Penzkofer, T. (2022) · European Radiology Experimental · doi:10.1186/s41747-022-00296-8 · Cited by 4
Development of an Injury Burden Prediction Model in Professional Baseball Pitchers
Bullock, G. S., Thigpen, C. A., Collins, P. G. S., Arden, N., Noonan, T. J., Kissenberth, M. J., & Shanley, E. (2022) · International Journal of Sports Physical Therapy · doi:10.26603/001c.39741 · Cited by 5
Intercontinental validation of a clinical prediction model for predicting 90-day and 2-year mortality in an Israeli cohort of 2033 patients with a femoral neck fracture aged 65 or above
Oosterhoff, J. H. F., Karhade, A. V., Groot, O. Q., Schwab, J. H., Heng, M. M., Klang, E., & Prat, D. (2023) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-023-02237-5 · Cited by 5
Development and external validation of a model to predict multidrug‐resistant bacterial infections in patients with cirrhosis
Marciano, S., Piano, S. S., Singh, V., Caraceni, P., Maiwall, R., Alessandria, C., Fernández, J., Kim, D. J., Kim, S. E., Soares, E. C., et al. (2024) · Liver International · doi:10.1111/liv.16063 · Cited by 6
Calibration, Clinical Utility, and Specificity of Clinical Decision Support Tools in Inflammatory Bowel Disease
Alsoud, D., Sabino, J., Ferrante, M., Verstockt, B., & Vermeire, S. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.09.020 · Cited by 6
Prognostic models for COVID-19 needed updating to warrant transportability over time and space
van Klaveren, D., Zanos, T. P., Nelson, J., Levy, T. J., Park, J. G., Retel Helmrich, I. R. A., Rietjens, J. A. C., Basile, M. J., Hajizadeh, N., Lingsma, H. F., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02651-3 · Cited by 5
Machine Learning Algorithms Can Be Reliably Leveraged to Identify Patients at High Risk of Prolonged Postoperative Opioid Use Following Orthopedic Surgery: A Systematic Review
Krivicich, L. M., Jan, K., Kunze, K. N., Rice, M. W., & Nho, S. J. (2023) · HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery · doi:10.1177/15563316231164138 · Cited by 5
Integrating Sigmoid Calibration Function into Entropy Thresholding Segmentation for Enhanced Recognition of Potholes Imaged Using a UAV Multispectral Sensor
Nomqupu, S., Sali, A., Nyamugama, A., & Ndou, N. N. (2024) · Applied Sciences · doi:10.3390/app14072670 · Cited by 5
“Uncalibrated models tend to either underestimate or overestimate the probability of outcomes [24], subsequently making insignificant decisions regarding pothole size.”
Background
Development and validation of a web-based nomogram for acute kidney injury in acute non-variceal upper gastrointestinal bleeding patients
Wei, C., Cao, H., Huang, L., & Feng, L.-H. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1474311 · Cited by 5
Preoperative Prediction of Occult Level V Lymph Node Metastasis in Papillary Thyroid Carcinoma: Development and Validation of a Radiomics-Driven Nomogram Model
Feng, J., Zheng, F., Liu, S., Qi, G.-F., Ye, X., Ye, J., & Jiang, Y. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.10.001 · Cited by 5
Les nomogrammes dans la pratique clinique : méthodologie, limites et intérêts
Filleron, T., Chaltiel, L., Jouve, E., Cabarrou, B., Gilhodes, J., Lusque, A., Mery-Lamarche, E., Dalenc, F., & Martínez, A. (2017) · Bulletin du Cancer · doi:10.1016/j.bulcan.2017.10.024 · Cited by 7
Prognostic model for predicting survival in very preterm infants: an external validation study
van Beek, P. E., Groenendaal, F., Onland, W., Koole, S., Dijk, P. H., Dijkman, K. P., van den Dungen, F., van Heijst, A., Kornelisse, R. F., Schuerman, F., et al. (2021) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17010 · Cited by 5
A model of multiple tumor marker for lymph node metastasis assessment in colorectal cancer: a retrospective study
Fu, J., Tu, M., Zhang, Y., Zhang, Y., Wang, J., Zeng, Z., Li, J., & Zeng, F. (2022) · PeerJ · doi:10.7717/peerj.13196 · Cited by 5
Decision Curves and Relative Utility Curves
Baker, S. G. (2019) · Medical Decision Making · doi:10.1177/0272989x19850762 · Cited by 5
Adapting a Risk Prediction Tool for Neonatal Opioid Withdrawal Syndrome
Reese, T. J., Wiese, A. D., Leech, A. A., Domenico, H. J., McNeer, E. A., Davis, S. E., Matheny, M. E., Wright, A. T., & Patrick, S. W. (2025) · PEDIATRICS · doi:10.1542/peds.2024-068673 · Cited by 4
Fully Interpretable Gaussian Centralized TSK Fuzzy Classifier From Probabilistic Perspective: Concepts, Output-Stability-Based Learning, and Ensemble
Li, Y., Chung, F.-L., Nojima, Y., & Wang, S. (2025) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2025.3600911 · Cited by 4
Clustered flexible calibration plots for binary outcomes using random effects modeling
Barreñada, L., Cock, B. D., Wynants, L., & CALSTER, B. V. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10046 · Cited by 5
Risk prediction of multiple cardiometabolic diseases in middle-aged and older adults using lipid metabolism biomarkers and derived composite measures based on machine learning: a prospective cohort study of CHARLS
Li, Y., Chen, P., & Gao, B. (2025) · Lipids in Health and Disease · doi:10.1186/s12944-025-02820-2 · Cited by 5
Prediction models for maltreatment risk: TRIPOD/PROBAST compliance, calibration, and fairness—A systematic review
Ahmed, R. S., & Shaban, M. (2026) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2026.107923 · Cited by 4
Calibration-informed metrics for instance-level predictive reliability in medical AI
Cabitza, F. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103366 · Cited by 5
A novel nomogram to predict psoriatic arthritis in patients with plaque psoriasis
Tan, M., Chen, J., Cheng, J., Hu, J., Hu, K., Yang, J. X., Li, X., Zhang, M., Zhu, W., Liao, L., et al. (2024) · JDDG Journal der Deutschen Dermatologischen Gesellschaft · doi:10.1111/ddg.15446 · Cited by 4
TripleSurv: Triplet Time-Adaptive Coordinate Learning Approach for Survival Analysis
Zhang, L., Zhong, L., Yang, F., Tang, L., Dong, D., Hui, H., & Tian, J. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3450910 · Cited by 4
An interrater reliability analysis of preoperative mortality risk calculators used for elective high-risk noncardiac surgical patients shows poor to moderate reliability
Vernooij, J. E. M., Roovers, L., van der Zwan, R., Preckel, B., Kalkman, C. J., & Koning, N. J. (2024) · BMC Anesthesiology · doi:10.1186/s12871-024-02771-8 · Cited by 4
Hierarchical Bayesian models to mitigate systematic disparities in prediction with proxy outcomes
Mikhaeil, J. M., Gelman, A., & Greengard, P. (2024) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1093/jrsssa/qnae142 · Cited by 4
Artificial Intelligence in Nuclear Cardiology
Miller, R. J. H. (2025) · Heart Failure Clinics · doi:10.1016/j.hfc.2025.01.015 · Cited by 4
AI in Healthcare: Bridging the Gap between Research and Clinical Implementation
Lanka, S., & Madala, P. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24oct089 · Cited by 5
A novel model with nutrition-related parameters for predicting overall survival of cancer patients
Zhang, Q., Zhang, K., Li, X., Zhang, X., Song, M., Liu, T., Song, C., Barazzoni, R., Wang, K., Xu, H., et al. (2021) · Supportive Care in Cancer · doi:10.1007/s00520-021-06272-z · Cited by 4
Early warning model construction and validation for urinary tract infection in patients with neurogenic lower urinary tract dysfunction (NLUTD): a retrospective study
Zhou, L., Liang, S., Qin, S., Fan, C., Gao, L., & Cai, W. (2022) · PeerJ · doi:10.7717/peerj.13388 · Cited by 4
Global Interpretable Calibration Index, a New Metric to Estimate Machine Learning Models’ Calibration
Cabitza, F., Campagner, A., & Famiglini, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-14463-9_6 · Cited by 4
Clinical utility gains from incorporating comorbidity and geographic location information into risk estimation equations for atherosclerotic cardiovascular disease
Xu, Y.-Z., Foryciarz, A., Steinberg, E., & Shah, N. (2022) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocad017 · Cited by 3
External validation and improvement of the scoring system for predicting the prognosis in hepatocellular carcinoma after interventional therapy
Qiao, W., Wang, Q., Mei, T., Wang, Q., Wang, W. J., & Zhang, Y. (2023) · Frontiers in Surgery · doi:10.3389/fsurg.2023.1045213 · Cited by 4
A radiomics nomogram based on contrast-enhanced CT for preoperative prediction of Lymphovascular invasion in esophageal squamous cell carcinoma
Wang, Y., Bai, G., Huang, W. J., Zhang, H., & Chen, W. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1208756 · Cited by 4
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and
Moons, E. G. M., Altman, D. G., Reitsma, J. B., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, P. G. S. (2015) · Cited by 4
Development and validation of ischemia risk scores
Miller, R. J. H., Rozanski, A., Slomka, P. J., Han, D., Gransar, H., Hayes, S. W., Friedman, J. D., Thomson, L., & Berman, D. S. (2022) · Journal of Nuclear Cardiology · doi:10.1007/s12350-022-02976-9 · Cited by 4
Validación del índice de salud prostática en un modelo predictivo de cáncer de próstata
Bonet, A., Barrionuevo-González, M., Bajo, A. M., Pulido-Fonseca, L., Polledo, L. E. O., Ruíz, J. C. T., & Sánchez‐Chapado, M. (2017) · Actas Urológicas Españolas · doi:10.1016/j.acuro.2017.06.003 · Cited by 4
Assessing a patient’s individual risk of biopsy-detectable prostate cancer: Be aware of case mix heterogeneity and a priori likelihood
Verbeek, J. F. M., Nieboer, D., Steyerberg, E. W., & Roobol, M. J. (2019) · European Urology Oncology · doi:10.1016/j.euo.2019.07.012 · Cited by 4
Targeted Development and Validation of Clinical Prediction Models in Secondary Care Settings: Opportunities and Challenges for Electronic Health Record Data
van Maurik, I. S., Doodeman, H. J., Nuijens, B. W., Möhringer, R. P. M., Sudiono, D. R., Jongbloed, W., & van Soelen, E. (2024) · JMIR Medical Informatics · doi:10.2196/57035 · Cited by 3
Classification and predictive models using supervised machine learning: A conceptual review
Pienaar, M. A., & Naidoo, K. D. (2025) · Southern African Journal of Critical Care · doi:10.7196/sajcc.2025.v411.2937 · Cited by 5
“[27,35,36] Depending on the size of the data, this is achieved by breaking up the data into bands of predicted probability called bins.”
Prospective prediction of first onset of major depressive disorder in midlife using machine learning
Massell, J., Preisig, M., Miché, M., Strippoli, M. F., Pistis, G., & Lieb, R. (2025) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-025-02942-z · Cited by 4
Development and validation of a prognostic prediction model for lumbar-disc herniation based on machine learning and fusion of clinical text data and radiomic features
Wang, Z., Zhang, H., Li, Y., Zhang, X., Liu, J., Ren, Z., Daping, Q., & Xiyun, Z. (2025) · European Spine Journal · doi:10.1007/s00586-025-09102-6 · Cited by 3
Computed tomography-based radiomics predicts prognostic and treatment-related levels of immune infiltration in the immune microenvironment of clear cell renal cell carcinoma
Song, S., Ge, W., Qi, X., Che, X., Wang, Q., & Wu, G. (2025) · BMC Medical Imaging · doi:10.1186/s12880-025-01749-3 · Cited by 3
Implementing parallel spreadsheet models for health policy decisions: The impact of unintentional errors on model projections
Bailey, S. L., Bono, R. S., Nash, D., & Kimmel, A. D. (2018) · PLoS ONE · doi:10.1371/journal.pone.0194916 · Cited by 3
Model for Predicting Risk of Postpartum Purulent-Inflammatory Complications after Cesarean Section: Cohort Retrospective Study
Куценко, И. И., Олегович, Б. И., Сергеевич, М. А., Bulgakova, V. P., & Borovikova, O. I. (2023) · Кубанский научный медицинский вестник · doi:10.25207/1608-6228-2023-30-1-26-36 · Cited by 3
Two web-based dynamically interactive nomograms and risk stratification systems for predicting survival outcomes and guiding treatment in non-metastatic nasopharyngeal carcinoma
Zuo, H., & Li, M. (2023) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-023-05363-0 · Cited by 3
Development and internal validation of a risk prediction model for stress urinary incontinence throughout pregnancy: A multicenter retrospective longitudinal study in Indonesia
Liang, S., Huang, S., Andarini, E., Wang, Y., Li, Y., & Cai, W. (2023) · Neurourology and Urodynamics · doi:10.1002/nau.25364 · Cited by 3
“14 In addition to calibration, the predictive ability of the model was evaluated using the area under the receiver operating characteristic (AUROC) curve and box plots.”
Method
Evaluating the clinical utility of an easily applicable prediction model of suicide attempts, newly developed and validated with a general community sample of adults
Miché, M., Strippoli, M. F., Preisig, M., & Lieb, R. (2024) · BMC Psychiatry · doi:10.1186/s12888-024-05647-w · Cited by 3
Relative performance evaluation of four bleeding risk scores in atrial fibrillation patients. What does the new DOAC score provide?
Abu-Assi, E., Castro, A. L., Fernández, M. C., Bermúdez, I. G., & Roubín, S. R. (2024) · International Journal of Cardiology · doi:10.1016/j.ijcard.2024.132018 · Cited by 3
Nomogram for Predicting Risk of Digestive Carcinoma Among Patients with Type 2 Diabetes
Feng, L.-H., Bu, K., Ren, S., Yang, Z., Bixun, L., & Deng, C.-E. (2020) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s251063 · Cited by 3
Biomarker Evaluation and Clinical Development
Vickers, A. J., & Assel, M. J. (2020) · Société Internationale d’Urologie Journal · doi:10.48083/zcjs3811 · Cited by 3
Clinical Prediction Models in Epidemiological Studies: Lessons from the Application of QRISK3 to UK Biobank Data
Parsons, R. E., Colopy, G. W., Clifton, D. A., & Clifton, L. (2022) · Journal of Data Science · doi:10.6339/22-jds1037 · Cited by 3
Machine Learning Does Not Improve Humeral Torsion Prediction Compared to Regression in Baseball Pitchers
Bullock, G. S., Thigpen, C. A., Collins, P. G. S., Arden, N., Noonan, T. K., Kissenberth, M. J., & Shanley, E. (2022) · International Journal of Sports Physical Therapy · doi:10.26603/001c.32380 · Cited by 3
Data-driven approaches to generating knowledge: Machine learning, artificial intelligence, and predictive modeling
Matheny, M. E., Lucila Ohno-machado, A., Davis, S. E., & Nemati, S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-91200-6.00031-0 · Cited by 3
Calibration of Prognostic Risk Scores
CALSTER, B. V., & Steyerberg, E. W. (2018) · Wiley eBooks · doi:10.1002/9781118445112.stat08078 · Cited by 3
Comment on: “A Machine Learning Approach to Concussion Risk Estimation Among Players Exhibiting Visible Signs in Professional Hockey”
Klemp, M., & Rein, R. (2025) · Sports Medicine · doi:10.1007/s40279-025-02211-8 · Cited by 3
Importance of Transplant Era on Post–Heart Transplant Predictive Models: A UNOS Cohort Analysis
Lescroart, M., Kransdorf, E. P., Scuppa, M. F., Patel, J. K., & Coutance, G. (2024) · Clinical Transplantation · doi:10.1111/ctr.15403 · Cited by 2
Development and validation of a prognosis risk score model for neonatal mortality in the Amhara region, Ethiopia. A prospective cohort study
Asaye, M. M., Matebe, Y. H., Lindgren, H. E., Erlandsson, K., & Gelaye, K. A. (2024) · Global Health Action · doi:10.1080/16549716.2024.2392354 · Cited by 2
Predictors of breakthrough invasive fungal infections (BIFI) in pediatric acute leukemia: a retrospective analysis and predictive model development
Li, Y., Qu, L., Wang, J., Chen, P., Jiang, A.-S., & Liu, H. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1488514 · Cited by 2
Development and validation of a risk score to predict neonatal mortality among NICU admissions in Southern Ethiopia: a retrospective follow-up study
Adane, S. M., Teshale, A. B., Belay, D. G., & Nigatu, S. G. (2025) · Frontiers in Pediatrics · doi:10.3389/fped.2025.1496019 · Cited by 3
Development and evaluation of machine learning algorithms for the prediction of opioid-related deaths among UK patients with non-cancer pain
Benitez-Aurioles, J., Medina, C. R. R., Jenkins, D. A., Peek, N. B., & Jani, M. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001190 · Cited by 4
Prognostic prediction models for psoriatic arthritis in psoriasis: A systematic review
Khoda, F., Khan, Y., Haw, W. Y., Nahar, S., Hodkinson, C., Chinoy, H., Warren, R. B., Martin, G. P., & Yiu, Z. Z. N. (2026) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.70561 · Cited by 2
A Closer Look at Probability Calibration of Knowledge Graph Embedding
Zhu, R., Wang, F., Bundy, A., Li, X., Nuamah, K., Xu, L., Mauceri, S., & Pan, J. Z. (2022) · International Joint Conference on Knowledge Graphs (IJCKG) · doi:10.1145/3579051.3579072 · Cited by 2
Optimal cut-off values and diagnostic significance of clinical laboratory indicators in newly diagnosed multiple myeloma
Li, M., Wu, H., Shou, C., Ye, P., Song, X., Wang, Y., Chen, Y., & Tong, X. (2024) · Discover Oncology · doi:10.1007/s12672-024-01254-z · Cited by 2
The mC2HEST Score for Incident Atrial Fibrillation
Li, Y., Li, Q., Wang, L., Zhang, T., Gao, H., Pastori, D., Liang, Z., Lip, G. Y. H., & Wang, Y. (2025) · JACC Advances · doi:10.1016/j.jacadv.2024.101521 · Cited by 4
Impact of pectoral muscle removal on deep-learning-based breast cancer risk prediction
Klaneček, Ž., Wang, Y., Wagner, T., Cockmartin, L., Marshall, N., Schott, B., Deatsch, A. E., Studen, A., Jarm, K., Krajc, M., et al. (2025) · Physics in Medicine and Biology · doi:10.1088/1361-6560/adb367 · Cited by 2
A calibrated fully interpretable fuzzy classifier via Vapnik–Chervonenkis-dimension minimization learning
Li, Y., Chung, F.-L., Nojima, Y., & Wang, S. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122142 · Cited by 2
Diagnostic tests for ovarian cancer in premenopausal women with non-specific symptoms (ROCkeTS): prospective, multicentre, cohort study
Sundar, S. S., Agarwal, R., Scandrett, K., Davenport, C. F., CALSTER, B. V., Johnson, S., Sengupta, P., Vikram, R., Kwong, F. L., Mallett, S., et al. (2026) · BMJ · doi:10.1136/bmj-2024-083912 · Cited by 2
Update to living systematic review
Wynants, L., CALSTER, B. V., Collins, G., Riley, R. D., Heinze, G., Schuit, E., Albu, E., Arshi, B., Bellou, V., Bonten, M. J. M., et al. (2020) · BMJ · doi:10.1136/bmj.m2204 · Cited by 3
Re-calibrating Machine Learning Models Using Confidence Interval Bounds
Campagner, A., Famiglini, L., & Cabitza, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13448-7_11 · Cited by 2
A methodology to assess the population size and estimate the needed resources for new licensed medications by combining clinical and administrative databases: The example of glycated haemoglobin in type 2 diabetes
Lapi, F., Bianchini, E., Marconi, E., Medea, G., Piccinni, C. P., Maggioni, A. P., Dondi, L., Pedrini, A., Martini, N., & Cricelli, C. A. (2023) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5641 · Cited by 2
Predicting EGFR Mutation in Lung Adenocarcinoma: Development and Validation of the EGFR Mutation Predictive Score (EMPS) in Bali, Indonesia
Njoto, E. N., Kusumawardani, I. A. J. D., & Rai, I. B. N. (2023) · Asian Pacific Journal of Cancer Prevention · doi:10.31557/apjcp.2023.24.8.2903 · Cited by 2
How many longitudinal covariate measurements are needed for risk prediction?
Reinikainen, J., Karvanen, J., & Tolonen, H. K. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.022 · Cited by 2
Retraction Note: The diagnostic nomogram of platelet-based score models for hepatic alveolar echinococcosis and atypical liver cancer
Du, Q., Wang, Y., Guan, S., Hu, C., Li, M., Zhou, L., Zhang, M., Chen, Y., Mei, X., Sun, J., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-73758-x · Cited by 2
Retracted
Benefit‐harm ratio of the diagnostic workup in patients with prostate cancer of Gleason score from 9 to 10
Ferraro, S., Rossi, R. S., & Biganzoli, E. M. (2021) · Cancer · doi:10.1002/cncr.33811 · Cited by 2
Beyond Discrimination: A Call for Comprehensive Assessment of Clinical Prediction Models in Inflammatory Bowel Disease
Alsoud, D., & CALSTER, B. V. (2024) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izae048 · Cited by 2
Predicting Early Hospital Discharge Following Revision Total Hip Arthroplasty: An Analysis of a Large National Database Using Machine Learning
Yeramosu, T., Farrar, J. M., Malik, A., Satpathy, J., Golladay, G. J., & Patel, N. K. (2024) · The Journal of Arthroplasty · doi:10.1016/j.arth.2024.12.006 · Cited by 1
Secondary Prevention of AFAIS: Deploying Traditional Regression, Machine Learning, and Deep Learning Models to Validate and Update CHA2DS2-VASc for 90-Day Recurrence
Simon, J. E., Kraiński, Ł., Karlinski, M. A., & Niewada, M. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14207327 · Cited by 1
Comparative validation of PREDICT versions 3.1 and 2.2 for overall survival in the Dutch breast cancer population
Vreven, L. W. A., Verheul, E. M., van Maaren, M. C., Doornkamp, F., Schipper, R., Siesling, S., Pharoah, P. D. P., Tjan‐Heijnen, V. C. G., & Voogd, A. C. (2025) · The Breast · doi:10.1016/j.breast.2025.104681 · Cited by 1
Development and interpretation prediction model for depressive symptoms in patients with chronic kidney disease: a cross-sectional study based on NHANES
Cao, F., Li, E., Ai, F., Wen, H., Tang, P., & Guo, B. (2026) · Psychology Health & Medicine · doi:10.1080/13548506.2025.2611359 · Cited by 1
Development and internal validation of a nomogram to predict in-hospital mortality in patients with cirrhosis and acute kidney injury receiving continuous renal replacement therapy: A retrospective cohort study
Liao, T., Su, T., Yang, L., Zhou, X., Feng, L.-H., & Yang, Z. (2026) · Journal of International Medical Research · doi:10.1177/03000605261421028 · Cited by 1
Personalised treatment in biologically naive patients with Crohn’s disease: a systematic review of the last 10 years
Buhl, E., Kraaer, M. T., Kjeldsen, J., & Ainsworth, M. A. (2026) · Scandinavian Journal of Gastroenterology · doi:10.1080/00365521.2026.2645095 · Cited by 1
Application of contrast-enhanced CT-based radiomics with machine learning to predict WHO/ISUP grading in clear cell renal cell carcinoma
Li, X., Song, S., Ge, W., Wang, H., Liu, W., Qi, X., & Zhao, Y. (2026) · BMC Cancer · doi:10.1186/s12885-026-15794-w · Cited by 1
Peer review: historical evolution and reviewers’ learning
Gómez, M. P. A., & Sardanelli, F. (2026) · Insights into Imaging · doi:10.1186/s13244-026-02302-8 · Cited by 1
Multidomain prediction of education-stratified MoCA-defined mild cognitive impairment in community-dwelling older adults in urban China
Wu, Z., Zhang, F., & Zhao, F. (2026) · BMC Geriatrics · doi:10.1186/s12877-026-07656-8 · Cited by 1
Explainable Predictions of Renal Cell Carcinoma with Interpretable Tree Ensembles from Contrast-enhanced CT Images
Han, F., Liao, S., Wu, R., Liu, S., Zhao, Y., & Xie, Y. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534147 · Cited by 1
“Although such models with better discrimination and calibration performance would theoretically be a better guide to clinical management [16-18], statistical measures cannot inform us whether the risk model improves clinical decision making.”
Background

Authors: Van Calster, B., & Vickers, A. J.
Affiliations: KU Leuven, Department of Development and Regeneration, Leuven, Belgium (BVC)
Memorial Sloan Kettering Cancer Center (US)
ORCID: Vickers, Andrew J.
Venue: Medical Decision Making
Publisher: SAGE Publications
Published: 2014-08-25
Volume: 35
Issue: 2
Pages: 162-169
ISSN: 0272-989X (ISSN-L)
1552-681X
DOI: 10.1177/0272989x14547233
OpenAlex: W2054193584
Cited by: 371
References: 29
Citations by year: 2026: 44 · 2025: 39 · 2024: 48 · 2023: 40 · 2022: 57 · 2021: 40 · 2020: 31 · 2019: 29 · 2018: 10 · 2017: 19 · 2016: 7 · 2015: 6 · 2014: 1
FWCI: 11.61
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Medical Decision Making (publishedVersion)
PubMed (publishedVersion)
Lirias (submittedVersion, other-oa)
RePEc: Research Papers in Economics (submittedVersion)
Funders: Fonds Wetenschappelijk Onderzoek
Vlaamse regering
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: risk prediction model, model calibration, net benefit, clinical utility, calibration error, overfitting, underfitting, risk overestimation, risk underestimation, risk thresholds, model discrimination, testicular cancer diagnosis
MeSH terms: Computer Simulation, Decision Support Techniques, Humans, Male, Models, Theoretical, Norway, Predictive Value of Tests, ROC Curve, Testicular Neoplasms, Bias, Linear Models, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks