Cited by 371Statistical Methods in EpidemiologyClinical Reasoning and Diagnostic SkillsMeta-analysis and systematic reviews

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.

Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance

Open access

Van Calster, Ben, Vickers, Andrew J.·2015·Medical Decision Making

DOI 10.1177/0272989x14547233

Open this paper in CoChat — free account Open PDF

Abstract

Decision-analytic measures to assess clinical utility of prediction models and diagnostic tests incorporate the relative clinical consequences of true and false positives without the need for external information such as monetary costs. Net Benefit is a commonly used metric that weights the relative consequences in terms of the risk threshold at which a patient would opt for treatment. Theoretical results demonstrate that clinical utility is affected by a model';s calibration, the extent to which estimated risks correspond to observed event rates. We analyzed the effects of different types of miscalibration on Net Benefit and investigated whether and under what circumstances miscalibration can make a model clinically harmful. Clinical harm is defined as a lower Net Benefit compared with classifying all patients as positive or negative by default. We used simulated data to investigate the effect of overestimation, underestimation, overfitting (estimated risks too extreme), and underfitting (estimated risks too close to baseline risk) on Net Benefit for different choices of the risk threshold. In accordance with theory, we observed that miscalibration always reduced Net Benefit. Harm was sometimes observed when models underestimated risk at a threshold below the event rate (as in underestimation and overfitting) or overestimated risk at a threshold above event rate (as in overestimation and overfitting). Underfitting never resulted in a harmful model. The impact of miscalibration decreased with increasing discrimination. Net Benefit was less sensitive to miscalibration for risk thresholds close to the event rate than for other thresholds. We illustrate these findings with examples from the literature and with a case study on testicular cancer diagnosis. Our findings strengthen the importance of obtaining calibrated risk models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1177/0272989x14547233

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers
Pencina, M. J., D’Agostino, R. B., & Steyerberg, E. W. (2010) · Statistics in Medicine · doi:10.1002/sim.4085 · Cited by 2,483
Clinical Prediction Models
Steyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491
Criteria for Evaluation of Novel Markers of Cardiovascular Risk
Hlatky, M. A., Greenland, P., Arnett, D. K., Ballantyne, C. M., Criqui, M. H., Elkind, M. S. V., Go, A. S., Harrell, F. E., Hong, Y., Howard, B. V., et al. (2009) · Circulation · doi:10.1161/circulationaha.109.192278 · Cited by 1,146
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600
Two further applications of a model for binary regression
COX, D. R. (1958) · Biometrika · doi:10.1093/biomet/45.3-4.562 · Cited by 556
Therapeutic decision making: a cost-benefit analysis
Pauker, S. G., & Kassirer, J. P. (1975) · New England Journal of Medicine · doi:10.1056/nejm197507312930505 · Cited by 507
Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers
Austin, P. C., & Steyerberg, E. W. (2014) · Statistics in Medicine · doi:10.1002/sim.5941 · Cited by 362

Flagged first, then by citations

Show all 29 references

Research works printed but not linked: 1

Vickers AJ, 2010, Semin Oncol, 76, 1298

Sorted by citations

Items 101–200 of 371

A model to predict risk of blood transfusion after gynecologic surgery
Stanhiser, J., Chagin, K. M., & Jelovsek, J. E. (2017) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2017.01.004 · Cited by 21
Development and validation of an ultrasound-based nomogram for preoperative prediction of cervical central lymph node metastasis in papillary thyroid carcinoma
Huang, C., Cong, S., Liang, T., Feng, Z., Gan, K., Zhou, R., Guo, Y., Luo, S., Liang, K., & Wang, Q. (2020) · Gland Surgery · doi:10.21037/gs-20-75 · Cited by 21
Prediction models for self-harm and suicide: a systematic review and critical appraisal
Seyedsalehi, A., Bailey, J. P. M., Ogonah, M. G. T., Fanshawe, T. R., & Fazel, S. (2025) · BMC Medicine · doi:10.1186/s12916-025-04367-6 · Cited by 25
Impact of correlation of predictors on discrimination of risk models in development and external populations
Kundu, S., Mazumdar, M., & Ferket, B. S. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0345-1 · Cited by 21
A Pre-operative Nomogram for Prediction of Lymph Node Metastasis in Bladder Urothelial Carcinoma
Lu, X., Wang, Y., Jiang, L., Gao, J., Zhu, Y., Hu, W., Wang, J., Ruan, X., Xu, Z., Meng, X., et al. (2019) · Frontiers in Oncology · doi:10.3389/fonc.2019.00488 · Cited by 20
Estimating the decision curve and its precision from three study designs
Pfeiffer, R. M., & Gail, M. H. (2019) · Biometrical Journal · doi:10.1002/bimj.201800240 · Cited by 20
“Van Calster and Vickers (2015) showed that miscalibrated models lead to smaller NB than the correct model, but they did not investigate the estimation of NB under a miscalibrated model.”
Background
Development and Validation of Clinical-Radiomics Nomogram for Preoperative Prediction of Central Lymph Node Metastasis in Papillary Thyroid Carcinoma
Feng, J., Liu, S., Qi, G.-F., Ye, J., Hong, L., Wu, W.-X., & Jiang, Y. (2024) · Academic Radiology · doi:10.1016/j.acra.2023.12.008 · Cited by 20
What is Quality End‐of‐Life Care for Patients With Heart Failure? A Qualitative Study With Physicians
Hutchinson, R. N., Gutheil, C. M., Wessler, B. S., Prevatt, H., Sawyer, D. B., & Han, P. K. J. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.120.016505 · Cited by 20
An Explainable Artificial Intelligence Text Classifier for Suicidality Prediction in Youth Crisis Text Line Users: Development and Validation Study
Thomas, J., Lucht, A., Segler, J., Wundrack, R., Miché, M., Lieb, R., Kuchinke, L., & Meinlschmidt, G. (2024) · JMIR Public Health and Surveillance · doi:10.2196/63809 · Cited by 22
The Role of Artificial Intelligence in Chronic Liver Diseases and Liver Transplantation
Spann, A., Strauss, A. T., Davis, S. E., & Bhat, M. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.05.012 · Cited by 21
A Multivariable Predictive Model for Success of External Cephalic Version
Dahl, C. M., Zhang, Y., Ong, J. X., Yeh, C., Son, M., Miller, E. S., Roy, A., & Grobman, W. A. (2021) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000004518 · Cited by 19
Statistical methods for validation of predictive models
Diniz, M. A. (2022) · Journal of Nuclear Cardiology · doi:10.1007/s12350-022-02994-7 · Cited by 22
Discordance and Performance of the ARC-HBR and PRECISE-DAPT High Bleeding Risk Definitions After Coronary Stenting
Lim, C.-E., Simonsson, M., Pasternak, B., Jernberg, T., Edgren, G., & Ueda, P. (2025) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2024.10.032 · Cited by 19
Artificial Intelligence in Gerontology: Data-Driven Health Management and Precision Medicine
Zhang, S., Wu, L., Zhao, Z., Fernández, J. R., & Chen, M. (2024) · Advances in Gerontology · doi:10.1134/s2079057024600691 · Cited by 20
Re‐CHARGE‐AF: Recalibration of the CHARGE‐AF Model for Atrial Fibrillation Risk Prediction in Patients With Acute Stroke
Ashburner, J. M., Wang, X., Li, X., Khurshid, S. S., Ko, D., Lipsanopoulos, A. T., Lee, P. R., Carmichael, T., Turner, A. C., Jackson, C. R., et al. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.121.022363 · Cited by 18
HINT: Integration Testing for AI-based features with Humans in the Loop
Chen, Q. Z., Schnabel, T., Nushi, B., & Amershi, S. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511141 · Cited by 24
A Review on Predictive Systems and Data Models for COVID-19
Khan, F. N., Khanam, A., Ramlal, A., & Ahmad, S. (2020) · Studies in computational intelligence · doi:10.1007/978-981-15-8534-0_7 · Cited by 18
Development and validation of early warning score systems for COVID‐19 patients
Youssef, A., Kouchaki, S., Shamout, F. E., Armstrong, J., El-Bouri, R., Taylor, T. K. F., Birrenkott, D. A., Vasey, B., Soltan, A. A. S., Zhu, T., et al. (2021) · Healthcare Technology Letters · doi:10.1049/htl2.12009 · Cited by 19
Managing the impact of inter-method bias of prostate specific antigen assays on biopsy referral: the key to move towards precision health in prostate cancer management
Ferraro, S., Biganzoli, G., Bussetti, M., Castaldi, S., Biganzoli, E. M., & Plebani, M. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0874 · Cited by 18
Assessment of gastric emptying of maltodextrin, coffee with milk and orange juice during labour at term using point of care ultrasound: a non‐inferiority randomised clinical trial
Nascimento, A. C., Govêia, C. S., Guimarães, G. M. N., Filho, R. P. L., Ladeira, L. C. A., & da Silva, H. B. G. (2019) · Anaesthesia · doi:10.1111/anae.14671 · Cited by 18
Decision Curves, Calibration, and Subgroups
Vickers, A. J., CALSTER, B. V., & Steyerberg, E. W. (2017) · Journal of Clinical Oncology · doi:10.1200/jco.2016.69.1576 · Cited by 18
Understanding the Value of Individualized Information: The Impact of Poor Calibration or Discrimination in Outcome Prediction Models
Olchanski, N., Cohen, J. T., Neumann, P. J., Wong, J. B., & Kent, D. M. (2017) · Medical Decision Making · doi:10.1177/0272989x17704855 · Cited by 18
“We have verified what others have observed for well-calibrated models: the EVIC increases as discriminatory power increases.(9) As model discrimination improves, the risk heterogeneity increases, and, consequently, there is also greater heterogeneity in individual ICER estimates.”
MethodResultInfluential
Development of machine-learning algorithms for 90-day and one-year mortality prediction in the elderly with femoral neck fractures based on the HEALTH and FAITH trials
Dijkstra, H., Oosterhoff, J. H. F., van de Kuit, A., IJpma, F. F. A., Schwab, J. H., Poolman, R. W., Sprague, S., Bzovsky, S., Bhandari, M., Swiontkowski, M. F., et al. (2023) · Bone & Joint Open · doi:10.1302/2633-1462.43.bjo-2022-0162.r1 · Cited by 17
Prognostic Models in Nephrology: Where Do We Stand and Where Do We Go from Here? Mapping Out the Evidence in a Scoping Review
Milders, J., Ramspek, C. L., Janse, R. J., Bos, W. J. W., Rotmans, J. I., Dekker, F. W., & van Diepen, M. (2023) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000000000285 · Cited by 20
Nomogram based on preoperative CT imaging predicts the EGFR mutation status in lung adenocarcinoma
Zhang, G., Zhang, J., Cao, Y., Zhao, Z., Li, S., Deng, L., & Zhou, J. (2020) · Translational Oncology · doi:10.1016/j.tranon.2020.100954 · Cited by 16
Short-term risk prediction after major lower limb amputation: PERCEIVE study
Gwilym, B. L., Pallmann, P., Waldron, C.-A., Thomas‐Jones, E., Milosevic, S., Brookes‐Howell, L., Harris, D., Massey, I., Burton, J., Stewart, P., et al. (2022) · British journal of surgery · doi:10.1093/bjs/znac309 · Cited by 16
Reliable prediction of difficult airway for tracheal intubation from patient preoperative photographs by machine learning methods
García-García, F., Lee, D., Mendoza-Garcés, F. J., & García‐Gutiérrez, S. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108118 · Cited by 16
Simpson's paradox in the integrated discrimination improvement
Chipman, J., & Braun, D. (2016) · Statistics in Medicine · doi:10.1002/sim.6862 · Cited by 16
Establishment and validation of a prediction model for vaginal delivery after cesarean and its pregnancy outcomes—Based on a prospective study
Lin, J., Hou, Y., Ke, Y., Zeng, W., & Gu, W. (2019) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2019.09.015 · Cited by 16
Development and Validation of a Nomogram Based on Motoric Cognitive Risk Syndrome for Cognitive Impairment
Liu, Y., Wei, K., Cao, X., Jiang, L., Gu, N., Feng, L., & Li, C. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.618833 · Cited by 16
Estimating Calibrated Individualized Survival Curves with Deep Learning
Kamran, F., & Wiens, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16098 · Cited by 17
Clinical Prediction Models for Valvular Heart Disease
Wessler, B. S., Lundquist, C. M., Koethe, B. C., Park, J. G., Brown, K., Williamson, T., Ajlan, M., Natto, Z. S., Lutz, J. S., Paulus, J. K., et al. (2019) · Journal of the American Heart Association · doi:10.1161/jaha.119.011972 · Cited by 18
Risk Prediction for Individuals
CALSTER, B. V., Steyerberg, E. W., & Harrell, F. H. (2015) · JAMA · doi:10.1001/jama.2015.12215 · Cited by 16
External validation and clinical usefulness of three commonly used cardiovascular risk prediction scores in an Emirati population: a retrospective longitudinal cohort study
Al-Shamsi, S., Govender, R. D., & King, J. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040680 · Cited by 15
Does the SORG Orthopaedic Research Group Hip Fracture Delirium Algorithm Perform Well on an Independent Intercontinental Cohort of Patients With Hip Fractures Who Are 60 Years or Older?
Oosterhoff, J. H. F., Oberai, T., Karhade, A. V., Doornberg, J. N., Kerkhoffs, G. M. M. J., Jaarsma, R. L., Schwab, J. H., & Heng, M. M. (2022) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002246 · Cited by 16
Polygenic risk scores for the prediction of common cancers in East Asians: A population-based prospective cohort study
Ho, P. J., Tan, I. B., Chong, D. Q., Khor, C. C., Yuan, J., Koh, W., Dorajoo, R., & Li, J. (2023) · eLife · doi:10.7554/elife.82608 · Cited by 15
A nomogram for the prediction of cerebrovascular disease among patients with brain necrosis after radiotherapy for nasopharyngeal carcinoma
Cai, J., Cheng, J., Li, H., Lin, W., Li, Y., Zhuo, X., Huang, X., Simone, C. B., Aronow, W. S., Chow, E., et al. (2018) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2018.11.008 · Cited by 15
A clinical prediction nomogram to assess risk of colorectal cancer among patients with type 2 diabetes
Feng, L.-H., Su, T., Bu, K., Ren, S., Yang, Z., Deng, C.-E., Bixun, L., & Wei, W. (2020) · Scientific Reports · doi:10.1038/s41598-020-71456-2 · Cited by 15
Post hoc calibration of medical segmentation models
Rousseau, A.-J., Becker, T., Appeltans, S., Blaschko, M. B., & Valkenborg, D. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-06587-0 · Cited by 16
A Simple Logistic Regression Model for Predicting the Likelihood of Recurrence of Atrial Fibrillation Within 1 Year After Initial Radio-Frequency Catheter Ablation Therapy
Jia, S., Mou, H., Wu, Y., Lin, W., Zeng, Y., Chen, Y., Chen, Y., Zhang, Q., Wang, W., Feng, C., et al. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.819341 · Cited by 14
Clinical Utility of Melanoma Sentinel Lymph Node Biopsy Nomograms
Drebin, H. M., Hosein, S., Kurtansky, N. R., Nadelmann, E. R., Moy, A. P., Ariyan, C. E., Bello, D. M., Brady, M. S., Coit, D. G., Marchetti, M. A., et al. (2023) · Journal of the American College of Surgeons · doi:10.1097/xcs.0000000000000886 · Cited by 14
Decision Curve Analysis of In-Hospital Mortality Prediction Models: The Relative Value of Pre- and Intraoperative Data For Decision-Making
Huber, M., Bello, C., Schober, P. R., Filipovic, M. G., & Luedi, M. M. (2024) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006874 · Cited by 14
Risk Projection for Time-to-Event Outcome Leveraging Summary Statistics With Source Individual-Level Data
Zheng, J., Zheng, Y., & Hsu, L.-I. (2021) · Journal of the American Statistical Association · doi:10.1080/01621459.2021.1895810 · Cited by 14
“Therefore, model recalibration is critically needed for accurate risk projection in the target population, but it is frequently under-appreciated (Van Calster and Vickers 2015).”
Background
Localization-adjusted diagnostic performance and assistance effect of a computer-aided detection system for pneumothorax and consolidation
Lee, S. Y., Ha, S., Jeon, M. G., Li, H., Choi, H., Kim, H. P., Choi, Y. R., Hoseok, I., Jeong, Y. J., Park, Y. H., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00658-x · Cited by 13
Predicting New-onset Exertional and Resting Hypoxemia in Fibrotic Interstitial Lung Disease
Saleem, F., Ryerson, C. J., Sarma, N., Johannson, K. A., Marcoux, V. S., Fisher, J. H., Assayag, D., Manganas, H., Khalil, N., Morisset, J., et al. (2023) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202303-208oc · Cited by 13
Prognostic Models for Chronic Low Back Pain Outcomes in Primary Care Are at High Risk of Bias and Lack Validation—High-Quality Studies Are Needed: A Systematic Review
Fu, Y., Feller, D., Koes, B. W., & Chiarotto, A. (2024) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2024.12081 · Cited by 16
CT-based whole lung radiomics nomogram for identification of PRISm from non-COPD subjects
Zhou, T., Guan, Y., Lin, X., Zhou, X., Mao, L., Ma, Y., Fan, B., Li, J., Liu, S., & Li, F. (2024) · Respiratory Research · doi:10.1186/s12931-024-02964-2 · Cited by 13
Applying Risk Prediction Models to Optimize Lung Cancer Screening: Current Knowledge, Challenges, and Future Directions
Sakoda, L. C., Henderson, L. M., Caverly, T. J., Wernli, K. J., & Katki, H. A. (2017) · Current Epidemiology Reports · doi:10.1007/s40471-017-0126-8 · Cited by 13
Leveraging Decision Curve Analysis to Improve Clinical Application of Surgical Risk Calculators
Dadashzadeh, E. R., Bou‐Samra, P., Huckaby, L. V., Nebbia, G., Handzel, R. M., Varley, P. R., Wu, S., & Tsung, A. (2021) · Journal of Surgical Research · doi:10.1016/j.jss.2020.11.059 · Cited by 13
Development and validation of a nomogram to predict in-hospital cardiac arrest among patients admitted with acute coronary syndrome
Wu, T., Yang, D., Li, H., & Guo, Y. (2021) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.05.082 · Cited by 13
Cost Effectiveness of Daclatasvir Plus Asunaprevir Therapy for Chinese Patients with Chronic Hepatitis C Virus Genotype 1b
Liu, Y., Wang, Z., Gai, R. T., Lin, H., & Wu, B. Y. (2018) · Clinical Drug Investigation · doi:10.1007/s40261-018-0621-9 · Cited by 13
A diagnostic prediction model for hypertension in Han and Yugur population from the China National Health Survey (CNHS)
Yu, C., Ren, X., Cui, Z. Q., Pan, L., Zhao, H., Sun, J., Wang, Y., Chang, L., Cao, Y., He, H., et al. (2022) · Chinese Medical Journal · doi:10.1097/cm9.0000000000001989 · Cited by 13
Patients With Femoral Neck Fractures Are at Risk for Conversion to Arthroplasty After Internal Fixation: A Machine‐learning Algorithm
van de Kuit, A., Oosterhoff, J. H. F., Dijkstra, H., Sprague, S., Bzovsky, S., Bhandari, M., Swiontkowski, M. F., Schemitsch, E. H., IJpma, F. F. A., Poolman, R. W., et al. (2022) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002283 · Cited by 12
Artificial Intelligence in Nuclear Cardiology
Miller, R. J. H. (2023) · Cardiology Clinics · doi:10.1016/j.ccl.2023.01.004 · Cited by 12
Decision curve analysis based on summary data
Hozo, I., Guyatt, G. H., & Djulbegović, B. (2023) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13945 · Cited by 13
Machine Learning Did Not Outperform Conventional Competing Risk Modeling to Predict Revision Arthroplasty
Oosterhoff, J. H. F., de Hond, A., Peters, R. M., van Steenbergen, L. N., Sorel, J. C., Zijlstra, W. P., Poolman, R. W., Ring, D., Jutte, P. C., Kerkhoffs, G. M. M. J., et al. (2024) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000003018 · Cited by 12
Application of a nomogram to radiomics labels in the treatment prediction scheme for lumbar disc herniation
Yu, G., Yang, W., Zhang, J., Zhang, Q., Zhou, J., Hong, Y., Luo, J., Shi, Q., Yang, Z., Zhang, K., et al. (2022) · BMC Medical Imaging · doi:10.1186/s12880-022-00778-6 · Cited by 12
Predicting bacterial infections among pediatric cancer patients with febrile neutropenia: External validation of the PICNICC model
Ojha, R. P., Asdahl, P. H., Steyerberg, E. W., & Schrøeder, H. (2017) · Pediatric Blood & Cancer · doi:10.1002/pbc.26935 · Cited by 12
A Bayesian method to estimate the optimal threshold of a marker used to select patients' treatment
Blangero, Y., Rabilloud, M., Écochard, R., & Subtil, F. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218821394 · Cited by 12
Statistical methods for building better biomarkers of chronic kidney disease
Pencina, M. J., Parikh, C. R., Kimmel, P. L., Cook, N. R., Coresh, J., Feldman, H. I., Foulkes, A. S., Gimotty, P. A., Hsu, C., Lemley, K. V., et al. (2019) · Statistics in Medicine · doi:10.1002/sim.8091 · Cited by 12
Calibrating variant-scoring methods for clinical decision making
Benevenuta, S., Capriotti, E., & Fariselli, P. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btaa943 · Cited by 11
Effectiveness and cost effectiveness of pharmacological thromboprophylaxis for medical inpatients: decision analysis modelling study
Davis, S., Goodacre, S. W., Horner, D. E., Pandor, A., Holland, M., de Wit, K., Hunt, B. J., & Griffin, X. L. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2022-000408 · Cited by 10
Development and internal validation of machine-learning models for predicting survival in patients who underwent surgery for spinal metastases
Santipas, B., Veerakanjana, K., Ittichaiwong, P., Chavalparit, P., Wilartratsami, S., & Luksanapruksa, P. (2024) · Asian Spine Journal · doi:10.31616/asj.2023.0314 · Cited by 11
Multitask machine learning-based tumor-associated collagen signatures predict peritoneal recurrence and disease-free survival in gastric cancer
Fu, M., Lin, Y., Yang, J., Cheng, J., Lin, L., Wang, G., Long, C., Xu, S., Lu, J., Li, G., et al. (2024) · Gastric Cancer · doi:10.1007/s10120-024-01551-0 · Cited by 13
The relative data hungriness of unpenalized and penalized logistic regression and ensemble-based machine learning methods: the case of calibration
Austin, P. C., Lee, D. S., & Wang, B. (2024) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00179-z · Cited by 11
Stacked Ensemble of Extremely Interpretable Takagi-Sugeno–Kang Fuzzy Classifiers for High-Dimensional Data
Li, Y., Zhou, E., Vong, C., & Wang, S. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3516857 · Cited by 12
Prediction of iatrogenic preterm birth in patients with scarred uterus: a retrospective cohort study in Northeast China
Zhang, L., Li, H., Li, J., Hou, Y., Xu, B., Li, N., Yang, T., Liu, C., & Qiao, C. (2020) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-020-03165-7 · Cited by 11
An Artificial Neural Network Model for Pediatric Mortality Prediction in Two Tertiary Pediatric Intensive Care Units in South Africa. A Development Study
Pienaar, M. A., Sempa, J. B., Luwes, N. J., & Solomon, L. J. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.797080 · Cited by 13
A Unifying Theory of Distance from Calibration
Blasiok, J., Gopalan, P., Hu, L., & Nakkiran, P. (2022) · Symposium on the Theory of Computing · doi:10.1145/3564246.3585182 · Cited by 71
A model to predict risk of postpartum infection after Caesarean delivery*
Moulton, L. J., Jelovsek, J. E., Lachiewicz, M. P., Chagin, K. M., & Goje, O. (2017) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.1080/14767058.2017.1344632 · Cited by 11
Dissecting contributions of individual systemic inflammatory response syndrome criteria from a prospective algorithm to the prediction and diagnosis of sepsis in a polytrauma cohort
Schefzik, R., Hahn, B., & Schneider‐Lindner, V. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1227031 · Cited by 10
An Attitude Survey and Assessment of the Feasibility, Acceptability, and Usability of a Traumatic Brain Injury Decision Support Tool in Uganda
Elahi, C., Spears, C. A., Williams, S., Dunn, T., Najjuma, J. N., Staton, C. A., Vissoci, J. R. N., Fuller, A. T., Kitya, D., & Haglund, M. M. (2020) · World Neurosurgery · doi:10.1016/j.wneu.2020.04.193 · Cited by 11
Prognostic Value of Combined Detection of Preoperative Albumin-to-Fibrinogen Ratio and Neutrophil-to-Lymphocyte Ratio in Operable Esophageal Squamous Cell Carcinoma Patients without Neoadjuvant Therapy
Zheng, Z., Lin, D., Chen, Q., Zheng, B., Liang, M., Chen, C., & Zheng, W. (2021) · Cancer Management and Research · doi:10.2147/cmar.s296266 · Cited by 10
Cost-effectiveness of novel regimens for Chinese patients with chronic hepatitis C
Wu, B. Y., Wang, Z., & Xie, Q. (2018) · Current Medical Research and Opinion · doi:10.1080/03007995.2018.1546678 · Cited by 10
Can We Geographically Validate a Natural Language Processing Algorithm for Automated Detection of Incidental Durotomy Across Three Independent Cohorts From Two Continents?
Karhade, A. V., Oosterhoff, J. H. F., Groot, O. Q., Agaronnik, N. D., Ehresman, J. S., Bongers, M. E. R., Jaarsma, R. L., Poonnoose, S. I., Sciubba, D. M., Tobert, D. G., et al. (2022) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002200 · Cited by 10
Against reflexive recalibration: towards a causal framework for addressing miscalibration
Swaminathan, A., Srivastava, U., Tu, L., López, I., Shah, N. H., & Vickers, A. J. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00184-2 · Cited by 9
Machine learning-based prediction of short- and long-term mortality for shared decision-making in older hip fracture patients: the Dutch Hip Fracture Audit algorithms in 74,396 cases
Dijkstra, H., Parsons, C. S., van Bremen, H.-E., Willems, H. C., de Hond, A., Munster, B. C. V., Doornberg, J. N., & Oosterhoff, J. H. F. (2025) · Acta Orthopaedica · doi:10.2340/17453674.2025.44248 · Cited by 10
Anticipating outcomes for patients with COVID-19 and identifying prognosis patterns
Darmon, M., & Dumas, G. (2021) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(21)00073-6 · Cited by 9
Predicting mortality in moderate-severe TBI patients without early withdrawal of life-sustaining treatments including ICU complications: The MYSTIC-score
Chang, H. Y. M., Flahive, J., Bose, A., Goostrey, K. J., Osgood, M. L., Carandang, R. A., Hall, W., & Muehlschlegel, S. (2022) · Journal of Critical Care · doi:10.1016/j.jcrc.2022.154147 · Cited by 9
A Predictive Nomogram for Red Blood Cell Transfusion in Pheochromocytoma Surgery: A Study on Improving the Preoperative Management of Pheochromocytoma
Guo, Y., You, L., Hu, H., Tong, A., Zhang, X., Yan, L., & Zhang, S. (2021) · Frontiers in Endocrinology · doi:10.3389/fendo.2021.647610 · Cited by 9
Using anti-Müllerian hormone to predict premature ovarian insufficiency: a retrospective cross-sectional study
Huang, Y., Kuang, X., Jiangzhou, H., Li, M., Yang, D., & Lai, D. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1454802 · Cited by 8
Cause-specific mortality after spousal bereavement in a Danish register-based cohort
Sloth, M. M. B., Hruza, J., Mortensen, L. H., Bhatt, S., & Katsiferis, A. (2025) · Scientific Reports · doi:10.1038/s41598-025-90657-1 · Cited by 8
Validation and update of a multivariable prediction model for the identification and management of patients at risk for hepatocellular carcinoma
Li, B., Zhao, Y., Cai, W., An-ping, M., & Li, H. (2021) · Clinical Proteomics · doi:10.1186/s12014-021-09326-w · Cited by 8
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration. Translation in to Russian
Moons, K. G. M., Altman, D. G., Reitsma, J. B., loannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, P. G. S. (2022) · Digital Diagnostics · doi:10.17816/dd110794 · Cited by 10
Five-Year Prognostic Value of DAPT Score in Older Patients undergoing Percutaneous Coronary Intervention: A Large-Sample Study in the Real World
Yan, K., Zhu, P., Tang, X., Li, Y., Li, J., Yuan, D., Yang, W., Yang, Y., Gao, R., Xu, B., et al. (2023) · Journal of Atherosclerosis and Thrombosis · doi:10.5551/jat.64212 · Cited by 8
Predicting Beta-Lactam Target Non-Attainment in ICU Patients at Treatment Initiation: Development and External Validation of Three Novel (Machine Learning) Models
Wieringa, A., Ewoldt, T. M. J., Gangapersad, R. N., Gijsen, M., Parolya, N., Kats, C. J. A. R., Spriet, I., Endeman, H., Haringman, J. J., van Hest, R. M., et al. (2023) · Antibiotics · doi:10.3390/antibiotics12121674 · Cited by 8
Development a nomogram to predict fertilisation rate of infertile males with borderline semen by using semen parameters combined with AMH and INHB
Li, J., Hu, T., Wang, Y., Fu, Y.-X., Wang, F., & Hu, R. (2021) · Andrologia · doi:10.1111/and.14182 · Cited by 8
Comparison of predictive models for cumulative live birth rate after treatment with ART
Bardet, L., Excoffier, J.-B., Salaün-Penquer, N., Ortala, M., Pasquier, M., d’Argent, E. M., & Massin, N. (2022) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2022.03.020 · Cited by 9
Effects of Radiotherapy on Survival of Esophageal Cancer Patients Receiving Immunotherapy: Propensity Score Analysis and Nomogram Construction
Nie, Y., Yao, G., Li, L., Feng, A., Zhang, W., Xu, X., Li, Q., & Yang, Z. (2022) · Cancer Management and Research · doi:10.2147/cmar.s375821 · Cited by 8
External validation of the ProVent score for prognostication of 1-year mortality of critically ill patients with prolonged mechanical ventilation: a single-centre, retrospective observational study in Austria
Dibiasi, C., Kimberger, O., Bologheanu, R., Staudinger, T., Heinz, G., Zauner, C., Sengölge, G., & Schaden, E. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-066197 · Cited by 7
Flood projections over the White Volta Basin under the shared socioeconomic pathways: an analytical hierarchical approach
Kabo-bah, A. T., Cobbina, N. A., Siabi, E. K., Siabi, S. E., & Ahiada, W. B. (2025) · Frontiers in Climate · doi:10.3389/fclim.2025.1564665 · Cited by 7
Validation and comparison of three mortality prediction scores in emergency department patients with community-acquired pneumonia
Hincapié, C., Ascuntar, J., León, A. L., & Jaimes, F. A. (2021) · Colombia medica · doi:10.25100/cm.v52i4.4287 · Cited by 7
Establishment and validation of a prediction model for self-absorption probability of chronic subdural hematoma
Tian, Y., Wang, D., Zhang, X., Wei, H., Wei, Y., An, S., Gao, C., Huang, J., Sun, J., Jiang, R., et al. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.913495 · Cited by 7
Regression-based prognostic models for functional independence after postacute brain injury rehabilitation are not transportable: a systematic review
Pommerich, U. M., Stubbs, P. W., Eggertsen, P. P., Fabricius, J., & Nielsen, J. F. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.02.009 · Cited by 7
Development and validation of a nomogram for prediction of recovery from moderate-severe xerostomia post-radiotherapy in nasopharyngeal carcinoma patients
Zhao, D., Teng, F., Meng, L., Fan, W., Luo, Y., Jiang, H.-Y., Nan-xiang, C., Zhang, X., Yu, W., Cai, B.-N., et al. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2023.109683 · Cited by 7
Development and validation of prognostic nomogram for cirrhotic patients with acute kidney injury upon ICU admission
Liao, T., Lu, Y.-Y., Su, T., Bei, L., Li, X., Yang, L., Ren, S., Huang, L., & Feng, L.-H. (2023) · Internal and Emergency Medicine · doi:10.1007/s11739-023-03436-z · Cited by 7
Identifying future high healthcare utilization in patients with multimorbidity – development and internal validation of machine learning prediction models using electronic health record data
Weil, L. I., Zwerwer, L. R., Chu, H., Verhoeff, M., Jeurissen, P., & Munster, B. C. V. (2024) · Health and Technology · doi:10.1007/s12553-024-00824-z · Cited by 7
Development of an Empirically Calibrated Model of Esophageal Squamous Cell Carcinoma in High-Risk Regions
Wang, Z., Zhang, Q., & Wu, B. (2019) · BioMed Research International · doi:10.1155/2019/2741598 · Cited by 7
“Currently, the growing consensus is that a systematic and reproducible approach for model calibration should be recommended [32].”
Background
Clinical Features and Serological Markers Risk Model Predicts Overall Survival in Patients Undergoing Breast Cancer and Bone Metastasis Surgeries
Mou, H., Wang, Z., Zhang, W., Li, G., Zhou, H., Yinwang, E., Wang, F., Sun, H., Xue, Y., Wang, Z., et al. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.693689 · Cited by 7
Identifying patients at risk of unplanned re-hospitalisation using statewide electronic health records
Brankovic, A., Rolls, D. A., Boyle, J., Niven, P., & Khanna, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-20907-z · Cited by 7

Authors: Van Calster, B., & Vickers, A. J.
Affiliations: KU Leuven, Department of Development and Regeneration, Leuven, Belgium (BVC)
Memorial Sloan Kettering Cancer Center (US)
ORCID: Vickers, Andrew J.
Venue: Medical Decision Making
Publisher: SAGE Publications
Published: 2014-08-25
Volume: 35
Issue: 2
Pages: 162-169
ISSN: 0272-989X (ISSN-L)
1552-681X
DOI: 10.1177/0272989x14547233
OpenAlex: W2054193584
Cited by: 371
References: 29
Citations by year: 2026: 44 · 2025: 39 · 2024: 48 · 2023: 40 · 2022: 57 · 2021: 40 · 2020: 31 · 2019: 29 · 2018: 10 · 2017: 19 · 2016: 7 · 2015: 6 · 2014: 1
FWCI: 11.61
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Medical Decision Making (publishedVersion)
PubMed (publishedVersion)
Lirias (submittedVersion, other-oa)
RePEc: Research Papers in Economics (submittedVersion)
Funders: Fonds Wetenschappelijk Onderzoek
Vlaamse regering
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: risk prediction model, model calibration, net benefit, clinical utility, calibration error, overfitting, underfitting, risk overestimation, risk underestimation, risk thresholds, model discrimination, testicular cancer diagnosis
MeSH terms: Computer Simulation, Decision Support Techniques, Humans, Male, Models, Theoretical, Norway, Predictive Value of Tests, ROC Curve, Testicular Neoplasms, Bias, Linear Models, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks