Validation of prediction models in the presence of competing risks: a guide through modern methods

Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis

Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878 · see citing works

Decision curve analysis: a novel method for evaluating prediction models

Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341 · see citing works

Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures

Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170 · see citing works

Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement

Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388 · see citing works

Evaluating the Yield of Medical Tests

Harrell, F. E. (1982) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1982.03320430047030 · Cited by 3,490 · see citing works

Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating

Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281 · see citing works

Tutorial in biostatistics: competing risks and multi‐state models

Putter, H., Fiocco, M. F., & Geskus, R. B. (2006) · Statistics in Medicine · doi:10.1002/sim.2712 · Cited by 2,458 · see citing works

Estimating and comparing time‐dependent areas under receiver operating characteristic curves for censored event times with competing risks

Blanche, P., Dartigues, J., & Jacqmin‐Gadda, H. (2013) · Statistics in Medicine · doi:10.1002/sim.5958 · Cited by 1,877 · see citing works

On the C‐statistics for evaluating overall adequacy of risk prediction procedures with censored survival data

Uno, H., Cai, T., Pencina, M. J., D'Agostino, R. B., & Wei, L. J. (2011) · Statistics in Medicine · doi:10.1002/sim.4154 · Cited by 1,535 · see citing works

Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research

Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508 · see citing works

Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers

Vickers, A. J., Cronin, A. M., Elkin, E. B., & Gönen, M. (2008) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-8-53 · Cited by 1,377 · see citing works

The Threshold Approach to Clinical Decision Making

Pauker, S. G., & Kassirer, J. P. (1980) · New England Journal of Medicine · doi:10.1056/nejm198005153022003 · Cited by 1,371 · see citing works

Prognosis and prognostic research: what, why, and how?

Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337 · see citing works

Prediction models need appropriate internal, internal-external, and external validation

Steyerberg, E. W., & Harrell, F. E. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.005 · Cited by 1,304 · see citing works

Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

Vickers, A. J., Van Calster, B., & Steyerberg, E. W. (2016) · BMJ · doi:10.1136/bmj.i6 · Cited by 1,163 · see citing works

External validation of a Cox prognostic model: principles and methods

Royston, P. J., & Altman, D. G. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-33 · Cited by 911 · see citing works

An Empirical Transition Matrix for Non-homogeneous Markov Chains Based on Censored Observations

Johansen, R. (1978) · Cited by 866 · see citing works

Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use

Kerr, K. F., Brown, M. D., Zhu, K., & Janes, H. (2016) · Journal of Clinical Oncology · doi:10.1200/jco.2015.65.5654 · Cited by 697 · see citing works

Sample size considerations for the external validation of a multivariable prognostic model: A resampling study

Collins, G. S., Ogundimu, E. O., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6787 · Cited by 665 · see citing works

Recovery of Information and Adjustment for Dependent Censoring Using Surrogate Markers

Robins, J. M., & Rotnitzky, A. G. (1992) · Birkhäuser Boston eBooks · doi:10.1007/978-1-4757-1229-2_14 · Cited by 646 · see citing works

Substantial effective sample sizes were required for external validation studies of predictive logistic regression models

Vergouwe, Y., Steyerberg, E. W., Eijkemans, M. J. C., & Habbema, J. D. F. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.06.017 · Cited by 636 · see citing works

Prognostic Models With Competing Risks

Wolbers, M., Koller, M. T., Witteman, J. C. M., & Steyerberg, E. W. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181a39056 · Cited by 628 · see citing works

Validation and updating of predictive logistic regression models: a study on sample size and shrinkage

Steyerberg, E. W., Borsboom, G. J. J. M., van Houwelingen, H. C., Eijkemans, M. J. C., & Habbema, J. D. F. (2004) · Statistics in Medicine · doi:10.1002/sim.1844 · Cited by 552 · see citing works

Consistent Estimation of the Expected Brier Score in General Survival Models with Right‐Censored Event Times

Gerds, T., & Schumacher, M. (2006) · Biometrical journal. Biometrische Zeitschrift · doi:10.1002/bimj.200610301 · Cited by 541 · see citing works

Dynamic Prediction in Clinical Survival Analysis

van Houwelingen, H. C., & Putter, H. (2011) · CRC Press eBooks · doi:10.1201/b11311 · Cited by 352 · see citing works

Pseudo-observations in survival analysis

Andersen, P. K., & Perme, M. P. (2009) · Statistical Methods in Medical Research · doi:10.1177/0962280209105020 · Cited by 351 · see citing works

Competing risks and the clinical community: irrelevance or ignorance?

Koller, M. T., Raatz, H., Steyerberg, E. W., & Wolbers, M. (2011) · Statistics in Medicine · doi:10.1002/sim.4384 · Cited by 302 · see citing works

Estimating a time‐dependent concordance index for survival prediction models with covariate dependent censoring

Gerds, T. A., Kattan, M. W., Schumacher, M. C., & Yu, C. (2012) · Statistics in Medicine · doi:10.1002/sim.5681 · Cited by 231 · see citing works

Prognosis Research in Health Care

(2019) · Oxford University Press eBooks · doi:10.1093/med/9780198796619.001.0001 · Cited by 221 · see citing works

Graphical calibration curves and the integrated calibration index (ICI) for survival models

Austin, P. C., Harrell, F. E., & Klaveren, D. V. (2020) · Statistics in Medicine · doi:10.1002/sim.8570 · Cited by 218 · see citing works

Concordance for prognostic models with competing risks

Wolbers, M., Blanche, P., Koller, M. T., Witteman, J. C. M., & Gerds, T. A. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt059 · Cited by 208 · see citing works

The c-index is not proper for the evaluation of $t$-year predicted risks

Blanche, P., Kattan, M. W., & Gerds, T. A. (2018) · Biostatistics · doi:10.1093/biostatistics/kxy006 · Cited by 158 · see citing works

The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models

Kattan, M. W., & Gerds, T. A. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0029-2 · Cited by 149 · see citing works

STRengthening Analytical Thinking for Observational Studies: the STRATOS initiative

Sauerbrei, W., Abrahamowicz, M., Altman, D. G., le Cessie, S., Carpenter, J. R., & initiative, O. B. O. T. S. (2014) · Statistics in Medicine · doi:10.1002/sim.6265 · Cited by 135 · see citing works

Calibration plots for risk prediction models in the presence of competing risks

Gerds, T. A., Andersen, P. K., & Kattan, M. W. (2014) · Statistics in Medicine · doi:10.1002/sim.6152 · Cited by 114 · see citing works

Time‐Dependent Predictive Accuracy in the Presence of Competing Risks

Saha‐Chaudhuri, P., & Heagerty, P. J. (2010) · Biometrics · doi:10.1111/j.1541-0420.2009.01375.x · Cited by 94 · see citing works

Kidney Failure Prediction Models: A Comprehensive External Validation Study in Patients with Advanced CKD

Ramspek, C. L., Evans, M., Wanner, C., Drechsler, C., Chesnaye, N. C., Szymczak, M., Krajewska, M., Torino, C., Porto, G., Hayward, S., et al. (2021) · Journal of the American Society of Nephrology · doi:10.1681/asn.2020071077 · Cited by 91 · see citing works

A spline-based tool to assess and visualize the calibration of multiclass risk predictions

Van Hoorde, K., Van Huffel, S., Timmerman, D., Bourne, T., & Van Calster, B. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.12.016 · Cited by 85 · see citing works

Quantifying the predictive accuracy of time‐to‐event models in the presence of competing risks

Schoop, R., Beyersmann, J., Schumacher, M., & Binder, H. (2011) · Biometrical journal. Biometrische Zeitschrift · doi:10.1002/bimj.201000073 · Cited by 80 · see citing works

Construction, validation and updating of a prognostic model for kidney graft survival

van Houwelingen, H. C., & Thorogood, J. (1995) · Statistics in Medicine · doi:10.1002/sim.4780141806 · Cited by 74 · see citing works

Estimation of required sample size for external validation of risk models for binary outcomes

Pavlou, M., Chen, Q., Omar, R., Seaman, S. R., Steyerberg, E. W., White, I. R., & Ambler, G. (2021) · Statistical Methods in Medical Research · doi:10.1177/09622802211007522 · Cited by 58 · see citing works

Tools for Checking Calibration of a Cox Model in External Validation: Approach Based on Individual Event Probabilities

Royston, P. J. (2014) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1401400403 · Cited by 57 · see citing works

A tool for predicting breast carcinoma mortality in women who do not receive adjuvant therapy

Kattan, M. W., Giri, D. D., Panageas, K. S., et al. (2004) · Cancer · doi:10.1002/cncr.20635 · Cited by 49 · see citing works

Evaluating Prognostic Accuracy of Biomarkers under Competing Risk

Zheng, Y., Cai, T., Jin, Y., et al. (2011) · Biometrics · doi:10.1111/j.1541-0420.2011.01671.x · Cited by 47 · see citing works

Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models

Ramspek, C. L., Teece, L., Snell, K. I. E., Evans, M., Riley, R. D., van Smeden, M., van Geloven, N., & van Diepen, M. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyab256 · Cited by 45 · see citing works

Comparing predictions among competing risks models with time‐dependent covariates

Cortese, G., Gerds, T. A., & Andersen, P. K. (2013) · Statistics in Medicine · doi:10.1002/sim.5773 · Cited by 44 · see citing works

Graphical calibration curves and the integrated calibration index (ICI) for competing risk models

Austin, P. C., Putter, H., Giardiello, D., & Klaveren, D. V. (2022) · Diagnostic and Prognostic Research · doi:10.1186/s41512-021-00114-6 · Cited by 44 · see citing works

Evaluation of competing risks prediction models using polytomous discrimination index

Ding, M., Ning, J., & Li, R. (2020) · Canadian Journal of Statistics · doi:10.1002/cjs.11583 · Cited by 7 · see citing works

Untitled work

(1999) · doi:10.1002/(sici)1097-0258(19990915/30)18:17/18

Verification of forecasts expressed in terms of probability

BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2

Developing clinical prediction models: a step-by-step guide

Efthimiou, O., Seo, M., Chalkou, K., Debray, T., Egger, M., & Salanti, G. (2024) · BMJ · doi:10.1136/bmj-2023-078276 · Cited by 544 · see citing works

Clinical prediction models using machine learning in oncology: challenges and recommendations

Collins, G. S., Chester-Jones, M., Gerry, S., Ma, J., Matos, J., Sehjal, J., Tsegaye, B., & Dhiman, P. (2025) · BMJ Oncology · doi:10.1136/bmjonc-2025-000914 · Cited by 27 · see citing works

Accurate roughness measurements by dynamic calibration, VFM-uncertainty calculations and a special calibration specimen

(2005) · doi:10.1088/1742-6596/13/1/054 · Cited by 5 · see citing works

Verification of wafer-level calibration accuracy at high temperatures

(2008) · doi:10.1109/arftg.2008.4633313 · Cited by 3 · see citing works

UNCERTAINTY EVALUATION OF CALIBRATION SPECIFICATION FOR PRESSURE REFERENCE LEAK

(2015)

Verification of wafer-level calibration accuracy at high temperatures

(2008) · doi:10.1109/mwsym.2008.4633313 · Cited by 3 · see citing works

Multi-frequency approach to the coaxial multiline through-reflect-line calibration

Lewandowski, A., Wiatr, W., & Dobrowolski, J. A. (2010) · International Conference on Microwaves, Radar & Wireless Communications · Cited by 3 · see citing works