Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use
Open accessAbstract
The decision curve is a graphical summary recently proposed for assessing the potential clinical impact of risk prediction biomarkers or risk models for recommending treatment or intervention. It was applied recently in an article in Journal of Clinical Oncology to measure the impact of using a genomic risk model for deciding on adjuvant radiation therapy for prostate cancer treated with radical prostatectomy. We illustrate the use of decision curves for evaluating clinical- and biomarker-based models for predicting a man's risk of prostate cancer, which could be used to guide the decision to biopsy. Decision curves are grounded in a decision-theoretical framework that accounts for both the benefits of intervention and the costs of intervention to a patient who cannot benefit. Decision curves are thus an improvement over purely mathematical measures of performance such as the area under the receiver operating characteristic curve. However, there are challenges in using and interpreting decision curves appropriately. We caution that decision curves cannot be used to identify the optimal risk threshold for recommending intervention. We discuss the use of decision curves for miscalibrated risk models. Finally, we emphasize that a decision curve shows the performance of a risk model in a population in which every patient has the same expected benefit and cost of intervention. If every patient has a personal benefit and cost, then the curves are not useful. If subpopulations have different benefits and costs, subpopulation-specific decision curves should be used. As a companion to this article, we released an R software package called DecisionCurve for making decision curves and related graphics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI has expanded clinical decision-making beyond diagnosis by enabling prediction of future health outcomes through advanced prognostic modelling.
“AI has expanded clinical decision-making beyond diagnosis by enabling prediction of future health outcomes through advanced prognostic modelling [30,35].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [35]
The Introduction (page unavailable) explains that risk prediction models estimate outcomes such as prostate cancer mortality and can guide treatment decisions. It does not attribute this capability to AI or establish that AI expanded clinical decision-making beyond diagnosis. (LLM, conf=0.86)
Experimental — accuracy not yet measuredAI is increasingly applied to treatment planning by generating individualised therapeutic recommendations based on patient-specific clinical profiles and longitudinal data patterns.
“AI is increasingly applied to treatment planning by generating individualised therapeutic recommendations based on patient-specific clinical profiles and longitudinal data patterns [35,34].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [35]
The Introduction describes models that use an individual’s biomarkers or other characteristics to estimate risk and help guide treatment decisions. It does not describe AI, longitudinal data patterns, or increasing use of AI in treatment planning. The Concluding Remarks also caution that decision curves assess treatment policies in populations, not individual benefits. (LLM, conf=0.86)
Experimental — accuracy not yet measuredDecision-curve analysis estimates net benefit across threshold probabilities and determines whether a model improves decision-making compared to standard approaches.
“Decision-curve analysis estimates net benefit across threshold probabilities and determines whether a model improves decision-making compared to standard approaches [35].”
Beyond imaging, AI is applied to laboratory data and digital pathology for disease detection and characterisation.
“Beyond imaging, AI is applied to laboratory data and digital pathology for disease detection and characterisation [2,35].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [35]
The Introduction discusses decision curves for clinical and biomarker-based prostate cancer risk models, not AI applications to laboratory data or digital pathology. Searches for AI, machine learning, digital pathology, and laboratory-based disease detection found no passage addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measuredThe maximum value of net benefit equals the prevalence.
“The maximum value of net benefit equals the prevalence.”
From Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance (2025) · cites this paper as [84]
Standardised net benefit equals net benefit divided by prevalence, and its maximum value is 1.
“Standardised net benefit equals net benefit divided by prevalence, and its maximum value is 1.”
From Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance (2025) · cites this paper as [84]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341 · see citing works
Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance
Van Calster, B., & Vickers, A. J. (2015) · Medical Decision Making · doi:10.1177/0272989x14547233 · Cited by 371 · see citing works
Beyond Diagnostic Accuracy: The Clinical Utility of Diagnostic Tests
Bossuyt, P. M., Reitsma, J. B., Linnet, K., & Moons, K. G. (2012) · Clinical Chemistry · doi:10.1373/clinchem.2012.182576 · Cited by 330 · see citing works
The numerical measure of the success of predictions
Peirce, C. S. (1884) · Science · doi:10.1126/science.ns-4.93.453-a · Cited by 315 · see citing works
Den, R. B., Yousefi, K., Trabulsi, E. J., Abdollah, F., Choeurng, V., Feng, F. Y.-C., Dicker, A. P., Lallas, C. D., Gomella, L. G., Davicioni, E., et al. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2014.59.0026 · Cited by 212 · see citing works
Pepe, M. S., Fan, J., Feng, Z., Gerds, T., & Hilden, J. (2015) · Statistics in Biosciences · doi:10.1007/s12561-014-9118-0 · Cited by 191 · see citing works
Bryant, R. J., Sjoberg, D. D., Vickers, A. J., Robinson, M., Kumar, R., Marsden, L., Davis, M., Scardino, P. T., Donovan, J., Neal, D. E., et al. (2015) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djv095 · Cited by 186 · see citing works
Raji, O. Y., Duffy, S. W., Agbaje, O. F., Baker, S. G., Christiani, D. C., Cassidy, A., & Field, J. K. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-4-201208210-00004 · Cited by 177 · see citing works
Using Relative Utility Curves to Evaluate Risk Prediction
Baker, S. G., Cook, N. R., Vickers, A., & Kramer, B. S. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2009.00592.x · Cited by 142 · see citing works
McMahon, P. J., Panczykowski, D. M., Yue, J. K., et al. (2014) · Journal of Neurotrauma · doi:10.1089/neu.2014.3635 · Cited by 125 · see citing works
Vickers, A. J., & Cronin, A. M. (2010) · Seminars in Oncology · doi:10.1053/j.seminoncol.2009.12.004 · Cited by 108 · see citing works
Scattoni, V., Lazzeri, M., Lughezzani, G., et al. (2013) · The Journal of Urology · doi:10.1016/j.juro.2013.02.3184 · Cited by 99 · see citing works
Translation: Beyond Diagnostic Accuracy: The Clinical Utility of Diagnostic Tests
Bossuyt, P. M., Reitsma, J. B., Línnet, K., & Moons, K. G. M. (2013) · Laboratory Medicine Online · doi:10.3343/lmo.2013.3.4.269 · Cited by 90 · see citing works
Putting Risk Prediction in Perspective: Relative Utility Curves
Baker, S. G. (2009) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djp353 · Cited by 79 · see citing works
A nomogram predicting pulmonary metastasis of hepatocellular carcinoma following partial hepatectomy
Li, J., Liu, Y., Yan, Z.-P., Wan, X., Xia, Y., Wang, K., Liu, J., Lau, W. Y., Wu, M., & Shen, F. (2014) · British Journal of Cancer · doi:10.1038/bjc.2014.19 · Cited by 67 · see citing works
Slankamenac, K., Beck‐Schimmer, B., Breitenstein, S., Puhan, M. A., & Clavien, P. (2013) · World Journal of Surgery · doi:10.1007/s00268-013-2159-6 · Cited by 66 · see citing works
Labidi, M., Lavoie, P., Lapointe, G. M., et al. (2015) · Journal of neurosurgery · doi:10.3171/2014.12.jns141240 · Cited by 65 · see citing works
Urinary Cell mRNA Profiles and Differential Diagnosis of Acute Kidney Graft Dysfunction
Matignon, M., Ding, R., Dadhania, D. M., Mueller, F. B., Hartono, C., Snopkowski, C., Li, C. Y., Lee, J. R., Sjoberg, D. D., Seshan, S. V., et al. (2014) · Journal of the American Society of Nephrology · doi:10.1681/asn.2013080900 · Cited by 50 · see citing works
Secin, F. P., Bianco, F. J., Cronin, A. M., et al. (2008) · The Journal of Urology · doi:10.1016/j.juro.2008.10.035 · Cited by 24 · see citing works
Baart, A. M., de Kort, W. L. A. M., Moons, K. G. M., et al. (2012) · Transfusion · doi:10.1111/j.1537-2995.2012.03957.x · Cited by 23 · see citing works
Ang, S. F., Ng, E. S.-H., Li, H., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128058 · Cited by 17 · see citing works
Lughezzani, G., Zorn, K. C., Budäus, L., et al. (2012) · European Urology · doi:10.1016/j.eururo.2012.04.022 · Cited by 13 · see citing works
Evaluating Prognostic Markers Using Relative Utility Curves and Test Tradeoffs
Baker, S. G., & Kramer, B. S. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2014.58.0092 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A simple, step-by-step guide to interpreting decision curve analysis
Vickers, A. J., van Calster, B., & Steyerberg, E. W. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0064-7 · Cited by 1,174 · see citing works
Verbakel, J. Y., Steyerberg, E. W., Uno, H., De Cock, B., Wynants, L., Collins, G. S., & Van Calster, B. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.028 · Cited by 138 · see citing works
Van Calster, B., Collins, G. S., Vickers, A. J., Wynants, L., Kerr, K. F., Barreñada, L., Varoquaux, G., Singh, K., Moons, K. G., Hernandez-Boussard, T., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100916 · Cited by 112 · see citing works
Self-citationValidation of prediction models in the presence of competing risks: a guide through modern methods
van Geloven, N., Giardiello, D., Bonneville, E. F., Teece, L., Ramspek, C. L., van Smeden, M., Snell, K. I. E., van Calster, B., Pohar-Perme, M., Riley, R. D., et al. (2022) · BMJ · doi:10.1136/bmj-2021-069249 · Cited by 106 · see citing works
Assessing the clinical impact of risk models for opting out of treatment
Kerr, K. F., Brown, M. D., Marsh, T. L., & Janes, H. (2019) · Medical Decision Making · doi:10.1177/0272989x18819479 · Cited by 23 · see citing works
Self-citationPatil, S. S., Jha, G. K., Mohsin, N., Patel, P., Ralte, L., & Shukla, A. K. (2026) · Cureus · doi:10.7759/cureus.109905
“AI is increasingly applied to treatment planning by generating individualised therapeutic recommendations based on patient-specific clinical profiles and longitudinal data patterns [35,34].”
- Authors
- Kerr, K. F., Brown, M. D., Zhu, K., & Janes, H.
- Affiliations
- University of Washington (US)
- Venue
- Journal of Clinical Oncology
- Published
- 2016-07-20
- Volume
- 34
- Issue
- 21
- Pages
- 2534-2540
- ISSN
- 0732-183X (ISSN-L)
- 1527-7755
- PMC
- PMC4962736
- OpenAlex
- W2417116130
- Cited by
- 697
- References
- 23
- Citations by year
- 2026: 79 · 2025: 100 · 2024: 90 · 2023: 95 · 2022: 98 · 2021: 122 · 2020: 55 · 2019: 36 · 2018: 12 · 2017: 10
- FWCI
- 19.89
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Clinical Oncology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- National Heart, Lung, and Blood Institute (R01HL085757)
- National Cancer Institute (R01CA152089)
- National Institute of General Medical Sciences (R01GM054438)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Prostate Cancer Diagnosis and Treatment
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- decision curves, risk prediction model, prostate cancer, prostate biopsy, adjuvant radiotherapy, radical prostatectomy
- MeSH terms
- Decision Making, Decision Support Techniques, Humans, Male, Models, Theoretical, Prostatectomy, Prostatic Neoplasms, Risk, Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
