CoChat

Evaluation of Markers and Risk Prediction Models: Overview of Relationships between NRI and Decision-Analytic Measures

Open access
Van Calster, Ben, Vickers, Andrew J., Pencina, Michael J., Baker, Stuart G., Timmerman, Dirk, Steyerberg, Ewout W. · 2013 · Medical Decision Making
Cited by 76

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

Background. For the evaluation and comparison of markers and risk prediction models, various novel measures have recently been introduced as alternatives to the commonly used difference in the area under the receiver operating characteristic (ROC) curve (ΔAUC). The net reclassification improvement (NRI) is increasingly popular to compare predictions with 1 or more risk thresholds, but decision-analytic approaches have also been proposed. Objective. We aimed to identify the mathematical relationships between novel performance measures for the situation that a single risk threshold T is used to classify patients as having the outcome or not. Methods. We considered the NRI and 3 utility-based measures that take misclassification costs into account: difference in net benefit (ΔNB), difference in relative utility (ΔRU), and weighted NRI (wNRI). We illustrate the behavior of these measures in 1938 women suspect of having ovarian cancer (prevalence 28%). Results. The 3 utility-based measures appear to be transformations of each other and hence always lead to consistent conclusions. On the other hand, conclusions may differ when using the standard NRI, depending on the adopted risk threshold T, prevalence P, and the obtained differences in sensitivity and specificity of the 2 models that are compared. In the case study, adding the CA-125 tumor marker to a baseline set of covariates yielded a negative NRI yet a positive value for the utility-based measures. Conclusions. The decision-analytic measures are each appropriate to indicate the clinical usefulness of an added marker or compare prediction models since these measures each reflect misclassification costs. This is of practical importance as these measures may thus adjust conclusions based on purely statistical measures. A range of risk thresholds should be considered in applying these measures.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/0272989x12470757

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 7 references

Cited by

See all 76

Metadata

Authors
Van Calster, B., Vickers, A. J., Pencina, M. J., Baker, S. G., Timmerman, D., & Steyerberg, E. W.
Venue
Medical Decision Making
Published
2013-01-11
Volume
33
Issue
4
Pages
490-501
ISSN
  • 0272-989X (ISSN-L)
  • 1552-681X
OpenAlex
W2024583141
Cited by
76
References
7
FWCI
8.89
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluation of Markers and Risk Prediction Models: Overview of Relationships between NRI and Decision-Analytic Measures. https://app.cochat.ai/papers/10.1177/0272989x12470757
The paper
Van Calster, B., Vickers, A. J., Pencina, M. J., Baker, S. G., Timmerman, D., & Steyerberg, E. W. (2013). Evaluation of Markers and Risk Prediction Models: Overview of Relationships between NRI and Decision-Analytic Measures. Medical Decision Making. https://doi.org/10.1177/0272989x12470757

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.