CoChat

Does the Net Reclassification Improvement Help Us Evaluate Models and Markers?

Vickers, Andrew J., Pepe, Margaret · 2014 · Annals of Internal Medicine
Cited by 48

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Abstract

Editorials21 January 2014Does the Net Reclassification Improvement Help Us Evaluate Models and Markers?Andrew J. Vickers, PhD and Margaret Pepe, PhDAndrew J. Vickers, PhDFrom Memorial Sloan-Kettering Cancer Center, New York, New York; and Fred Hutchinson Cancer Research Center, Seattle, Washington.Search for more papers by this author and Margaret Pepe, PhDFrom Memorial Sloan-Kettering Cancer Center, New York, New York; and Fred Hutchinson Cancer Research Center, Seattle, Washington.Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/M13-2841 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Few statistical methods have been as rapidly adopted as the net reclassification improvement (NRI). The reasons that the NRI has become so popular so quickly are not hard to fathom. First, it is an appealing concept to investigators. Instead of reporting that their new marker improved the area under the receiver-operating characteristic curve (AUC)—a statistical abstraction at the best of times—they can say that their marker led to “reclassification,” suggesting clinical value. The popularity of the NRI also stems from a belief that the AUC is insensitive. For example, in an early paper on the NRI, Pencina and colleagues stated ...References1. Pencina MJ, D'Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30:11-21. [PMID: 21204120] CrossrefMedlineGoogle Scholar2. Begg CB, Gonen M, Seshan VE. Testing the incremental predictive accuracy of new markers. Clin Trials. 2013;10:690-2. [PMID: 23881367] CrossrefMedlineGoogle Scholar3. Leening MJG, Vedder MM, Witteman JCM, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies. A literature review and clinician's guide. Ann Intern Med. 2014;160:122-131. LinkGoogle Scholar4. Vickers AJ, Elkin EB, Steyerberg E. Net reclassification improvement and decision theory [Letter]. Stat Med. 2009;28:525-6. [PMID: 17907248] CrossrefMedlineGoogle Scholar5. Kerr KF, Wang Z, Janes H, McClelland RL, Psaty BM, Pepe MS. Net reclassification indices for evaluating risk prediction instruments: a critical review. Epidemiology. 2014;25:114-21. [PMID: 24240655] CrossrefMedlineGoogle Scholar6. Hilden J, Gerds TA. A note on the evaluation of novel biomarkers: do not rely on integrated discrimination improvement and net reclassification index. Stat Med. 2013;:. [PMID: 23553436] MedlineGoogle Scholar7. Pepe MS, Janes H, Kerr KF, Psaty BM. Net Reclassification Index: a Misleading Measure of Prediction Improvement. UW Biostatistics Working Paper Series; Working Paper 394. 2013. Google Scholar8. Vickers AJ. Decision Curve Analysis. New York: Memorial Sloan-Kettering Cancer Center; 2013. Accessed at www.mskcc.org/research/epidemiology-biostatistics/health-outcomes/decision-curve-analysis-0 on 6 December 2013. Google Scholar9. Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making [Editorial]. Ann Intern Med. 2012;157:294-5. [PMID: 22910942] LinkGoogle Scholar10. Van Calster B, Vickers AJ, Pencina MJ, Baker SG, Timmerman D, Steyerberg EW. Evaluation of markers and risk prediction models: overview of relationships between NRI and decision-analytic measures. Med Decis Making. 2013;33:490-501. [PMID: 23313931] CrossrefMedlineGoogle Scholar Author, Article, and Disclosure InformationAffiliations: From Memorial Sloan-Kettering Cancer Center, New York, New York; and Fred Hutchinson Cancer Research Center, Seattle, Washington.Grant Support: In part by funds from David H. Koch provided through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers, and SPORE grant P50-CA92629 from the National Cancer Institute to Dr. Howard Scher. This work was also supported in part by a National Institutes of Health/National Cancer Institute Cancer Center Support Grant to Memorial Sloan-Kettering Cancer Center under award P30 CA00874 and by grant R01CA152089 from the National Institutes of Health.Disclosures: None disclosed. Forms can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M13-2841.Corresponding Author: Andrew J. Vickers, PhD, Department of Epidemiology & Biostatistics, Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street, 2nd Floor, New York, NY 10021; e-mail, [email protected]org.Current Author Addresses: Dr. Vickers: Department of Epidemiology & Biostatistics, Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street, 2nd Floor, New York, NY 10021.Dr. Pepe: Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, PO Box 19024, Seattle, WA 98109-1024. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoNet Reclassification Improvement: Computation, Interpretation, and Cont

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7326/m13-2841

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 48

Metadata

Authors
Vickers, A. J., & Pepe, M.
Venue
Annals of Internal Medicine
Published
2014-01-21
Volume
160
Issue
2
Pages
136-137
ISSN
  • 0003-4819 (ISSN-L)
  • 1539-3704
OpenAlex
W2068583723
Cited by
48
References
1
FWCI
10.65
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.