The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Net Reclassification Indices for Evaluating Risk Prediction Instruments: A Critical Review
Open accessAbstract
Net reclassification indices have recently become popular statistics for measuring the prediction increment of new biomarkers. We review the various types of net reclassification indices and their correct interpretations. We evaluate the advantages and disadvantages of quantifying the prediction increment with these indices. For predefined risk categories, we relate net reclassification indices to existing measures of the prediction increment. We also consider statistical methodology for constructing confidence intervals for net reclassification indices and evaluate the merits of hypothesis testing based on such indices. We recommend that investigators using net reclassification indices should report them separately for events (cases) and nonevents (controls). When there are two risk categories, the components of net reclassification indices are the same as the changes in the true- and false-positive rates. We advocate the use of true- and false-positive rates and suggest it is more useful for investigators to retain the existing, descriptive terms. When there are three or more risk categories, we recommend against net reclassification indices because they do not adequately account for clinically important differences in shifts among risk categories. The category-free net reclassification index is a new descriptive device designed to avoid predefined risk categories. However, it experiences many of the same problems as other measures such as the area under the receiver operating characteristic curve. In addition, the category-free index can mislead investigators by overstating the incremental value of a biomarker, even in independent validation data. When investigators want to test a null hypothesis of no prediction increment, the well-established tests for coefficients in the regression model are superior to the net reclassification index. If investigators want to use net reclassification indices, confidence intervals should be calculated using bootstrap methods rather than published variance formulas. The preferred single-number summary of the prediction increment is the improvement in net benefit.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Gail, M. H., Brinton, L. A., Byar, D. P., Corle, D. K., Green, S. B., Schairer, C., & Mulvihill, J. J. (1989) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/81.24.1879 · Cited by 3,334
Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers
Pencina, M. J., D'Agostino, R. B., & Steyerberg, E. W. (2011) · Statistics in Medicine · doi:10.1002/sim.4085 · Cited by 2,453
Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,112
Coronary Artery Calcium Score and Risk Classification for Coronary Heart Disease Prediction
Polonsky, T. S. (2010) · JAMA · doi:10.1001/jama.2010.461 · Cited by 1,119
Interpreting Incremental Value of Markers Added to Risk Prediction Models
Pencina, M. J., D'Agostino, R. B., Pencina, K. M., Janssens, A. C. J. W., & Greenland, P. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kws207 · Cited by 506
The numerical measure of the success of predictions
PEIRCE, C. S. (1884) · Science · doi:10.1126/science.ns-4.93.453-a · Cited by 403
Uno, H., Tian, L., Cai, T., Kohane, I. S., & Wei, L. J. (2013) · Statistics in Medicine · doi:10.1002/sim.5647 · Cited by 385
Pencina, M. J., D'Agostino, R. B., & Demler, O. V. (2012) · Statistics in Medicine · doi:10.1002/sim.4348 · Cited by 349
Cited by
Matsushita, K., Coresh, J., Sang, Y., Chalmers, J., Fox, C., Guallar, E., Jafar, T., Jassal, S. K., Landman, G. W. D., Muntner, P., et al. (2015) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(15)00040-6 · Cited by 903
Developing and evaluating polygenic risk prediction models for stratified disease prevention
Chatterjee, N., Shi, J., & García-Closas, M. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.27 · Cited by 697
Ohkuma, T., Ninomiya, T., Tomiyama, H., Kario, K., Hoshide, S., Kita, Y., Inoguchi, T., Maeda, Y., Kohara, K., Tabara, Y., et al. (2017) · Hypertension · doi:10.1161/hypertensionaha.117.09097 · Cited by 541
Analysis of machine learning techniques for heart failure readmissions
Mortazavi, B. J., Downing, N. S., Bucholz, E. M., Dharmarajan, K., Manhapra, A., Li, S.-X., Negahban, S. N., & Krumholz, H. M. (2016) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.116.003039 · Cited by 401
Pabinger, I., van Es, N., Heinze, G., Posch, F., Riedl, J., Reitter, E.-M., Di Nisio, M., Cesarman-Maus, G., Kraaijpoel, N., Zielinski, C. C., et al. (2018) · The Lancet Haematology · doi:10.1016/s2352-3026(18)30063-2 · Cited by 335
Metadata
- Authors
- Kerr, K. F., Wang, Z., Janes, H., McClelland, R. L., Psaty, B. M., & Pepe, M. S.
- Venue
- Epidemiology
- Published
- 2014-01
- Volume
- 25
- Issue
- 1
- Pages
- 114-121
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2074847887
- Cited by
- 446
- References
- 23
- FWCI
- 25.52
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Net Reclassification Indices for Evaluating Risk Prediction Instruments: A Critical Review. https://app.cochat.ai/papers/10.1097/ede.0000000000000018
- The paper
- Kerr, K. F., Wang, Z., Janes, H., McClelland, R. L., Psaty, B. M., & Pepe, M. S. (2014). Net Reclassification Indices for Evaluating Risk Prediction Instruments: A Critical Review. Epidemiology. https://doi.org/10.1097/ede.0000000000000018
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.