The DOI matches its Crossref record; 140 of the 141 references it cites were found in a registry or an index.
Net Reclassification Improvement: Computation, Interpretation, and Controversies: A Literature Review and Clinician's Guide
Open accessAbstract
The net reclassification improvement (NRI) is an increasingly popular measure for evaluating improvements in risk predictions. This article details a review of 67 publications in high-impact general clinical journals that considered the NRI. Incomplete reporting of NRI methods, incorrect calculation, and common misinterpretations were found. To aid improved applications of the NRI, the article elaborates on several aspects of the computation and interpretation in various settings. Limitations and controversies are discussed, including the effect of miscalibration of prediction models, the use of the continuous NRI and “clinical NRI,” and the relation with decision analytic measures. A systematic approach toward presenting NRI analysis is proposed: Detail and motivate the methods used for computation of the NRI, use clinically meaningful risk cutoffs for the category-based NRI, report both NRI components, address issues of calibration, and do not interpret the overall NRI as a percentage of the study population reclassified. Promising NRI findings need to be followed with decision analytic or formal cost-effectiveness evaluations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Dedicated measures, such as the widely used but improper net reclassification improvement, are available for evaluating competing models.”
From Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance (2025) · cites this paper as [119]
Citations
Cites
Grundy, S. M., Bilheimer, D. W., Chait, A., Clark, L. T., Denke, M. A., Havel, R. J., Hazzard, W. R., Hulley, S. B., Hunninghake, D. B., Kreisberg, R. A., et al. (1993) · JAMA · doi:10.1001/jama.269.23.3015 · Cited by 27,944
Expert Panel on Detection, E. (2001) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.285.19.2486 · Cited by 26,011
Detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III)
Grundy, S. M., Becker, D. M., Clark, L. T., Cooper, R., Denke, M. A., Howard, J. L., Hunninghake, D. B., Illingworth, D. R., Luepker, R. V., McBride, P. E., et al. (2002) · Civil War Book Review · Cited by 12,198
European Guidelines on cardiovascular disease prevention in clinical practice (version 2012)
Authors/Task Force Members:, Perk, J., De Backer, G., Gohlke, H., Graham, I., Reiner, Ž., Verschuren, M., Albus, C., Benlian, P., Boysen, G., et al. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs092 · Cited by 8,490
Roffi, M., Patrono, C., Collet, J.-P., Mueller, C., Valgimigli, M., Andreotti, F., Bax, J. J., Borger, M. A., Brotons, C., Chew, D. P., et al. (2016) · European Heart Journal · doi:10.1093/eurheartj/ehv320 · Cited by 8,261
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Self-citationDecision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341
Kearon (2012) · doi:10.1378/chest.11-2301 · Cited by 6,119
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Self-citationClinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Self-citationAuthors Task Force Members, Hamm, C. W., Bassand, J., Agewall, S., Bax, J. J., Boersma, E. H., Bueno, H., Caso, P., Dudek, D., Gielen, S., et al. (2011) · European Heart Journal · doi:10.1093/eurheartj/ehr236 · Cited by 3,096
Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers
Pencina, M. J., D’Agostino, R. B., & Steyerberg, E. W. (2010) · Statistics in Medicine · doi:10.1002/sim.4085 · Cited by 2,483
Self-citationUse and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,156
General and abdominal adiposity and risk of death in Europe
Pischon, T., Boeing, H., Hoffmann, K., Bergmann, M., Schulze, M. B., Overvad, K., van der Schouw, Y. T., Spencer, E., Moons, K. G. M., Tjønneland, A., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0801891 · Cited by 2,155
Mosca, L., Benjamin, E. J., Berra, K., Bezanson, J. L., Dolor, R. J., Lloyd-Jones, D. M., Newby, L. K., Piña, I. L., Roger, V. L., Shaw, L. J., et al. (2011) · Circulation · doi:10.1161/cir.0b013e31820faaf8 · Cited by 2,143
2010 ACCF/AHA Guideline for Assessment of Cardiovascular Risk in Asymptomatic Adults
Greenland, P., Alpert, J. S., Beller, G. A., Benjamin, E. J., Budoff, M. J., Fayad, Z. A., Foster, E., Hlatky, M. A., Hodgson, J. M., Kushner, F. G., et al. (2010) · Circulation · doi:10.1161/cir.0b013e3182051b4c · Cited by 1,976
Cholesterol efflux capacity, high-density lipoprotein function, and atherosclerosis
Khera, A. V., Cuchel, M., de la Llera-Moya, M., Rodrigues, A., Burke, M. F., Jafri, K., French, B. C., Phillips, J. A., Mucksavage, M. L., Wilensky, R. L., et al. (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1001689 · Cited by 1,943
KANNEL, W. B., DAWBER, T. R., KAGAN, A., REVOTSKIE, N., & STOKES, J. (1961) · Annals of Internal Medicine · doi:10.7326/0003-4819-55-1-33 · Cited by 1,837
Modern Epidemiology 3rd edition
Rothman, K. J., Lash, T. L., & Greenland, S. (1986) · Cited by 1,736
Glycated hemoglobin, diabetes, and cardiovascular risk in nondiabetic adults
Selvin, E., Steffes, M. W., Zhu, H., Matsushita, K., Wagenknecht, L., Pankow, J., Coresh, J., & Brancati, F. L. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0908359 · Cited by 1,544
Genomewide Association Studies and Assessment of the Risk of Disease
Manolio, T. A. (2010) · New England Journal of Medicine · doi:10.1056/nejmra0905980 · Cited by 1,512
Hyponatremia and Mortality among Patients on the Liver-Transplant Waiting List
Kim, W. R., Biggins, S. W., Kremers, W. K., Wiesner, R. H., Kamath, P. S., Benson, J. T., Edwards, E. B., & Therneau, T. M. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0801209 · Cited by 1,485
Severe hypoglycemia and risks of vascular events and death
Zoungas, S., Patel, A., Chalmers, J., de Galan, B. E., Li, Q., Billot, L., Woodward, M., Ninomiya, T., Neal, B., MacMahon, S., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa1003795 · Cited by 1,471
A Predictive Model for Progression of Chronic Kidney Disease to Kidney Failure
Tangri, N. (2011) · JAMA · doi:10.1001/jama.2011.451 · Cited by 1,390
Greenland, P. (2007) · Statistics in Medicine · doi:10.1002/sim.2976 · Cited by 1,300
Pepe, M. S. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh101 · Cited by 1,276
The Emerging Risk Factors Collaboration (2011) · The Lancet · doi:10.1016/s0140-6736(11)60105-0 · Cited by 1,216
Gulati, A., Jabbour, A., Ismail, T. F., Guha, K., Khwaja, J., Raza, S., Morarji, K., Brown, T. D. H., Ismail, N. A., Dweck, M. R., et al. (2013) · JAMA · doi:10.1001/jama.2013.1363 · Cited by 1,183
C-reactive protein, fibrinogen, and cardiovascular disease prediction
The Emerging Risk Factors Collaboration (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1107477 · Cited by 1,150
Criteria for Evaluation of Novel Markers of Cardiovascular Risk
Hlatky, M. A., Greenland, P., Arnett, D. K., Ballantyne, C. M., Criqui, M. H., Elkind, M. S. V., Go, A. S., Harrell, F. E., Hong, Y., Howard, B. V., et al. (2009) · Circulation · doi:10.1161/circulationaha.109.192278 · Cited by 1,146
Coronary Artery Calcium Score and Risk Classification for Coronary Heart Disease Prediction
Polonsky, T. S. (2010) · JAMA · doi:10.1001/jama.2010.461 · Cited by 1,123
Yeboah, J., McClelland, R. L., Polonsky, T. S., Burke, G. L., Sibley, C. T., O’Leary, D., Carr, J. J., Goff, D. C., Greenland, P., & Herrington, D. M. (2012) · JAMA · doi:10.1001/jama.2012.9624 · Cited by 1,122
de Lemos, J. A., Drazner, M. H., Omland, T., Ayers, C. R., Khera, A., Rohatgi, A. K., Hashim, I. A., Berry, J. D., Das, S. R., Morrow, D. A., et al. (2010) · JAMA · doi:10.1001/jama.2010.1768 · Cited by 1,103
Devereaux, P. J., Chan, M. T. V., Alonso‐Coello, P., Walsh, M., Berwanger, O. B., Villar, J. C., Wang, C. Y., Garutti, R. I., Jacka, M. J., Sigamani, A., et al. (2012) · JAMA · doi:10.1001/jama.2012.5502 · Cited by 1,102
Schnabel, R. B., SULLIVAN, L. M., Levy, D. A., Pencina, M. J., Massaro, J. M., D’Agostino, R. B., Newton‐Cheh, C., Yamamoto, J. F., Magnani, J. W., Tadros, T. M., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60443-8 · Cited by 1,038
Self-citationSelection Criteria for Lung-Cancer Screening
Tammemägi, M. C., Katki, H. A., Hocking, W. G., Church, T. R., Caporaso, N. E., Kvale, P. A., Chaturvedi, A. K., Silvestri, G. A., Riley, T. L., Commins, J. M., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1211776 · Cited by 1,023
Matsushita, K., Mahmoodi, B. K., Woodward, M., Emberson, J., Jafar, T. H., Jee, S. H., Polkinghorne, K. R., Shankar, A., Smith, D. H., Tonelli, M. A., et al. (2012) · JAMA · doi:10.1001/jama.2012.3954 · Cited by 987
Clinical risk factors, DNA variants, and the development of type 2 diabetes
Lyssenko, V., Jonsson, A., Almgren, P., Pulizzi, N., Isomaa, B., Tuomi, T., Berglund, G., Altshuler, D., Nilsson, P., & Groop, L. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0801869 · Cited by 938
A Sensitive Cardiac Troponin T Assay in Stable Coronary Artery Disease
Omland, T., de Lemos, J. A., Sabatine, M. S., Christophi, C. A., Rice, M. M., Jablonski, K. A., Tjora, S., DOMANSKI, M. J., Gersh, B. J., Rouleau, J. L., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0805299 · Cited by 909
Use of Multiple Biomarkers to Improve the Prediction of Death from Cardiovascular Causes
Zethelius, B., Berglund, L., Sundström, J., Ingelsson, E., Basu, S., Larsson, A. O., Venge, P., & Ärnlöv, J. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0707064 · Cited by 893
Kramer, A. A., & Zimmerman, J. E. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000275267.64078.b0 · Cited by 886
Predicting the 30-Year Risk of Cardiovascular Disease
Pencina, M. J., D’Agostino, R. B., Larson, M. G., Massaro, J. M., & Vasan, R. S. (2009) · Circulation · doi:10.1161/circulationaha.108.816694 · Cited by 885
Self-citationAuthors/Task Force Members: Joep Perk (Chairperson) (Sweden)*, G. D. B. (. (2012) · European Journal of Preventive Cardiology · doi:10.1177/2047487312450228 · Cited by 863
Farooq, V., Klaveren, D. V., Steyerberg, E. W., Meliga, E., Vergouwe, Y., Chieffo, A., Kappetein, A. P., Colombo, A., Holmes, D. R., Mack, M. J., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)60108-7 · Cited by 853
Self-citationCarotid-Wall Intima–Media Thickness and Cardiovascular Events
Polak, J. F., Pencina, M. J., Pencina, K. M., O’Donnell, C. J., Wolf, P. A., & D’Agostino, R. B. (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1012592 · Cited by 829
Self-citationDen Ruijter, H. M., Peters, S. A. E., Anderson, T. J., Britton, A. R., Dekker, J. M., Eijkemans, M. J., Engström, G., Evans, G. W., de Graaf, J., Grobbee, D. E., et al. (2012) · JAMA · doi:10.1001/jama.2012.9630 · Cited by 760
Self-citationGenotype score in addition to common risk factors for prediction of type 2 diabetes
Meigs, J. B., Shrader, P., Sullivan, L. M., McAteer, J. B., Fox, C. S., Dupuis, J., Manning, A. K., Florez, J. C., Wilson, P. W. F., D'Agostino, R. B., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0804742 · Cited by 759
deFilippi, C. R., de Lemos, J. A., Christenson, R. H., Gottdiener, J. S., Kop, W. J., Zhan, M., & Seliger, S. L. (2010) · JAMA · doi:10.1001/jama.2010.1708 · Cited by 715
Polymorphisms Associated with Cholesterol and Risk of Cardiovascular Events
Kathiresan, S., Melander, O., Anevski, D., Guiducci, C., Burtt, N. P., Roos, C., Hirschhorn, J. N., Berglund, G., Hedblad, B., Groop, L., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0706728 · Cited by 696
Bates, S. M., Jaeschke, R., Stevens, S. M., Goodacre, S. W., Wells, P. S., Stevenson, M. D., Kearon, C., Schünemann, H. J., Crowther, M. A., Pauker, S. G., et al. (2012) · CHEST Journal · doi:10.1378/chest.11-2299 · Cited by 687
Ripatti, S., Tikkanen, E., Orho-Melander, M., Havulinna, A. S., Silander, K., Sharma, A., Guiducci, C., Perola, M., Jula, A., Sinisalo, J., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(10)61267-6 · Cited by 562
Buckley, D., Fu, R. F., Freeman, M., Rogers, K. R., & Helfand, M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-7-200910060-00009 · Cited by 561
Two further applications of a model for binary regression
COX, D. R. (1958) · Biometrika · doi:10.1093/biomet/45.3-4.562 · Cited by 556
Novel and Conventional Biomarkers for Prediction of Incident Cardiovascular Events in the Community
Melander, O. (2009) · JAMA · doi:10.1001/jama.2009.943 · Cited by 549
Lipid-Related Markers and Cardiovascular Disease Prediction
The Emerging Risk Factors Collaboration (2012) · JAMA · doi:10.1001/jama.2012.6571 · Cited by 547
Predictive accuracy of the Framingham coronary risk score in British men:prospective cohort study
Brindle, P. M., Emberson, J., Lampe, F. C., Walker, M., Whincup, P. H., Fahey, T. P., & Ebrahim, S. B. J. (2003) · BMJ · doi:10.1136/bmj.327.7426.1267 · Cited by 524
Serial Changes in Highly Sensitive Troponin I Assay and Early Diagnosis of Myocardial Infarction
Keller, T., Zeller, T., Ojeda, F., Tzikas, S., Lillpopp, L., Sinning, C., Wild, P. S., Genth‐Zotz, S., Warnholtz, A., Giannitsis, E., et al. (2011) · JAMA · doi:10.1001/jama.2011.1896 · Cited by 508
Interpreting Incremental Value of Markers Added to Risk Prediction Models
Pencina, M. J., D'Agostino, R. B., Pencina, K. M., Janssens, A. C. J. W., & Greenland, P. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kws207 · Cited by 506
Self-citationThe effect of including C-reactive protein in cardiovascular risk prediction models for women
Cook, N. R., Buring, J. E., & Ridker, P. M. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-1-200607040-00128 · Cited by 504
Cook, N. R., & Ridker, P. M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-11-200906020-00007 · Cited by 458
Performance of Common Genetic Variants in Breast-Cancer Risk Models
Wacholder, S., Hartge, P. A., Prentice, R. L., García‐Closas, M., Feigelson, H. S., Diver, W. R., Thun, M. J., Cox, D. G., Hankinson, S. E., Kraft, P. L., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0907727 · Cited by 416
Authors/Task Force Members:, Perk, J., Backer, G. D., Gohlke, H., Graham, I. D., Reiner, Ž., Verschuren, W. M. M., Albus, C., Benlian, P., Boysen, G., et al. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs254 · Cited by 390
Helfand, M., Buckley, D., Freeman, M., Fu, R. F., Rogers, K. R., Fleming, C., & Humphrey, L. L. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-7-200910060-00010 · Cited by 383
Hense, H. (2003) · European Heart Journal · doi:10.1016/s0195-668x(03)00081-2 · Cited by 370
Association between a literature-based genetic risk score and cardiovascular events in women
Paynter, N. P. (2010) · JAMA · doi:10.1001/jama.2010.119 · Cited by 361
Evaluation of Newer Risk Markers for Coronary Heart Disease Risk Classification: A Cohort Study
Kavousi, M., Elias-Smale, S., Rutten, J. H. W., Leening, M. J. G., Vliegenthart, R., Verwoert, G. C., Krestin, G. P., Oudkerk, M., de Maat, M. P. M., Leebeek, F. W. G., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-6-201203200-00006 · Cited by 360
Self-citationPencina, M. J., D’Agostino, R. B., & Demler, O. (2011) · Statistics in Medicine · doi:10.1002/sim.4348 · Cited by 348
Self-citationStatistical methods for assessment of added usefulness of new biomarkers
Pencina, M. J., D’Agostino, R. B., & Vasan, R. S. (2010) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm.2010.340 · Cited by 335
Self-citationAssociation Between Familial Atrial Fibrillation and Risk of New-Onset Atrial Fibrillation
Lubitz, S. A., Yin, X., Fontes, J. D., Magnani, J. W., Rienstra, M., Pai, M., Villalon, M. L., Vasan, R. S., Pencina, M. J., Levy, D., et al. (2010) · JAMA · doi:10.1001/jama.2010.1690 · Cited by 313
Self-citationMunshi, N. C., Anderson, K. C., Bergsagel, P. L., Shaughnessy, J. D., Palumbo, A. P., Durie, B. G. M., Fonseca, R., Stewart, A. K., Harousseau, J., Dimopoulos, M. A., et al. (2011) · Blood · doi:10.1182/blood-2010-10-300970 · Cited by 312
Comparisons of established risk prediction models for cardiovascular disease: systematic review
Siontis, G. C. M., Tzoulaki, I., Siontis, K. C., & Ioannidis, J. P. A. (2012) · BMJ · doi:10.1136/bmj.e3318 · Cited by 300
Visvanathan, K., Chlebowski, R. T., Hurley, P. A., Col, N. F., Ropka, M. E., Collyar, D. E., Morrow, M., Runowicz, C. D., Pritchard, K. I., Hagerty, K. L., et al. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2008.20.5179 · Cited by 285
Schelbert, E. B., Cao, J. J., Sigurdsson, S. O., Aspelund, T., Kellman, P., Aletras, A. H., Dyke, C. K., Þorgeirsson, G., Eiríksdóttir, G., Launer, L. J., et al. (2012) · JAMA · doi:10.1001/2012.jama.11089 · Cited by 284
Assessment of claims of improved prediction beyond the Framingham risk score.
Tzoulaki, I., Liberopoulos, G., & Ioannidis, J. (2009) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.2009.1757 · Cited by 281
Cook, N. R., Paynter, N. P., Eaton, C. B., Manson, J. E., Martin, L. W., Robinson, J. G., Rossouw, J. E., Wassertheil-Smoller, S. W., & Ridker, P. M. (2012) · Circulation · doi:10.1161/circulationaha.111.075929 · Cited by 266
Sudden Death After Myocardial Infarction
Adabağ, S. (2008) · JAMA · doi:10.1001/jama.2008.553 · Cited by 263
Paynter, N. P., Chasman, D. I., Buring, J. E., Shiffman, D., Cook, N. R., & Ridker, P. M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-2-200901200-00003 · Cited by 259
Assessing the Value of Risk Predictions by Using Risk Stratification Tables
Janes, H., PEPE, M. S., & Gu, W. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-10-200811180-00009 · Cited by 249
Vergouwe, Y., Moons, K. G. M., & Steyerberg, E. W. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq223 · Cited by 243
Self-citationTesting for improvement in prediction model performance
PEPE, M. S., Kerr, K. F., Longton, G., & Wang, Z. (2013) · Statistics in Medicine · doi:10.1002/sim.5727 · Cited by 239
Cornelis, M. C., Qi, L., Zhang, C., Kraft, P. L., Manson, J. E., Cai, T., Hunter, D. J., & Hu, F. B. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-8-200904210-00008 · Cited by 230
Association of Major and Minor ECG Abnormalities With Coronary Heart Disease Events
Bauer, D. C. (2012) · JAMA · doi:10.1001/jama.2012.434 · Cited by 213
Hilden, J., & Gerds, T. A. (2013) · Statistics in Medicine · doi:10.1002/sim.5804 · Cited by 200
One statistical test is sufficient for assessing new predictive markers
Vickers, A. J., Cronin, A. M., & Begg, C. B. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-13 · Cited by 198
Fonarow, G. C., Pan, W., Saver, J. L., Smith, E. E., Reeves, M. J., Broderick, J. P., Kleindorfer, D., Sacco, R. L., Olson, D. M., Hernandez, A. F., et al. (2012) · JAMA · doi:10.1001/jama.2012.7870 · Cited by 181
Rosenberg, S. J., Elashoff, M. R., Beineke, P., Daniels, S. E., Wingrove, J. A., Tingley, W. G., Sager, P. T., Sehnert, A. J., Yau, M., Kraus, W. E., et al. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-7-201010050-00005 · Cited by 173
Association Between 9p21 Genomic Markers and Heart Disease
Palomaki, G. E., Melillo, S. N., & Bradley, L. (2010) · JAMA · doi:10.1001/jama.2010.118 · Cited by 157
Beyond the Usual Prediction Accuracy Metrics: Reporting Results for Clinical Decision Making
Localio, A. R., & Goodman, S. N. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-4-201208210-00014 · Cited by 139
Individualized Guidelines: The Potential for Increasing Quality and Reducing Costs
Eddy, D. M., Adler, J. S., Patterson, B., Lucas, D., Smith, K. A., & Morris, M. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-154-9-201105030-00008 · Cited by 139
Chou, R., Arora, B., Dana, T., Fu, R. F., Walker, M., & Humphrey, L. L. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-6-201109200-00006 · Cited by 136
Issues in methodological research: perspectives from researchers and commissioners
Lilford, R. J., Richardson, A., Stevens, A., Fitzpatrick, R., Edwards, S., Rock, F., & Hutton, J. L. (2001) · Health Technology Assessment · doi:10.3310/hta5080 · Cited by 132
Assessing New Biomarkers and Predictive Models for Use in Clinical Practice
McGeechan, K., Macaskill, P., Irwig, L., Liew, G., & Wong, T. Y. (2008) · Archives of Internal Medicine · doi:10.1001/archinte.168.21.2304 · Cited by 132
Performance of reclassification statistics in comparing risk prediction models
Cook, N. R., & Paynter, N. P. (2011) · Biometrical Journal · doi:10.1002/bimj.201000078 · Cited by 121
Comments on ‘Integrated discrimination and net reclassification improvements—Practical advice’
Pencina, M. J., D’Agostino, R. B., D’Agostino, R. B., & Vasan, R. S. (2007) · Statistics in Medicine · doi:10.1002/sim.3106 · Cited by 112
Self-citationMcGeechan, K., Liew, G., Macaskill, P., Irwig, L., Klein, R., Sharrett, A. R., Klein, B. E. K., Wang, J. J., Chambless, L. E., & Wong, T. Y. (2008) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2008.02.094 · Cited by 112
Pepe, M. S., Feng, Z., & Gu, J. W. (2007) · Statistics in Medicine · doi:10.1002/sim.2991 · Cited by 105
de Boer, I. H., Levin, G., Robinson‐Cohen, C., Biggs, M. L., Hoofnagle, A. N., Siscovick, D. S., & Kestenbaum, B. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-9-201205010-00004 · Cited by 102
Cook, N. R. (2007) · Statistics in Medicine · doi:10.1002/sim.2987 · Cited by 102
Problems With Risk Reclassification Methods for Evaluating Prediction Models
Pepe, M. S. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwr013 · Cited by 102
Ganna, A., Reilly, M., Fairé, U. D., Pedersen, N., Magnusson, P. K. E., & Ingelsson, E. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr374 · Cited by 90
Reclassification Calculations for Persons With Incomplete Follow-up
Steyerberg, E. W., & Pencina, M. J. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-3-201002020-00019 · Cited by 88
Self-citationUsing Additional Information on Working Hours to Predict Coronary Heart Disease
Mika Kivimaki, B., Batty, G. D., Hamer, M., Ferrie, J. E., Vahtera, J., Virtanen, M., Marmot, M. G., Singh‐Manoux, A., & Shipley, M. J. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-154-7-201104050-00003 · Cited by 87
Greenland, S. (2007) · Statistics in Medicine · doi:10.1002/sim.2995 · Cited by 80
Evaluation of Markers and Risk Prediction Models
CALSTER, B. V., Vickers, A. J., Pencina, M. J., Baker, S. G., Timmerman, D., & Steyerberg, E. W. (2013) · Medical Decision Making · doi:10.1177/0272989x12470757 · Cited by 76
Self-citationKoller, M., Leening, M. J. G., Wolbers, M., Steyerberg, E. W., Hunink, M. G. M., Schoop, R., Albert Hofman, D., Bucher, H. C., Psaty, B. M., Lloyd‐Jones, D. M., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00002 · Cited by 74
Self-citationUse of reclassification for assessment of improved prediction: an empirical evaluation
Tzoulaki, I., Liberopoulos, G., & Ioannidis, J. P. A. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr013 · Cited by 70
Mihăescu, R., van Zitteren, M., van Hoek, M., Sijbrands, E. J. G., Uitterlinden, A. G., Witteman, J. C. M., Albert Hofman, D., Hunink, M. G. M., van Duijn, C. M., & Janssens, A. C. J. W. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq122 · Cited by 69
Self-citationValidity of the Framingham point scores in the elderly: Results from the Rotterdam study
Koller, M., Steyerberg, E. W., Wolbers, M., Stijnen, T., Bucher, H. C., Hunink, M. G. M., & Witteman, J. C. M. (2007) · American Heart Journal · doi:10.1016/j.ahj.2007.03.022 · Cited by 54
Self-citationBiases Introduced by Choosing Controls to Match Risk Factors of Cases in Biomarker Research
PEPE, M. S., Fan, J., Seymour, C. W., Li, C., Huang, Y., & Feng, Z. (2012) · Clinical Chemistry · doi:10.1373/clinchem.2012.186007 · Cited by 48
Austin, P. C., & Steyerberg, E. W. (2012) · Statistics in Medicine · doi:10.1002/sim.5598 · Cited by 47
Self-citationStrengthening the reporting of genetic risk prediction studies: the GRIPS statement
Janssens, A. C. J. W., Ioannidis, J. P. A., van Duijn, C. M., Little, J., Khoury, M. J., & the GRIPS Group (2011) · Genome Medicine · doi:10.1186/gm230 · Cited by 44
One-Year Risk for Advanced Colorectal Neoplasia: U.S. Versus U.K. Risk-Stratification Guidelines
Martínez, M. E., Thompson, P. A., Messer, K. S., Ashbeck, E. L., Lieberman, D. A., Baron, J. A., Ahnen, D. J., Robertson, D. J., Jacobs, E. T., Greenberg, E. R., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-12-201212180-00005 · Cited by 42
A Bias-Corrected Net Reclassification Improvement for Clinical Subgroups
Paynter, N. P., & Cook, N. R. (2012) · Medical Decision Making · doi:10.1177/0272989x12461856 · Cited by 39
Strengthening the reporting of Genetic RIsk Prediction Studies: the GRIPS Statement
Janssens, A. C. J. W., Ioannidis, J. P. A., van Duijn, C. M., Little, J., & Khoury, M. J. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.02.004 · Cited by 37
Use of risk stratification to guide ambulatory management of neutropenic fever
Worth, L. J., Lingaratnam, S. M., Taylor, A., Hayward, A., Morrissey, S., Cooney, J. P., Bastick, P. A., Eek, R. W., Wei, A. H., & Thursky, K. (2011) · Internal Medicine Journal · doi:10.1111/j.1445-5994.2010.02339.x · Cited by 36
Primary Prevention of Cardiovascular Disease
Hingorani, A. D., & Psaty, B. M. (2009) · JAMA · doi:10.1001/jama.2009.1698 · Cited by 36
Mühlenbruch, K., Heraclides, A., Steyerberg, E. W., Joost, H., Boeing, H., & Schulze, M. B. (2012) · European Journal of Epidemiology · doi:10.1007/s10654-012-9744-0 · Cited by 34
Self-citationChallenges to Improve Coronary Heart Disease Risk Assessment
Wilson, P. W. F. (2009) · JAMA · doi:10.1001/jama.2009.1765 · Cited by 26
Commentary: Reporting standards are needed for evaluations of risk reclassification
Pepe, M. S., & Janes, H. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr083 · Cited by 26
Mapping Out Biomarkers for Alzheimer Disease
Breteler, M. M. B. (2011) · JAMA · doi:10.1001/jama.2010.2017 · Cited by 24
Net reclassification improvement and decision theory
Vickers, A. J., Elkin, E. B., & Steyerberg, E. W. (2007) · Statistics in Medicine · doi:10.1002/sim.3087 · Cited by 24
Self-citationCook, N. R., & Paynter, N. P. (2011) · Statistics in Medicine · doi:10.1002/sim.4209 · Cited by 24
Merry, A. H. H., Boer, J. M. A., Schouten, L. J., Ambergen, T., Steyerberg, E. W., Feskens, E. J. M., Verschuren, W. M. M., Gorgels, A. P. M., & van den Brandt, P. A. (2011) · European Journal of Preventive Cardiology · doi:10.1177/1741826711410256 · Cited by 22
Self-citationStrengthening the Reporting of Genetic Risk Prediction Studies: The GRIPS Statement
Janssens, A. C. J. W., Ioannidis, J. P. A., van Duijn, C. M., Little, J., Khoury, M. J., & for the GRIPS Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-154-6-201103150-00008 · Cited by 16
Framework for Evaluating Novel Risk Markers
Hlatky, M. A. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-6-201203200-00013 · Cited by 13
Biomarkers for Prediction of Cardiovascular Events
Cook, N. R. (2009) · JAMA · doi:10.1001/jama.2009.1636 · Cited by 12
Fibrosis and Mortality in Patients With Dilated Cardiomyopathy
Leening, M. J. G., & Steyerberg, E. W. (2013) · JAMA · doi:10.1001/jama.2013.6470 · Cited by 11
Self-citationNet reclassification improvement: a link between statistics and clinical practice
Leening, M. J. G., & Cook, N. R. (2013) · European Journal of Epidemiology · doi:10.1007/s10654-012-9759-6 · Cited by 11
Self-citationCoronary Artery Calcium Score and Cardiovascular Event Prediction
McEvoy, J. W. (2010) · JAMA · doi:10.1001/jama.2010.1143 · Cited by 6
Complications Associated With Use of Bone-Morphogenetic Proteins in Spinal Fusion Procedures
Smoljanović, T. (2009) · JAMA · doi:10.1001/jama.2009.1639 · Cited by 6
Comparing costs associated with risk stratification rules for t-year survival
Cai, T., Tian, L., & Lloyd‐Jones, D. M. (2011) · Biostatistics · doi:10.1093/biostatistics/kxr001 · Cited by 6
Markers for Prediction of Cardiovascular Disease Risk
Kavousi, M., Leening, M. J. G., & Witteman, J. C. M. (2012) · JAMA · doi:10.1001/jama.2012.25786 · Cited by 5
Self-citationCoronary artery calcium for guiding statin treatment
Kengne, A. P., Echouffo-Tcheugui, J. B., & Sobngwi, E. (2012) · The Lancet · doi:10.1016/s0140-6736(12)60140-8 · Cited by 4
Risk categorization for calculating net reclassification improvement
Takahara, M., Katakami, N., Kaneto, H., & Shimomura, I. (2013) · European Journal of Epidemiology · doi:10.1007/s10654-013-9824-9 · Cited by 4
Modeling Cardiovascular Disease Prevention
Pletcher, M. J. (2010) · JAMA · doi:10.1001/jama.2010.188 · Cited by 2
Body-mass index, abdominal adiposity, and cardiovascular risk – Authors' reply
Wormser, D., Angelantonio, E. D., Sattar, N. A., Collins, R., Thompson, S. G., & Danesh, J. (2011) · The Lancet · doi:10.1016/s0140-6736(11)61122-7 · Cited by 2
Biomarkers for Prediction of Cardiovascular Events—Reply
Melander, O. (2009) · JAMA · doi:10.1001/jama.2009.1638 · Cited by 1
Flagged first, then by citations
1 of the 141 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Di Angelantonio E, 2012, JAMA, 307, 2499
- Melander O, 2009, JAMA, 302, 2090
- Greenland P, 2010, Circulation, 122, 584
- Hilden J, 2013, Stat Med
Cited by
Sorted by citations
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationTraumatic brain injury: integrated approaches to improve prevention, clinical care, and research
Maas, A. I. R., Menon, D. K., Adelson, P. D., Andelic, N., Bell, M. J., Belli, A., Bragge, P., Brazinova, A., Büki, A., Chesnut, R. M., et al. (2017) · The Lancet Neurology · doi:10.1016/s1474-4422(17)30371-x · Cited by 2,609
Self-citationTowards better clinical prediction models: seven steps for development and an ABCD for validation
Steyerberg, E. W., & Vergouwe, Y. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu207 · Cited by 1,928
Self-citationLi, X. S., Obeid, S., Klingenberg, R., Gencer, B., Mach, F., Räber, L., Windecker, S., Rodondi, N., Nanchen, D., Muller, O., et al. (2017) · European Heart Journal · doi:10.1093/eurheartj/ehw582 · Cited by 479
Lammers, W. J., Hirschfield, G. M., Corpechot, C., Nevens, F., Lindor, K. D., Janssen, H. L. A., Floreani, A., Ponsioen, C. Y., Mayo, M. J., Invernizzi, P., et al. (2015) · Gastroenterology · doi:10.1053/j.gastro.2015.07.061 · Cited by 441
Metadata
- Authors
- Leening, M. J. G., Vedder, M. M., Witteman, J. C. M., Pencina, M. J., & Steyerberg, E. W.
- Affiliations
- Duke University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2014-01-21
- Volume
- 160
- Issue
- 2
- Pages
- 122-131
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2103645914
- Cited by
- 594
- References
- 141
- Citations by year
- 2026: 38 · 2025: 30 · 2024: 41 · 2023: 59 · 2022: 59 · 2021: 53 · 2020: 56 · 2019: 39 · 2018: 49 · 2017: 44 · 2016: 55 · 2015: 47 · 2014: 23
- FWCI
- 7.11
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- net reclassification improvement, risk prediction model, model calibration, cost-effectiveness, reporting quality
- MeSH terms
- Data Interpretation, Statistical, Decision Support Techniques, Humans, Models, Statistical, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review