Cited by 2,453

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers

Open access

Pencina, Michael J., D'Agostino, Ralph B., Steyerberg, Ewout W.·2011·Statistics in Medicine

Open this paper in CoChat — free account

Abstract

Appropriate quantification of added usefulness offered by new markers included in risk prediction algorithms is a problem of active research and debate. Standard methods, including statistical significance and c statistic are useful but not sufficient. Net reclassification improvement (NRI) offers a simple intuitive way of quantifying improvement offered by new markers and has been gaining popularity among researchers. However, several aspects of the NRI have not been studied in sufficient detail. In this paper we propose a prospective formulation for the NRI which offers immediate application to survival and competing risk data as well as allows for easy weighting with observed or perceived costs. We address the issue of the number and choice of categories and their impact on NRI. We contrast category‐based NRI with one which is category‐free and conclude that NRIs cannot be compared across studies unless they are defined in the same manner. We discuss the impact of differing event rates when models are applied to different samples or definitions of events and durations of follow‐up vary between studies. We also show how NRI can be applied to case–control data. The concepts presented in the paper are illustrated in a Framingham Heart Study example. In conclusion, NRI can be readily calculated for survival, competing risk, and case–control data, is more objective and comparable across studies using the category‐free version, and can include relative costs for classifications. We recommend that researchers clearly define and justify the choices they make when choosing NRI for their application. Copyright © 2010 John Wiley & Sons, Ltd.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1002/sim.4085

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581
Self-citation
General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study
D’Agostino, R. B., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008) · Circulation · doi:10.1161/circulationaha.107.699579 · Cited by 7,491
Self-citation
Projecting Individualized Probabilities of Developing Breast Cancer for White Females Who Are Being Examined Annually
Gail, M. H., Brinton, L. A., Byar, D. P., Corle, D. K., Green, S. B., Schairer, C., & Mulvihill, J. J. (1989) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/81.24.1879 · Cited by 3,334
C-Reactive Protein and Parental History Improve Global Cardiovascular Risk Prediction: The Reynolds Risk Score for Men
Ridker, P. M., Paynter, N. P., Rifai, N., Gaziano, J. M., & Cook, N. R. (2008) · Circulation · doi:10.1161/circulationaha.108.814251 · Cited by 841
Genotype score in addition to common risk factors for prediction of type 2 diabetes
Meigs, J. B., Shrader, P., Sullivan, L. M., McAteer, J. B., Fox, C. S., Dupuis, J., Manning, A. K., Florez, J. C., Wilson, P. W. F., D'Agostino, R. B., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0804742 · Cited by 759 · cited 1×
“On the other hand, the NRI seems to have gained popularity in medical journals, most likely due to its simplicity [7, 13].”
Self-citation

Show all 5 references

Research works printed but not linked: 24

10.1161/01.STR.25.1.40 · DOI · Paper page
10.1016/S0140-6736(09)60443-8 · DOI · Paper page
10.7326/0003-4819-148-2-200801150-00005 · DOI · Paper page
10.1161/CIRCULATIONAHA.106.672402 · DOI · Paper page
10.1002/sim.2929 · DOI · Paper page
10.1002/sim.2991 · DOI
10.1002/sim.3106 · DOI
10.1001/jama.298.7.776 · DOI · Paper page
Cook NR, 2009, Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures, Annals of Internal Medicine, 150, 795, 10.7326/0003-4819-150-11-200906020-00007 · DOI · Paper page
Steyerberg EW, 2010, Reclassification calculations with incomplete follow‐up, Annals of Internal Medicine, 152, 195, 10.7326/0003-4819-152-3-201002020-00019 · DOI
10.1161/CIRCULATIONAHA.108.816694 · DOI · Paper page
10.1002/sim.2995 · DOI
10.1002/sim.3087 · DOI
10.1080/01621459.1993.10476289 · DOI · Paper page
10.1126/science.ns-4.93.453-a · DOI · Paper page
10.1177/0272989X06295361 · DOI · Paper page
10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3 · doi:10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3
10.1002/sim.2300 · DOI
10.1002/sim.2987 · DOI
10.1097/EDE.0b013e3181c30fb2 · DOI · Paper page
10.1093/biomet/asp040 · DOI
10.1080/01621459.1996.10476660 · DOI · Paper page
10.1002/sim.1802 · DOI · Paper page
10.1002/sim.4780141909 · DOI

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citation
New insights into the genetic etiology of Alzheimer’s disease and related dementias
Bellenguez, C., Küçükali, F., Jansen, I. E., Kleineidam, L., Moreno-Grau, S., Amin, N., Naj, A. C., Campos-Martin, R., Grenier-Boley, B., Andrade, V., et al. (2022) · Nature Genetics · doi:10.1038/s41588-022-01024-z · Cited by 2,498
Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
“Reclassification measures, such as net reclassification improvement or index (NRI), evaluate whether a single biomarker has any incremental value to a prediction model [124,126].”
Background
Towards better clinical prediction models: seven steps for development and an ABCD for validation
Steyerberg, E. W., & Vergouwe, Y. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu207 · Cited by 1,928
Self-citation
Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer
Huang, Y.-Q., Liang, C.-H., He, L., Tian, J., Liang, C.-S., Chen, X., Ma, Z.-L., & Liu, Z.-Y. (2016) · Journal of Clinical Oncology · doi:10.1200/jco.2015.65.9128 · Cited by 1,765
PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508
Self-citation
HDL cholesterol efflux capacity and incident cardiovascular events
Rohatgi, A., Khera, A., Berry, J. D., Givens, E. G., Ayers, C. R., Wedin, K. E., Neeland, I. J., Yuhanna, I. S., Rader, D. R., de Lemos, J. A., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1409065 · Cited by 1,390
Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker
Moons, K. G. M., Kengne, A. P., Woodward, M., Royston, P., Vergouwe, Y., Altman, D. G., & Grobbee, D. E. (2012) · Heart · doi:10.1136/heartjnl-2011-301246 · Cited by 889
A New Risk Scheme to Predict Warfarin-Associated Hemorrhage
Fang, M. C., Go, A. S., Chang, Y., Borowsky, L. H., Pomernacki, N. K., Udaltsova, N., & Singer, D. E. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.03.031 · Cited by 852
Common Carotid Intima-Media Thickness Measurements in Cardiovascular Risk Prediction: A Meta-analysis
Den Ruijter, H. M., Peters, S. A. E., Anderson, T. J., Britton, A. R., Dekker, J. M., Eijkemans, M. J., Engström, G., Evans, G. W., de Graaf, J., Grobbee, D. E., et al. (2012) · JAMA · doi:10.1001/jama.2012.9630 · Cited by 760
Metabolite Profiling and Cardiovascular Event Risk: A Prospective Study of 3 Population-Based Cohorts
Würtz, P., Havulinna, A. S., Soininen, P., Tynkkynen, T., Prieto-Merino, D., Tillin, T., Ghorbani, A., Artati, A., Wang, Q., Tiainen, M., et al. (2015) · Circulation · doi:10.1161/circulationaha.114.013116 · Cited by 677
Net Reclassification Improvement: Computation, Interpretation, and Controversies: A Literature Review and Clinician's Guide
Leening, M. J. G., Vedder, M. M., Witteman, J. C. M., Pencina, M. J., & Steyerberg, E. W. (2014) · Annals of Internal Medicine · doi:10.7326/m13-1522 · Cited by 594
Self-citation
An open approach to systematically prioritize causal variants and genes at all published human GWAS trait-associated loci
Mountjoy, E., Schmidt, E. M., Carmona, M., Schwartzentruber, J., Peat, G., Miranda, A., Fumis, L., Hayhurst, J., Buniello, A., Karim, M. A., et al. (2021) · Nature Genetics · doi:10.1038/s41588-021-00945-5 · Cited by 581
Development of a novel frailty index to predict mortality in patients with end‐stage liver disease
Lai, J. C., Covinsky, K. E., Dodge, J. L., Boscardin, W. J., Segev, D. L., Roberts, J. P., & Feng, S. (2017) · Hepatology · doi:10.1002/hep.29219 · Cited by 556
Interpreting Incremental Value of Markers Added to Risk Prediction Models
Pencina, M. J., D'Agostino, R. B., Pencina, K. M., Janssens, A. C. J. W., & Greenland, P. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kws207 · Cited by 506
Self-citation
Prognostic Utility of Novel Biomarkers of Cardiovascular Stress: The Framingham Heart Study
Wang, T. J., Wollert, K. C., Larson, M. G., Coglianese, E., McCabe, E. L., Cheng, S., Ho, J. E., Fradley, M. G., Ghorbani, A., Xanthakis, V., et al. (2012) · Circulation · doi:10.1161/circulationaha.112.129437 · Cited by 482
Prevalence, Impact, and Predictive Value of Detecting Subclinical Coronary and Carotid Atherosclerosis in Asymptomatic Adults
Baber, U., Mehran, R., Sartori, S., Schoos, M. M., Sillesen, H., Muntendam, P., Garcia, M. J., Gregson, J., Pocock, S., Falk, E., et al. (2015) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.01.017 · Cited by 461
Trans-ancestry genome-wide association meta-analysis of prostate cancer identifies new susceptibility loci and informs genetic risk prediction
Conti, D. V., Darst, B. F., Moss, L. C., Saunders, E. J., Sheng, X., Chou, A., Schumacher, F. R., Olama, A. A. A., Benlloch, S., Dadaev, T., et al. (2021) · Nature Genetics · doi:10.1038/s41588-020-00748-0 · Cited by 462
Independent and Incremental Value of Deformation Indices for Prediction of Trastuzumab-Induced Cardiotoxicity
Negishi, K., Negishi, T., Hare, J. L., Haluska, B. A., Plana, J. C., & Marwick, T. H. (2013) · Journal of the American Society of Echocardiography · doi:10.1016/j.echo.2013.02.008 · Cited by 454
Arterial Wave Reflections and Incident Cardiovascular Events and Heart Failure
Chirinos, J. A., Kips, J. G., Jacobs, D. R., Brumback, L., Duprez, D. A., Kronmal, R., Bluemke, D. A., Townsend, R. R., Vermeersch, S., & Segers, P. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.07.054 · Cited by 431
Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance
Van Calster, B., & Vickers, A. J. (2015) · Medical Decision Making · doi:10.1177/0272989x14547233 · Cited by 371
Development and Validation of a Protein-Based Risk Score for Cardiovascular Outcomes Among Patients With Stable Coronary Heart Disease
Ganz, P., Heidecker, B., Hveem, K., Jonasson, C., Kato, S., Segal, M. R., Sterling, D. G., & Williams, S. A. (2016) · JAMA · doi:10.1001/jama.2016.5951 · Cited by 364
A metabolic profile of all-cause mortality risk identified in an observational study of 44,168 individuals
Deelen, J., Kettunen, J., Fischer, K., van der Spek, A., Trompet, S., Kastenmüller, G., Boyd, A., Zierer, J., van den Akker, E. B., Ala-Korpela, M., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-11311-9 · Cited by 362
Evaluation of Newer Risk Markers for Coronary Heart Disease Risk Classification: A Cohort Study
Kavousi, M., Elias-Smale, S., Rutten, J. H. W., Leening, M. J. G., Vliegenthart, R., Verwoert, G. C., Krestin, G. P., Oudkerk, M., de Maat, M. P. M., Leebeek, F. W. G., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-6-201203200-00006 · Cited by 360
Self-citation
Novel metrics for evaluating improvement in discrimination: net reclassification and integrated discrimination improvement for normal variables and nested models
Pencina, M. J., D'Agostino, R. B., & Demler, O. V. (2012) · Statistics in Medicine · doi:10.1002/sim.4348 · Cited by 349
Self-citation
Biomarker Profiling by Nuclear Magnetic Resonance Spectroscopy for the Prediction of All-Cause Mortality: An Observational Study of 17,345 Persons
Fischer, K., Kettunen, J., Würtz, P., Haller, T., Havulinna, A. S., Kangas, A. J., Soininen, P., Esko, T., Tammesoo, M.-L., Mägi, R., et al. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001606 · Cited by 344
Temporal Relationship and Predictive Value of Urinary Acute Kidney Injury Biomarkers After Pediatric Cardiopulmonary Bypass
Krawczeski, C. D., Goldstein, S. L., Woo, J. G., Wang, Y., Piyaphanee, N., Ma, Q., Bennett, M., & Devarajan, P. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.08.017 · Cited by 332
Cardiovascular Disease Risk Assessment: Insights from Framingham
D’Agostino Sr., R. B., Pencina, M. J., Massaro, J. M., & Coady, S. (2013) · Global Heart · doi:10.1016/j.gheart.2013.01.001 · Cited by 329
Self-citation
ECG-Based Deep Learning and Clinical Risk Factors to Predict Atrial Fibrillation
Khurshid, S., Friedman, S., Reeder, C., Di Achille, P., Diamant, N., Singh, P., Harrington, L. X., Wang, X., Al-Alusi, M. A., Sarma, G., et al. (2022) · Circulation · doi:10.1161/circulationaha.121.057480 · Cited by 322
Association Between Familial Atrial Fibrillation and Risk of New-Onset Atrial Fibrillation
Lubitz, S. A., Yin, X., Fontes, J. D., Magnani, J. W., Rienstra, M., Pai, M., Villalon, M. L., Vasan, R. S., Pencina, M. J., Levy, D., et al. (2010) · JAMA · doi:10.1001/jama.2010.1690 · Cited by 313
Self-citation
Discrimination and Net Reclassification of Cardiovascular Risk With Lipoprotein(a)
Willeit, P., Kiechl, S., Kronenberg, F., Witztum, J. L., Santer, P., Mayr, M., Xu, Q., Mayr, A., Willeit, J., & Tsimikas, S. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2014.03.061 · Cited by 286
Prevalence and Prognosis of Unrecognized Myocardial Infarction Determined by Cardiac Magnetic Resonance in Older Adults
Schelbert, E. B., Cao, J. J., Sigurdsson, S., Aspelund, T., Kellman, P., Aletras, A. H., Dyke, C. K., Thorgeirsson, G., Eiriksdottir, G., Launer, L. J., et al. (2012) · JAMA · doi:10.1001/2012.jama.11089 · Cited by 284
Prevalence and Prognostic Impact of the Coexistence of Multiple Frailty Domains in Elderly Patients with Heart Failure: The FRAGILE-HF Cohort Study
Matsue, Y., Kamiya, K., Saito, H., Saito, K., Ogasahara, Y., Maekawa, E., Konishi, M., Kitai, T., Iwata, K., Jujo, K., et al. (2020) · European Journal of Heart Failure · doi:10.1002/ejhf.1926 · Cited by 283
Trimethylamine N-oxide and prognosis in acute heart failure
Suzuki, T., Heaney, L. M., Bhandari, S. S., Jones, D. J. L., & Ng, L. L. (2016) · Heart · doi:10.1136/heartjnl-2015-308826 · Cited by 253
Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review
Binuya, M. A. E., Engelhardt, E. G., Schats, W., Schmidt, M. K., & Steyerberg, E. W. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01801-8 · Cited by 246
Self-citation
Predicting Coronary Heart Disease Using Risk Factor Categories for a Japanese Urban Population, and Comparison with the Framingham Risk Score: The Suita Study
Nishimura, K., Okamura, T., Watanabe, M., Nakai, M., Takegami, M., Higashiyama, A., Kokubo, Y., Okayama, A., & Miyamoto, Y. (2014) · Journal of Atherosclerosis and Thrombosis · doi:10.5551/jat.19356 · Cited by 228
Assessing the incremental value of diagnostic and prognostic markers: a review and illustration
Steyerberg, E. W., Pencina, M. J., Lingsma, H. F., Kattan, M. W., Vickers, A. J., & Van Calster, B. (2012) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2011.02562.x · Cited by 219
Self-citation
Nontraditional Risk Factors in Cardiovascular Disease Risk Assessment: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force
Lin, J. S., Evans, C. V., Johnson, E., Redmond, N., Coppola, E. L., & Smith, N. (2018) · JAMA · doi:10.1001/jama.2018.4242 · Cited by 216
Coronary Artery Calcium Score for Long-term Risk Classification in Individuals With Type 2 Diabetes and Metabolic Syndrome From the Multi-Ethnic Study of Atherosclerosis
Malik, S., Zhao, Y., Budoff, M., Nasir, K., Blumenthal, R. S., Bertoni, A. G., & Wong, N. D. (2017) · JAMA Cardiology · doi:10.1001/jamacardio.2017.4191 · Cited by 215
A Genetic Risk Score Is Associated With Incident Cardiovascular Disease and Coronary Artery Calcium: The Framingham Heart Study
Thanassoulis, G., Peloso, G. M., Pencina, M. J., Hoffmann, U., Fox, C. S., Cupples, L. A., Levy, D., D'Agostino, R. B., Hwang, S.-J., & O'Donnell, C. J. (2012) · Circulation: Cardiovascular Genetics · doi:10.1161/circgenetics.111.961342 · Cited by 211
Self-citation
Comparison of Conventional Lipoprotein Tests and Apolipoproteins in the Prediction of Cardiovascular Disease: Data From UK Biobank
Welsh, C., Celis-Morales, C. A., Brown, R., Mackay, D. F., Lewsey, J., Mark, P. B., Gray, S. R., Ferguson, L. D., Anderson, J. J., Lyall, D. M., et al. (2019) · Circulation · doi:10.1161/circulationaha.119.041149 · Cited by 191
Carotid plaque thickness and carotid plaque burden predict future cardiovascular events in asymptomatic adult Americans
Sillesen, H., Sartori, S., Sandholt, B., Baber, U., Mehran, R., & Fuster, V. (2018) · European Heart Journal - Cardiovascular Imaging · doi:10.1093/ehjci/jex239 · Cited by 204
Multivariable Modeling Strategies
Harrell, F. E. (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7_4 · Cited by 221
Serum Brain–Derived Neurotrophic Factor and Vascular Endothelial Growth Factor Levels Are Associated With Risk of Stroke and Vascular Brain Injury: Framingham Study
Pikula, A., Beiser, A. S., Chen, T. C., Preis, S. R., Vorgias, D., DeCarli, C., Au, R., Kelly-Hayes, M., Kase, C. S., Wolf, P. A., et al. (2013) · Stroke · doi:10.1161/strokeaha.113.001447 · Cited by 200
Added Predictive Value of Night-Time Blood Pressure Variability for Cardiovascular Events and Mortality: The Ambulatory Blood Pressure–International Study
Palatini, P., Reboldi, G., Beilin, L. J., Casiglia, E., Eguchi, K., Imai, Y., Kario, K., Ohkubo, T., Pierdomenico, S. D., Schwartz, J. E., et al. (2014) · Hypertension · doi:10.1161/hypertensionaha.114.03694 · Cited by 193
The Net Reclassification Index (NRI): a misleading measure of prediction improvement even with independent test data sets
Pepe, M. S., Fan, J., Feng, Z., Gerds, T., & Hilden, J. (2015) · Statistics in Biosciences · doi:10.1007/s12561-014-9118-0 · Cited by 191
“large portion of simulations the NRI statistic was positive and statistically significant in the test dataset using the standard test based on the empirically estimated NRI divided by the standard error provided in [11,12].”
Radiomics signature of computed tomography imaging for prediction of survival and chemotherapeutic benefits in gastric cancer
Jiang, Y., Chen, C., Xie, J., Wang, W., Zha, X., Lv, W., Chen, H., Hu, Y., Li, T., Yu, J., et al. (2018) · EBioMedicine · doi:10.1016/j.ebiom.2018.09.007 · Cited by 184
Genetic Risk Prediction and a 2-Stage Risk Screening Strategy for Coronary Heart Disease
Tikkanen, E., Havulinna, A. S., Palotie, A., Salomaa, V., & Ripatti, S. (2013) · Arteriosclerosis, Thrombosis, and Vascular Biology · doi:10.1161/atvbaha.112.301120 · Cited by 175
Extracellular matrix proteomics identifies molecular signature of symptomatic carotid plaques
Langley, S. R., Willeit, K., Didangelos, A., Matic, L. P., Skroblin, P., Barallobre-Barreiro, J., Lengquist, M., Rungger, G., Kapustin, A., Kedenko, L., et al. (2017) · Journal of Clinical Investigation · doi:10.1172/jci86924 · Cited by 174
Prediction of Outcome in Patients With Acute Ischemic Stroke Based on Initial Severity and Improvement in the First 24 h
Wouters, A., Nysten, C., Thijs, V., & Lemmens, R. (2018) · Frontiers in Neurology · doi:10.3389/fneur.2018.00308 · Cited by 162
Balancing stroke and bleeding risks in patients with atrial fibrillation and renal failure: the Swedish Atrial Fibrillation Cohort study
Friberg, L., Benson, L., & Lip, G. Y. H. (2015) · European Heart Journal · doi:10.1093/eurheartj/ehu139 · Cited by 161
Progression of diabetic kidney disease and trajectory of kidney function decline in Chinese patients with Type 2 diabetes
Jiang, G., Luk, A. O. Y., Tam, C. H. T., Xie, F., Carstensen, B., Lau, E. S. H., Lim, C. K. P., Lee, H. M., Ng, A. C. W., Ng, M. C. Y., et al. (2019) · Kidney International · doi:10.1016/j.kint.2018.08.026 · Cited by 159
A Polygenic Risk Score to Refine Risk Stratification and Prediction for Severe Liver Disease by Clinical Fibrosis Scores
De Vincentis, A., Tavaglione, F., Jamialahmadi, O., Picardi, A., Antonelli Incalzi, R., Valenti, L., Romeo, S., & Vespasiani-Gentilucci, U. (2022) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2021.05.056 · Cited by 158
Fructosamine and glycated albumin and the risk of cardiovascular outcomes and death
Selvin, E., Rawlings, A. M., Lutsey, P. L., Maruthur, N., Pankow, J. S., Steffes, M., & Coresh, J. (2015) · Circulation · doi:10.1161/circulationaha.115.015415 · Cited by 151
Hypertensive Disorders of Pregnancy and 10-Year Cardiovascular Risk Prediction
Stuart, J. J., Tanz, L. J., Cook, N. R., Spiegelman, D., Missmer, S. A., Rimm, E. B., Rexrode, K. M., Mukamal, K. J., & Rich-Edwards, J. W. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.05.077 · Cited by 154
Novel genetic markers improve measures of atrial fibrillation risk prediction
Everett, B. M., Cook, N. R., Conen, D., Chasman, D. I., Ridker, P. M., & Albert, C. M. (2013) · European Heart Journal · doi:10.1093/eurheartj/eht033 · Cited by 141
Deep Learning to Assess Long-term Mortality From Chest Radiographs
Lu, M. T., Ivanov, A., Mayrhofer, T., Hosny, A., Aerts, H. J. W. L., & Hoffmann, U. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.7416 · Cited by 140
Enhanced liver fibrosis score predicts transplant-free survival in primary sclerosing cholangitis
Vesterhus, M., Hov, J. R., Holm, A., Schrumpf, E., Nygård, S., Godang, K., Andersen, I. M., Næss, S., Thorburn, D., Saffioti, F., et al. (2015) · Hepatology · doi:10.1002/hep.27825 · Cited by 138
Reproductive Risk Factors and Coronary Heart Disease in the Women’s Health Initiative Observational Study
Parikh, N. I., Jeppson, R. P., Berger, J. S., Eaton, C. B., Kroenke, C. H., LeBlanc, E. S., Lewis, C. E., Loucks, E. B., Parker, D. R., Rillamas-Sun, E., et al. (2016) · Circulation · doi:10.1161/circulationaha.115.017854 · Cited by 134
Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: a pilot usability study.
Brennan, M., Puri, S., Ozrazgat-Baslanti, T., Feng, Z., Ruppert, M., Hashemighouchani, H., Momcilovic, P., Li, X., Wang, D. Z., & Bihorac, A. (2019) · Surgery · doi:10.1016/j.surg.2019.01.002 · Cited by 131
Mini-Cog Performance: Novel Marker of Post Discharge Risk Among Patients Hospitalized for Heart Failure
Patel, A., Parikh, R., Howell, E. H., Hsich, E., Landers, S. H., & Gorodeski, E. Z. (2015) · Circulation: Heart Failure · doi:10.1161/circheartfailure.114.001438 · Cited by 133
Prognostic value of cardiovascular biomarkers in the population
Neumann, J. T., Twerenbold, R., Weimann, J., Ballantyne, C. M., Benjamin, E. J., Costanzo, S., de Lemos, J. A., deFilippi, C. R., Di Castelnuovo, A., Donfrancesco, C., et al. (2024) · JAMA · doi:10.1001/jama.2024.5596 · Cited by 125
Prognostic utility of novel biomarkers of cardiovascular stress in patients with aortic stenosis undergoing valve replacement
Lindman, B. R., Breyley, J. G., Schilling, J. D., Vatterott, A. M., Zajarias, A., Maniar, H. S., Damiano, R. J., Moon, M. R., Lawton, J. S., Gage, B. F., et al. (2015) · Heart · doi:10.1136/heartjnl-2015-307742 · Cited by 114
Twelve-single nucleotide polymorphism genetic risk score identifies individuals at increased risk for future atrial fibrillation and stroke
Tada, H., Shiffman, D., Smith, J. G., Sjögren, M., Lubitz, S. A., Ellinor, P. T., Louie, J. Z., Catanese, J. J., Engström, G., Devlin, J. J., et al. (2014) · Stroke · doi:10.1161/strokeaha.114.006072 · Cited by 110
Deep Learning to Estimate Biological Age From Chest Radiographs
Raghu, V. K., Weiss, J., Hoffmann, U., Aerts, H. J. W. L., & Lu, M. T. (2021) · JACC: Cardiovascular Imaging · doi:10.1016/j.jcmg.2021.01.008 · Cited by 101
Prevalence and prognostic implications of malnutrition as defined by GLIM criteria in elderly patients with heart failure
Hirose, S., Matsue, Y., Kamiya, K., Kagiyama, N., Hiki, M., Dotare, T., Sunayama, T., Konishi, M., Saito, H., Saito, K., et al. (2021) · Clinical Nutrition · doi:10.1016/j.clnu.2021.01.014 · Cited by 101
Improved predictive models for acute kidney injury with IDEA: Intraoperative Data Embedded Analytics.
Adhikari, L., Ozrazgat-Baslanti, T., Ruppert, M., Madushani, R. W. M. A., Paliwal, S., Hashemighouchani, H., Zheng, F., Tao, M., Lopes, J. M., Li, X., et al. (2019) · PLOS ONE · doi:10.1371/journal.pone.0214904 · Cited by 98
Machine learning improves risk stratification after acute coronary syndrome
Myers, P. D., Scirica, B. M., & Stultz, C. M. (2017) · Scientific Reports · doi:10.1038/s41598-017-12951-x · Cited by 75
High-sensitivity cardiac troponin T and the risk of incident atrial fibrillation: The Atherosclerosis Risk in Communities (ARIC) study
Filion, K. B., Agarwal, S. K., Ballantyne, C. M., Eberg, M., Hoogeveen, R. C., Huxley, R. R., Loehr, L. R., Nambi, V., Soliman, E. Z., & Alonso, A. (2015) · American Heart Journal · doi:10.1016/j.ahj.2014.10.005 · Cited by 73
Haemoglobin A1c even within non-diabetic level is a predictor of cardiovascular disease in a general Japanese population: the Hisayama Study
Ikeda, F., Doi, Y., Ninomiya, T., Hirakawa, Y., Mukai, N., Hata, J., Shikata, K., Yoshida, D., Matsumoto, T., Kitazono, T., et al. (2013) · Cardiovascular Diabetology · doi:10.1186/1475-2840-12-164 · Cited by 67
Artificial Intelligence–Enabled Prediction of Heart Failure Risk From Single-Lead Electrocardiograms
Dhingra, L. S., Aminorroaya, A., Pedroso, A. F., Khunte, A., Sangha, V., McIntyre, D., Chow, C. K., Asselbergs, F. W., Brant, L. C. C., Barreto, S. M., et al. (2025) · JAMA Cardiology · doi:10.1001/jamacardio.2025.0492 · Cited by 65
Comparative studies of 2168 plasma proteins measured by two affinity-based platforms in 4000 Chinese adults
Wang, B., Pozarickij, A., Mazidi, M., Wright, N., Yao, P., Said, S., Iona, A., Kartsonaki, C., Fry, H., Lin, K., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56935-2 · Cited by 64
Protein markers and risk of type 2 diabetes and prediabetes: a targeted proteomics approach in the KORA F4/FF4 study
Huth, C., von Toerne, C., Schederecker, F., de las Heras Gala, T., Herder, C., Kronenberg, F., Meisinger, C., Rathmann, W., Koenig, W., Waldenberger, M., et al. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-018-0475-8 · Cited by 56
Baseline and long-term fibrinogen levels and risk of sudden cardiac death: A new prospective study and meta-analysis
Kunutsor, S. K., Kurl, S., Zaccardi, F., & Laukkanen, J. A. (2016) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2015.12.020 · Cited by 58
Circulating fatty acids and risk of gestational diabetes mellitus: prospective analyses in China
Pan, X.-F., Huang, Y., Li, X., Wang, Y., Ye, Y., Chen, H., Marklund, M., Wen, Y., Liu, Y., Zeng, H., et al. (2021) · European Journal of Endocrinology · doi:10.1530/eje-21-0118 · Cited by 57
Circulating IGFBP‐2: a novel biomarker for incident dementia
McGrath, E. R., Himali, J. J., Levy, D., Conner, S. C., DeCarli, C. S., Pase, M. P., Courchesne, P., Satizabal, C. L., Vasan, R. S., Beiser, A. S., et al. (2019) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.50854 · Cited by 49
Population-Based Evidence that Survival in Amyotrophic Lateral Sclerosis Is Related to Weight Loss at Diagnosis
Marin, B., Arcuti, S., Jesus, P., Logroscino, G., Copetti, M., Fontana, A., Nicol, M., Raymondeau, M., Desport, J. C., Preux, P. M., et al. (2016) · Neurodegenerative Diseases · doi:10.1159/000442444 · Cited by 48
Non-calcific aortic tissue quantified from computed tomography angiography improves diagnosis and prognostication of patients referred for transcatheter aortic valve implantation
Grodecki, K., Tamarappoo, B. K., Huczek, Z., Jedrzejczyk, S., Cadet, S., Kwiecinski, J., Rymuza, B., Parma, R., Olasinska-Wisniewska, A., Fijalkowska, J., et al. (2021) · European Heart Journal - Cardiovascular Imaging · doi:10.1093/ehjci/jeaa304 · Cited by 38
Chapter 12: Systematic Review of Prognostic Tests
Rector, T. S., Taylor, B. C., & Wilt, T. J. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-011-1899-y · Cited by 37
Circulating peroxiredoxin 4 and type 2 diabetes risk: the Prevention of Renal and Vascular Endstage Disease (PREVEND) study
Abbasi, A., Corpeleijn, E., Gansevoort, R. T., Gans, R. O. B., Struck, J., Schulte, J., Hillege, H. L., van der Harst, P., Stolk, R. P., Navis, G., et al. (2014) · Diabetologia · doi:10.1007/s00125-014-3278-9 · Cited by 27
Artificial intelligence-enhanced electrocardiography to predict regurgitant valvular heart diseases: an international study
Liang, Y., Sau, A., Zeidaabadi, B., Barker, J., Patlatzoglou, K., Pastika, L., Sieliwonczyk, E., Whinnett, Z., Peters, N. S., Yu, Z., et al. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf448 · Cited by 26
Associations Between Macrophage Colony‐Stimulating Factor and Monocyte Chemotactic Protein 1 in Plasma and First‐Time Coronary Events: A Nested Case–Control Study
Schiopu, A., Bengtsson, E., Gonçalves, I., Nilsson, J., Fredrikson, G. N., & Björkbacka, H. (2016) · Journal of the American Heart Association · doi:10.1161/jaha.115.002851 · Cited by 25
Artificial Intelligence in Health Data Analysis: The Darwinian Evolution Theory Suggests an Extremely Simple and Zero-Cost Large-Scale Screening Tool For Prediabetes and Type 2 Diabetes
Buccheri, E., Dell'Aquila, D., & Russo, M. (2021) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2021.108722 · Cited by 24
Prognostic significance of on-treatment home and clinic blood pressure for predicting cardiovascular events in hypertensive patients in the HONEST study
Shimada, K., Kario, K., Kushiro, T., Teramukai, S., Zenimura, N., Ishikawa, Y., Okuda, Y., & Saito, I. (2016) · Journal of Hypertension · doi:10.1097/hjh.0000000000000966 · Cited by 24
Remnant cholesterol and long-term incidence of death in coronary artery disease patients
Drexel, H., Mader, A., Larcher, B., Festa, A., Vonbank, A., Fraunberger, P., Leiherer, A., & Saely, C. H. (2025) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2024.119048 · Cited by 17
Predicting Heart Failure From 12-Lead ECGs Using AI
Desai, A. S., Pandey, A., Suratekar, R., Ahmad, F. S., Alger, H. M., Anto, A. G., Nujum G, S., Lopez-Jimenez, F., Oh, J. K., Asfahan, S., et al. (2026) · JACC · doi:10.1016/j.jacc.2025.10.065 · Cited by 16
Updated risk factors should be used to predict development of diabetes
Bethel, M. A., Hyland, K. A., Chacra, A. R., Deedwania, P., Fulcher, G. R., Holman, R. R., Jenssen, T., Levitt, N. S., McMurray, J. J. V., Boutati, E., et al. (2017) · Journal of Diabetes and its Complications · doi:10.1016/j.jdiacomp.2017.02.012 · Cited by 12
Artificial Intelligence-Enabled ECG Analysis to Predict Incident Heart Failure
Khurshid, S., Friedman, S. F., Kany, S., D’Souza, V., Roshandelpoor, A., Kosyakovsky, L. B., Chitsazan, M., Cunningham, J. W., Singh, P., Lau, E. S., et al. (2026) · Circulation: Heart Failure · doi:10.1161/circheartfailure.125.013927 · Cited by 2

Authors: Pencina, M. J., D'Agostino, R. B., & Steyerberg, E. W.
Venue: Statistics in Medicine
Publisher: Wiley
Published: 2010-11-05
Volume: 30
Issue: 1
Pages: 11-21
ISSN: 0277-6715 (ISSN-L)
1097-0258
DOI: 10.1002/sim.4085
OpenAlex: W2106421416
Cited by: 2,453
References: 5
FWCI: 108.81
Type: journal article
Language: en
Open access: green
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Self-check

Citations

Cites

Cited by

Metadata

Record

About these checks

Data sources