Quantifying discrimination of Framingham risk functions with different survival C statistics
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Cardiovascular risk prediction functions offer an important diagnostic tool for clinicians and patients themselves. They are usually constructed with the use of parametric or semi‐parametric survival regression models. It is essential to be able to evaluate the performance of these models, preferably with summaries that offer natural and intuitive interpretations. The concept of discrimination, popular in the logistic regression context, has been extended to survival analysis. However, the extension is not unique. In this paper, we define discrimination in survival analysis as the model's ability to separate those with longer event‐free survival from those with shorter event‐free survival within some time horizon of interest. This definition remains consistent with that used in logistic regression, in the sense that it assesses how well the model‐based predictions match the observed data. Practical and conceptual examples and numerical simulations are employed to examine four C statistics proposed in the literature to evaluate the performance of survival models. We observe that they differ in the numerical values and aspects of discrimination that they capture. We conclude that the index proposed by Harrell is the most appropriate to capture discrimination described by the above definition. We suggest researchers report which C statistic they are using, provide a rationale for their selection, and be aware that comparing different indices across studies may not be meaningful. Copyright © 2012 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study
D’Agostino, R. B., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008) · Circulation · doi:10.1161/circulationaha.107.699579 · Cited by 7,491 · cited 2×
“have presented new risk functions (also based on the Cox model [10]) that can be used for CVD and its component diseases [9].”
Self-citationPencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493 · cited 1×
“Finally, the word ‘separate’ may, but need not be, equated with ranking — distances can be built into measures that capture the correctness of ordering in a manner exemplified by discrimination slope used in the binary outcome case [29].”
Self-citationEvaluating the Yield of Medical Tests
Harrell, F. E. (1982) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1982.03320430047030 · Cited by 3,490
Epidemiological approaches to heart disease: the Framingham study
Dawber, T. R., Meadors, G. F., & Moore, F. E. (1951) · American Journal of Public Health and the Nations Health · doi:10.2105/ajph.41.3.279 · Cited by 2,148
Uno, H., Cai, T., Pencina, M. J., D'Agostino, R. B., & Wei, L. J. (2011) · Statistics in Medicine · doi:10.1002/sim.4154 · Cited by 1,535 · cited 5×
“[26] showed that formula (6) provides a consistent estimator for the AUC defined by (4), whereas estimator (5) converges to a quantity that still depends on censoring.”
InfluentialSelf-citationA new measure of prognostic separation in survival data
Royston, P., & Sauerbrei, W. (2004) · Statistics in Medicine · doi:10.1002/sim.1621 · Cited by 450
Cited by
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Baber, U., Mehran, R., Sartori, S., Schoos, M. M., Sillesen, H., Muntendam, P., Garcia, M. J., Gregson, J., Pocock, S., Falk, E., et al. (2015) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.01.017 · Cited by 461
“Discrimination was evaluated with the Harrell’s c-index (24).”
MethodKang, L., Chen, W., Petrick, N. A., & Gallas, B. D. (2015) · Statistics in Medicine · doi:10.1002/sim.6370 · Cited by 420
Ganz, P., Heidecker, B., Hveem, K., Jonasson, C., Kato, S., Segal, M. R., Sterling, D. G., & Williams, S. A. (2016) · JAMA · doi:10.1001/jama.2016.5951 · Cited by 364
Ho, J. E., Enserro, D., Brouwers, F. P., Kizer, J. R., Shah, S. J., Psaty, B. M., Bartz, T. M., Santhanakrishnan, R., Lee, D. S., Chan, C., et al. (2016) · Circulation: Heart Failure · doi:10.1161/circheartfailure.115.003116 · Cited by 362
Metadata
- Authors
- Pencina, M. J., D'Agostino, R. B., & Song, L.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2012-02-17
- Volume
- 31
- Issue
- 15
- Pages
- 1543-1553
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2134770445
- Cited by
- 95
- References
- 6
- FWCI
- 7.19
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.