Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Background: Clinicians who are using the Framingham Risk Score ( FRS ) or the American College of Cardiology/American Heart Association Pooled Cohort Equations ( PCE ) to estimate risk for their patients based on electronic health data ( EHD ) face 4 questions. (1) Do published risk scores applied to EHD yield accurate estimates of cardiovascular risk? (2) Are FRS risk estimates, which are based on data that are up to 45 years old, valid for a contemporary patient population seeking routine care? (3) Do the PCE make the FRS obsolete? (4) Does refitting the risk score using EHD improve the accuracy of risk estimates? Methods and Results: Data were extracted from the EHD of 84 116 adults aged 40 to 79 years who received care at a large healthcare delivery and insurance organization between 2001 and 2011. We assessed calibration and discrimination for 4 risk scores: published versions of FRS and PCE and versions obtained by refitting models using a subset of the available EHD . The published FRS was well calibrated (calibration statistic K=9.1, miscalibration ranging from 0% to 17% across risk groups), but the PCE displayed modest evidence of miscalibration (calibration statistic K=43.7, miscalibration from 9% to 31%). Discrimination was similar in both models (C‐index=0.740 for FRS , 0.747 for PCE ). Refitting the published models using EHD did not substantially improve calibration or discrimination. Conclusions: We conclude that published cardiovascular risk models can be successfully applied to EHD to estimate cardiovascular risk; the FRS remains valid and is not obsolete; and model refitting does not meaningfully improve the accuracy of risk estimates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Stone, N. J., Robinson, J. G., Lichtenstein, A. H., Bairey Merz, C. N., Blum, C. B., Eckel, R. H., Goldberg, A. C., Gordon, D., Levy, D., Lloyd-Jones, D. M., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.002 · Cited by 3,845
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697
Explaining the decrease in U.S. deaths from coronary disease, 1980–2000
Ford, E. S., Ajani, U. A., Croft, J. B., Critchley, J. A., Labarthe, D. R., Kottke, T. E., Giles, W. H., & Capewell, S. (2007) · New England Journal of Medicine · doi:10.1056/nejmsa053935 · Cited by 2,757
Self-citationD'Agostino, R. B., Grundy, S., Sullivan, L. M., Wilson, P., & for the CHD Risk Prediction Group (2001) · JAMA · doi:10.1001/jama.286.2.180 · Cited by 2,108 · cited 5×
“(4) Refitting models using EHD did not offer substantive improvements in calibration or discrimination and is unlikely to be necessary in practice.”
BackgroundMethodInfluentialA REVIEW OF GOODNESS OF FIT STATISTICS FOR USE IN THE DEVELOPMENT OF LOGISTIC REGRESSION MODELS1
LEMESHOW, S., & HOSMER, D. W. (1982) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a113284 · Cited by 1,947
Validation of the Atherosclerotic Cardiovascular Disease Pooled Cohort Risk Equations
Muntner, P., Colantonio, L. D., Cushman, M., Goff, D. C., Howard, G., Howard, V. J., Kissela, B., Levitan, E. B., Lloyd-Jones, D. M., & Safford, M. M. (2014) · JAMA · doi:10.1001/jama.2014.2630 · Cited by 603
Prognostic Utility of Novel Biomarkers of Cardiovascular Stress: The Framingham Heart Study
Wang, T. J., Wollert, K. C., Larson, M. G., Coglianese, E., McCabe, E. L., Cheng, S., Ho, J. E., Fradley, M. G., Ghorbani, A., Xanthakis, V., et al. (2012) · Circulation · doi:10.1161/circulationaha.112.129437 · Cited by 482 · cited 2×
“(40,45] (45,50] (50,55] (55,60] (60,65] (65,70] (70,75] (75,80] Age group Av er ag e pr ed ic te d 5− ye ar C V ri sk”
de Ruijter, W., Westendorp, R. G. J., Assendelft, W. J. J., den Elzen, W. P. J., de Craen, A. J. M., le Cessie, S., & Gussekloo, J. (2009) · BMJ · doi:10.1136/bmj.a3083 · Cited by 339
C-Reactive Protein and Reclassification of Cardiovascular Risk in the Framingham Heart Study
Wilson, P. W. F., Pencina, M., Jacques, P., Selhub, J., D'Agostino, R., & O'Donnell, C. J. (2008) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.108.831198 · Cited by 299
Cited by
Grundy, S. M., Stone, N. J., Bailey, A. L., Beam, C., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Circulation · doi:10.1161/cir.0000000000000625 · Cited by 6,185
Arnett, D. K., Blumenthal, R. S., Albert, M. A., Buroker, A. B., Goldberger, Z. D., Hahn, E. J., Himmelfarb, C. D., Khera, A., Lloyd-Jones, D., McEvoy, J. W., et al. (2019) · Circulation · doi:10.1161/cir.0000000000000678 · Cited by 4,536
(2019) · doi:10.1161/cir.0000000000000677 · Cited by 4,501
Grundy, S. M., Stone, N. J., Bailey, A. L., Beam, C., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.11.003 · Cited by 2,691
2019 ACC/AHA Guideline on the Primary Prevention of Cardiovascular Disease
Arnett, D. K., Blumenthal, R. S., Albert, M. A., Buroker, A. B., Goldberger, Z. D., Hahn, E. J., Himmelfarb, C. D., Khera, A., Lloyd-Jones, D., McEvoy, J. W., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2019.03.010 · Cited by 1,896
Metadata
- Authors
- Wolfson, J., Vock, D. M., Bandyopadhyay, S., Kottke, T., Vazquez‐Benitez, G., Johnson, P., Adomavicius, G., & O'Connor, P. J.
- Venue
- Journal of the American Heart Association
- Published
- 2017-04-05
- Volume
- 6
- Issue
- 4
- ISSN
- 2047-9980 (ISSN-L)
- OpenAlex
- W2610062134
- Cited by
- 33
- References
- 9
- FWCI
- 5.42
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data. https://app.cochat.ai/papers/10.1161/jaha.116.003670
- The paper
- Wolfson, J., Vock, D. M., Bandyopadhyay, S., Kottke, T., Vazquez‐Benitez, G., Johnson, P., Adomavicius, G., & O'Connor, P. J. (2017). Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data. Journal of the American Heart Association. https://doi.org/10.1161/jaha.116.003670
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.