CoChat
Sign Up
Cited by 112Machine Learning in HealthcareArtificial Intelligence in HealthcareHealth Promotion and Cardiovascular Prevention

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.

Machine learning and atherosclerotic cardiovascular disease risk prediction in a multi-ethnic population

Open access
Ward, Andrew, Sarraju, Ashish, Chung, Sukyung, Li, Jiang, Harrington, Robert, Heidenreich, Paul, Palaniappan, Latha, Scheinker, David, Rodriguez, Fatima·2020·npj Digital Medicine
DOI 10.1038/s41746-020-00331-1PMC PMC7511400
Open this paper in CoChat — free accountOpen PDF

Abstract

The pooled cohort equations (PCE) predict atherosclerotic cardiovascular disease (ASCVD) risk in patients with characteristics within prespecified ranges and has uncertain performance among Asians or Hispanics. It is unknown if machine learning (ML) models can improve ASCVD risk prediction across broader diverse, real-world populations. We developed ML models for ASCVD risk prediction for multi-ethnic patients using an electronic health record (EHR) database from Northern California. Our cohort included patients aged 18 years or older with no prior CVD and not on statins at baseline (n = 262,923), stratified by PCE-eligible (n = 131,721) or PCE-ineligible patients based on missing or out-of-range variables. We trained ML models [logistic regression with L 2 penalty and L 1 lasso penalty, random forest, gradient boosting machine (GBM), extreme gradient boosting] and determined 5-year ASCVD risk prediction, including with and without incorporation of additional EHR variables, and in Asian and Hispanic subgroups. A total of 4309 patients had ASCVD events, with 2077 in PCE-ineligible patients. GBM performance in the full cohort, including PCE-ineligible patients (area under receiver-operating characteristic curve (AUC) 0.835, 95% confidence interval (CI): 0.825–0.846), was significantly better than that of the PCE in the PCE-eligible cohort (AUC 0.775, 95% CI: 0.755–0.794). Among patients aged 40–79, GBM performed similarly before (AUC 0.784, 95% CI: 0.759–0.808) and after (AUC 0.790, 95% CI: 0.765–0.814) incorporating additional EHR data. Overall, ML models achieved comparable or improved performance compared to the PCE while allowing risk discrimination in a larger group of patients including PCE-ineligible patients. EHR-trained ML models may help bridge important gaps in ASCVD risk prediction.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-020-00331-1

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Consort diagram. N number, ASCVD atherosclerotic cardiovascular disease, CVD cardiovascular disease, PCE pooled cohort equation, HDL-C high-density lipoprotein cholesterol, BP blood pressure. *Patients with an outcome event within the 5-year follow-up window were not excluded. †Pre-existing cardiovascular disease was defined by International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) codes, including: atrial fibrillation: 427.31; heart failure: 428*; coronary artery disease: 411*, 413*, 414*; myocardial infarction: 410*; and stroke: 430–434*, 436* (refer to Supplementary Table 1).
Fig. 1 Consort diagram. N number, ASCVD atherosclerotic cardiovascular disease, CVD cardiovascular disease, PCE pooled cohort equation, HDL-C high-density lipoprotein cholesterol, BP blood pressure. *Patients with an outcome event within the 5-year follow-up window were not excluded. †Pre-existing cardiovascular disease was defined by International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) codes, including: atrial fibrillation: 427.31; heart failure: 428*; coronary artery disease: 411*, 413*, 414*; myocardial infarction: 410*; and stroke: 430–434*, 436* (refer to Supplementary Table 1).CC BY · PubMed Central
Fig. 2 ROC curves for PCE and ML model performance. ROC curves are outlined for PCE versus model ML performance in the PCE-eligible cohort (in a cross-validation and b held-out test data) and ML model performance on the full cohort (in c cross-validation and d held-out test data). Legend entries denote the AUC for each method with 95% confidence intervals. ROC receiver-operating characteristic, PCE pooled cohort equations, ML machine learning, AUC area under receiver-operating characteristic curve, LRL2 logistic regression with an L2 penalty, LRLasso logistic regression with an L1 (lasso) penalty, RF random forest, GBM gradient boosting machine, XGBoost extreme gradient boosting.
Fig. 2 ROC curves for PCE and ML model performance. ROC curves are outlined for PCE versus model ML performance in the PCE-eligible cohort (in a cross-validation and b held-out test data) and ML model performance on the full cohort (in c cross-validation and d held-out test data). Legend entries denote the AUC for each method with 95% confidence intervals. ROC receiver-operating characteristic, PCE pooled cohort equations, ML machine learning, AUC area under receiver-operating characteristic curve, LRL2 logistic regression with an L2 penalty, LRLasso logistic regression with an L1 (lasso) penalty, RF random forest, GBM gradient boosting machine, XGBoost extreme gradient boosting.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 27 references

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Ward, A., Sarraju, A., Chung, S., Li, J., Harrington, R., Heidenreich, P., Palaniappan, L., Scheinker, D., & Rodriguez, F.
Affiliations
  • Stanford University (US)
  • Stanford Medicine (US)
  • Palo Alto Medical Foundation (US)
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2020-09-23
Volume
3
Issue
1
Pages
125-125
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-020-00331-1
PMC
PMC7511400
OpenAlex
W3088053597
Cited by
112
References
27
Citations by year
2026: 18 · 2025: 17 · 2024: 22 · 2023: 28 · 2022: 16 · 2021: 9 · 2020: 1
FWCI
5.30
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
U.S. Department of Health & Human Services | NIH | National Heart, Lung, and Blood Institute (1K01HL144607)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health Promotion and Cardiovascular Prevention
Keywords
atherosclerotic cardiovascular disease, cardiovascular risk prediction, Pooled Cohort Equations, machine learning, gradient boosting, extreme gradient boosting, random forest, electronic health records, multi-ethnic population, Asian patients, Hispanic patients, risk discrimination, statin-naive patients

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review