CoChat
Sign Up
Cited by 313ECG Monitoring and AnalysisAtrial Fibrillation Management and OutcomesMachine Learning in Healthcare

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.

Deep neural networks can predict new-onset atrial fibrillation from the 12-lead ECG and help identify those at risk of atrial fibrillation-related stroke

Open access
Raghunath, Sushravya, Pfeifer, John M., Ulloa-Cerna, Alvaro E., Nemani, Arun, Carbonati, Tanner, Jing, Linyuan, vanMaanen, David P., Hartzel, Dustin N., Ruhl, Jeffery A., Lagerman, Braxton F., Rocha, Daniel B., Stoudt, Nathan J., Schneider, Gargi, Johnson, Kipp W., Zimmerman, Noah, Leader, Joseph B., Kirchner, H. Lester, Griessenauer, Christoph J., Hafez, Ashraf, Good, Christopher W., Fornwalt, Brandon K., Haggerty, Christopher M.·2021·Circulation
DOI 10.1161/circulationaha.120.047829PMC PMC7996054

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Atrial fibrillation (AF) is associated with substantial morbidity, especially when it goes undetected. If new-onset AF could be predicted, targeted screening could be used to find it early. We hypothesized that a deep neural network could predict new-onset AF from the resting 12-lead ECG and that this prediction may help identify those at risk of AF-related stroke. METHODS: We used 1.6 M resting 12-lead digital ECG traces from 430 000 patients collected from 1984 to 2019. Deep neural networks were trained to predict new-onset AF (within 1 year) in patients without a history of AF. Performance was evaluated using areas under the receiver operating characteristic curve and precision-recall curve. We performed an incidence-free survival analysis for a period of 30 years following the ECG stratified by model predictions. To simulate real-world deployment, we trained a separate model using all ECGs before 2010 and evaluated model performance on a test set of ECGs from 2010 through 2014 that were linked to our stroke registry. We identified the patients at risk for AF-related stroke among those predicted to be high risk for AF by the model at different prediction thresholds. RESULTS: The area under the receiver operating characteristic curve and area under the precision-recall curve were 0.85 and 0.22, respectively, for predicting new-onset AF within 1 year of an ECG. The hazard ratio for the predicted high- versus low-risk groups over a 30-year span was 7.2 (95% CI, 6.9-7.6). In a simulated deployment scenario, the model predicted new-onset AF at 1 year with a sensitivity of 69% and specificity of 81%. The number needed to screen to find 1 new case of AF was 9. This model predicted patients at high risk for new-onset AF in 62% of all patients who experienced an AF-related stroke within 3 years of the index ECG. CONCLUSIONS: Deep learning can predict new-onset AF from the 12-lead ECG in patients with no previous history of AF. This prediction may help identify patients at risk for AF-related strokes.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1161/circulationaha.120.047829

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1. Flow chart illustrating the study design and data summary. A, Data exclusions and data definition of proof-of-concept model. B, Data flow for deployment model. AF indicates atrial fibrillation.
Figure 1. Flow chart illustrating the study design and data summary. A, Data exclusions and data definition of proof-of-concept model. B, Data flow for deployment model. AF indicates atrial fibrillation.CC BY-NC-ND · PubMed Central
Figure 2. Illustration of model performance for all, normal, and abnormal subpopulations. The model performance is represented for proof-of-concept model as area under the receiver operating characteristic (left) and precision-recall curves (right). The bars represent the mean performance across the 5-fold cross-validation with error bars showing 95% CIs. The black circle represents the M0 model performance on the holdout set. The 3 bars represent model performance for extreme gradient boosting (XGBoost) model with age and sex as inputs (gray); DNN model with digital ECG traces as input (DNN-ECG; orange); and DNN model with digital ECG traces, age, and sex as inputs (DNN-ECG-AS; blue). AF indicates atrial fibrillation; and ROC, receiver operating characteristic.
Figure 2. Illustration of model performance for all, normal, and abnormal subpopulations. The model performance is represented for proof-of-concept model as area under the receiver operating characteristic (left) and precision-recall curves (right). The bars represent the mean performance across the 5-fold cross-validation with error bars showing 95% CIs. The black circle represents the M0 model performance on the holdout set. The 3 bars represent model performance for extreme gradient boosting (XGBoost) model with age and sex as inputs (gray); DNN model with digital ECG traces as input (DNN-ECG; orange); and DNN model with digital ECG traces, age, and sex as inputs (DNN-ECG-AS; blue). AF indicates atrial fibrillation; and ROC, receiver operating characteristic.CC BY-NC-ND · PubMed Central
Figure 3. ROC curves, incidence-free KM survival curves, and HRs in subpopulations for the 3 models evaluated on the holdout set. The 3 models are XGBoost model with age and sex only (blue); DNN model with ECG traces only (DNN-ECG; red); and DNN model with ECG traces, age, and sex (DNN-ECG-AS; black) for all ECGs in the holdout set. A, ROC curves with operating points marked for the 3 models. B, Incidence-free KM curves for the high- and low-risk groups for the operating point shown in A for a follow-up of 30 years. Note that curves corresponding to the low-risk group for all 3 models overlap. C, The plot of HR with 95% CIs for the 3 models in subpopulations defined by age groups, sex, and normal or abnormal ECG label. Note that there is no HR for age <50 years for the first model as there was no subject classified as high-risk for new-onset atrial fibrillation by the model for that subpopulation. DNN indicates deep neural network; HR, hazard ratio; KM, Kaplan-Meier; and ROC, receiver operating characteristic.
Figure 3. ROC curves, incidence-free KM survival curves, and HRs in subpopulations for the 3 models evaluated on the holdout set. The 3 models are XGBoost model with age and sex only (blue); DNN model with ECG traces only (DNN-ECG; red); and DNN model with ECG traces, age, and sex (DNN-ECG-AS; black) for all ECGs in the holdout set. A, ROC curves with operating points marked for the 3 models. B, Incidence-free KM curves for the high- and low-risk groups for the operating point shown in A for a follow-up of 30 years. Note that curves corresponding to the low-risk group for all 3 models overlap. C, The plot of HR with 95% CIs for the 3 models in subpopulations defined by age groups, sex, and normal or abnormal ECG label. Note that there is no HR for age <50 years for the first model as there was no subject classified as high-risk for new-onset atrial fibrillation by the model for that subpopulation. DNN indicates deep neural network; HR, hazard ratio; KM, Kaplan-Meier; and ROC, receiver operating characteristic.CC BY-NC-ND · PubMed Central
Figure 4. Incidence-free KM survival curves within the holdout set for subpopulations defined by sex, and age groups. The top row shows the KM curves for subpopulations in age groups: <50 years, 50 to 65 years, and ≥65 years for men (left) and women (right). The bottom row shows the KM curves for the model-predicted (model M0 trained with ECG traces, age, and sex; DNN-ECG-AS) low-risk groups and high-risk groups for new-onset atrial fibrillation for each age group for men and women. It also reflects relative hazards between age groups. The horizontal dotted gray line represents incidence-free proportion of 50%, and vertical lines represent the median survival time for the respective curves. DNN indicates deep neural network; and KM, Kaplan-Meier.
Figure 4. Incidence-free KM survival curves within the holdout set for subpopulations defined by sex, and age groups. The top row shows the KM curves for subpopulations in age groups: <50 years, 50 to 65 years, and ≥65 years for men (left) and women (right). The bottom row shows the KM curves for the model-predicted (model M0 trained with ECG traces, age, and sex; DNN-ECG-AS) low-risk groups and high-risk groups for new-onset atrial fibrillation for each age group for men and women. It also reflects relative hazards between age groups. The horizontal dotted gray line represents incidence-free proportion of 50%, and vertical lines represent the median survival time for the respective curves. DNN indicates deep neural network; and KM, Kaplan-Meier.CC BY-NC-ND · PubMed Central
Figure 5. Illustration of model sensitivity to detect patients at risk of AF-related strokes as a function of the proportion of the population flagged as high risk to develop new-onset AF. Colored curves denote patients with strokes occurring within 1 (blue), 2 (orange), and 3 (green) years after ECG in the deployment test set. Gray dotted lines represent the corresponding optimal operating thresholds from Table II in the Data Supplement. AF indicates atrial fibrillation.
Figure 5. Illustration of model sensitivity to detect patients at risk of AF-related strokes as a function of the proportion of the population flagged as high risk to develop new-onset AF. Colored curves denote patients with strokes occurring within 1 (blue), 2 (orange), and 3 (green) years after ECG in the deployment test set. Gray dotted lines represent the corresponding optimal operating thresholds from Table II in the Data Supplement. AF indicates atrial fibrillation.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 50 references

Research works printed but not linked: 1

  1. 10.2307/2281868 · DOI · Paper page

Cited by

Sorted by citations

Items 301–313 of 313

Related

See all

Metadata

Authors
Raghunath, S., Pfeifer, J. M., Ulloa-Cerna, A. E., Nemani, A., Carbonati, T., Jing, L., vanMaanen, D. P., Hartzel, D. N., Ruhl, J. A., Lagerman, B. F., Rocha, D. B., Stoudt, N. J., Schneider, G., Johnson, K. W., Zimmerman, N., Leader, J. B., Kirchner, H. L., Griessenauer, C. J., Hafez, A., Good, C. W., Fornwalt, B. K., & Haggerty, C. M.
Affiliations
  • Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA.
  • Heart and Vascular Center, Evangelical Hospital, Lewisburg, PA (J.M.P.).
  • Tempus Labs (United States) (US)
  • Phenomic Analytics and Clinical Data Core (D.N.H., B.F.L., D.B.R., J.B.L.), Geisinger, Danville, PA.
  • Geisinger Health System (US)
  • Paracelsus Medical University (AT)
  • UPMC Hamot (US)
Corresponding author
Haggerty, Christopher M.
ORCID
Venue
Circulation
Publisher
Ovid Technologies (Wolters Kluwer Health)
Published
2021-03-30
Volume
143
Issue
13
Pages
1287-1298
ISSN
  • 0009-7322 (ISSN-L)
  • 1524-4539
DOI
10.1161/circulationaha.120.047829
PMC
PMC7996054
OpenAlex
W3133470659
Cited by
312
References
50
Citations by year
2026: 56 · 2025: 74 · 2024: 65 · 2023: 51 · 2022: 40 · 2021: 25
FWCI
30.50
Type
journal article
Language
en
Open access
Licence
cc-by-nc-nd
Locations
  • Circulation (publishedVersion, cc-by-nc-nd)
  • medRxiv (acceptedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Atrial Fibrillation Management and Outcomes
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
new-onset atrial fibrillation, atrial fibrillation prediction, deep neural network, 12-lead ECG, stroke risk stratification, atrial fibrillation screening, digital ECG
MeSH terms
Deep Learning, Atrial Fibrillation, Electrocardiography, Female, Humans, Male, Survival Analysis, Neural Networks, Computer, Stroke

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review