The DOI matches its Crossref record; 57 of the 59 references it cites were found in a registry or an index.
Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network
Open accessAbstract
The electrocardiogram (ECG) is a widely used medical test, consisting of voltage versus time traces collected from surface recordings over the heart 1. Here we hypothesized that a deep neural network (DNN) can predict an important future clinical event, 1-year all-cause mortality, from ECG voltage-time traces. By using ECGs collected over a 34-year period in a large regional health system, we trained a DNN with 1,169,662 12-lead resting ECGs obtained from 253,397 patients, in which 99,371 events occurred. The model achieved an area under the curve (AUC) of 0.88 on a held-out test set of 168,914 patients, in which 14,207 events occurred. Even within the large subset of patients (n = 45,285) with ECGs interpreted as 'normal' by a physician, the performance of the model in predicting 1-year mortality remained high (AUC = 0.85). A blinded survey of cardiologists demonstrated that many of the discriminating features of these normal ECGs were not apparent to expert reviewers. Finally, a Cox proportional-hazard model revealed a hazard ratio of 9.5 (P < 0.005) for the two predicted groups (dead versus alive 1 year after ECG) over a 25-year follow-up period. These results show that deep learning can add substantial prognostic information to the interpretation of 12-lead resting ECGs, even in cases that are interpreted as normal by physicians.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Emerging evidence suggests that ECG waveforms contain latent signatures of ventricular dysfunction [4], structural heart disease [5], metabolic abnormalities [6], and broader physiologic characteristics such as biological age [7] and future mortality risk [8].”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“Cardiologists reviewing selected normal ECGs were unable to reliably identify the mortality-associated patterns detected by the model [8].”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“maintained strong performance even among ECGs interpreted as normal by physicians, with an AUC of 0.85.”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“As discussed above, ECG-age [7], mortality-prediction [8], and disease-specific models illustrate how AI-derived outputs may function as quantitative markers of physiologic risk rather than simple diagnostic labels.”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“Raghunath et al. trained a deep neural network using more than 1.1 million 12-lead ECGs from over 250,000 patients to predict one-year all-cause mortality.”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“This study demonstrated that AI-ECG can extract prognostic information from ECG waveforms that may not be visually apparent”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“The model achieved an AUC of 0.88 in the held-out test set”
From Deep Learning Applied to 12-Lead ECGs for Detection of Structural, Metabolic and Systemic Disease (2026) · cites this paper as [8]
In the citing paper
“Contemporary examples of these clinical applications in cardiology include the use of ECGs to predict left ventricular dysfunction or future development of atrial fibrillation (AF).[13][14][15][16]”
From Unlocking Tomorrow’s Health Care: Expanding the Clinical Scope of Wearables by Applying Artificial Intelligence (2024) · cites this paper as [14]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
Nonparametric Estimation from Incomplete Observations
Kaplan, E. L., & Meier, P. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4380-9_25 · Cited by 45,239
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
Nonparametric Estimation from Incomplete Observations
Kaplan, E. L., & Meier, P. (1958) · Journal of the American Statistical Association · doi:10.1080/01621459.1958.10501452 · Cited by 39,158
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
(2010) · International Conference on Machine Learning · Cited by 13,995
mice: Multivariate Imputation by Chained Equations in R
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870
Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., Saunders, L. D., Beck, C. A., Feasby, T. E., & Ghali, W. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000182534.19832.83 · Cited by 10,669
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581
A Real-Time QRS Detection Algorithm
Pan, J., & Tompkins, W. J. (1985) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.1985.325532 · Cited by 7,855
General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study
D’Agostino, R. B., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008) · Circulation · doi:10.1161/circulationaha.107.699579 · Cited by 7,491
Heart Disease and Stroke Statistics—2021 Update: A Report From the American Heart Association
Virani, S. S., Alonso, A., Aparicio, H. J., Benjamin, E. J., Bittencourt, M. S., Callaway, C. W., Carson, A. P., Chamberlain, A. M., Cheng, S., Delling, F. N., et al. (2021) · Circulation · doi:10.1161/cir.0000000000000950 · Cited by 5,784
Hannun, A. Y., Rajpurkar, P., Haghpanahi, M., Tison, G. H., Bourn, C., Turakhia, M. P., & Ng, A. Y. (2019) · Nature Medicine · doi:10.1038/s41591-018-0268-3 · Cited by 3,148
Obesity and the risk of heart failure
Kenchaiah, S., Evans, J. C., Levy, D., Wilson, P. W. F., Benjamin, E. J., Larson, M. G., Kannel, W. B., & Vasan, R. S. (2002) · New England Journal of Medicine · doi:10.1056/nejmoa020245 · Cited by 3,093
Multifactorial Index of Cardiac Risk in Noncardiac Surgical Procedures
Goldman, L., Caldera, D. L., Nussbaum, S. R., Southwick, F. S., Krogstad, D. J., Murray, B. E., Burke, D. S., O’Malley, T. A., Goroll, A. H., Caplan, C. H., et al. (1977) · New England Journal of Medicine · doi:10.1056/nejm197710202971601 · Cited by 2,738
Chesebro, J. H., Knatterud, G. L., Roberts, R., BORER, J. S., Cohen, L. S., Dalen, J. E., Dodge, H. T., Francis, C. K., Hillis, D., & Ludbrook, P. A. (1987) · Circulation · doi:10.1161/01.cir.76.1.142 · Cited by 2,240
The Seattle Heart Failure Model
Levy, W. C., Mozaffarian, D., Linker, D. T., Sutradhar, S., Anker, S. D., Cropp, A. B., Anand, I. S., Maggioni, A. P., Burton, P. B. J., Sullivan, M. D., et al. (2006) · Circulation · doi:10.1161/circulationaha.105.584102 · Cited by 2,024
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649
Attia, Z. I., Noseworthy, P. A., Lopez-Jimenez, F., Asirvatham, S. J., Deshmukh, A. J., Gersh, B. J., Carter, R. E., Yao, X., Rabinstein, A. A., Erickson, B. J., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(19)31721-0 · Cited by 1,613
A Validated Prediction Model for All Forms of Acute Coronary Syndrome
Eagle, K. A., Lim, M. J., Dabbous, O. H., Pieper, K. S., Goldberg, R. J., de Werf, F. V., Goodman, S. G., Granger, C. B., Steg, P. G., Gore, J. M., et al. (2004) · JAMA · doi:10.1001/jama.291.22.2727 · Cited by 1,590
A deep convolutional neural network model to classify heartbeats
Acharya, U. R., Oh, S. L., Hagiwara, Y., Tan, J. H., Adam, M., Gertych, A., & Tan, R. (2017) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.08.022 · Cited by 1,479
Attia, Z. I., Kapa, S., Lopez-Jimenez, F., McKie, P. M., Ladewig, D. J., Satam, G., Pellikka, P. A., Enriquez-Sarano, M., Noseworthy, P. A., Munger, T. M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-018-0240-2 · Cited by 1,389
The principles of software QRS detection
Kohler, B.-U., Hennig, C., & Orglmeister, R. (2002) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/51.993193 · Cited by 1,260
Phenomapping for Novel Classification of Heart Failure With Preserved Ejection Fraction
Shah, S. J., Katz, D. H., Selvaraj, S., Burke, M. A., Yancy, C. W., Gheorghiade, M., Bonow, R. O., Huang, C.-C., & Deo, R. C. (2015) · Circulation · doi:10.1161/circulationaha.114.010637 · Cited by 1,157
Wavelet transforms and the ECG: a review
Addison, P. S. (2005) · Physiological Measurement · doi:10.1088/0967-3334/26/5/r01 · Cited by 908
ECG-based heartbeat classification for arrhythmia detection: A survey
Luz, E. J. D. S., Schwartz, W. R., Cámara-Chávez, G., & Menotti, D. (2015) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.12.008 · Cited by 899
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.1.1 · Cited by 704
Motwani, M., Dey, D., Berman, D. S., Germano, G., Achenbach, S., Al-Mallah, M. H., Andreini, D., Budoff, M. J., Cademartiri, F., Callister, T. Q., et al. (2016) · European Heart Journal · doi:10.1093/eurheartj/ehw188 · Cited by 692
Deep learning approach for active classification of electrocardiogram signals
Rahhal, M. M. A., Bazi, Y., Alhichri, H. S., Alajlan, N., Melgani, F., & Yager, R. R. (2016) · Information Sciences · doi:10.1016/j.ins.2016.01.082 · Cited by 626
Lloyd-Jones, D. M., Braun, L. T., Ndumele, C. E., Smith, S. C., Sperling, L. S., Virani, S. S., & Blumenthal, R. S. (2019) · Circulation · doi:10.1161/cir.0000000000000638 · Cited by 554
Peake, D. (1988) · Journal of Emergency Medicine · doi:10.1016/0736-4679(88)90388-5 · Cited by 534
Improving palliative care with deep learning
Avati, A., Jung, K., Harman, S. M., Downing, N. L., Ng, A. Y., & Shah, N. H. (2017) · BMC Medical Informatics and Decision Making · doi:10.1109/bibm.2017.8217669 · Cited by 472
Asl, B. M., Setarehdan, S. K., & Mohebbi, M. (2008) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2008.04.007 · Cited by 392
Davie, A. P., Francis, C. M., Love, M. P., Caruana, L., Starkey, I. R., Shaw, T. R. D., Sutherland, G. R., & McMurray, J. J. V. (1996) · BMJ · doi:10.1136/bmj.312.7025.222 · Cited by 323
Lloyd-Jones, D. M., Braun, L. T., Ndumele, C. E., Smith, S. C., Sperling, L. S., Virani, S. S., & Blumenthal, R. S. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.11.005 · Cited by 302
Yu, S., & Chen, Y.-H. (2007) · Pattern Recognition Letters · doi:10.1016/j.patrec.2007.01.017 · Cited by 266
Participation in Cardiac Rehabilitation, Readmissions, and Death After Acute Myocardial Infarction
Dunlay, S. M., Pack, Q. R., Thomas, R. J., Killian, J. M., & Roger, V. L. (2014) · The American Journal of Medicine · doi:10.1016/j.amjmed.2014.02.008 · Cited by 262
Heartbeat Time Series Classification With Support Vector Machines
Kampouraki, A., Manis, G., & Nikou, C. (2009) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2008.2003323 · Cited by 258
A History of the origin, evolution, and impact of electrocardiography
Fye, W. B. (1994) · The American Journal of Cardiology · doi:10.1016/0002-9149(94)90135-x · Cited by 230
Attia, Z. I., Kapa, S., Yao, X., López-Jiménez, F. A., Mohan, T. L., Pellikka, P. A., Carter, R. E., Shah, N. D., Friedman, P. A., & Noseworthy, P. A. (2019) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.13889 · Cited by 199
Real-Time Multilead Convolutional Neural Network for Myocardial Infarction Detection
Liu, W., Zhang, M., Zhang, Y., Liao, Y., Huang, Q., Chang, S., Wang, H., & He, J. L. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2771768 · Cited by 183
Curry (2018) · doi:10.1001/jama.2018.6848 · Cited by 150
Recommendations for the Standardization and Interpretation of the Electrocardiogram
Mason, J. W., Hancock, E. W., & Gettes, L. S. (2007) · Circulation · doi:10.1161/circulationaha.106.180201 · Cited by 137
The Seattle Heart Failure Model: Prediction of Survival in Heart Failure
Cheitlin, M. D. (2007) · Yearbook of Cardiology · doi:10.1016/s0145-4145(08)70196-5 · Cited by 134
MULTIFACTORIAL INDEX OF CARDIAC RISK IN NONCARDIAC SURGICAL PROCEDURES
Goldman, L., Caldera, D. L., Nussbaum, S. R., Southwick, F. S., Krogstad, D. J., Murray, B. E., Burke, D. S., Malley, T. A. O., Goroll, A. H., Caplan, C. H., et al. (1978) · Survey of Anesthesiology · doi:10.1097/00132586-197810000-00051 · Cited by 130
Smith, S. W., Walsh, B., Grauer, K., Wang, K., Rapin, J., Li, J., Fennell, W. H., & Taboulet, P. (2018) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2018.11.013 · Cited by 107
An innovative approach of QRS segmentation based on first-derivative, Hilbert and Wavelet Transforms
Madeiro, J. P. D. V., Cortez, P. C., Marques, J. A. L., Seisdedos, C. R. V., & Sobrinho, C. R. M. R. (2012) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2011.12.011 · Cited by 105
Effects of integrated delivery system on cost and quality.
Hwang, W., Chang, J., LaClair, M., & Paz, H. L. (2013) · PubMed · Cited by 93
A Contribution to the Automatic Processing of Electrocardiograms Using Syntactic Methods
Belforte, G., Mori, R. D., & Ferraris, F. (1979) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.1979.326470 · Cited by 58
Goodfellow, S. D., Goodwin, A. J., Greer, R., Laussen, P. C., Mazwi, M., & Eytan, D. (2018) · Machine Learning for Healthcare Conference · Cited by 50
The Electrocardiogram as a Prognostic Tool for Predicting Major Cardiac Events
Lanza, G. A. (2007) · Progress in Cardiovascular Diseases · doi:10.1016/j.pcad.2007.03.003 · Cited by 38
Political and economic illusions of socialism
Lavoie, D. (1986) · Critical Review · doi:10.1080/08913818608459473 · Cited by 34
Classification of ECG Signals Using Extreme Learning Machine
Karpagachelvi, S., Arthanari, M., & Sivakumar, M. (2011) · Computer and Information Science · doi:10.5539/cis.v4n1p42 · Cited by 33
Hedberg, P., Lönnberg, I., Jonason, T., Nilsson, G., Pehrsson, K., & Ringqvist, I. (2004) · American Heart Journal · doi:10.1016/j.ahj.2004.03.034 · Cited by 29
Hedberg, P., Lönnberg, I., Jonason, T., Nilsson, G. H., Pehrsson, K., & Ringqvist, I. (2005) · ACC Current Journal Review · doi:10.1016/j.accreview.2004.12.095 · Cited by 2
Research works printed but not linked: 5
- LeBlanc, A. R. Quantitative analysis of cardiac arrhythmias. Crit. Rev. Biomed. Eng. 14, 1–43 (1986).
- Chen, T. & He, T. Higgs Boson discovery with boosted trees. JMLR Work. Conf. Proc. 42, 69–80 (2015).
- Springenberg, J. T., Dosovitskiy, A., Brox, T. & Riedmiller, M. Striving for simplicity: the all convolutional net. Preprint at https://arxiv.org/abs/1412.6806 (2014). · arXiv
- Lin, M., Chen, Q. & Yan, S. Network in network. Preprint at https://arxiv.org/abs/1312.4400 (2013). · arXiv
- Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014). · arXiv
Cited by
Sorted by citations
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
Artificial intelligence-enhanced electrocardiography in cardiovascular disease management
Siontis, K. C., Noseworthy, P. A., Attia, Z. I., & Friedman, P. A. (2021) · Nature Reviews Cardiology · doi:10.1038/s41569-020-00503-2 · Cited by 914
Raghunath, S., Pfeifer, J. M., Ulloa-Cerna, A. E., Nemani, A., Carbonati, T., Jing, L., vanMaanen, D. P., Hartzel, D. N., Ruhl, J. A., Lagerman, B. F., et al. (2021) · Circulation · doi:10.1161/circulationaha.120.047829 · Cited by 312
Systems biology in cardiovascular disease: a multiomics approach
Joshi, A., Rienks, M., Theofilatos, K., & Mayr, M. (2020) · Nature Reviews Cardiology · doi:10.1038/s41569-020-00477-1 · Cited by 309
Deep learning and the electrocardiogram: review of the current state-of-the-art
Somani, S. S., Russak, A. J., Richter, F., Zhao, S., Vaid, A., Chaudhry, F. F., Freitas, J. K. D., Naik, N., Miotto, R., Nadkarni, G. N., et al. (2020) · EP Europace · doi:10.1093/europace/euaa377 · Cited by 276
Metadata
- Authors
- Raghunath, S., Ulloa Cerna, A. E., Jing, L., vanMaanen, D. P., Stough, J., Hartzel, D. N., Leader, J. B., Kirchner, H. L., Stumpe, M. C., Hafez, A., Nemani, A., Carbonati, T., Johnson, K. W., Young, K., Good, C. W., Pfeifer, J. M., Patel, A. A., Delisle, B. P., Alsaid, A., Beer, D., Haggerty, C. M., & Fornwalt, B. K.
- Affiliations
- Bucknell University (US)
- Geisinger Medical Center (US)
- Tempus Labs (United States) (US)
- University of Kentucky (US)
- Corresponding author
- Fornwalt, Brandon K.
- ORCID
- Venue
- Nature Medicine
- Published
- 2020-05-11
- Volume
- 26
- Issue
- 6
- Pages
- 886-891
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W3104075323
- Cited by
- 314
- References
- 59
- Citations by year
- 2026: 41 · 2025: 49 · 2024: 61 · 2023: 51 · 2022: 53 · 2021: 49 · 2020: 9
- FWCI
- 25.00
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Nature Medicine (publishedVersion)
- arXiv (Cornell University) (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Geisinger Health System Foundation
- Geisinger Health Plan Tempus Labs Inc
- Pennsylvania Department of Health (SAP 4100070267)
- American Heart Association (17CCRG33700289)
- Tempus Labs Inc
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- 12-lead ECG, all-cause mortality prediction, deep neural network, resting ECG, prognostic stratification, Cox proportional hazards model, cardiovascular risk prediction
- MeSH terms
- Cardiologists, Deep Learning, Adult, Aged, Aged, 80 and over, Algorithms, Cause of Death, Electrocardiography, Female, Humans, Male, Middle Aged, Mortality, Prognosis, Retrospective Studies, ROC Curve, Proportional Hazards Models, Neural Networks, Computer, Risk Assessment, Area Under Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
