Cited by 241Cardiac Imaging and DiagnosticsArtificial Intelligence in Healthcare and EducationRadiomics and Machine Learning in Medical Imaging

The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index; 40 of the 57 sources its claims cite were checked for support.

Deep convolutional neural networks to predict cardiovascular risk from computed tomography

Open access

Abstract

Coronary artery calcium is an accurate predictor of cardiovascular events. While it is visible on all computed tomography (CT) scans of the chest, this information is not routinely quantified as it requires expertise, time, and specialized equipment. Here, we show a robust and time-efficient deep learning system to automatically quantify coronary calcium on routine cardiac-gated and non-gated CT. As we evaluate in 20,084 individuals from distinct asymptomatic (Framingham Heart Study, NLST) and stable and acute chest pain (PROMISE, ROMICAT-II) cohorts, the automated score is a strong predictor of cardiovascular events, independent of risk factors (multivariable-adjusted hazard ratios up to 4.3), shows high correlation with manual quantification, and robust test-retest reliability. Our results demonstrate the clinical value of a deep learning system for the automated prediction of cardiovascular events. Implementation into clinical practice would address the unmet need of automating proven imaging biomarkers to guide management and improve population health.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

A deep learning system automatically quantified coronary artery calcium from routine cardiac-gated and non-gated CT and predicted cardiovascular events across four large cohorts. The automated score was strongly associated with events, correlated highly with manual scoring, and showed robust test–retest reliability.

Key findings

  • In 20,084 test individuals from FHS-CT2, NLST, PROMISE, and ROMICAT-II, the automated calcium score was a predictor of cardiovascular events independent of risk factors, with multivariable-adjusted hazard ratios up to 4.3 in NLST and odds ratios up to 57.11 in ROMICAT-II.
  • NLST (n = 14,959; ASCVD death n = 288; median follow-up 6.7 years): adjusted for age, sex, diabetes, hypertension, past heart disease, and stroke, HRs versus very low risk were 1.57 (95% CI 0.96–2.57, P = 0.069) for low, 2.79 (1.70–4.57, P < 0.001) for moderate, and 3.87 (2.45–6.11, P < 0.001) for high.
  • PROMISE (n = 4,021; all-cause mortality, MI, or unstable angina n = 130; median 25 months): adjusted for Framingham Risk Score, HRs were 1.90 (1.15–3.15, P = 0.012), 2.57 (1.47–4.50, P = 0.001), and 2.95 (1.71–5.08, P < 0.001) for low, moderate, and high versus very low.
  • ROMICAT-II (n = 441; ACS n = 38): adjusted for TIMI risk score, ORs were 7.70 (2.44–24.30, P = 0.001), 16.20 (4.44–59.13, P < 0.001), and 57.11 (16.55–197.12, P < 0.001) for low, moderate, and high versus very low. Automated scores correlated with manual scores in 5,521 patients (Spearman 0.92, P < 0.0001), combined predicted AUC was 0.74 versus manual 0.75 (P = 0.544), and test–retest ICC was 0.993 versus manual 0.997 in 252 pairs.

Methods

  • Retrospective secondary analysis of a longitudinal primary prevention cohort (FHS-CT1 and FHS-CT2) and three randomized trials (NLST, PROMISE, ROMICAT-II). The system was trained and tuned on 1,636 FHS-CT1 CT scans and tested on 20,084 scans: FHS-CT2 n = 663, NLST n = 14,959, PROMISE n = 4,021, ROMICAT-II n = 441.
  • The deep learning system used four consecutive U-Net-based steps: heart localization, heart segmentation, coronary calcium segmentation, and Agatston calcium score calculation. Scores were stratified as very low (0), low (1–100), moderate (101–300), and high (>300).
  • Analyses included Cox proportional hazards and log-rank tests for NLST and PROMISE, odds ratios for ROMICAT-II, Spearman correlation, ICC, Cohen’s kappa, concordance, and random-effects meta-analysis of AUCs. Manual calcium scores were available for 5,521 subjects.

Limitations

  • The paper states that most participants were non-Hispanic white: FHS 100%, NLST 91%, PROMISE 77%, and ROMICAT-II 66%; generalizability to other demographics requires future study.
  • The paper states that the system evaluated CAC on non-contrast cardiac and chest CT and therefore could not detect noncalcified plaque, which can be present even with a calcium score of zero.
  • Evident from the text: manual reference scores were available for only 5,521 of 20,084 test subjects, and NLST manual validation used a randomly selected subset of 396; FHS-CT2 testing included all CTs, including those initially considered non-diagnostic, while training used only diagnostic-quality CTs.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-021-20966-2

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Overview of the deep-learning framework, the training and test cohorts, and the implemented evaluation steps. a The deep-learning framework was trained and tuned on 1636 computed tomography (CT) scans from Framingham Heart Study (FHS)-CT1. In four consecutive steps a coronary calcium risk score was calculated in a fully automatic fashion. Independent testing was performed on 20,084 CT scans from four different clinical cohorts. b The performance of the framework was evaluated with respect to its clinical value and robustness. c CT scans of three representative patients of FHS-CT2 outlined with the deep learning system heart (blue contours) and coronary calcium (orange contours). FHS-CT117, FHS-CT217 Framingham Heart Study, (CT1) participants from the seventh examination cycle of the Offspring Cohort or first examination cycle of the Third Generation Cohort (2002–05) and (CT2) participants from the second examination cycle of the Third Generation Cohort (2008–11), NLST18 National Lung Screening Trial, PROMISE19 Prospective Multicenter Imaging Study for Evaluation of Chest Pain, ROMICAT-II20 Rule out Myocardial Infarction using Computer Assisted Tomography II, ECG electrocardiographic, CT computed tomography, ASCVD atherosclerotic cardiovascular disease, ACS acute coronary syndrome.
Fig. 1 Overview of the deep-learning framework, the training and test cohorts, and the implemented evaluation steps. a The deep-learning framework was trained and tuned on 1636 computed tomography (CT) scans from Framingham Heart Study (FHS)-CT1. In four consecutive steps a coronary calcium risk score was calculated in a fully automatic fashion. Independent testing was performed on 20,084 CT scans from four different clinical cohorts. b The performance of the framework was evaluated with respect to its clinical value and robustness. c CT scans of three representative patients of FHS-CT2 outlined with the deep learning system heart (blue contours) and coronary calcium (orange contours). FHS-CT117, FHS-CT217 Framingham Heart Study, (CT1) participants from the seventh examination cycle of the Offspring Cohort or first examination cycle of the Third Generation Cohort (2002–05) and (CT2) participants from the second examination cycle of the Third Generation Cohort (2008–11), NLST18 National Lung Screening Trial, PROMISE19 Prospective Multicenter Imaging Study for Evaluation of Chest Pain, ROMICAT-II20 Rule out Myocardial Infarction using Computer Assisted Tomography II, ECG electrocardiographic, CT computed tomography, ASCVD atherosclerotic cardiovascular disease, ACS acute coronary syndrome.CC BY · PubMed Central
Fig. 2 Outcome analysis for deep learning predicted calcium scores. Kaplan–Meier survival analysis of CAC risk groups for a cardiovascular disease-related death for 14,959 subjects of the National Lung Screening Trial (NLST)18 and b all-cause mortality, myocardial infarction and unstable angina for 4021 subjects of the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE)19. A two-sided log rank test was used to calculate the p values (*p value ≤ 0.05; ***p value ≤ 0.001) in panels (a) and (b). c Thirty-day acute coronary syndrome (ACS) rate across CAC groups for 441 subjects from the Rule Out Myocardial Infarction using Computer Assisted Tomography II (ROMICAT-II)20 trial. The shaded area corresponds to the 95% confidence interval of the 30-day ACS rate across CAC groups. A two-sided Fisher’s exact test was used to estimate differences in the ACS rate between the very low risk group and the low, moderate, and high risk groups (***p value ≤ 0.001). CAC coronary artery calcium, ASCVD atherosclerotic cardiovascular disease, ACS acute coronary syndrome. CAC risk groups: Very low: 0; Low: 1–100; Moderate: 101–300; High: >30021.
Fig. 2 Outcome analysis for deep learning predicted calcium scores. Kaplan–Meier survival analysis of CAC risk groups for a cardiovascular disease-related death for 14,959 subjects of the National Lung Screening Trial (NLST)18 and b all-cause mortality, myocardial infarction and unstable angina for 4021 subjects of the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE)19. A two-sided log rank test was used to calculate the p values (*p value ≤ 0.05; ***p value ≤ 0.001) in panels (a) and (b). c Thirty-day acute coronary syndrome (ACS) rate across CAC groups for 441 subjects from the Rule Out Myocardial Infarction using Computer Assisted Tomography II (ROMICAT-II)20 trial. The shaded area corresponds to the 95% confidence interval of the 30-day ACS rate across CAC groups. A two-sided Fisher’s exact test was used to estimate differences in the ACS rate between the very low risk group and the low, moderate, and high risk groups (***p value ≤ 0.001). CAC coronary artery calcium, ASCVD atherosclerotic cardiovascular disease, ACS acute coronary syndrome. CAC risk groups: Very low: 0; Low: 1–100; Moderate: 101–300; High: >30021.CC BY · PubMed Central
Fig. 3 Confusion matrices to compare automatic and manual CAC quantification and to assess test–retest repeatability. a Comparison of CAC classes calculated by the deep-learning framework and expert readers, combining data from FHS-CT2, NLST, PROMISE, and ROMICAT-II (n = 5521). The robustness of b the deep learning framework and c expert readers to quantify CAC was assessed in 252 FHS-CT1 subjects who underwent two subsequent CT scans within 1 h (Scan 1 and Scan 2). CAC coronary artery calcium, FHS-CT117, FHS-CT217 Framingham Heart Study, (CT1) participants from the seventh examination cycle of the offspring cohort or first examination cycle of the Third Generation Cohort (2002–05) and (CT2) participants from the second examination cycle of the Third Generation Cohort (2008–11), NLST18 National Lung Screening Trial, PROMISE19 Prospective Multicenter Imaging Study for Evaluation of Chest Pain, ROMICAT-II20 Rule Out Myocardial Infarction using Computer Assisted Tomography II, CAC risk groups: very low: 0; low: 1–100; moderate: 101–300; high: >30021.
Fig. 3 Confusion matrices to compare automatic and manual CAC quantification and to assess test–retest repeatability. a Comparison of CAC classes calculated by the deep-learning framework and expert readers, combining data from FHS-CT2, NLST, PROMISE, and ROMICAT-II (n = 5521). The robustness of b the deep learning framework and c expert readers to quantify CAC was assessed in 252 FHS-CT1 subjects who underwent two subsequent CT scans within 1 h (Scan 1 and Scan 2). CAC coronary artery calcium, FHS-CT117, FHS-CT217 Framingham Heart Study, (CT1) participants from the seventh examination cycle of the offspring cohort or first examination cycle of the Third Generation Cohort (2002–05) and (CT2) participants from the second examination cycle of the Third Generation Cohort (2008–11), NLST18 National Lung Screening Trial, PROMISE19 Prospective Multicenter Imaging Study for Evaluation of Chest Pain, ROMICAT-II20 Rule Out Myocardial Infarction using Computer Assisted Tomography II, CAC risk groups: very low: 0; low: 1–100; moderate: 101–300; high: >30021.CC BY · PubMed Central

Verification

Claims the paper makes

21 Own result · 20 Own method · 12 Partly supported · 10 Own interpretation · 7 No full text of the sources · 4 No basis given · 3 Couldn’t confirm · 2 Supported · 2 Supported so far · 1 Sources not found · accuracy not yet measured

Show all 43 claims

  • It may state two separate claims as one. 15
  • Probably not a substantial claim. 30
39 of 39
  • The paper relies on this without support.

    The measurement of coronary calcium score requires radiological expertise, time, and specialized coronary calcium quantification equipment.

    “However, the measurement requires radiological expertise, time, and specialized coronary calcium quantification equipment.”
  • The paper relies on this without support.

    This essentially free available information is usually not reported.

    “As a result, this essentially free available information is usually not reported.”
  • The paper relies on this without support.

    Deep learning can automate complex assessments that previously could only be done by radiologists, at a scale with higher speed and lower cost.

    “A major advantage is that deep learning can automate complex assessments that previously could only be done by radiologists, but now is feasible at a scale with a higher speed and lower cost.”
  • The paper relies on this without support.

    Traditionally, coronary calcium scoring requires special software, manual measurement by trained experts and dedicated ECG-gated cardiac CT.

    “Traditionally, coronary calcium scoring requires special software, manual measurement by trained experts and dedicated ECG-gated cardiac CT.”
  • The paper relies on this without support.

    As a consequence, the calcium score is often not reported on routine noncardiac chest CT.

    “As a consequence, the calcium score is often not reported on routine noncardiac chest CT”
  • The authors state this as an assumption.

    Before clinical introduction of automated systems for cardiovascular risk prediction, their generalizability across multiple clinical scenarios and institutions needs to be demonstrated.

    “This is essential, as before clinical introduction can be considered, the generalizability of these automated systems needs to be demonstrated as they need to be able to predict cardiovascular events of asymptomatic individuals across multiple clinical scenarios and work robustly on data from multiple institutions.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 59 references

Cited by

Sorted by citations

Metadata

Authors
Zeleznik, R., Foldyna, B., Eslami, P., Weiss, J., Alexander, I., Taron, J., Parmar, C., Alvi, R. M., Banerji, D., Uno, M., Kikuchi, Y., Karady, J., Zhang, L., Scholtz, J.-E., Mayrhofer, T., Lyass, A., Mahoney, T. F., Massaro, J. M., Vasan, R. S., Douglas, P. S., Hoffmann, U., Lu, M. T., & Aerts, H. J. W. L.
Affiliations
  • Brigham and Women's Hospital (US)
  • Harvard University (US)
  • Boston University (US)
  • Duke University (US)
Corresponding author
Aerts, Hugo J. W. L.
Venue
Nature Communications
Published
2021-01-29
Volume
12
Issue
1
Pages
715-715
ISSN
2041-1723 (ISSN-L)
OpenAlex
W3126008863
Cited by
240
References
59
Citations by year
2026: 33 · 2025: 48 · 2024: 61 · 2023: 36 · 2022: 42 · 2021: 20
FWCI
19.23
Type
journal article
Language
en
Open access
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Publications (Maastricht University) (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (submittedVersion, other-oa)
  • PubMed Central (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Research Publications (Maastricht University) (submittedVersion, other-oa)
Funders
  • American Heart Association (5T32HL076136, 5K24HL113128, 18UNPG34030172, 5R01-HL109711, HHSN268201500001I, 5U01HL123339)
  • American College of Radiology Imaging Network
  • European Commission
  • Deutsche Forschungsgemeinschaft
  • National Institutes of Health (75N92019D00031, NIH-USA R35CA22052, 5k24hl113128-09, 5u01hl123339-06, R35CA22052, n01hc25195-41-0-1, 18UNPG34030172, HHSN268201500001I, U01-CA209414, NIH/NHLBI 5K24HL113128, NIH‐USA U24CA194354, NIH-USA U01CA190234, NIH-USA U01CA209414, U24CA194354, NHLBI 5K24HL113128, NO1-HC-25195, NIH-USA U24CA194354, NIH-USA U01CA190234, U01CA190234, 5K24HL113128)
  • National Heart, Lung, and Blood Institute (K24HL113128, 5K24HL113128, HHSN268201500001I, 5U01HL123339, HHSN268201500001C, 75N92019D00031, N01-HC-025195, R01HL109711, U01 CA209414, T32HL076136, NO1‐HC‐25195, 5R01-HL109711, U01 HL123339, E0583118)
  • National Cancer Institute (U24-CA194354, U01-CA190234, U01-CA209414, R35CA22052)
  • H2020 European Research Council (866504)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Cardiac Imaging and Diagnostics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
coronary artery calcium, cardiovascular risk prediction, deep learning, computed tomography, cardiovascular events, imaging biomarkers, test-retest reliability, chest pain, Framingham Heart Study, promise
MeSH terms
Deep Learning, Heart Disease Risk Factors, Aged, Calcium, Cardiovascular Diseases, Chest Pain, Coronary Vessels, Female, Follow-Up Studies, Humans, Image Processing, Computer-Assisted, Male, Middle Aged, Retrospective Studies, Tomography, X-Ray Computed, Reproducibility of Results, Risk Assessment, Asymptomatic Diseases

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-27, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-27 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review