The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Deep Learning for Automatic Calcium Scoring in CT: Validation Using Multiple Cardiac CT and Chest CT Protocols
Open accessAbstract
Background Although several deep learning (DL) calcium scoring methods have achieved excellent performance for specific CT protocols, their performance in a range of CT examination types is unknown. Purpose To evaluate the performance of a DL method for automatic calcium scoring across a wide range of CT examination types and to investigate whether the method can adapt to different types of CT examinations when representative images are added to the existing training data set. Materials and Methods The study included 7240 participants who underwent various types of nonenhanced CT examinations that included the heart: coronary artery calcium (CAC) scoring CT, diagnostic CT of the chest, PET attenuation correction CT, radiation therapy treatment planning CT, CAC screening CT, and low-dose CT of the chest. CAC and thoracic aorta calcification (TAC) were quantified using a convolutional neural network trained with (a) 1181 low-dose chest CT examinations (baseline), (b) a small set of examinations of the respective type supplemented to the baseline (data specific), and (c) a combination of examinations of all available types (combined). Supplemental training sets contained 199–568 CT images depending on the calcium burden of each population. The DL algorithm performance was evaluated with intraclass correlation coefficients (ICCs) between DL and manual (Agatston) CAC and (volume) TAC scoring and with linearly weighted κ values for cardiovascular risk categories (Agatston score; cardiovascular disease risk categories: 0, 1–10, 11–100, 101–400, >400). Results At baseline, the DL algorithm yielded ICCs of 0.79–0.97 for CAC and 0.66–0.98 for TAC across the range of different types of CT examinations. ICCs improved to 0.84–0.99 (CAC) and 0.92–0.99 (TAC) for CT protocol–specific training and to 0.85–0.99 (CAC) and 0.96–0.99 (TAC) for combined training. For assignment of cardiovascular disease risk category, the κ value for all test CT scans was 0.90 (95% confidence interval [CI]: 0.89, 0.91) for the baseline training. It increased to 0.92 (95% CI: 0.91, 0.93) for both data-specific and combined training. Conclusion A deep learning calcium scoring algorithm for quantification of coronary and thoracic calcium was robust, despite substantial differences in CT protocol and variations in subject population. Augmenting the algorithm training with CT protocol–specific images further improved algorithm performance. © RSNA, 2020 See also the editorial by Vannier in this issue.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A combined solution capable of analyzing cardiac and non-gated chest CT as presented in our investigation has only been proposed by de Vos et al.36 and van Velzen et al.41.”
From Deep convolutional neural networks to predict cardiovascular risk from computed tomography (2021) · cites this paper as [41]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although other studies have investigated deep learning algorithms for automated coronary calcium quantification34–42, they used smaller cohorts or proprietary technologies.”
From Deep convolutional neural networks to predict cardiovascular risk from computed tomography (2021) · cites this paper as [41]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening
The National Lung Screening Trial Research Team (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1102873 · Cited by 11,064
Quantification of coronary artery calcium using ultrafast computed tomography
Agatston, A. S., Janowitz, W. R., Hildner, F. J., Zusmer, N. R., Viamonte, M., & Detrano, R. (1990) · Journal of the American College of Cardiology · doi:10.1016/0735-1097(90)90282-t · Cited by 7,786
The multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781
Grundy, S. M., Stone, N. J., Bailey, A. L., Beam, C., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Circulation · doi:10.1161/cir.0000000000000625 · Cited by 6,185
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277
Setio, A. A. A., Traverso, A., de Bel, T., Berens, M. S. N., van den Bogaard, C., Cerello, P., Chen, H., Dou, Q., Fantacci, M. E., Geurts, B. H. J., et al. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.06.015 · Cited by 1,456
Long-Term Prognosis Associated With Coronary Calcification
Budoff, M. J., Shaw, L. J., Liu, S. T., Weinstein, S. R., Tseng, P. H., Flores, F. R., Callister, T. Q., Raggi, P., Berman, D. S., & Mosler, T. P. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2006.10.079 · Cited by 1,341
Grundy, S. M., Stone, N. J., Bailey, A. L., Beam, C. T., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S. D., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Circulation · doi:10.1161/cir.0000000000000624 · Cited by 933
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894
Research works printed but not linked: 4
- Grundy SM, 2019, Circulation, 139, e1046
- Bank IEM, 2017, Ischaemic Heart Disease: Early Recognition and Risk Disparities [dissertation]. Vol. Chapter 3
- Taylor HA, 2005, Ethn Dis, 15, S6
- Cano-Espinosa C, 2018, Proceedings of SPIE: medical imaging 2018—image processing, 10574, 105742K
Cited by
Sorted by citations
Adams, S. J., Stone, E., Baldwin, D. R., Vliegenthart, R., Lee, P., & Fintelmann, F. J. (2023) · The Lancet · doi:10.1016/s0140-6736(22)01694-4 · Cited by 467
Machine Learning and the Future of Cardiovascular Care
Quer, G., Arnaout, R., Henne, M., & Arnaout, R. (2021) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2020.11.030 · Cited by 353
Deep convolutional neural networks to predict cardiovascular risk from computed tomography
Zeleznik, R., Foldyna, B., Eslami, P., Weiss, J., Alexander, I., Taron, J., Parmar, C., Alvi, R. M., Banerji, D., Uno, M., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-20966-2 · Cited by 240
Automated coronary calcium scoring using deep learning with multicenter external validation
Eng, D. K.-M., Chute, C., Khandwala, N., Rajpurkar, P., Long, J., Shleifer, S., Khalaf, M. H., Sandhu, A. T., Rodríguez, F., Maron, D. J., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00460-1 · Cited by 189
Eche, T., Schwartz, L. H., Mokrane, F.-Z., & Dercle, L. (2021) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2021210097 · Cited by 163
Metadata
- Authors
- van Velzen, S. G. M., Lessmann, N., Velthuis, B. K., Bank, I. E. M., van den Bongard, D. H. J. G., Leiner, T., de Jong, P. A., Veldhuis, W. B., Correa, A., Terry, J. G., Carr, J. J., Viergever, M. A., Verkooijen, H. M., & Išgum, I.
- Affiliations
- Radboud University Medical Center (NL)
- ORCID
- Venue
- Radiology
- Published
- 2020-04
- Volume
- 295
- Issue
- 1
- Pages
- 66-79
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- PMC
- PMC7106943
- OpenAlex
- W3005949594
- Cited by
- 244
- References
- 34
- Citations by year
- 2026: 29 · 2025: 26 · 2024: 40 · 2023: 36 · 2022: 48 · 2021: 47 · 2020: 18
- FWCI
- 18.14
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Radiology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Funders
- U.S. Department of Health and Human Services (HHSN268201800014I, HHSN268201800010I, HHSN268201800013I, HHSN268201800012I, HHSN268201800015I)
- Jackson State University (HHSN268201800015I/HHSN26800001, HHSN268201800013 I, HHSN268201800014 I)
- Mississippi State Department of Health (HHSN26800001, HHSN268201800015 I, HHSN268201800015I/HHSN26800001)
- National Institutes of Health (HHSN268201800010I, HHSN268201800012I, HHSN268201800014I, HHSN26800001, HHSN268201800013I, HHSN268201800015I)
- National Heart, Lung, and Blood Institute (HHSN268201800013I, HHSN26800001, HHSN268201800012C, HHSN268201800010 I, HHSN268201800014 I, HHSN268201800012 I, HHSN268201100011I, HHSN268201800014C, HHSN268201800015I, HHSN268201800011C, HHSN268201800011I)
- National Cancer Institute (HHSN261201800014I, HHSN261201800012I, HHSN261201800010I)
- National Institute on Minority Health and Health Disparities (HHSN268201800012I, HHSN268201800010I, HHSN268201800014I, HHSN268201800015I, HHSN268201800013I)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Cardiac Imaging and Diagnostics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- coronary artery calcium scoring, thoracic aortic calcification, deep learning, convolutional neural networks, chest CT, low-dose CT, cardiovascular risk stratification, Agatston score, cardiovascular risk prediction, cardiovascular risk prediction models
- MeSH terms
- Deep Learning, Aged, Clinical Protocols, Coronary Artery Disease, Female, Heart, Humans, Male, Middle Aged, Retrospective Studies, Thorax, Tomography, X-Ray Computed, Vascular Calcification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
