Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis
Open accessAbstract
AIMS: Traditional prognostic risk assessment in patients undergoing non-invasive imaging is based upon a limited selection of clinical and imaging findings. Machine learning (ML) can consider a greater number and complexity of variables. Therefore, we investigated the feasibility and accuracy of ML to predict 5-year all-cause mortality (ACM) in patients undergoing coronary computed tomographic angiography (CCTA), and compared the performance to existing clinical or CCTA metrics. METHODS AND RESULTS: The analysis included 10 030 patients with suspected coronary artery disease and 5-year follow-up from the COronary CT Angiography EvaluatioN For Clinical Outcomes: An InteRnational Multicenter registry. All patients underwent CCTA as their standard of care. Twenty-five clinical and 44 CCTA parameters were evaluated, including segment stenosis score (SSS), segment involvement score (SIS), modified Duke index (DI), number of segments with non-calcified, mixed or calcified plaques, age, sex, gender, standard cardiovascular risk factors, and Framingham risk score (FRS). Machine learning involved automated feature selection by information gain ranking, model building with a boosted ensemble algorithm, and 10-fold stratified cross-validation. Seven hundred and forty-five patients died during 5-year follow-up. Machine learning exhibited a higher area-under-curve compared with the FRS or CCTA severity scores alone (SSS, SIS, DI) for predicting all-cause mortality (ML: 0.79 vs. FRS: 0.61, SSS: 0.64, SIS: 0.64, DI: 0.62; P< 0.001). CONCLUSIONS: Machine learning combining clinical and CCTA data was found to predict 5-year ACM significantly better than existing clinical or CCTA metrics alone.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The SSS and SIS have demonstrated clinical validity in detecting major adverse cardiac events in multiple populations.
“The SSS and SIS have demonstrated clinical validity in detecting major adverse cardiac events in multiple populations.15–19”
From Lifelong endurance exercise and its relation with coronary atherosclerosis (2023) · cites this paper as [18]
The Prediction all-cause mortality section reports that both SSS and SIS predicted five-year all-cause mortality in patients with suspected coronary artery disease. It does not report their performance for major adverse cardiac events or establish validity across multiple populations; the Study limitations section says cardiac-specific endpoints were unavailable for all centres. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408 · see citing works
Hadamitzky, M., Freißmuth, B., Meyer, T., Hein, F., Kastrati, A., Martinoff, S., Schömig, A., & Hausleiter, J. (2009) · JACC: Cardiovascular Imaging · doi:10.1016/j.jcmg.2008.11.015 · Cited by 280 · see citing works
Self-citationVerification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2 · cited 1×
“Calibration of the LogitBoost model was assessed using the Brier score method (range: 0–1)—lower scores being consistent with better model calibration.(20) Calibration was also assessed graphically with plots showing the observed and predicted proportion of events, grouped by decile of risk.”
Method
Flagged first, then by citations
Research works printed but not linked: 25
- Budoff MJ, Dowe D, Jollis JG, Gitter M, Sutherland J, Halamert E, Scherer M, Bellinger R, Martin A, Benton R, Delago A, Min JK. Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evalu- ation of coronary artery stenosis in individuals without known coronary artery dis- ease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Inva- sive Coronary Angiography) trial. J Am Coll Cardiol 2008;52:1724 -1732.
- Hadamitzky M, Achenbach S, Al-Mallah M, Berman D, Budoff M, Cademartiri F, Callister T, Chang HJ, Cheng V, Chinnaiyan K, Chow BJW, Cury R, Delago A, Dunning A, Feuchtner G, Gomez M, Kaufmann P, Kim YJ, Leipsic J, Lin FY, Maffei E, Min JK, Raff G, Shaw LJ, Villines TC, Hausleiter J. Optimized prognostic score for coronary computed tomographic angiography: Results from the CON- FIRM registry (COronary CT angiography evaluation for clinical outcomes: An international multicenter registry). J Am Coll Cardiol 2013;62:468 -476.
- Chow BJ, Small G, Yam Y, Chen L, Achenbach S, Al-Mallah M, Berman DS, Budoff MJ, Cademartiri F, Callister TQ, Chang H-J, Cheng V, Chinnaiyan K, Delago A, Dunning A, Hadamitzky M, Hausleiter J, Kaufmann P, Lin F, Maffei E, Raff GL, Shaw LJ, Villines TC, Min JK. Incremental prognostic value of cardiac CT in CAD using CONFIRM (COroNary computed tomography angiography evalu- ation for clinical outcomes: an InteRnational Multicenter registry). Circ Cardiovasc Imaging 2011;4:463 -472.
- Waljee AK, Higgins PDR. Machine learning in medicine: a primer for physicians. Am J Gastroenterol 2010;105:1224 -1226.
- Deo RC. Machine learning in medicine. Circulation. 2015;132:1920 -1930.
- Min JK, Dunning A, Lin FY, Achenbach S, Al-Mallah MH, Berman DS, Budoff MJ, Cademartiri F, Callister TQ, Chang H-J, Cheng V, Chinnaiyan KM, Chow B, Delago A, Hadamitzky M, Hausleiter J, Karlsberg RP, Kaufmann P, Maffei E, Nasir K, Pencina MJ, Raff GL, Shaw LJ, Villines TC. Rationale and design of the CONFIRM (COronary CT Angiography EvaluatioN For Clinical Outcomes: An InteRnational Multicenter) Registry. J Cardiovasc Comput Tomogr 2011;5:84 -92.
- Abbara S, Arbab-Zadeh A, Callister TQ, Desai MY, Mamuya W, Thomson L, Weigold WG. SCCT guidelines for performance of coronary computed tomo- graphic angiography: a report of the Society of Cardiovascular Computed Tomog- raphy Guidelines Committee. J Cardiovasc Comput Tomogr 2009;3:190 -204.
- Wilson PWF, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Pre- diction of coronary heart disease using risk factor categories. Circulation 1998;97: 1837 -1847.
- Hall M, Holmes G. Benchmarking attribute selection techniques for discrete class data mining. IEEE Trans Knowledge Data Eng 2003;15:1437 -1447.
- Friedman J, Hastie T, Tibshirani R. Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann Stat 2000;28: 337 -407.
- Molinaro AM, Simon R, Pfeiffer RM. Prediction error estimation: a comparison of resampling methods. Bioinformatics 2005;21:3301 -3307.
- Arsanjani R, Dey D, Khachatryan T, Shalev A, Hayes S, Fish M, Nakanishi R, Germano G, Berman D, Slomka P. Prediction of revascularization after myocardial perfusion SPECT by machine learning in a large population. J Nucl Cardiol 2015;22: 877 -884.
- Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH. The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 2009;11:10-18.
- Kanamori T, Takenouchi T, Eguchi S, Murata N. Robust loss functions for boosting. Neural Comput. 2007;19:2183 -2244.
- Greiner M, Sohr D, Go ¨bel P. A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests. J Immunol Methods 1995;185:123 -132.
- Selvarajah S, Kaur G, Haniff J, Cheong KC, Hiong TG, van der Graaf Y, Bots ML. Comparison of the Framingham Risk Score, SCORE and WHO/ISH cardiovascular risk prediction models in an Asian population. Int J Cardiol 2014;176:211 -218.
- Franco M, Cooper RS, Bilal U, Fuster V. Challenges and opportunities for cardio- vascular disease prevention. Am J Med 2015;124:95 -102.
- Arbab-Zadeh A, Fuster V. The myth of the "vulnerable plaque". J Am Coll Cardiol 2015;65:846-855.
- Greenland P, Knoll MD, Stamler J, Neaton JD, Dyer AR, Garside DB, Wilson PW. Major risk factors as antecedents of fatal and nonfatal coronary heart disease events. JAMA 2003;290:891 -897.
- Budoff MJ, Shaw LJ, Liu ST, Weinstein SR, Mosler TP, Tseng PH, Flores FR, Callister TQ, Raggi P, Berman DS. Long-term prognosis associated with coronary calcification: observations from a registry of 25,253 patients. J Am Coll Cardiol 2007; 49:1860 -1870.
- Mjolsness E, DeCoste D. Machine learning for science: state of the art and future prospects. Science 2001;293:2051 -2055.
- Patel MR, Peterson ED, Dai D, Brennan JM, Redberg RF, Anderson HV, Brindis RG, Douglas PS. Low diagnostic yield of elective coronary angiography. N Engl J Med 2010;362:886 -895.
- Meijboom WB, Mieghem CA, van Pelt N, Weustink A, Pugliese F, Mollet NR, Boersma E, Regar E, van Geuns RJ, de Jaegere PJ, Serruys PW, Krestin GP, de Feyter PJ. Comprehensive assessment of coronary artery stenoses computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina. J Am Coll Cardiol 2008;52:636 -643.
- Kang X, Berman DS, Lewin HC, Cohen I, Friedman JD, Germano G, Hachamovitch R, Shaw LJ. Incremental prognostic value of myocardial perfusion single photon emission computed tomography in patients with diabetes mellitus. Am Heart J 1999;138:1025 -1032.
- Binder H, Allignol A, Schumacher M, Beyersmann J. Boosting for high-dimensional time-to-event data with competing risks. Bioinformatics 2009;25:890 -896.
Cited by
Sorted by citations
Machine-learning-based disease diagnosis: a comprehensive review
Ahsan, M. M., Luna, S. A., & Siddique, Z. (2022) · Healthcare · doi:10.3390/healthcare10030541 · Cited by 603 · see citing works
Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network
Raghunath, S., Ulloa Cerna, A. E., Jing, L., vanMaanen, D. P., Stough, J., Hartzel, D. N., Leader, J. B., Kirchner, H. L., Stumpe, M. C., Hafez, A., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0870-z · Cited by 314 · see citing works
Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.09.009 · Cited by 227 · see citing works
Machine Learning Outperforms ACC/AHA CVD Risk Calculator in MESA
Kakadiaris, I. A., Vrigkas, M., Yen, A. A., Kuznetsova, T., Budoff, M., & Naghavi, M. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.118.009476 · Cited by 227 · see citing works
Self-citationLifelong endurance exercise and its relation with coronary atherosclerosis
De Bosscher, R., Dausin, C., Claus, P., Bogaert, J., Dymarkowski, S., Goetschalckx, K., Ghekiere, O., Van De Heyning, C. M., Van Herck, P., Paelinck, B., et al. (2023) · European Heart Journal · doi:10.1093/eurheartj/ehad152 · Cited by 140 · see citing works
Cai, Y., Cai, Y.-Q., Tang, L.-Y., Wang, Y.-H., Gong, M., Jing, T.-C., Li, H.-J., Li-Ling, J., Hu, W., Yin, Z., et al. (2024) · BMC Medicine · doi:10.1186/s12916-024-03273-7 · Cited by 122 · see citing works
Smith, L. A., Oakden-Rayner, L., Bird, A., Zeng, M., To, M.-S., Mukherjee, S., & Palmer, L. J. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00177-2 · Cited by 116 · see citing works
Automation and artificial intelligence in the clinical laboratory
Naugler, C., & Church, D. L. (2019) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2018.1561640 · Cited by 112 · see citing works
Nielsen, J. C., Lin, Y.-J., de Oliveira Figueiredo, M. J., Sepehri Shamloo, A., Alfie, A., Boveda, S., Dagres, N., Di Toro, D., Eckhardt, L. L., Ellenbogen, K., et al. (2020) · EP Europace · doi:10.1093/europace/euaa065 · Cited by 98 · see citing works
Mastrodicasa, D., van Assen, M., Huisman, M., Leiner, T., Williamson, E. E., Nicol, E. D., Allen, B. D., Saba, L., Vliegenthart, R., & Hanneman, K. (2025) · Radiology · doi:10.1148/radiol.240516 · Cited by 69 · see citing works
Hanneman, K., Playford, D., Dey, D., van Assen, M., Mastrodicasa, D., Cook, T. S., Gichoya, J. W., Williamson, E. E., Rubin, G. D., & on behalf of the American Heart Association Council on Cardiovascular Radiology and Intervention; and Council on Lifelong Congenital Heart Disease and Heart Health in the Young (2024) · Circulation · doi:10.1161/cir.0000000000001202 · Cited by 72 · see citing works
Self-citationMachine intelligence in cardiovascular medicine
Miller, D. D. (2020) · Cardiology in Review · doi:10.1097/crd.0000000000000294 · Cited by 58 · see citing works
Suri, J. S., Bhagawati, M., Paul, S., Protogerou, A. D., Sfikakis, P. P., Kitas, G. D., Khanna, N. N., Ruzsa, Z., Sharma, A. M., Saxena, S., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12030722 · Cited by 55 · see citing works
The differences between suicide ideators and suicide attempters: simple, complicated, or complex
Huang, X., Ribeiro, J. D., & Franklin, J. C. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000498 · Cited by 50 · see citing works
Holistic AI analysis of hybrid cardiac perfusion images for mortality prediction
Marcinkiewicz, A. M., Zhang, W., Shanbhag, A., Miller, R. J. H., Lemley, M., Ramirez, G., Buchwald, M., Killekar, A., Kavanagh, P. B., Feher, A., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01526-0 · Cited by 12 · see citing works
Self-citation
- Authors
- Motwani, M., Dey, D., Berman, D. S., Germano, G., Achenbach, S., Al-Mallah, M. H., Andreini, D., Budoff, M. J., Cademartiri, F., Callister, T. Q., Chang, H.-J., Chinnaiyan, K., Chow, B. J. W., Cury, R. C., Delago, A., Gomez, M., Gransar, H., Hadamitzky, M., Hausleiter, J., Hindoyan, N., Feuchtner, G., Kaufmann, P. A., Kim, Y.-J., Leipsic, J., Lin, F. Y., Maffei, E., Marques, H., Pontone, G., Raff, G., Rubinshtein, R., Shaw, L. J., Stehli, J., Villines, T. C., Dunning, A., Min, J. K., & Slomka, P. J.
- Affiliations
- Cedars-Sinai Medical Center (US)
- Friedrich-Alexander-Universität Erlangen-Nürnberg (DE)
- King Saud bin Abdulaziz University for Health Sciences (SA)
- University of Milan (IT)
- Harbor–UCLA Medical Center (US)
- Montreal Heart Institute (CA)
- Severance Hospital (KR)
- Corewell Health William Beaumont University Hospital (US)
- Ottawa Heart Institute (CA)
- Capital Cardiology Associates (US)
- NewYork–Presbyterian Hospital (US)
- Deutsches Herzzentrum München (DE)
- Innsbruck Medical University (AT)
- University Hospital Zurich (CH)
- Seoul National University (KR)
- University of British Columbia (CA)
- Hospital da Luz (PT)
- Centro Cardiologico Monzino (IT)
- Technion – Israel Institute of Technology (IL)
- Emory University (US)
- Walter Reed National Military Medical Center (US)
- Duke Clinical Research Institute (US)
- Corresponding author
- Slomka, Piotr J.
- ORCID
- Motwani, Manish
- Dey, Damini
- Berman, Daniel S.
- Germano, Guido
- Achenbach, Stephan
- Al-Mallah, Mouaz H.
- Andreini, Daniele
- Budoff, Matthew J.
- Cademartiri, Filippo
- Chang, Hyuk-Jae
- Chinnaiyan, Kavitha
- Chow, Benjamin J.W.
- Cury, Ricardo C.
- Delago, Augustin
- Gomez, Millie
- Gransar, Heidi
- Hadamitzky, Martin
- Feuchtner, Gudrun
- Kaufmann, Philipp A.
- Kim, Yong-Jin
- Leipsic, Jonathon
- Lin, Fay Y.
- Maffei, Erica
- Marques, Hugo
- Pontone, Gianluca
- Raff, Gilbert
- Shaw, Leslee J.
- Stehli, Julia
- Villines, Todd C.
- Min, James K.
- Slomka, Piotr J.
- Venue
- European Heart Journal
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-06-01
- Volume
- 38
- Issue
- 7
- Pages
- ehw188
- ISSN
- 0195-668X (ISSN-L)
- 1522-9645
- PMC
- PMC5897836
- OpenAlex
- W2408866005
- Cited by
- 692
- References
- 4
- Citations by year
- 2026: 17 · 2025: 48 · 2024: 85 · 2023: 82 · 2022: 97 · 2021: 107 · 2020: 109 · 2019: 70 · 2018: 47 · 2017: 25 · 2016: 5
- FWCI
- 42.54
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Heart Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- eScholarship (California Digital Library) (submittedVersion)
- YUHSpace (Yonsei University Medical Library) (submittedVersion, cc-by-nc-nd)
- mediaTUM (Technical University of Munich) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Funders
- National Heart Lung and Blood Institute (R01-HL111141, R01-HL115150, R01-HL118019, R01-2R01HL089765)
- Dowager Countess Eleanor Peel Trust
- Dickinson Trust
- Dalio Foundation
- Michael Wolk Foundation
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Cardiac Imaging and Diagnostics
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- all-cause mortality prediction, coronary artery disease, coronary CT angiography, machine learning, prognostic risk assessment, segment involvement score, non-calcified plaque, Framingham risk score, non-invasive imaging
- MeSH terms
- Machine Learning, Computed Tomography Angiography, Cause of Death, Coronary Artery Disease, Feasibility Studies, Female, Humans, Male, Middle Aged, Prospective Studies, Registries, Risk Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
