Clinician's guide to trustworthy and responsible artificial intelligence in cardiovascular imaging
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
A growing number of artificial intelligence (AI)-based systems are being proposed and developed in cardiology, driven by the increasing need to deal with the vast amount of clinical and imaging data with the ultimate aim of advancing patient care, diagnosis and prognostication. However, there is a critical gap between the development and clinical deployment of AI tools. A key consideration for implementing AI tools into real-life clinical practice is their “trustworthiness” by end-users. Namely, we must ensure that AI systems can be trusted and adopted by all parties involved, including clinicians and patients. Here we provide a summary of the concepts involved in developing a “trustworthy AI system.” We describe the main risks of AI applications and potential mitigation techniques for the wider application of these promising techniques in the context of cardiovascular imaging. Finally, we show why trustworthy AI concepts are important governing forces of AI development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774
The potential for artificial intelligence in healthcare
Davenport, T., & Kalakota, R. (2019) · Future Healthcare Journal · doi:10.7861/futurehosp.6-2-94 · Cited by 3,800
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652 · cited 1×
“It has been envisioned that AI tools will take over or at least substitute the work of radiologists and cardiovascular imagers to a great extent and consequently necessitate fewer human resources creating cheaper and more accurate care in the future (52).”
BackgroundWynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., et al. (2020) · doi:10.1101/2020.03.24.20041020 · Cited by 3,070
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · cited 1×
“As Rajkomar et al. summarized: any type of bias depicted within the dataset is learned and adapted into model performance (90).”
BackgroundGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 2×
“posed a list of questions on how and why data was collected, what is the composition of the data, and how it was curated and labeled in their document entitled "Datasheets for Datasets” (102).”
BackgroundThe false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · cited 1×
“note that with the currently available methods, our best hope is to go through rigorous internal and external validation and use explainability models for troubleshooting and system audits (114).”
MethodAdversarial attacks on medical machine learning
Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., & Kohane, I. S. (2019) · Science · doi:10.1126/science.aaw4399 · Cited by 1,088 · cited 1×
“Moreover, in the field of medical imaging, particular attention is necessary toward dealing with potential adversarial attacks (97), including “one-pixel” attacks (98).”
BackgroundZhang, J., Gajjala, S., Agrawal, P., Tison, G. H., Hallock, L. A., Beussink-Nelson, L., Lassen, M. H., Fan, E., Aras, M. A., Jordan, C., et al. (2018) · Circulation · doi:10.1161/circulationaha.118.034338 · Cited by 1,006
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894 · cited 1×
“Given the degradation in performance reported for deep-learning algorithms for medical imaging, it is of paramount importance that the test set consists of independent cohorts to allow for external validation, a key requirement for ensuring the trustworthiness of AI systems (19, 20).”
BackgroundAutomated cardiovascular magnetic resonance image analysis with fully convolutional networks
Bai, W., Sinclair, M., Tarroni, G., Oktay, O., Rajchl, M., Vaillant, G., Lee, A. M., Aung, N., Lukaschuk, E., Sanghvi, M. M., et al. (2018) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/s12968-018-0471-x · Cited by 773 · cited 5×
“Transformative measures have already impacted many areas of cardiovascular medicine, from smart devices promising to diagnose arrhythmias based on single-lead ECG (4) to automatic image segmentation tools shortening manual image analysis (5, 6).”
BackgroundMethodInfluentialSelf-citationLarrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1919012117 · Cited by 685 · cited 1×
“Overrepresentation of a certain group leads to data collection bias (18), as exemplified by Larrazabal et al. (91).”
BackgroundIn AI We Trust: Ethics, Artificial Intelligence, and Reliability
Ryan, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00228-y · Cited by 600 · cited 1×
“It is easy to get lost in a philosophical discussion about how to define trust or if it is even possible outside the human realm (21, 22).”
BackgroundSecond opinion needed: Communicating uncertainty in medical machine learning
Kompa, B., Snoek, J., & Beam, A. L. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00367-3 · Cited by 514
Why digital medicine depends on interoperability
Lehne, M., Sass, J., Essenwanger, A., Schepers, J., & Thun, S. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0158-1 · Cited by 501
Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
Campello, V. M., Gkontra, P., Izquierdo, C., Martin-Isla, C., Sojoudi, A., Full, P. M., Maier-Hein, K., Zhang, Y., He, Z., Ma, J., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3090082 · Cited by 481
Self-citationUK Biobank's cardiovascular magnetic resonance protocol
Petersen, S. E., Matthews, P. M., Francis, J. M., Robson, M. D., Zemrak, F., Boubertakh, R., Young, A. A., Hudson, S., Weale, P., Garratt, S., et al. (2016) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/s12968-016-0227-4 · Cited by 481 · cited 1×
“Critically, DL-based segmentation tools are often trained and tested on images from single clinical centers, using one vendor with a well-defined protocol resulting in homogenous datasets (59, 60).”
MethodSelf-citationGeis, J. R., Brady, A. P., Wu, C. C., Spencer, J., Ranschaert, E., Jaremko, J. L., Langer, S. G., Borondy Kitts, A., Birch, J., Shields, W. F., et al. (2019) · Radiology · doi:10.1148/radiol.2019191586 · Cited by 388
Presenting machine learning model information to clinical end users with model facts labels
Sendak, M. P., Gao, M., Brajer, N., & Balu, S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0253-3 · Cited by 279
CheXclusion: Fairness gaps in deep chest X-ray classifiers
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y., & Ghassemi, M. (2020) · Biocomputing 2021 · doi:10.1142/9789811232701_0022 · Cited by 275 · cited 1×
“, true positive rate disparity, statistical parity group fairness, equalized odds, predictive/equality) (93, 94).”
BackgroundDeep convolutional neural networks to predict cardiovascular risk from computed tomography
Zeleznik, R., Foldyna, B., Eslami, P., Weiss, J., Alexander, I., Taron, J., Parmar, C., Alvi, R. M., Banerji, D., Uno, M., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-20966-2 · Cited by 240
What do medical students actually need to know about artificial intelligence?
McCoy, L. G., Nagaraj, S., Morgado, F., Harish, V., Das, S., & Celi, L. A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0294-7 · Cited by 240
Does “ AI ” stand for augmenting inequality in the era of covid-19 healthcare?
Leslie, D., Mazumder, A., Peppin, A., Wolters, M. K., & Hagerty, A. (2021) · BMJ · doi:10.1136/bmj.n304 · Cited by 177
Informed Consent and Medical Artificial Intelligence: What to Tell the Patient?
Cohen, I. G. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3529576 · Cited by 165
Grunhut, J., Wyatt, A. T., & Marques, O. (2021) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205211036836 · Cited by 161 · cited 1×
“(133) proposed crucial points to improve AI literacy in medical education programs.”
BackgroundDirect Automatic Coronary Calcium Scoring in Cardiac and Chest CT
de Vos, B. D., Wolterink, J. M., Leiner, T., de Jong, P. A., Lessmann, N., & Isgum, I. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2899534 · Cited by 134
Puyol-Antón, E., Ruijsink, B., Mariscal Harana, J., Piechnik, S. K., Neubauer, S., Petersen, S. E., Razavi, R., Chowienczyk, P., & King, A. P. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.859310 · Cited by 85 · cited 1×
“performed the first analysis of DL fairness in cardiovascular segmentation using the UKB dataset (92).”
MethodSelf-citationMinimising multi-centre radiomics variability through image normalisation: a pilot study
Campello, V. M., Martín-Isla, C., Izquierdo, C., Guala, A., Palomares, J. F. R., Viladés, D., Descalzo, M. L., Karakas, M., Çavuş, E., Raisi-Estabragh, Z., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-16375-0 · Cited by 24
Self-citation
Cited by
A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
Salih, A. M., Raisi‐Estabragh, Z., Galazzo, I. B., Radeva, P., Petersen, S. E., Lekadir, K., & Menegaz, G. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400304 · Cited by 897
“XAI is indispensable to increase the model transparency and the trust of end-users in the model outcome [6] [7].”
BackgroundSelf-citationExplainable Artificial Intelligence in Radiological Cardiovascular Imaging—A Systematic Review
Haupt, M., Maurer, M. H., & Thomas, R. P. (2025) · Diagnostics · doi:10.3390/diagnostics15111399 · Cited by 58
Marey, A., Serdysnki, K. C., Killeen, B. D., Unberath, M., & Umair, M. (2024) · BJR|Artificial Intelligence · doi:10.1093/bjrai/ubae008 · Cited by 7
Metadata
- Authors
- Szabo, L., Raisi-Estabragh, Z., Salih, A., McCracken, C., Ruiz Pujadas, E., Gkontra, P., Kiss, M., Maurovich-Horvath, P., Vago, H., Merkely, B., Lee, A. M., Lekadir, K., & Petersen, S. E.
- Venue
- Frontiers in Cardiovascular Medicine
- Publisher
- Frontiers Media SA
- Published
- 2022-11-08
- Volume
- 9
- Pages
- 1016032-1016032
- ISSN
- 2297-055X (ISSN-L)
- OpenAlex
- W4308460922
- Cited by
- 36
- References
- 33
- FWCI
- 1.59
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.