The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging
Open accessPublished version of a preprint
Abstract
Purpose To evaluate the trustworthiness of saliency maps for abnormality localization in medical imaging. Materials and Methods Using two large publicly available radiology datasets (Society for Imaging Informatics in Medicine–American College of Radiology Pneumothorax Segmentation dataset and Radiological Society of North America Pneumonia Detection Challenge dataset), the performance of eight commonly used saliency map techniques were quantified in regard to (a) localization utility (segmentation and detection), (b) sensitivity to model weight randomization, (c) repeatability, and (d) reproducibility. Their performances versus baseline methods and localization network architectures were compared, using area under the precision-recall curve (AUPRC) and structural similarity index measure (SSIM) as metrics. Results All eight saliency map techniques failed at least one of the criteria and were inferior in performance compared with localization networks. For pneumothorax segmentation, the AUPRC ranged from 0.024 to 0.224, while a U-Net achieved a significantly superior AUPRC of 0.404 (P < .005). For pneumonia detection, the AUPRC ranged from 0.160 to 0.519, while a RetinaNet achieved a significantly superior AUPRC of 0.596 (P <.005). Five and two saliency methods (of eight) failed the model randomization test on the segmentation and detection datasets, respectively, suggesting that these methods are not sensitive to changes in model parameters. The repeatability and reproducibility of the majority of the saliency methods were worse than localization networks for both the segmentation and detection datasets. Conclusion The use of saliency maps in the high-risk domain of medical imaging warrants additional scrutiny and recommend that detection or segmentation models be used if localization is the desired output of the network. Keywords: Technology Assessment, Technical Aspects, Feature Detection, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2021
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the performance on the testing datasets was not competitive [19].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [19]
In the citing paper
“Eight interpretable methods of saliency maps (+ Grad-CAM, guided backpropagation, and guided Grad-CAM) were evaluated [19].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [19]
In the citing paper
“Although we used established saliency attribution methods to interpret our model’s predictions, the technical limitations39,40 of these methods may restrict the applicability of the insights gained.”
From Foundation Model for Cancer Imaging Biomarkers (2024) · cites this paper as [40]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 12×
“RetinaNet trained to generate bounding boxes achieved a higher performance than all the saliency methods (AUPRC, 0.59 6 0.26; P , .001 for all comparisons).”
MethodInfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Rudin argues that instead of creating methods to interpret black box models trained for high-stakes decision-making, we should instead put our focus on designing models that are inherently interpretable (31).”
BackgroundDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 8×
“Institutional review board approval was not required for this retrospective study; the chest radiographs used in this study Abbreviations ACR = American College of Radiology, AUC = area under the receiver operating characteristic curve, AUPRC = area under the precision-recall curve, AVG = average of all masks across t…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- Adebayo J, 2018, Adv Neural Inf Process Syst, 25, 9505
- Boyd K, 2013, Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 451
- Alzantot M, 2019, In: 2019 IEEE International Conference on Smart Computing (SMARTCOMP), Conference Location, Conference date, 81
Cited by
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328
AI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631
A Review of Explainable Artificial Intelligence in Healthcare
Sadeghi, Z., Alizadehsani, R., CIFCI, M. A., Kausar, S., Rehman, R., Mahanta, P., Bora, P. K., Almasri, A., Alkhawaldeh, R. S., Hussain, S., et al. (2024) · Computers and Electrical Engineering · doi:10.1016/j.compeleceng.2024.109370 · Cited by 598
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
Benchmarking saliency methods for chest X-ray interpretation
Saporta, A., Gui, X., Agrawal, A., Pareek, A., Truong, S. Q. H., Nguyen, C. D. T., Ngo, V.-D., Seekins, J., Blankenberg, F. G., Ng, A. Y., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00536-x · Cited by 270
“Our work builds upon several studies investigating the validity of saliency maps for localization 59–61 and upon some early work on the trustworthiness of saliency methods to explain DNNs in medical imaging 47 .”
Metadata
- Authors
- Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., Hoebel, K., Gupta, S., Patel, J., Gidwani, M., Adebayo, J., Li, M. D., & Kalpathy-Cramer, J.
- Affiliations
- Harvard University (US)
- Georgia Institute of Technology (US)
- Massachusetts Institute of Technology (US)
- ORCID
- Venue
- Radiology: Artificial Intelligence
- Published
- 2021-11-01
- Volume
- 3
- Issue
- 6
- Pages
- e200267-e200267
- ISSN
- 2638-6100 (ISSN-L)
- PMC
- PMC8637231
- OpenAlex
- W3202023533
- Cited by
- 342
- References
- 55
- Citations by year
- 2026: 76 · 2025: 66 · 2024: 59 · 2023: 61 · 2022: 60 · 2021: 19
- FWCI
- 17.89
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Radiology Artificial Intelligence (publishedVersion)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Biomedical Imaging and Bioengineering (5T32EB1680)
- National Institutes of Health
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Keywords
- saliency maps, anomaly localization, trustworthiness, pneumonia detection, repeatability, reproducibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging. https://app.cochat.ai/papers/10.1148/ryai.2021200267
- The paper
- Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., Hoebel, K., Gupta, S., Patel, J., Gidwani, M., Adebayo, J., Li, M. D., & Kalpathy-Cramer, J. (2021). Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging. Radiology: Artificial Intelligence. https://doi.org/10.1148/ryai.2021200267
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.