Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging
Open accessPublished version of a preprint
The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
Purpose To evaluate the trustworthiness of saliency maps for abnormality localization in medical imaging. Materials and Methods Using two large publicly available radiology datasets (Society for Imaging Informatics in Medicine–American College of Radiology Pneumothorax Segmentation dataset and Radiological Society of North America Pneumonia Detection Challenge dataset), the performance of eight commonly used saliency map techniques were quantified in regard to (a) localization utility (segmentation and detection), (b) sensitivity to model weight randomization, (c) repeatability, and (d) reproducibility. Their performances versus baseline methods and localization network architectures were compared, using area under the precision-recall curve (AUPRC) and structural similarity index measure (SSIM) as metrics. Results All eight saliency map techniques failed at least one of the criteria and were inferior in performance compared with localization networks. For pneumothorax segmentation, the AUPRC ranged from 0.024 to 0.224, while a U-Net achieved a significantly superior AUPRC of 0.404 (P < .005). For pneumonia detection, the AUPRC ranged from 0.160 to 0.519, while a RetinaNet achieved a significantly superior AUPRC of 0.596 (P <.005). Five and two saliency methods (of eight) failed the model randomization test on the segmentation and detection datasets, respectively, suggesting that these methods are not sensitive to changes in model parameters. The repeatability and reproducibility of the majority of the saliency methods were worse than localization networks for both the segmentation and detection datasets. Conclusion The use of saliency maps in the high-risk domain of medical imaging warrants additional scrutiny and recommend that detection or segmentation models be used if localization is the desired output of the network. Keywords: Technology Assessment, Technical Aspects, Feature Detection, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2021
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the performance on the testing datasets was not competitive [19].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [19]
In the citing paper
“Eight interpretable methods of saliency maps (+ Grad-CAM, guided backpropagation, and guided Grad-CAM) were evaluated [19].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [19]
In the citing paper
“Although we used established saliency attribution methods to interpret our model’s predictions, the technical limitations39,40 of these methods may restrict the applicability of the insights gained.”
From Foundation Model for Cancer Imaging Biomarkers (2024) · cites this paper as [40]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 12×
“RetinaNet trained to generate bounding boxes achieved a higher performance than all the saliency methods (AUPRC, 0.59 6 0.26; P , .001 for all comparisons).”
MethodInfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Rudin argues that instead of creating methods to interpret black box models trained for high-stakes decision-making, we should instead put our focus on designing models that are inherently interpretable (31).”
BackgroundDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 8×
“Institutional review board approval was not required for this retrospective study; the chest radiographs used in this study Abbreviations ACR = American College of Radiology, AUC = area under the receiver operating characteristic curve, AUPRC = area under the precision-recall curve, AVG = average of all masks across t…”
BackgroundMethodInfluentialAxiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · cited 10×
“Institutional review board approval was not required for this retrospective study; the chest radiographs used in this study Abbreviations ACR = American College of Radiology, AUC = area under the receiver operating characteristic curve, AUPRC = area under the precision-recall curve, AVG = average of all masks across t…”
BackgroundMethodInfluentialStriving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · cited 2×
“For model interpretability, we evaluated the following saliency methods: gradient explanation (GRAD) (18), Smoothgrad (SG) (19), integrated gradients (IG) (20), smooth IG (SIG) (19,20), gradient-weighted class activation mapping (GCAM) (9), XRAI (21), guided backpropagation (GBP) (22), and guided GCAM (GGCAM) (9).”
MethodClinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Narin, A., Kaya, C., & Pamuk, Z. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00984-y · Cited by 2,054 · cited 1×
“Additionally, we only performed tests on two CNN architectures, though these are commonly used networks in the literature for chest radiograph analysis (32,33).”
MethodDeep Learning of Representations for Unsupervised and Transfer Learning.
Bengio, Y. (2011) · Cited by 1,703
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467
Rajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449 · cited 1×
“Additionally, we only performed tests on two CNN architectures, though these are commonly used networks in the literature for chest radiograph analysis (32,33).”
MethodFacial Landmark Detection by Deep Multi-task Learning
Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_7 · Cited by 1,365
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · cited 1×
“…activation mapping (GCAM) ( A backpropagation-based method that uses the feature maps of the final convolutional layer to generate heatmaps XRAI (21) Builds on IG by starting with a baseline image and incrementally adding regions that offer maximal at-tribution gain Guided backpropagation (GBP)…”
MethodSelf-citationCheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 1,219
Adversarial Examples Are Not Bugs, They Are Features
Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., & Madry, A. (2019) · arXiv · doi:10.48550/arxiv.1905.02175 · Cited by 971
AI can be sexist and racist — it’s time to make it fair
Zou, J., & Schiebinger, L. (2018) · Nature · doi:10.1038/d41586-018-05707-8 · Cited by 765
Bien, N., Rajpurkar, P., Ball, R. L., Irvin, J., Park, A., Jones, E., Bereket, M., Patel, B. N., Yeom, K. W., Shpanskaya, K., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002699 · Cited by 734 · cited 13×
“First, we determined if there were differences in the utilities between saliency map methods derived from the trained classification models compared with the utility of the localization networks (U-Net for segmentation and RetinaNet for detection).”
BackgroundInfluentialZar, J. H. (2005) · Encyclopedia of Biostatistics · doi:10.1002/0470011815.b2a15150 · Cited by 566 · cited 1×
“We used the SSIM to assess the change of the original saliency map with the saliency maps generated from the model after each randomization step (26).”
MethodDeep learning interpretation of echocardiograms
Ghorbani, A., Ouyang, D., Abid, A., He, B., Chen, J. H., Harrington, R. A., Liang, D. H., Ashley, E. A., & Zou, J. Y. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0216-8 · Cited by 518
On the interpretability of artificial intelligence in radiology: challenges and opportunities
Reyes, M., Meier, R., Pereira, S., Silva, C. A., Dahlweid, F.-M., Tengg-Kobligk, H. V., Summers, R. M., & Wiest, R. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020190043 · Cited by 484
Automated deep-neural-network surveillance of cranial images for acute neurologic events
Titano, J. J., Badgeley, M., Schefflein, J., Pain, M., Su, A., Cai, M., Swinburne, N., Zech, J., Kim, J., Bederson, J., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0147-y · Cited by 458
Sayres, R., Taly, A., Rahimy, E., Blumer, K., Coz, D., Hammel, N., Krause, J., Narayanaswamy, A., Rastegar, Z., Wu, D., et al. (2019) · Ophthalmology · doi:10.1016/j.ophtha.2018.11.016 · Cited by 418 · cited 1×
“A recent study also showed that saliency maps did not provide additional performance improvement in assisted clinician interpretation compared with only providing a model prediction (11).”
BackgroundHidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411 · cited 1×
“For example, a chest tube in an image would imply the presence of a pneumothorax, but much of the chest tube may not be in the region of interest (34).”
BackgroundArea under the Precision-Recall Curve: Point Estimates and Confidence Intervals
Boyd, K., Eng, K. H., & Page, C. D. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40994-3_29 · Cited by 381 · cited 1×
“…or bounding boxes in the training and validation datasets (AVG) and a high baseline determined by the area under the precision recall curve (AUPRC [23]) of segmentation (U-Net) and detection networks (RetinaNet); (b) in model weight randomization, the average value of the structural similarity…”
MethodOzenne, B., Subtil, F., & Maucort-Boulch, D. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.02.010 · Cited by 379 · cited 1×
“In the context of findings on medical images, the area under the receiver operating characteristic curve (AUC) can be skewed by the presence of a large number of true-negative findings (25).”
BackgroundWinkler, J. K., Fink, C., Toberer, F., Enk, A., Deinlein, T., Hofmann-Wellenhof, R., Thomas, L., Lallas, A., Blum, A., Stolz, W., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1735 · Cited by 365
Jeyaraj, P. R., & Samuel Nadar, E. R. (2019) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-018-02834-7 · Cited by 360 · cited 1×
“Many CNNs have outperformed conventional methods for various medical tasks (4,5).”
MethodShih, G., Wu, C. C., Halabi, S. S., Kohli, M. D., Prevedello, L. M., Cook, T. S., Sharma, A., Amorosa, J. K., Arteaga, V., Galperin-Aizenberg, M., et al. (2019) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2019180041 · Cited by 337
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
A deep CNN based multi-class classification of Alzheimer's disease using MRI
Farooq, A., Anwar, S., Awais, M., & Rehman, S. (2017) · 2017 IEEE International Conference on Imaging Systems and Techniques (IST) · doi:10.1109/ist.2017.8261460 · Cited by 303
3D Deep Learning for Multi-modal Imaging-Guided Survival Time Prediction of Brain Tumor Patients
Nie, D., Zhang, H., Adeli, E., Liu, L., & Shen, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_25 · Cited by 282
Saba, T., Khan, M. A., Rehman, A., & Marie-Sainte, S. L. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1413-3 · Cited by 250 · cited 1×
“Many CNNs have outperformed conventional methods for various medical tasks (4,5).”
MethodShould Health Care Demand Interpretable Artificial Intelligence or Accept “Black Box” Medicine?
Wang, F., Kaushal, R., & Khullar, D. (2020) · Annals of Internal Medicine · doi:10.7326/m19-2548 · Cited by 244
Detection of anaemia from retinal fundus images via deep learning
Mitani, A., Huang, A., Venugopalan, S., Corrado, G. S., Peng, L., Webster, D. R., Hammel, N., Liu, Y., & Varadarajan, A. V. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0487-z · Cited by 230 · cited 1×
“Although saliency maps were not originally intended for either segmentation or detection, they have been used this way in clinical research to identify areas of abnormality from trained neural networks (6–8).”
MethodRegulation of predictive analytics in medicine
Parikh, R. B., Obermeyer, Z., & Navathe, A. S. (2019) · Science · doi:10.1126/science.aaw0029 · Cited by 223
Lou, B., Doken, S., Zhuang, T., Wingerter, D., Gidwani, M., Mistry, N., Ladic, L., Kamen, A., & Abazeed, M. E. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30058-5 · Cited by 206 · cited 1×
“One approach would be to train CNNs that output traditional handcrafted features (such as shape and texture) as intermediates (30).”
BackgroundSelf-citationXRAI: Better Attributions Through Regions
Kapishnikov, A., Bolukbasi, T., Viegas, F., & Terry, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00505 · Cited by 198
Brooks, A. C., Zhao, X., & Pappas, T. N. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.926161 · Cited by 175
Li, M., Chang, K., Bearce, B., Chang, C. Y., Huang, A. J., Campbell, J. P., Brown, J. M., Singh, P., Hoebel, K., Erdogmus, D., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0255-1 · Cited by 169
Self-citationStructural similarity index family for image quality assessment in radiological images
Renieblas, G. P., Nogués, A. T., González, A. M., Gómez-Leon, N., & del Castillo, E. G. (2017) · Journal of Medical Imaging · doi:10.1117/1.jmi.4.3.035501 · Cited by 166 · cited 1×
“Note that a low base-line of 0.5 for SSIM was chosen because SSIM ranges from 0 (for lack of any structural similarity) to 1 (for identical structural similarity), and 0.5 marks the midpoint for whether the SSIM is more structurally similar or dissimilar (24).”
BackgroundInfluentialLi, M. D., Arun, N. T., Gidwani, M., Chang, K., Deng, F., Little, B. P., Mendoza, D. P., Lang, M., Lee, S. I., O’Shea, A., et al. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020200079 · Cited by 161
Self-citationShin, Y., & Balasingham, I. (2017) · 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2017.8037556 · Cited by 108
Eitel, F., Ritter, K., & Initiative, A. D. N. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_1 · Cited by 96
Deep Neural Network or Dermatologist?
Young, K., Booth, G., Simpson, B., Dutton, R., & Shrapnel, S. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-33850-3_6 · Cited by 79
Evaluation of Explainable Deep Learning Methods for Ophthalmic Diagnosis
Singh, A., Balaji, J. J., Rasheed, M. A., Jayakumar, V., Raman, R., & Lakshminarayanan, V. (2021) · Clinical ophthalmology · doi:10.2147/opth.s312236 · Cited by 79
Chang, K., Beers, A. L., Brink, L., Patel, J., Singh, P., Arun, N., Hoebel, K., Gaw, N. B., Shah, M., Pisano, E. D., et al. (2020) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2020.05.015 · Cited by 58
Self-citationDeep Learning Interpretation of EchocardiogramsPreprint
Ghorbani, A., Ouyang, D., Abid, A., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/681676 · Cited by 58
Tolkachev, A., Sirazitdinov, I., Kholiavchenko, M., Mustafaev, T., & Ibragimov, B. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3023476 · Cited by 57
A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
Nie, W., Zhang, Y., & Patel, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07039 · Cited by 51
Assessing the validity of saliency maps for abnormality localization in medical imaging
Arun, N., Gaw, N. B., Singh, P., Chang, K., Hoebel, K., Patel, J., Gidwani, M., & Kalpathy–Cramer, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.00063 · Cited by 16
Self-citationNeuroMask: Explaining Predictions of Deep Neural Networks through Mask Learning
Alzantot, M., Widdicombe, A., Julier, S., & Srivastava, M. B. (2019) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp.2019.00033 · Cited by 1 · cited 1×
“We could also investigate incorporating saliency maps as a part of neural network training and evaluate if this type of approach results in maps that have higher utility than maps that are generated after model training (35).”
Background
Research works printed but not linked: 3
- Adebayo J, 2018, Adv Neural Inf Process Syst, 25, 9505
- Boyd K, 2013, Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 451
- Alzantot M, 2019, In: 2019 IEEE International Conference on Smart Computing (SMARTCOMP), Conference Location, Conference date, 81
Cited by
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328
AI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631
A Review of Explainable Artificial Intelligence in Healthcare
Sadeghi, Z., Alizadehsani, R., CIFCI, M. A., Kausar, S., Rehman, R., Mahanta, P., Bora, P. K., Almasri, A., Alkhawaldeh, R. S., Hussain, S., et al. (2024) · Computers and Electrical Engineering · doi:10.1016/j.compeleceng.2024.109370 · Cited by 598
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
Benchmarking saliency methods for chest X-ray interpretation
Saporta, A., Gui, X., Agrawal, A., Pareek, A., Truong, S. Q. H., Nguyen, C. D. T., Ngo, V.-D., Seekins, J., Blankenberg, F. G., Ng, A. Y., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00536-x · Cited by 270
“Our work builds upon several studies investigating the validity of saliency maps for localization 59–61 and upon some early work on the trustworthiness of saliency methods to explain DNNs in medical imaging 47 .”
Metadata
- Authors
- Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., Hoebel, K., Gupta, S., Patel, J., Gidwani, M., Adebayo, J., Li, M. D., & Kalpathy-Cramer, J.
- Affiliations
- Harvard University (US)
- Georgia Institute of Technology (US)
- Massachusetts Institute of Technology (US)
- ORCID
- Venue
- Radiology: Artificial Intelligence
- Published
- 2021-11-01
- Volume
- 3
- Issue
- 6
- Pages
- e200267-e200267
- ISSN
- 2638-6100 (ISSN-L)
- PMC
- PMC8637231
- OpenAlex
- W3202023533
- Cited by
- 342
- References
- 55
- Citations by year
- 2026: 76 · 2025: 66 · 2024: 59 · 2023: 61 · 2022: 60 · 2021: 19
- FWCI
- 17.89
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Radiology Artificial Intelligence (publishedVersion)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Biomedical Imaging and Bioengineering (5T32EB1680)
- National Institutes of Health
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Keywords
- saliency maps, anomaly localization, trustworthiness, pneumonia detection, repeatability, reproducibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.