A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Deep learning has demonstrated remarkable accuracy analyzing images for cancer detection tasks in recent years. The accuracy that has been achieved rivals radiologists and is suitable for implementation as a clinical tool. However, a significant problem is that these models are black-box algorithms therefore they are intrinsically unexplainable. This creates a barrier for clinical implementation due to lack of trust and transparency that is a characteristic of black box algorithms. Additionally, recent regulations prevent the implementation of unexplainable models in clinical settings which further demonstrates a need for explainability. To mitigate these concerns, there have been recent studies that attempt to overcome these issues by modifying deep learning architectures or providing after-the-fact explanations. A review of the deep learning explanation literature focused on cancer detection using MR images is presented here. The gap between what clinicians deem explainable and what current methods provide is discussed and future suggestions to close this gap are provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Explanation is also needed for trust and transparency [22–28].”
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · cited 1×
“Beyond ethical and legal issues, clinicians and patients need to be able to trust the classifications provided by these systems [21].”
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · cited 1×
“Although the research volume is expanding rapidly around explainability [ 35– 62 ] a review focusing on image intepretability”
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097
Large scale deep learning for computer aided detection of mammographic lesions
Kooi, T., Litjens, G., van Ginneken, B., Gubern-Mérida, A., Sánchez, C. I., Mann, R., den Heeten, A., & Karssemeijer, N. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.07.007 · Cited by 965 · cited 1×
“A part of this growth is the application of deep learning algorithms to big data in the healthcare sector [1–11] as depicted in Figure 1.”
Explainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718 · cited 1×
“In the review by [76], the authors provide a review of explainable deep learning models for medical imaging analysis.”
Nam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539
Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017) · arXiv · doi:10.48550/arxiv.1702.04595 · Cited by 487
Cruz-Roa, A. A., Arevalo Ovalle, J. E., Madabhushi, A., & González Osorio, F. A. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_50 · Cited by 403
Langlotz, C. P., Allen, B., Erickson, B. J., Kalpathy-Cramer, J., Bigelow, K., Cook, T. S., Flanders, A. E., Lungren, M. P., Mendelson, D. S., Rudie, J. D., et al. (2019) · Radiology · doi:10.1148/radiol.2019190613 · Cited by 368
Becker, A. S., Marcon, M., Ghafoor, S., Wurnig, M. C., Frauenfelder, T., & Boss, A. (2017) · Investigative Radiology · doi:10.1097/rli.0000000000000358 · Cited by 352
Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging
Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., Hoebel, K., Gupta, S., Patel, J., Gidwani, M., et al. (2021) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2021200267 · Cited by 342
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
Kwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294
Towards Reverse-Engineering Black-Box Neural Networks
Oh, S. J., Schiele, B., & Fritz, M. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_7 · Cited by 291
Shen, S., Han, S. X., Aberle, D. R., Bui, A. A., & Hsu, W. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.01.048 · Cited by 268
How far have we come? Artificial intelligence for chest radiograph interpretation
Kallianos, K., Mongan, J., Antani, S., Henry, T., Taylor, A., Abuya, J., & Kohli, M. (2019) · Clinical Radiology · doi:10.1016/j.crad.2018.12.015 · Cited by 173
Local interpretable model-agnostic explanations for classification of lymph node metastases
Palatnik de Sousa, I., Maria Bernardes Rebuzzi Vellasco, M., & Costa da Silva, E. (2019) · Sensors · doi:10.3390/s19132969 · Cited by 155
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153
Understanding Intra-Class Knowledge Inside CNN
Wei, D., Zhou, B., Torrabla, A., & Freeman, W. (2015) · arXiv · doi:10.48550/arxiv.1507.02379 · Cited by 98
Global Model Interpretation Via Recursive Partitioning
Yang, C., Rangarajan, A., & Ranka, S. (2018) · 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc/smartcity/dss.2018.00256 · Cited by 93
Hao, J., Kim, Y., Mallavarapu, T., Oh, J. H., & Kang, M. (2019) · BMC Medical Genomics · doi:10.1186/s12920-019-0624-2 · Cited by 82
The Stakes of Uncertainty: Developing and Integrating Machine Learning in Clinical Care
ELISH, M. C. (2018) · Ethnographic Praxis in Industry Conference Proceedings · doi:10.1111/1559-8918.2018.01213 · Cited by 59
Kumar, D., Sankar, V., Clausi, D., Taylor, G. W., & Wong, A. (2019) · IEEE Access · doi:10.1109/access.2019.2945524 · Cited by 26
ClinicalVis: Supporting Clinical Task-Focused Design Evaluation
Ghassemi, M., Pushkarna, M., Wexler, J., Johnson, J., & Varghese, P. (2018) · arXiv · doi:10.48550/arxiv.1810.05798 · Cited by 26
Cited by
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783
Deep Learning for Medical Image-Based Cancer Diagnosis
Jiang, X., Hu, Z., Wang, S., & Zhang, Y. (2023) · Cancers · doi:10.3390/cancers15143608 · Cited by 345
Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 311
Chen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300
“Likewise, transparent ML in cancer detection is also reviewed in [52] and structured following the same aspects of generic transparent ML techniques, such as Local vs. Global and Ad-Hoc vs. Post-Hoc. distinctions The guidelines and systematic review of the state of the field presented here aim at…”
MethodNazir, S., Dickson, D. M., & Akram, M. U. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106668 · Cited by 283
“A review of XAI approaches for cancer detection with MRI for DNNs concluded that explainability can be improved with counter examples, and intrinsically explainable models [258].”
BackgroundInfluential
Metadata
- Authors
- Gulum, M. A., Trombley, C. M., & Kantardzic, M.
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-05-17
- Volume
- 11
- Issue
- 10
- Pages
- 4573
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3161558656
- Cited by
- 145
- References
- 34
- FWCI
- 8.82
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.