Explainable Artificial Intelligence for Bias Detection in COVID CT-Scan Classifiers
Open accessThe DOI matches its Crossref record; 22 of the 24 references it cites were found in a registry or an index.
Abstract
Problem: An application of Explainable Artificial Intelligence Methods for COVID CT-Scan classifiers is presented. Motivation: It is possible that classifiers are using spurious artifacts in dataset images to achieve high performances, and such explainable techniques can help identify this issue. Aim: For this purpose, several approaches were used in tandem, in order to create a complete overview of the classificatios. Methodology: The techniques used included GradCAM, LIME, RISE, Squaregrid, and direct Gradient approaches (Vanilla, Smooth, Integrated). Main results: Among the deep neural networks architectures evaluated for this image classification task, VGG16 was shown to be most affected by biases towards spurious artifacts, while DenseNet was notably more robust against them. Further impacts: Results further show that small differences in validation accuracies can cause drastic changes in explanation heatmaps for DenseNet architectures, indicating that small changes in validation accuracy may have large impacts on the biases learned by the networks. Notably, it is important to notice that the strong performance metrics achieved by all these networks (Accuracy, F1 score, AUC all in the 80 to 90% range) could give users the erroneous impression that there is no bias. However, the analysis of the explanation heatmaps highlights the bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, the black box nature of AI-model architecture restricts the generalizability and interoperability of the model which may cause incorrect diagnosis in the prediction [55,104].”
From Artificial intelligence bias in medical system designs: a systematic review (2023) · cites this paper as [104]
In the citing paper
“Applications to specific medical domains include mechanical ventilation [45], COVID CT-Scans [104], ECG [97] and COVID X-rays [144], whereas generic refers to assessing bias at different stages of AImodel in medical domain [19,52,107].”
From Artificial intelligence bias in medical system designs: a systematic review (2023) · cites this paper as [104]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“[8] study the specific case of a VGG16 classifier [9] trained on a CXR datasets.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 4×
“However, better performance was reached by using transfer-learning from Imagenet, which is a common procedure for this type of classification.”
BackgroundInfluentialDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
SLIC Superpixels Compared to State-of-the-Art Superpixel Methods
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.120 · Cited by 9,201 · cited 2×
“The segmentation algorithm chosen for this study was the Simple Interactive Linear Clustering (SLIC), developed by Achanta et al [23].”
MethodWang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Wang, L., Lin, Z. Q., & Wong, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-76550-z · Cited by 3,113
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467
Research works printed but not linked: 5
- Zhao, J., Zhang, Y., He, X., and Xie, P. (2020). COVID-CT-Dataset: A CT scan dataset about COVID-19. arXiv.
- Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv.
- Tan, M., and Le, Q.V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv.
- Petsiuk, V., Das, A., and Saenko, K. (2018). Rise: Randomized input sampling for explanation of black-box models. arXiv.
- Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. arXiv.
Cited by
S Band, S., Yarahmadi, A., Hsu, C.-C., Biyari, M., Sookhak, M., Ameri, R., Dehzangi, I., Chronopoulos, A. T., & Liang, H.-W. (2023) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2023.101286 · Cited by 297
Recent Applications of Explainable AI (XAI): A Systematic Literature Review
Saarela, M., & Podgorelec, V. (2024) · Applied Sciences · doi:10.3390/app14198884 · Cited by 183
Borys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110787 · Cited by 169
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168
Explainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review
Giuste, F., Shi, W., Zhu, Y., Naren, T., Isgut, M., Sha, Y., Tong, L., Gupte, M., & Wang, M. D. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2022.3185953 · Cited by 128
“[21] also demonstrated the utility of adding random colored artifacts to CT images to identify model architecture which are most robust to such perturbation.”
BackgroundHassan, S. U., Abdulkadir, S. J., Zahid, M. S. M., & Al-Selwi, S. M. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109569 · Cited by 108
Explainable artificial intelligence (XAI) in radiology and nuclear medicine: a literature review
de Vries, B. M., Zwezerijnen, G. J. C., Burchell, G. L., van Velden, F. H. P., Menke-van der Houven van Oordt, C. W., & Boellaard, R. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1180773 · Cited by 106
“For example measuring the signal-to-noise-ratio for data quality harmonization, DL-based artifact detection model (62, 63) or simple visual inspection can be proposed to provide information of the quality of the data before utilization in the diagnostic DL systems.”
BackgroundMethodChampendal, M., Müller, H., Prior, J. O., & dos Reis, C. S. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.111159 · Cited by 116
Jung, J., Lee, H., Jung, H., & Kim, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16110 · Cited by 87
Hamida, S. U., Chowdhury, M. J. M., Chakraborty, N. R., Biswas, K., & Sami, S. K. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8110149 · Cited by 95
Grad-CAM (Gradient-weighted Class Activation Mapping): A systematic literature review
Fayyaz, A. M., Abdulkadir, S. J., Talpur, N., Al-Selwi, S. M., Hassan, S. U., & Sumiea, E. H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111200 · Cited by 94
Artificial intelligence bias in medical system designs: a systematic review
Kumar, A., Aelgani, V., Vohra, R., Gupta, S. K., Bhagawati, M., Paul, S., Saba, L., Suri, N., Khanna, N. N., Laird, J. R., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16029-x · Cited by 65
Evaluation Metrics for XAI: A Review, Taxonomy, and Practical Applications
Kadir, M. A., Mosavi, A. H., & Sonntag, D. (2023) · IEEE International Conference on Intelligent Engineering Systems (INES) · doi:10.1109/ines59282.2023.10297629 · Cited by 81
Khan, M. A., Azhar, M., Ibrar, K., Alqahtani, A., Alsubai, S., Binbusayyis, A., Kim, Y. J., & Chang, B. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/4254631 · Cited by 49
“[37] identified the issue of usage of false or doubtful artifacts in dataset images by classifiers; they suggested a new AI-based technique for COVID-19 classification on the COVID-CT dataset [38].”
MethodExplainable quantum clustering method to model medical data
Deshmukh, S., Behera, B. K., Mulay, P., Ahmed, E. A., Al-Kuwari, S. M., Tiwari, P., & Farouk, A. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110413 · Cited by 52
Stanley, E. A. M., Wilms, M., Mouchès, P., & Forkert, N. D. (2022) · Journal of medical imaging · doi:10.1117/1.jmi.9.6.061102 · Cited by 56
“14 in the context of COVID computed tomography scan classifiers.”
BackgroundA Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Wang, Z., Huang, C., & Yao, X. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3702004 · Cited by 51
Explainable AI for Sensor Signal Interpretation to Revolutionize Human Health Monitoring: A Review
Alharthi, A. S., Alqurashi, A., Alharbi, T. E. A., Alammar, M. M., Aldosari, N., Bouchekara, H. R. E. H., Sha’aban, Y. A., Shahriar, M. S., & Ayidh, A. A. (2025) · IEEE Access · doi:10.1109/access.2025.3585764 · Cited by 44
“…[78], [79], [80], [81], [82], [83], [84], [85], [86], [87], [88], [89], [90], [91], [92], [93], [94], [95], [96], [97], [98], [99], [100], [101], [102], [103], [104], [105], [106], [107], [108], [109], [110], [111], [112], [113], [114], [115], [116], [117], [118], [119], [120], [121], [122],…”
InfluentialToward Human-centered XAI in Practice: A survey
Kong, X., Liu, S., & Zhu, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-022-1407-3 · Cited by 50
Walker, M. C., Willner, I., Miguel, O. X., Murphy, M. S. Q., El-Chaâr, D., Moretti, F., Dingwall Harvey, A. L. J., Rennicks White, R., Muldoon, K. A., Carrington, A. M., et al. (2022) · PLOS ONE · doi:10.1371/journal.pone.0269323 · Cited by 31
Explainable artificial intelligence in ophthalmology
Tan, T. F., Dai, P., Zhang, X., Jin, L., Poh, S. S. J., Hong, D. C., Lim, J., Lim, G., Teo, Z. L., Liu, N., et al. (2023) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000983 · Cited by 29
“[42] found that despite strong performance [area under the curve (AUC) > 0.”
BackgroundValidating Automatic Concept-Based Explanations for AI-Based Digital Histopathology
Sauter, D., Lodde, G., Nensa, F., Schadendorf, D., Livingstone, E., & Kukuk, M. (2022) · Sensors · doi:10.3390/s22145346 · Cited by 26
“Some evaluations can be found for MMD-critic [36], Grad-CAM [36,37], Kernel SHapley Additive exPlanation (SHAP) [38], testing with concept activation vectors (TCAV) [37], LRP [7], and further saliency methods [39].”
MethodArtificial Intelligence and Infectious Disease Imaging
Chu, W. T., Reza, S. M. S., Anibal, J., Landa, A., Crozier, I., Bağcı, U., Wood, B. J., & Solomon, J. M. (2023) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiad158 · Cited by 27
Aquino, G., Costa, M. G. F., & Filho, C. F. F. C. (2022) · Sensors · doi:10.3390/s22155644 · Cited by 23
“[28] used XAI methods for a COVID-19 classifier on CT images.”
MethodAI image generation technology in ophthalmology: Use, misuse and future applications
Phipps, B., Hadoux, X., Sheng, B., Campbell, J. P., Liu, T. Y. A., Keane, P. A., Cheung, C. Y.-L., Chung, T. Y., Wong, T. Y., & van Wijngaarden, P. (2025) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2025.101353 · Cited by 22
Chauhan, T., & Sonawane, S. S. (2022) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v10i4.5538 · Cited by 14
An Overview and Comparison of XAI Methods for Object Detection in Computer Tomography
Sahatova, K., & Balabaeva, K. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.11.005 · Cited by 14
Explainable optimization of deep learning model for COVID-19 detection using chest images
Riyadi, S., Pramudya, E. N., Damarjati, C., Molina, J. M., & García, J. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101559 · Cited by 8
Quantitative Assessment of XAI Methods for COVID-19 Detection: A Comparative Approach
Rajpoot, R., Jain, S., Semwal, V. B., & Singh, D. (2025) · SN Computer Science · doi:10.1007/s42979-025-03663-5 · Cited by 5
“Sousa et al. [23] investigate bias in CT-scan classifiers using explainable AI methods.”
Explicable AI for surveillance and interpretation of Coronavirus using X-ray imaging
Chauhan, T., & Sonawane, S. S. (2023) · International Conference on Emerging Smart Computing and Informatics (ESCI) · doi:10.1109/esci56872.2023.10099633 · Cited by 5
Open your black box classifier
Lisboa, P. J. G. (2023) · Healthcare Technology Letters · doi:10.1049/htl2.12050 · Cited by 4
“Explanation methods were central to identifying the bias in the models due to spurious effects that happened to correlate with class membership in the data set, despite good practice by splitting the data into three groups for model inference, optimisation and performance estimation [5].”
MethodThe role of explainability throughout the MLOps lifecycle: review and research agenda
Tekkesinoglu, S., Wagner, M., & Runeson, P. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1737008 · Cited by 3
Saleh, B. J., Omar, Z., As’ari, M. A., Bhateja, V., & Izhar, L. I. (2025) · Applied Computational Intelligence and Soft Computing · doi:10.1155/acis/8829294 · Cited by 2
Explainable Artificial Intelligence in Healthcare Applications: A Systematic Review
Costa, B., & Georgieva, P. (2023) · International Scientific Conference on Computer Science (COMSCI) · doi:10.1109/comsci59259.2023.10315829 · Cited by 3
“Cancers 2021 8 [14] Sensors 2021 9 [15] The Lancet Digital Health 168 2022 [16] IEEE Journal of Translational Engineering in Health and Medicine 22 [17] Procedia Computer Science 0 [18] Computer Vision in Co-clinical Medical Imaging for Precision Medicine 4 [19] IEEE Access 2 [20] IEEE Reviews in…”
BackgroundTumor Detection Using Deep Learning and Explainable Artificial Intelligence
Mehra, S. D., Aswani, S., & Shetty, S. D. (2024) · International Conference on Computational Intelligence and Network Systems (CINS) · doi:10.1109/cins63881.2024.10864458 · Cited by 2
Díaz, E., Lakshminarayanan, V., & Calderón-Benavides, L. (2025) · Machine Learning Health · doi:10.1088/3049-477x/ae209d · Cited by 1
Zhang, Z., & Lu, M. (2026) · Engineered Science · doi:10.30919/es2245 · Cited by 2
Delgado, C. F., Ferretti, M. T., & Carnevale, A. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13681-8.00012-6 · Cited by 2
Kabachenko, F., Samarina, A., & Mikhaylik, Y. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-19032-2_37
Artificial intelligence and radiography methods for diagnostic and distinguish of COVID-19: Review
Sameer, H. A., Mutlag, A. H., & Gharghan, S. K. (2023) · AIP conference proceedings · doi:10.1063/5.0154498
MEHAK: Explainable AI Classifier for COVID-19 Early Detection Based on LIME
Rana, M., Hussain, M., & Saeed, U. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-77620-5_29
Analysis of AI-Bias in Modern Healthcare Systems
Kumar, A., & Singh, D. (2025) · Transactions on computer systems and networks · doi:10.1007/978-981-96-6703-1_14 · Cited by 1
Ali, F. A. R., Ahmed, S. A., Hasan, M. S., Sahi, S., Samir, M., Mahmod, S. A., & Ahmed, J. M. (2025) · International Conference on Cyber Resilience (ICCR) · doi:10.1109/iccr67387.2025.11292208
Explainable Adaboost Model for the Early Prediction of Covid-19 Mortality
Aslam, N., Alharthi, H., Alwazzeh, M. J., Khan, I. U., Alshakhs, F., AlRajeh, L. S., Alduailej, H. H., Alansari, A., Alturaif, R. K., & Bashamakh, A. (2025) · International Conference on Computational Intelligence, Security, and Artificial Intelligence (IntelliSecAI) · doi:10.1109/intellisecai66368.2025.11472969
“There are very few studies that used EAI with COVID research work; however, in the [32] study, they focused on the importance of EAI, as their results show that even though these classifiers can perform very well in complex medical data, a chance of bias can still be present [32].”
XAI-Powered Micro-Expression Detection Against Deepfakes
Ramzia, S., Pamba, R. V., Subhash, N., Noorain, S., & Safir, S. (2025) · International Conference on Algorithms, Computing and Systems (ICACS) · doi:10.1145/3789418.3789425
Explainable AI-based techniques for disease classification: a systematic review and future work
Memon, S. A., Ebrahimi, A., Shaikh, M., & Wiil, U. K. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03750-4
Metadata
- Authors
- Palatnik de Sousa, I., Vellasco, M. M. B. R., & Costa da Silva, E.
- Affiliations
- Department of Electrical Engineering, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro 22453-900, Brazil
- Pontifícia Universidade Católica do Rio de Janeiro (BR)
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2021-08-23
- Volume
- 21
- Issue
- 16
- Pages
- 5657
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC8402377
- OpenAlex
- W3194757962
- Cited by
- 43
- References
- 24
- Citations by year
- 2025: 11 · 2024: 9 · 2023: 14 · 2022: 9
- FWCI
- 2.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (001)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- explainable AI, bias detection, Grad-CAM, lime, RISE, gradient-based explanation, VGG16, DenseNet, model robustness
- MeSH terms
- COVID-19, SARS-CoV-2, Artificial Intelligence, Humans, Tomography, X-Ray Computed, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.