From Blackbox to Explainable AI in Healthcare: Existing Tools and Case Studies
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Introduction. Artificial intelligence (AI) models have been employed to automate decision-making, from commerce to more critical fields directly affecting human lives, including healthcare. Although the vast majority of these proposed AI systems are considered black box models that lack explainability, there is an increasing trend of attempting to create medical explainable Artificial Intelligence (XAI) systems using approaches such as attention mechanisms and surrogate models. An AI system is said to be explainable if humans can tell how the system reached its decision. Various XAI-driven healthcare approaches and their performances in the current study are discussed. The toolkits used in local and global post hoc explainability and the multiple techniques for explainability pertaining the Rational, Data, and Performance explainability are discussed in the current study. Methods. The explainability of the artificial intelligence model in the healthcare domain is implemented through the Local Interpretable Model-Agnostic Explanations and Shapley Additive Explanations for better comprehensibility of the internal working mechanism of the original AI models and the correlation among the feature set that influences decision of the model. Results. The current state-of-the-art XAI-based and future technologies through XAI are reported on research findings in various implementation aspects, including research challenges and limitations of existing models. The role of XAI in the healthcare domain ranging from the earlier prediction of future illness to the disease’s smart diagnosis is discussed. The metrics considered in evaluating the model’s explainability are presented, along with various explainability tools. Three case studies about the role of XAI in the healthcare domain with their performances are incorporated for better comprehensibility. Conclusion. The future perspective of XAI in healthcare will assist in obtaining research insight in the healthcare domain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Kumar, Y., Koul, A., Singla, R., & Ijaz, M. F. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03612-z · Cited by 1,040
RetractedArrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · cited 1×
“Table 3 represents the various characteristics like the acceptable input type, a local and global explainable toolkit feature, and the model-specific and agnostic feature associated with the toolkits [38].”
BackgroundExplainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · cited 1×
“Explaining Articial Intelligence (XAI) [1, 2] is a eld of machine intelligence engineering that ensures that complicated technics are made accessible and customizable for eective decision-making in the eld of science and technology.”
BackgroundDefinitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · cited 1×
“In these contexts, explainable approaches must contend that individual features lack contextual significance, rendering the task more difficult than on datasets with more meaningful features [74].”
BackgroundCausability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · cited 1×
“*e current models in healthcare and medicine have limitations in grappling with real-world complexities and uncertainties [7].”
BackgroundPotential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395 · cited 1×
“Also, some of the predictive models are biased and tend to produce misleading outcomes [6]. challenges of the adaptability, transparency, and biased behavior of the existing models have provoked transparency in the design models in conventional Artificial Intelligence. goal is to assist the…”
BackgroundLee, D., & Yoon, S. N. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18010271 · Cited by 1,017 · cited 1×
“Severalof the most efficient machine learning systems’ properties seem suspicious in making the decisions [57].”
Background
Cited by
Shaping the future of healthcare: ethical clinical challenges and pathways to trustworthy AI
Goktas, P., & Grzybowski, A. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14051605 · Cited by 336
“In this process, internal organizational efforts bolster reliability and fairness while managing potentially harmful external influences such as regulatory gaps or privacy risks.”
Large Language Models in Healthcare and Medical Applications: A Review
Maity, S., & Saikia, M. J. (2025) · Bioengineering · doi:10.3390/bioengineering12060631 · Cited by 249
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152
Moazemi, S., Vahdati, S., Li, J., Kalkhoff, S., Castano, L. J. V., Dewitz, B., Bibo, R., Sabouniaghdam, P., Tootooni, M. S., Bundschuh, R. A., et al. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1109411 · Cited by 97
Shifa, N., Saleh, M., Akbari, Y., & Al Maadeed, S. (2025) · Clinical Imaging · doi:10.1016/j.clinimag.2025.110492 · Cited by 45
Metadata
- Authors
- Srinivasu, P. N., Sandhya, N., Jhaveri, R. H., & Raut, R.
- Venue
- Mobile Information Systems
- Publisher
- Wiley
- Published
- 2022-06-13
- Volume
- 2022
- Pages
- 1-20
- ISSN
- 1574-017X (ISSN-L)
- 1875-905X
- OpenAlex
- W4282821541
- Cited by
- 171
- References
- 25
- FWCI
- 11.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: From Blackbox to Explainable AI in Healthcare: Existing Tools and Case Studies. https://app.cochat.ai/papers/10.1155/2022/8167821
- The paper
- Srinivasu, P. N., Sandhya, N., Jhaveri, R. H., & Raut, R. (2022). From Blackbox to Explainable AI in Healthcare: Existing Tools and Case Studies. Mobile Information Systems. https://doi.org/10.1155/2022/8167821
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.