The DOI matches its Crossref record; 72 of the 78 references it cites were found in a registry or an index; none of the 20 sources its claims cite is checked for support yet.
Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare
Open accessPublished version of a preprint
Abstract
Artificial Intelligence (AI) describes computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. Examples of AI techniques are machine learning, neural networks, and deep learning. AI can be applied in many different areas, such as econometrics, biometry, e-commerce, and the automotive industry. In recent years, AI has found its way into healthcare as well, helping doctors make better decisions (“clinical decision support”), localizing tumors in magnetic resonance images, reading and analyzing reports written by radiologists and pathologists, and much more. However, AI has one big risk: it can be perceived as a “black box”, limiting trust in its reliability, which is a very big issue in an area in which a decision can mean life or death. As a result, the term Explainable Artificial Intelligence (XAI) has been gaining momentum. XAI tries to ensure that AI algorithms (and the resulting decisions) can be understood by humans. In this narrative review, we will have a look at some central concepts in XAI, describe several challenges around XAI in healthcare, and discuss whether it can really help healthcare to advance, for example, by increasing understanding and trust. Finally, alternatives to increase trust in AI are discussed, as well as future research possibilities in the area of XAI.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This narrative review surveys the current status of Explainable Artificial Intelligence (XAI) in healthcare, covering the "black box" trust problem, legal/regulatory drivers, transparent versus post-hoc explainability, privacy and security trade-offs, and the tension between explainability and accuracy. The author argues that in healthcare accuracy may outweigh explainability, and that trust might better be built through robustness, fairness, reproducibility, and human–AI collaboration than through explanation alone.
Key findings
- XAI is growing rapidly in the literature: the first mention of "explainable artificial intelligence" in a PubMed title, abstract or keyword was in 2018, and it has since been mentioned 511 times, of which more than half (310) appeared in 2022 or the first months of 2023; Embase shows a similar trend (queries performed 2023-03-06).
- Explainability and accuracy can trade off: making a model fully transparent (a "glass box") may require simplification that reduces the accuracy of a highly predictive model (a "crystal ball"). Van der Veer et al. concluded citizens may value explainability in healthcare less than in non-healthcare domains and less than professionals assume, especially weighed against accuracy.
- XAI is double-edged for privacy and security: it can help detect errors and malicious behavior, but explanations may reveal sensitive information or enable manipulation via reverse engineering. Saifullah et al. recommend differential private federated learning plus perturbation-based XAI methods when both privacy and explainability are required.
- Explanations do not automatically produce trust: Le Merrer and Trédan show remote explainability cannot prevent a service provider from lying about the true reasons for its decisions, while Druce et al. found a statistically significant increase in user trust and acceptance using a three-fold explanation.
Methods
- Design: narrative review of XAI concepts, issues, and challenges in healthcare, with no systematic review protocol described.
- Data: no new data were created or analyzed; the paper draws on cited literature and a bibliometric query of PubMed and Embase for the term "explainable artificial intelligence" in titles, abstracts and keywords, performed on 2023-03-06.
- Analysis: thematic discussion organized into sections covering transparency, legal/regulatory compliance (GDPR, the EU AI Act, HIPAA, PIPL), transparent vs. post-hoc explainability, privacy/security, human–AI collaboration, trust, scientific XAI (sXAI), the explainability–accuracy balance, measurement of explainability, and future complexity.
Limitations
- The paper states no formal limitations section; it is explicitly a narrative review, and the text notes that explainability is difficult to measure because "the quality of an explanation is somewhat subjective."
- Evident from the text: the review is a non-systematic narrative synthesis by a single author, so selection of cited work is not governed by a documented search protocol.
- Evident from the text: the version provided is a preprint (preprints.org, posted 7 March 2023) marked "NOT PEER-REVIEWED," and the author is an employee of Philips Research (declared under Acknowledgements).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 20 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results for the Embase database show a similar trend.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 1. Introduction
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIn the paper
“Its first mention in a PubMed title, abstract or keywords was in 2018, in a paper about machine learning in neuroscience [11].”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 1. Introduction
What the sources say
- Not checked yetcites [11] Vu, 2018, A Shared Vision for Machine Learning in Neuroscience, J. Neurosci. source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XAI is being mentioned more and more in scientific publications, as can be seen in figure 1.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 1. Introduction
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Another advantage of XAI is that it can help organizations comply with laws and regulations that require transparency and explainability in AI systems.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 3. Legal and regulatory compliance
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This shows the growing importance of XAI in (bio)medicine and healthcare.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 1. Introduction
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIn the paper
“Within the General Data Protection Regulation (GDPR) of the European Union, transparency is a fundamental principle for data processing [15].”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 3. Legal and regulatory compliance
What the sources say
- Not checked yetcites [15] Jung, 2023, Essential properties and explanation effectiveness of explainable artificial intelligence in healthcare: A systematic review, Heliyon source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“With the advent of deep learning (using multiple layers) and new types of algorithms such as Deep Belief Networks (DBNs) [46] and Generative Adversarial Networks (GANs) [47], made possible by the increasing computer power, artificial intelligence algorithms are gaining complexity.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 11. Increasing complexity in the future
What the sources say
- Not checked yetcites [46] Bell, A., Nov, O., and Stoyanovich, J. (2022). Think about the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance. arXiv. source
- Not checked yetcites [47] HHS Office for Civil Rights (2002). Standards for privacy of individually identifiable health information—Final rule. Fed. Regist., 67, 53181–53273.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In healthcare, this balance might shift more to the 'crystal ball' as accuracy might be considered more important than transparency and explainability.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 9. 'Glass box' vs. 'crystal ball': balance between explainability and accuracy/performance
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Explainability is much more difficult to be measured because the quality of an explanation is somewhat subjective.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 10. How to measure explainability?
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In some cases, the need for explainability can come at the cost of reduced performance of the model.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 9. 'Glass box' vs. 'crystal ball': balance between explainability and accuracy/performance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 11
- Probably not a substantial claim. 16
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since then, it has been mentioned a total of 511 times, of which more than half (310) in papers from 2022 or the first months of 2023.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 1. Introduction
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“This trust can exist on two levels: trust in the model versus trust in the prediction [14].”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 2. From 'black box' to '(translucent) glass box'
What the sources say
- Not checked yetcites [14] Tjoa, 2020, A survey on explainable artificial intelligence (xai): Toward medical xai, IEEE Trans. Neural Netw. Learn. Syst. source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A low opacity of the translucent glass box (or high transparency of the model) can lead to a better understanding of the model, which in turn could increase trust.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 2. From 'black box' to '(translucent) glass box'
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Rests on an assumption: AI algorithms can be perceived as black boxes, and this limits trust in their reliability, which is a major issue in healthcare where decisions can be life-or-death.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, explainability might be considered of less importance in healthcare, compared to accuracy.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
In the paper
“The post-hoc models need to explained by resorting to diverse means to enhance their interpretability, such as text explanations, visual explanations, local explanations, explanations by example, explanations by simplification and feature relevance explanations techniques.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 4. Explainability: transparent or post-hoc
What the sources say
- Not checked yetcites [22] Vale, 2022, Explainable artificial intelligence (XAI) post-hoc explainability methods: Risks and limitations in non-discrimination law, AI Ethics source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Within healthcare, trust is a fundamental issue because important decisions might be taken based on the output of the AI algorithm.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 7. Do explanations always raise trust?
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: AI algorithms can be perceived as black boxes, and this limits trust in their reliability, which is a major issue in healthcare where decisions can be life-or-death.
In the paper
“The EU is currently working on the Artificial Intelligence Act [17] which makes a distinction between non-high-risk and high-risk AI systems.”
- Also stated in: § 3. Legal and regulatory compliance · plain
“On non-high-risk systems only limited transparency obligations are imposed, while for high-risk systems many restrictions are imposed on quality, documentation, traceability, transparency, human oversight, accuracy and robustness.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 3. Legal and regulatory compliance
What the sources say
- Not checked yetcites [17] 2019, Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view, IEEE Access source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Also stated in: § 3. Legal and regulatory compliance · plain
In the paper
“Linear models and rule-based models are very transparent, but usually have a lower performance than deep learning algorithms (figure 5 of [39]).”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 9. 'Glass box' vs. 'crystal ball': balance between explainability and accuracy/performance
What the sources say
- Not checked yetcites [39] Solayman, 2023, Automatic COVID-19 prediction using explainable machine learning techniques, Int. J. Cogn. Comput. Eng.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Combining explainability of algorithms with privacy-preserving methods such as federated learning [27] might help.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 5. Privacy and security: a mixed bag
What the sources say
- Not checked yetcites [27] 2021, Dissecting scientific explanation in AI (sXAI): A case for medicine and healthcare, Artif. Intell. source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A bias could be introduced by having a training dataset with low diversity, or by subjective responses of clinicians to a questionnaire.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“End users just want to be sure that the predictions done by the algorithm are accurate, which can be proven by showing them correct predictions from the past.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Robustness of an algorithm in healthcare can be proven by presenting good results based on long-term use in different patient populations.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This increasing complexity might make it almost impossible for end users in healthcare (clinicians as well as patients) to understand and trust the algorithms.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: AI algorithms can be perceived as black boxes, and this limits trust in their reliability, which is a major issue in healthcare where decisions can be life-or-death.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, making AI explainable is a difficult task, and it will be even more difficult when the complexity of AI algorithms will continue to increase.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With algorithms getting more complex, it might also be more difficult to make them explainable.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 11. Increasing complexity in the future
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
In the paper
“For example, trust in algorithms could also be raised by ensuring robustness and by encouraging fairness [50].”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
What the sources say
- Not checked yetcites [50] Charmet, 2022, Explainable artificial intelligence for cybersecurity: A literature survey, Ann. Telecommun. source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Van der Veer et al. [40] concluded that citizens may indeed value explainability of AI systems in healthcare less than in non-healthcare domains and less than often assumed by professionals, especially when weighed against system accuracy”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 9. 'Glass box' vs. 'crystal ball': balance between explainability and accuracy/performance
What the sources say
- Not checked yetcites [40] Gao, 2022, A review of the application of deep learning in the detection of Alzheimer's disease, Int. J. Cogn. Comput. Eng.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Fairness of an AI algorithm is concurrent with bias minimization.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Privacy laws such as GDPR, HIPAA and PIPL all include clauses that state that the handling of healthcare data should be transparent, which means that AI algorithms that work with these data should be transparent and explainable as well.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
What the sources say
- Not checked yetcites [15] Jung, 2023, Essential properties and explanation effectiveness of explainable artificial intelligence in healthcare: A systematic review, Heliyon source
- Not checked yetcites [19] Lim, 2022, Explainable AI, but explainable to whom? An exploratory case study of xAI in healthcare, Handbook of Artificial Intelligence in Healthcare: Practicalities and Prospects
- Not checked yetcites [21] Phillips, P.J., Hahn, C.A., Fontana, P.C., Broniatowski, D.A., and Przybocki, M.A. (2020). Four Principles of Explainable Artificial Intelligence, National Institute of Standards and Technology. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Arrieta et al. [22] classified studies on XAI into two approaches: some works focus on creating transparent models, while most works wrap black-box models with a layer of explainability, the so-called post-hoc models.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 4. Explainability: transparent or post-hoc
What the sources say
- Not checked yetcites [22] Vale, 2022, Explainable artificial intelligence (XAI) post-hoc explainability methods: Risks and limitations in non-discrimination law, AI Ethics source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Especially in healthcare, accuracy (and to a lesser extent performance) is crucial as it could be a matter of life and death.”
§ Explainable Artificial Intelligence (XAI) in Healthcare > 12. Discussion
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper relies on this without support.
AI algorithms can be perceived as black boxes, and this limits trust in their reliability, which is a major issue in healthcare where decisions can be life-or-death.
“However, AI has one big risk: it can be perceived as a 'black box', limiting trust in its reliability, which is a very big issue in an area in which a decision can mean life or death.”
“They can be perceived as a 'black box': something goes in, and something comes out, with no understanding of what happens inside.”
“This distrust in AI algorithms, their accuracy and reliability, is a very big issue in an area in which a decision could mean life or death of the patient.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Explainable AI can enhance trust by making AI recommendations understandable [304].”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [304]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Reference 31 gives conflicting conference and publication years
MinorThe reference identifies the proceedings as a 2020 conference but gives 2018 as the publication year.
“2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)”
“2018, 293-300.”
Citations
Cites
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 1×
“[20] classified studies on XAI into two approaches—some works focus on creating transparent models, while most works wrap black-box models with a layer of explainability, the so-called post-hoc models (Figure 2).”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
Zadeh, L. (1997) · Fuzzy Sets Syst. · doi:10.1016/s0165-0114(97)00077-8 · Cited by 2,771 · cited 1×
“[35] discuss the multi-granularity three-way decisions paradigm [36] and how this acts as a part of granular computing models, playing a significant role in explainable decision-making.”
BackgroundA Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203
XAI—Explainable artificial intelligence
Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.-Z. (2019) · Science Robotics · doi:10.1126/scirobotics.aay7120 · Cited by 2,053 · cited 1×
“As a result of this distrust, the term Explainable Artificial Intelligence (XAI) [10] has been gaining momentum as a possible solution.”
BackgroundSecure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,676 · cited 1×
“Combining the explainability of algorithms with privacy-preserving methods such as federated learning [52] might help.”
Method
Flagged first, then by citations
6 of the 78 references couldn’t be matched to a known paper.
Research works printed but not linked: 7
- Holzinger, A., Goebel, R., Fong, R., Moon, T., Müller, K.-R., and Samek, W. (2020, January 12–18). Explainable AI methods—A brief overview. Proceedings of the xxAI—Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, Vienna, Austria.
- Zadeh, L.A. (1979). Fuzzy Sets, Fuzzy Logic, and Fuzzy Systems: Selected Papers, World Scientific.
- Mencar, C., and Alonso, J.M. (2018, January 6–7). Paving the way to explainable artificial intelligence with fuzzy modeling: Tutorial. Proceedings of the Fuzzy Logic and Applications: 12th International Workshop (WILF 2018), Genoa, Italy.
- Fauvel, K., Masson, V., and Fromont, E. (2020). A performance-explainability framework to benchmark machine learning methods: Application to multivariate time series classifiers. arXiv.
- Huynh, T.D., Tsakalakis, N., Helal, A., Stalla-Bourdillon, S., and Moreau, L. (2022). Explainability-by-Design: A Methodology to Support Explanations in Decision-Making Systems. arXiv.
- Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv.
- Marcus, G. (2020). The next decade in AI: Four steps towards robust artificial intelligence. arXiv.
Cited by
Sorted by citations
Mienye, I. D., Obaido, G., Jere, N., Mienye, E., Aruleba, K., Emmanuel, I. D., & Ogbuokiri, B. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101587 · Cited by 212
A literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180
“As mentioned in [82], XAI enhances trust and usability in crop recommendation systems, as it provides farmers with the ability to interpret the model’s predictions based on specific agricultural needs.”
Mathew, D. E., Ebem, D. U., Ikegwu, A. C., Ukeoma, P. E., & Dibiaezue, N. F. (2025) · Neural Processing Letters · doi:10.1007/s11063-025-11732-2 · Cited by 168
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155
Santos, M. R., Guedes, L. A., & Sánchez-Gendriz, I. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6010016 · Cited by 104
“To address this challenge, researchers have introduced explainable artificial intelligence (XAI) [15].”
BackgroundAI-Reinforced Wearable Sensors and Intelligent Point-of-Care Tests
Yammouri, G., & Lahcen, A. A. (2024) · Journal of Personalized Medicine · doi:10.3390/jpm14111088 · Cited by 92
Rane, N., Choudhary, S., & Rane, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4637897 · Cited by 74
XAI-Based Clinical Decision Support Systems: A Systematic Review
Kim, S. Y., Kim, D. H., Kim, M., Ko, H.-J., & Jeong, O. (2024) · Applied Sciences · doi:10.3390/app14156638 · Cited by 89
Nikiforidis, K., Kyrtsoglou, A., Vafeiadis, T., Kotsiopoulos, T., Nizamis, A., Ioannidis, D., Votis, K., Tzovaras, D., & Sarigiannidis, P. G. (2024) · ICT Express · doi:10.1016/j.icte.2024.12.001 · Cited by 75
Establishing and evaluating trustworthy AI: overview and research challenges
Kowald, D., Scher, S., Pammer-Schindler, V., Müllner, P., Waxnegger, K., Demelius, L., Fessl, A., Toller, M., Mendoza Estrada, I. G., Šimić, I., et al. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1467222 · Cited by 59
“For a detailed overview of evaluation metrics for transparency and explainability, please also see Hulsen (2023), in which metrics such as simulatability, decomposability, coherence, or comprehensiveness are mentioned.”
Tomášik, J., Zsoldos, M., Oravcová, Ľ., Lifková, M., Pavleová, G., Strunga, M., & Thurzo, A. (2024) · AI · doi:10.3390/ai5010009 · Cited by 58
“One of the means of making AI more believable could be demystifying the AI algorithms (“the black box”) and making them comprehensible to humans, which may become quite challenging—especially with the ever-increasing complexity of used algorithms [67].”
BackgroundSarno, L. D., Caroselli, A., Tonin, G., Graglia, B., Pansini, V., Causio, F. A., Gatto, A., & Chiaretti, A. (2024) · Biomedicines · doi:10.3390/biomedicines12061220 · Cited by 63
“Overall, the next phases aim to bridge the current gap between research and clinical practice, striking a delicate balance between the transformative potential of AI and its current limitations.”
Kumar, R., Garg, S., Kaur, R., Johar, M. G. M., Singh, S., Menon, S. V., Kumar, P., Hadi, A. M., Hasson, S. A., & Šajić, J. L. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1583459 · Cited by 74
Zabala-Vargas, S. A., Jaimes-Quintanilla, M., & Jimenez-Barrera, M. H. (2023) · Buildings · doi:10.3390/buildings13122944 · Cited by 52
Grad-CAM (Gradient-weighted Class Activation Mapping): A systematic literature review
Fayyaz, A. M., Abdulkadir, S. J., Talpur, N., Al-Selwi, S. M., Hassan, S. U., & Sumiea, E. H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111200 · Cited by 76
AI in Radiology: Navigating Medical Responsibility
Contaldo, M. T., Pasceri, G., Vignati, G., Bracchi, L., Triggiani, S., & Carrafiello, G. (2024) · Diagnostics · doi:10.3390/diagnostics14141506 · Cited by 61
“Indeed, in a glass box model, full transparency represents the ideal scenario: every parameter is known In line with the previous discussion, the concept of Explainable Artificial Intelligence (XAI) is gaining significant attention in the scientific community [38–40].”
BackgroundKim, S.-D. (2024) · Technologies · doi:10.3390/technologies12050068 · Cited by 40
“While a significant risk or issue associated with AI is its potential to be perceived as a “black box,” limiting trust in its reliability [23], this study enhanced credibility by rigorously validating ChatGPT responses through comprehensive literature verification using reputable prior studies.”
Pawlicki, M., Pawlicka, A., Kozik, R., & Choraś, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127759 · Cited by 40
A Comprehensive Review of Multimodal Deep Learning for Enhanced Medical Diagnostics
Al‐Zoghby, A. M., Ebada, A. I., Saleh, A. S., Abdelhay, M., & Awad, W. A. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.065571 · Cited by 55
Reliable Autism Spectrum Disorder Diagnosis for Pediatrics Using Machine Learning and Explainable AI
Jeon, I., Kim, M., So, D., Kim, E. Y., Nam, Y., Kim, S., Shim, S.-H., Kim, J., & Moon, J. (2024) · Diagnostics · doi:10.3390/diagnostics14222504 · Cited by 40
An Activity System-based Perspective of Generative AI: Challenges and Research Directions
Nah, F. F., Cai, J., Zheng, R., & Pang, N. L.-S. (2023) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00190 · Cited by 42
Tomášik, J., Zsoldos, M., Majdáková, K., Fleischmann, A., Oravcová, Ľ., Ballová, D. S., & Thurzo, A. (2024) · Applied Sciences · doi:10.3390/app14177837 · Cited by 37
Explainable Artificial Intelligence: Importance, Use Domains, Stages, Output Shapes, and Challenges
Ullah, N., Khan, J. A., Falco, I. D., & Sannino, G. (2024) · ACM Computing Surveys · doi:10.1145/3705724 · Cited by 45
Threat Detection and Response Using AI and NLP in Cybersecurity
Ismail, D. W. S. (2024) · Journal of Internet Services and Information Security · doi:10.58346/jisis.2024.i1.013 · Cited by 34
Lee, S., Lee, J., Park, J., Park, J., Kim, D., Lee, J., & Oh, J. (2023) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2023.11.063 · Cited by 37
Introduction to Explainable AI (XAI) in E-Commerce
Chaudhary, M., Gaur, L., Singh, G., & Afaq, A. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-55615-9_1 · Cited by 32
Pesapane, F., Giambersio, E., Capetti, B., Monzani, D., Grasso, R., Nicosia, L., Rotili, A., Sorce, A., Meneghetti, L., Carriero, S., et al. (2024) · Life · doi:10.3390/life14040454 · Cited by 33
LaBoone, P. A., & Marques, O. (2024) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2024.100294 · Cited by 58
Raman Spectroscopy and AI Applications in Cancer Grading: An Overview
Conforti, P. M., Lazzini, G., Russo, P., & D’Acunto, M. (2024) · IEEE Access · doi:10.1109/access.2024.3388841 · Cited by 39
“For instance, in the healthcare sector, through XAI techniques physicians can understand which pathological features in input data guided the algorithm before exploiting the insight obtained from automatically generated diagnosis reports [194], [195].”
BackgroundAssudani, P. J., Bhurgy, A. S., Kollem, S., Bhurgy, B. S., Ahmad, M. O., Kulkarni, M. B., & Bhaiyya, M. L. (2025) · Microchemical Journal · doi:10.1016/j.microc.2025.115802 · Cited by 37
Explainable AI-driven model for gastrointestinal cancer classification
Binzagr, F. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1349373 · Cited by 30
“Moreover, XAI techniques aid in the identification of potential biases within the model, facilitating the prevention of misdiagnosis and promoting equitable healthcare outcomes (24, 25, 61).”
A comprehensive review of explainable AI for disease diagnosis
Biswas, A. A. (2024) · Array · doi:10.1016/j.array.2024.100345 · Cited by 39
Advances in automated anesthesia: a comprehensive review
Cai, X., Wang, X., Zhu, Y., Yao, Y., & Chen, J. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-024-00085-z · Cited by 40
“AI-driven systems must be designed for interpretability, ensuring that clinicians can understand the reasoning behind the system’s recommendations [168].”
Abdollahi, M., Jafarizadeh, A., Asbagh, A. G., Sobhi, N., Pourmoghtader, K., Pedrammehr, S., Asadi, H., Tan, R., Alizadehsani, R., & Acharya, U. R. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1560 · Cited by 32
“Explainable AI tackles the “ black-box ” character of many ML models, which are difficult to comprehend (Hulsen, 2023).”
Advancing clinical biochemistry: addressing gaps and driving future innovations
Cao, H., Oghenemaro, E. F., latypova, A., Abosaoda, M. K., Zaman, G. S., & Devi, A. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1521126 · Cited by 47
“Overreliance on AI-driven systems without adequate human oversight can introduce risks, particularly in cases in which AI misinterpretations lead to incorrect diagnoses or inappropriate treatment choices (194).”
Exploring explainable AI methods for bird sound-based species recognition systems
Das, N., Padhy, N., Dey, N., Paul, H., & Chowdhury, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17982-3 · Cited by 26
The crucial role of explainable artificial intelligence (XAI) in improving health care management
Johannssen, A., & Chukhrova, N. (2025) · Health Care Management Science · doi:10.1007/s10729-025-09720-y · Cited by 33
“As AI models are increasingly used in clinical decision-making, their opacity can obscure sources of error, which makes explainability essential for safe implementation [ 4, 13, 16].”
Explainable AI in Digestive Healthcare and Gastrointestinal Endoscopy
Mascarenhas, M., Mendes, F., Martins, M., Ribeiro, T., Afonso, J. P. L., Cardoso, P., Ferreira, J. P. S., Fonseca, J., & Macedo, G. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020549 · Cited by 30
“Consequently, the desire for models’ transparency has led to the birth of explainable AI (XAI) and the appearance of “glass box” models that enhance the explainability of AI models without sacrificing their performance [8,9].”
Govea, J., Gutierrez, R., & Villegas-Ch, W. E. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1410790 · Cited by 26
Pawlicki, M., Pawlicka, A., Kozik, R., & Choraś, M. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10972-3 · Cited by 24
“As Hulsen puts it, the choice between a ’glass box’ model, which prioritises explainabil-ity, and a ’crystal ball’ model, which emphasises accuracy, is a central theme in xAI (Hulsen 2023 ).”
InfluentialFrom AI to the Era of Explainable AI in Healthcare 5.0: Current State and Future Outlook
Rahman, A., Kundu, D., Debnath, T., Rahman, M., Das, U. K., Miah, A. S. M., & Muhammad, G. (2025) · Expert Systems · doi:10.1111/exsy.70060 · Cited by 28
“Hulsen (2023) 2023 Outline various impediments associated with executing explainable AI in the healthcare sector.”
Vidanagamachchi, S., & Waidyarathna, K. M. G. T. R. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1471200 · Cited by 29
Mahamud, E., Assaduzzaman, M., Islam, J., Fahad, N., Hossen, M. J., & Ramanathan, T. T. (2025) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2025.100240 · Cited by 26
Explainable AI for trustworthy intelligent process monitoring
Johannssen, A., Qiu, P., Yeganeh, A., & Chukhrova, N. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111407 · Cited by 31
AI-Powered Predictive Models in Implant Dentistry: Planning, Risk Assessment, and Outcomes
Neji, G., Gasparro, R., Tlili, M., Dhahri, A., Khanfir, F., Sammartino, G., Aliberti, A., Campana, M. D., & Amor, F. B. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm15010228 · Cited by 32
“Additionally, the interpretability of complex neural networks often described as “black box” systems poses an obstacle to full clinical integration [23].”
Incorporation of artificial intelligence into nursing research: A scoping review
Yasin, Y. M., Al‐Hamad, A., Metersky, K., & Kehyayan, V. (2025) · International Nursing Review · doi:10.1111/inr.13013 · Cited by 21
“These include a limited understanding of AI in healthcare and even society in general (Hulsen, 2023), lack of standardization in AI usage, ethical considerations, and fear of technology replacing human roles (Robert, 2019).”
Selvi, G. C., Murugan, R., Yenduri, G., Gurrammagari, D. R., Bhulakshmi, D., Kandati, D. R., Supriya, Y., Gadekallu, T. R., Rathore, R. S., & Jhaveri, R. H. (2023) · Recent Advances in Computer Science and Communications · doi:10.2174/0126662558266152231128060222 · Cited by 22
Explainable artificial intelligence for investigating the effect of lifestyle factors on obesity
Khater, T., Tawfik, H., & Singh, B. B. (2024) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2024.200427 · Cited by 21
Explainable AI for Symptom-Based Detection of Monkeypox: a machine learning approach
Setegn, G. M., & Dejene, B. E. (2025) · BMC Infectious Diseases · doi:10.1186/s12879-025-10738-4 · Cited by 23
Overcoming barriers in the use of artificial intelligence in point of care ultrasound
Vega, R., Dehghan, M., Nagdev, A. D., Buchanan, B. M., Kapur, J., Jaremko, J. L., & Zonoobi, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01633-y · Cited by 29
Rane, N. L., Choudhary, S., & Rane, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4637894 · Cited by 19
Adversarial Examples on XAI-Enabled DT for Smart Healthcare Systems
Imam, N. H. (2024) · Sensors · doi:10.3390/s24216891 · Cited by 17
Puts, S., Zegers, C. M. L., Dekker, A. L. A. J., & Bermejo, I. (2024) · JMIR Formative Research · doi:10.2196/60095 · Cited by 17
Piscitelli, M. S., Razzano, G., Buscemi, G., & Capozzoli, A. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.115115 · Cited by 16
Dembrower, K., Crippa, A., Eklund, M., & Strand, F. (2025) · Radiology · doi:10.1148/radiol.242566 · Cited by 17
Singh, S. S., Kumar, S., Ahuja, R., & Barua, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103217 · Cited by 22
Deva, R., Ramani, D., Divate, T., Jalota, S., & Ismail, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713362 · Cited by 32
“Explainability is also promoted and utmost importance is given to addressing the risks and ethical concerns [42, 91].”
Alves, E., Gurupadayya, B., & Prabhakaran, P. (2025) · Critical Reviews in Analytical Chemistry · doi:10.1080/10408347.2025.2575352 · Cited by 22
Explainable AI: The Quest for Transparency in Business and Beyond
DeSimone, H., & León, M. (2024) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1109/icict62343.2024.00093 · Cited by 20
Agrawal, R., & Agrawal, R. (2025) · Discover Mental Health · doi:10.1007/s44192-025-00232-3 · Cited by 22
“Accuracy and performance are important because they may mean the difference between life and death, especially in the healthcare industry [55].”
Villegas-Ch, W. E., Jaramillo-Alcázar, Á., Navarro, A. M., & Mera-Navarrete, A. (2025) · IEEE Access · doi:10.1109/access.2025.3541979 · Cited by 15
Machine learning versus clinicians for detection and classification of oral mucosal lesions
Schwärzler, J., Tolstaya, E. V., Tichý, A., Paris, S., Aarabi, G., Chaurasia, A., Malenova, Y., Steybe, D., & Schwendicke, F. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105992 · Cited by 22
Fuellen, G., Kulaga, A. Y., Lobentanzer, S., Unfried, M., Avelar, R. A., Palmer, D. H., & Kennedy, B. K. (2024) · Ageing Research Reviews · doi:10.1016/j.arr.2024.102617 · Cited by 15
“Sometimes a trade-off between explainability and correctness/accuracy is noted (Hulsen, 2023), but retrieving explanations can also improve correctness (Zhao et al., 2024).”
Zhang, Y., Tian, H., & Lu, J. (2025) · BMC Psychology · doi:10.1186/s40359-025-03404-6 · Cited by 18
Festivals in Age of AI: Smarter Crowds, Happier Fans
Lopes, J. M., Massano‐Cardoso, I., & Granadeiro, C. (2025) · Tourism and Hospitality · doi:10.3390/tourhosp6010035 · Cited by 21
“A key factor in building this trust is the trans-parency of AI processes, which enhances perceptions of effectiveness and fairness while reducing perceived risks (Hulsen, 2023; Yu & Li, 2022).”
Nimmy, S. F., Hussain, O. K., Chakrabortty, R. K., & Saha, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113333 · Cited by 15
Duan, J., Zhang, M., Song, M., Xu, X., & Lu, H. (2025) · Bioengineering · doi:10.3390/bioengineering12090954 · Cited by 13
“This inherent trade-off be-tween interpretability and performance represents a fundamental challenge in medical AI research [53].”
Explainable AI: Bridging the Gap between Machine Learning Models and Human Understanding
Avacharmal, R. (2024) · Journal of Informatics Education and Research · doi:10.52783/jier.v4i2.960 · Cited by 12
“The paper "Explainability in AI healthcare" by Hulsen (2023) targeted the concepts and challenges of explainable AI in healthcare, stressing its crucial importance for the presented decision-making processes to be explainable, transparent, accountable, and so well trusted [19].”
Asfahani, A. M. (2024) · Cogent Business & Management · doi:10.1080/23311975.2024.2395430 · Cited by 18
Artificial Intelligence in migrant health: a critical perspective on opportunities and risks
Matlin, S. A., Claron, I. M. M., Merone, J., Netto, G., Takian, A., Zaman, M. H., & Saso, L. (2025) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2025.101421 · Cited by 16
“Hulsen also questioned whether explanations always increase trust, noted a trade-off between explainability and accuracy, and pointed to alternative approaches like “Explainability-by-Design”, a method that incorporates explainability from the start rather than adding it later as a reactive measure.”
Explainable Machine Learning Models for Brain Diseases: Insights from a Systematic Review
Mallma, M. J. R., Zuloaga-Rotta, L., Borja-Rosales, R., Mallma, J. R. R., Vilca-Aguilar, M., Salas-Ojeda, M., & Mauricio, D. (2024) · Neurology International · doi:10.3390/neurolint16060098 · Cited by 15
“…be expanded in future research: • The present work did not find results related to the use of Large Language Models (LLM) or Generative AI (GenAI) as a topic related to explainability in the medical field, and it is something that should be studied given the popularity and novelty of these models.”
A Comprehensive Framework for Transparent and Explainable AI Sensors in Healthcare
Bouderhem, R. (2024) · International Electronic Conference on Sensors and Applications (ECSA) · doi:10.3390/ecsa-11-20524 · Cited by 16
“In healthcare, where decisions can have profound impacts on human lives, it is crucial that AI systems are explainable and transparent [5], allowing healthcare professionals and patients to understand the reasoning behind their outputs and recommendations [6].”
Rahman, E., Esfahlani, S. S., Rao, P., Garcia, P., Ioannidis, S., Nosta, J., Rahman, Z., & Webb, W. R. (2025) · European Journal of Plastic Surgery · doi:10.1007/s00238-025-02278-6 · Cited by 11
“XAI provides tools and methods that make the decision-making processes of AI models more transparent and understandable, thus fostering trust and enabling clinicians to make informed decisions based on AI recommendations [142].”
XAI-based Clinical Decision Support System: A Systematic Review
Kim, S., Kim, D., Kim, M., Ko, H., & Jeong, O. (2024) · doi:10.20944/preprints202406.0721.v1 · Cited by 72
An explainable AI-assisted web application in cancer drug value prediction
Kothari, S., Sharma, S., Shejwal, S., Kazi, A., DSilva, M., & Karthikeyan, M. (2024) · MethodsX · doi:10.1016/j.mex.2024.102696 · Cited by 11
“The authors of [3] discuss the importance of explainability in clinical decision support systems (CDSS) powered by artificial intelligence (AI).”
BackgroundExplainable Artificial Intelligence (XAI) Model for Cancer Image Classification
Singhal, A., Agrawal, K. K., Quezada, A., Aguiñaga, A. R., Jiménez, S., & Yadav, S. P. (2024) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2024.051363 · Cited by 16
Roychowdhury, S., Lanfranchi, V., & Mazumdar, S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110944 · Cited by 15
Phillips, V. J. (2025) · AI & Society · doi:10.1007/s00146-025-02337-3 · Cited by 11
“In the context of busy clinical practices, the seemingly modest addition of 30–40 s per patient case (Hulsen 2023) for generating and interpreting explanations can quickly become a non-trivial burden.”
InfluentialAlam, M. S. B., Lameesa, A., Sharmin, S., Afrin, S., Ahmed, S. F., Nikoo, M. R., & Gandomi, A. H. (2025) · Digital Communications and Networks · doi:10.1016/j.dcan.2025.04.001 · Cited by 9
Záhonyi, P., Fekete, D., Szabó, E., Nagy, Z. K., & Nagy, B. (2025) · European Journal of Pharmaceutical Sciences · doi:10.1016/j.ejps.2025.107173 · Cited by 10
Askr, H., Fayed, M. A. A., Farghaly, H. M., et al. (2025) · Scientific Reports · doi:10.1038/s41598-024-82483-8 · Cited by 8
“When the translucent glass box has low opacity (or when the model has high transparency), it promotes a better comprehension of the model 20 .”
Leveraging Explainable AI in Business and Further
DeSimone, H., & León, M. (2024) · IEEE Opportunity Research Scholars Symposium (ORSS) · doi:10.1109/orss62274.2024.10697961 · Cited by 9
“Regulations like the GDPR make it nearly impossible to use black-box models in various sectors, emphasizing the growing need for explainability and transparency [38], [39].”
MethodEmerging Trends in AI-Powered IoT: Innovations, Key Challenges, and Future Directions
AbdelRaouf, H., Abouyoussef, M. S., Fouda, M. M., Fadlullah, Z. M., & Ibrahem, M. I. (2025) · IEEE SoutheastCon · doi:10.1109/southeastcon56624.2025.10971687 · Cited by 7
“In healthcare, it enhances transparency in diagnostic tools, helping doctors trust AI-generated diagnoses and treat-ment recommendations [38].”
Hou, J., Zhang, Z., Cheng, X., & Wang, W. (2026) · Journal of Medical Internet Research · doi:10.2196/85663 · Cited by 8
“Given that the field of Explainable Artificial Intelligence remains in its developmental stages, fully visualizing algorithmic decision-making logic is not realistic in the short term; an incremental transparency strategy should therefore be adopted [107].”
Spanakis, M., Bakaros, E., Papadopoulou, S.-N., Fournaraki, A., & Symvoulakis, E. K. (2025) · Pharmacoepidemiology · doi:10.3390/pharma4040022 · Cited by 6
“XAI further enhances clinical utility by showing why an algorithm predicts elevated risk—for example, highlighting serotonin reuptake inhibition in SJW and SSRI, or sodium channel activation in aconite with antiarrhythmics [195,196].”
Review on Explainable AI by using LIME and SHAP Models for Healthcare Domain
Shaikh, A. S., Samant, R. M., Patil, K. S., Patil, N. R., & Mirkale, A. R. (2023) · International Journal of Computer Applications · doi:10.5120/ijca2023923263 · Cited by 9
From Anatomy to Algorithm: Scope of AI-Assisted Diagnostic Competencies in Health Sciences Education
Galdames, I. S. (2024) · International Journal of Medical and Surgical Sciences · doi:10.32457/ijmss.v11i3.2818 · Cited by 7
Personalized Financial Services through NLP and AI-Driven Innovations in FinTech
Ghosh, R., Ghosh, M., Roy, D., Gupta, B., Chakraborty, S., Biswas, A., & Basu, M. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-9944-6.ch019 · Cited by 7
Xu, Q., Adam, A., Abdullah, A., & Baharun, N. B. (2025) · Diagnostics · doi:10.3390/diagnostics15091150 · Cited by 7
“Interpretability also remains a major concern, as the opaque decision-making processes of deep learning models reduce clinical trust [19].”
Advanced Feature Selection Techniques in Medical Imaging—A Systematic Literature Review
Khan, S. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.066932 · Cited by 10
Implementation of Explainable AI on CNN- and ViT-Based Models for Classifying Breast Cancer
Hendry, A., Felicia, E., Natanael, Y., & Rumagit, R. Y. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.09.039 · Cited by 9
Narteni, S., Baiardini, I., Braido, F., & Mongelli, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0292980 · Cited by 6
“XAI comprehends a wide range of methodologies, which can be broadly categorized as post-hoc explanations of black box models and transparent-by-design techniques [4].”
BackgroundKathait, U. S., Rana, A., Chauhan, R. S., & Rawat, R. (2024) · International Conference on Sustainable Expert Systems (ICSES) · doi:10.1109/icses63445.2024.10763115 · Cited by 6
Karthiayani, A. (2024) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-3731-8.ch009 · Cited by 6
Nopour, R. (2025) · BMC Cancer · doi:10.1186/s12885-025-14369-5 · Cited by 6
“Explainable Artificial Intelligence (XAI) is incrementally cited in scientific publications and is essential in improving trust in AI models established in medicine and healthcare [42, 43].”
Daram, S. (2025) · International Journal of AI BigData Computational and Management Studies · doi:10.63282/3050-9416.ijaibdcms-v6i2p102 · Cited by 7
Novielli, P., Nazzicari, N., Pavan, S., Delvento, C., Diacono, D., Zoani, C., Bellotti, R., & Tangaro, S. (2025) · Applied System Innovation · doi:10.3390/asi8060164 · Cited by 5
“…applied across the life sciences to bridge predictive accuracy with biological insight: from clinical decision support and healthcare model design [21,22] to omics-driven discovery where interpretable attributions help prioritize variants, genes, and pathways for hypothesis generation [23].”
Empowering photodynamic therapy with artificial intelligence: current trends and future directions
Paul, A., Xavierselvan, M., Aebisher, D., Kubrak, T. P., Bartusik‐Aebisher, D., & Mallidi, S. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1771804 · Cited by 8
León, M., & DeSimone, H. (2024) · Advances in Science Technology and Engineering Systems Journal · doi:10.25046/aj090502 · Cited by 7
Application of Generative AI in Patient Engagement
ZareiNejad, M., & Tavana, P. (2025) · Springer eBooks · doi:10.1007/978-3-031-82963-5_5 · Cited by 4
Metadata
- Authors
- Hulsen, T.
- Affiliations
- Philips (Netherlands) (NL)
- Corresponding author
- Hulsen, Tim
- ORCID
- Hulsen, Tim
- Venue
- AI
- Publisher
- MDPI AG
- Published
- 2023-08-10
- Volume
- 4
- Issue
- 3
- Pages
- 652-666
- ISSN
- 2673-2688 (ISSN-L)
- OpenAlex
- W4385766743
- Semantic Scholar
- 747ce5b1d0178cb08dcf48e43f738f1476876ed5
- Cited by
- 305
- References
- 78
- Citations by year
- 2026: 92 · 2025: 112 · 2024: 85 · 2023: 16
- FWCI
- 26.99
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- AI (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- explainable AI, artificial intelligence in healthcare, clinical decision support, black box problem, trust in AI, interpretability, medical image analysis, radiology report analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
