How Explainable Artificial Intelligence Can Increase or Decrease Clinicians’ Trust in AI Applications in Health Care: Systematic Review
Open accessPublished version of a preprint
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Background: Artificial intelligence (AI) has significant potential in clinical practice. However, its “black box” nature can lead clinicians to question its value. The challenge is to create sufficient trust for clinicians to feel comfortable using AI, but not so much that they defer to it even when it produces results that conflict with their clinical judgment in ways that lead to incorrect decisions. Explainable AI (XAI) aims to address this by providing explanations of how AI algorithms reach their conclusions. However, it remains unclear whether such explanations foster an appropriate degree of trust to ensure the optimal use of AI in clinical practice. Objective: This study aims to systematically review and synthesize empirical evidence on the impact of XAI on clinicians’ trust in AI-driven clinical decision-making. Methods: A systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searching PubMed and Web of Science databases. Studies were included if they empirically measured the impact of XAI on clinicians’ trust using cognition- or affect-based measures. Out of 778 articles screened, 10 met the inclusion criteria. We assessed the risk of bias using standard tools appropriate to the methodology of each paper. Results: The risk of bias in all papers was moderate or moderate to high. All included studies operationalized trust primarily through cognitive-based definitions, with 2 also incorporating affect-based measures. Out of these, 5 studies reported that XAI increased clinicians’ trust compared with standard AI, particularly when the explanations were clear, concise, and relevant to clinical practice. In addition, 3 studies found no significant effect of XAI on trust, and the presence of explanations does not automatically improve trust. Notably, 2 studies highlighted that XAI could either enhance or diminish trust, depending on the complexity and coherence of the provided explanations. The majority of studies suggest that XAI has the potential to enhance clinicians’ trust in recommendations generated by AI. However, complex or contradictory explanations can undermine this trust. More critically, trust in AI is not inherently beneficial, as AI recommendations are not infallible. These findings underscore the nuanced role of explanation quality and suggest that trust can be modulated through the careful design of XAI systems. Conclusions: Excessive trust in incorrect advice generated by AI can adversely impact clinical accuracy, just as can happen when correct advice is distrusted. Future research should focus on refining both cognitive and affect-based measures of trust and on developing strategies to achieve an appropriate balance in terms of trust, preventing both blind trust and undue skepticism. Optimizing trust in AI systems is essential for their effective integration into clinical practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Transparency in AI models is essential for gaining clinician trust and enabling effective collaboration.”
From Enhancing Clinician Trust in AI Diagnostics: A Dynamic Framework for Confidence Calibration and Transparency (2025) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Explainable AI in healthcare has been surveyed extensively, yet no existing review addresses the challenges posed by decoder-based LLMs.”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Most evaluations focus on cognitive facets of trust (e.g., perceptions of reliability) rather than affective facets, despite the latter's influence on technology acceptance.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, fostering trust is not an unqualified good-excessive or unfounded trust (over-trust) in AI is risky [59].”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“Rosenbacke et al. [59] conducted a systematic review on XAI's impact on clinician trust and found considerable heterogeneity in how trust is assessed across studies.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [59]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Trust: The Social Virtues and the Creation of Prosperity
Schneider, P. A., & Fukuyama, F. (1996) · Journal of Marketing · doi:10.2307/1251846 · Cited by 10,501
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Trust, Reciprocity, and Social History
Berg, J., Dickhaut, J., & McCabe, K. (1995) · Games and Economic Behavior · doi:10.1006/game.1995.1027 · Cited by 5,548
Nudge: Improving Decisions About Health, Wealth and Happiness
Adkisson, R. V. (2008) · The Social Science Journal · doi:10.1016/j.soscij.2008.09.003 · Cited by 5,543
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · cited 1×
“AI can enhance clinical decision-making, reduce medical errors, and improve patient outcomes [1,2].”
BackgroundHuman trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1×
“In a recent systematic review on trust in AI, it was observed that transparency is critical for fostering trust among decision makers [5].”
Background
Research works printed but not linked: 4
- Seligman, AB, 2000, The Problem of Trust, 240
- Kahneman, D, 2011, Thinking, Fast and Slow
- Lewicki, RJ, 2011, Framing Trust: Trust as a Heuristic, 110
- MadsenMGregorSMeasuring human-computer trust2000Proceedings of the 11th Australasian Conference on Information SystemsJan 10, 2000Australia68
Cited by
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236
“The goal is not only to satisfy regulatory and ethical requirements but also to foster human–AI collaboration by improving the understanding and confidence of clinicians in AI-driven tools [15,16].”
Salimparsa, M., Sedig, K., Lizotte, D. J., Abdullah, S. S., Chalabianloo, N., & Muanda, F. T. (2025) · Informatics · doi:10.3390/informatics12040119 · Cited by 39
“Moreover, fostering trust is not an unqualified good–excessive or unfounded trust (over-trust) in AI is risky [59].”
InfluentialYu, Y., Gomez-Cabello, C. A., Haider, S. A., Genovese, A., Prabha, S., Trabilsy, M., Collaco, B. G., Wood, N. G., Bagaria, S., Tao, C., et al. (2025) · Diagnostics · doi:10.3390/diagnostics15172204 · Cited by 32
“Explainable AI (XAI) aims to clarify how AI systems arrive at decisions, thus helping clinicians feel more confident in AI-generated recommendations [4].”
Explainable artificial intelligence for multi-modal cancer analysis: from genomics to immunology
Zhang, M., & Yin, L. (2026) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2025.105040 · Cited by 5
Van Dort, B. A., Engelsma, T., Sneekes, P., Peute, L., & Medlock, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001417 · Cited by 9
“Previous reviews have explored XAI in healthcare, with a focus on explainability techniques [11,12], human computer interaction [13], and trust and intention to use systems [14].”
Metadata
- Authors
- Rosenbacke, R., Melhus, Å., McKee, M., & Stuckler, D.
- Affiliations
- Copenhagen Business School (DK)
- Uppsala University (SE)
- European Observatory on Health Systems and Policies (BE)
- Corresponding author
- Rosenbacke, Rikard
- Venue
- JMIR AI
- Publisher
- JMIR Publications Inc.
- Published
- 2024-10-30
- Volume
- 3
- Pages
- e53207
- ISSN
- 2817-1705 (ISSN-L)
- PMC
- PMC11561425
- OpenAlex
- W4403914562
- Cited by
- 305
- References
- 38
- Citations by year
- 2026: 223 · 2025: 81 · 2024: 1
- FWCI
- 10.33
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- JMIR AI (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Diva portal (Dalarna University Library) (submittedVersion, other-oa)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- CBS Research Portal (Copenhagen Business School) (submittedVersion, cc-by)
- Funders
- European Observatory on Health Systems and Policies
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- explainable AI, clinician trust in AI, clinical decision support, AI explanations, explanation quality, trust in AI recommendations, black box problem, clinical judgment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
