The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
How Explainable Artificial Intelligence Can Increase or Decrease Clinicians’ Trust in AI Applications in Health Care: Systematic Review
Open accessPublished version of a preprint
Abstract
Background: Artificial intelligence (AI) has significant potential in clinical practice. However, its “black box” nature can lead clinicians to question its value. The challenge is to create sufficient trust for clinicians to feel comfortable using AI, but not so much that they defer to it even when it produces results that conflict with their clinical judgment in ways that lead to incorrect decisions. Explainable AI (XAI) aims to address this by providing explanations of how AI algorithms reach their conclusions. However, it remains unclear whether such explanations foster an appropriate degree of trust to ensure the optimal use of AI in clinical practice. Objective: This study aims to systematically review and synthesize empirical evidence on the impact of XAI on clinicians’ trust in AI-driven clinical decision-making. Methods: A systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searching PubMed and Web of Science databases. Studies were included if they empirically measured the impact of XAI on clinicians’ trust using cognition- or affect-based measures. Out of 778 articles screened, 10 met the inclusion criteria. We assessed the risk of bias using standard tools appropriate to the methodology of each paper. Results: The risk of bias in all papers was moderate or moderate to high. All included studies operationalized trust primarily through cognitive-based definitions, with 2 also incorporating affect-based measures. Out of these, 5 studies reported that XAI increased clinicians’ trust compared with standard AI, particularly when the explanations were clear, concise, and relevant to clinical practice. In addition, 3 studies found no significant effect of XAI on trust, and the presence of explanations does not automatically improve trust. Notably, 2 studies highlighted that XAI could either enhance or diminish trust, depending on the complexity and coherence of the provided explanations. The majority of studies suggest that XAI has the potential to enhance clinicians’ trust in recommendations generated by AI. However, complex or contradictory explanations can undermine this trust. More critically, trust in AI is not inherently beneficial, as AI recommendations are not infallible. These findings underscore the nuanced role of explanation quality and suggest that trust can be modulated through the careful design of XAI systems. Conclusions: Excessive trust in incorrect advice generated by AI can adversely impact clinical accuracy, just as can happen when correct advice is distrusted. Future research should focus on refining both cognitive and affect-based measures of trust and on developing strategies to achieve an appropriate balance in terms of trust, preventing both blind trust and undue skepticism. Optimizing trust in AI systems is essential for their effective integration into clinical practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Transparency in AI models is essential for gaining clinician trust and enabling effective collaboration.”
From Enhancing Clinician Trust in AI Diagnostics: A Dynamic Framework for Confidence Calibration and Transparency (2025) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Explainable AI in healthcare has been surveyed extensively, yet no existing review addresses the challenges posed by decoder-based LLMs.”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Most evaluations focus on cognitive facets of trust (e.g., perceptions of reliability) rather than affective facets, despite the latter's influence on technology acceptance.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, fostering trust is not an unqualified good-excessive or unfounded trust (over-trust) in AI is risky [59].”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“Rosenbacke et al. [59] conducted a systematic review on XAI's impact on clinician trust and found considerable heterogeneity in how trust is assessed across studies.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [59]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Trust: The Social Virtues and the Creation of Prosperity
Schneider, P. A., & Fukuyama, F. (1996) · Journal of Marketing · doi:10.2307/1251846 · Cited by 10,501
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Trust, Reciprocity, and Social History
Berg, J., Dickhaut, J., & McCabe, K. (1995) · Games and Economic Behavior · doi:10.1006/game.1995.1027 · Cited by 5,548
Nudge: Improving Decisions About Health, Wealth and Happiness
Adkisson, R. V. (2008) · The Social Science Journal · doi:10.1016/j.soscij.2008.09.003 · Cited by 5,543
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Artificial intelligence and the future of global health
Schwalbe, N., & Wahl, B. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30226-9 · Cited by 876
UNCERTAINTY AND THE WELFARE ECONOMICS OF MEDICAL CARE
ARROW, K. J. (1978) · Uncertainty in Economics · doi:10.1016/b978-0-12-214850-7.50028-0 · Cited by 834
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629
Artificial Unintelligence: How Computers Misunderstand the World
Broussard, M. (2018) · The MIT Press eBooks · doi:10.7551/mitpress/11022.001.0001 · Cited by 559
The Functions and Limitations of Trust in the Provision of Medical Care
Mechanic, D. (1998) · Journal of Health Politics, Policy and Law · doi:10.1215/03616878-23-4-661 · Cited by 413
Cutillo, C. M., Sharma, K. R., Foschini, L., Kundu, S., Mackintosh, M., Mandl, K. D., MI in Healthcare Workshop Working Group, Beck, T., Collier, E., Colvis, C., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0254-2 · Cited by 337
Nazar, M., Alam, M. M., Yafi, E., & Su'ud, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3127881 · Cited by 312
Trusting Automation: Designing for Responsivity and Resilience
Chiou, E. K., & Lee, J. D. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211009995 · Cited by 301 · cited 1×
“For the trust, they used the terminology used by Chiou and Lee [36] of appropriate XSL • FO RenderX reliance.”
MethodNaiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102941 · Cited by 170 · cited 4×
“Naiseh et al [28,29] noted that explanations affected both cognitive and affect-based trust and could result in either overtrust or undertrust.”
BackgroundInfluentialExplainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review
Giuste, F., Shi, W., Zhu, Y., Naren, T., Isgut, M., Sha, Y., Li, T., Gupte, M., & Wang, M. D. (2022) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2022.3185953 · Cited by 162
Kliegr, T., Bahník, Š., & Fürnkranz, J. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103458 · Cited by 139
The explainability paradox: Challenges for xAI in digital pathology
Evans, T., Retzlaff, C. O., Geißler, C., Kargl, M., Plass, M., Müller, H., Kiehl, T.-R., Zerbe, N., & Holzinger, A. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.03.009 · Cited by 135
Sanchez‐Martinez, S., Cámara, Ó., Piella, G., Čikeš, M., González‐Ballester, M. Á., Miron, M., Vellido, A., Gómez, E., Fraser, A. G., & Bijnens, B. H. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.765693 · Cited by 127
Does AI explainability affect physicians’ intention to use AI?
Liu, C., Chen, Z., Kuo, S.-C., & Lin, T.-C. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104884 · Cited by 119
Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays
Gaube, S., Suresh, H., Raue, M., Lermer, E., Koch, T. K., Hudecek, M. F. C., Ackery, A. D., Grover, S. C., Coughlin, J. F., Frey, D., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-28633-w · Cited by 92 · cited 2×
“Gaube et al [33] found no difference in trust between experts and nonexperts but reported that the performance of nonexperts who drew upon XAI was superior in clinical practice.”
BackgroundJung, J., Lee, H., Jung, H., & Kim, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16110 · Cited by 87 · cited 1×
“…field, these do not include the most recent empirical evidence from clinical settings, although they did consistently speculate that XAI could increase users’ trust and thus the intention to use AI tools [20,21], as well as enhance confidence in decisions and thus, the trust of clinicians [22,23].”
BackgroundExplainable recommendation: when design meets trust calibration
Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2021) · World Wide Web · doi:10.1007/s11280-021-00916-0 · Cited by 68 · cited 4×
“Naiseh et al [28,29] noted that explanations affected both cognitive and affect-based trust and could result in either overtrust or undertrust.”
BackgroundMethodInfluentialTowards Global Explanations of Convolutional Neural Networks With Concept Attribution
Wu, W., Su, Y., Chen, X., Zhao, S., King, I., Lyu, M. R.-T., & Tai, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00868 · Cited by 63
As if sand were stone. New concepts and metrics to probe the ground on which to build trustable AI
Cabitza, F., Campagner, A., & Sconfienza, L. M. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01224-9 · Cited by 57
Kumar, A., Manikandan, R., Köse, U., Gupta, D. K., & Satapathy, S. C. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3457187 · Cited by 55
Rainey, C., O’Regan, T., Matthew, J., Skelton, E., Woznitza, N., Chu, K.-Y., Goodman, S., McConnell, J., Hughes, C., Bond, R., et al. (2022) · Radiography · doi:10.1016/j.radi.2022.06.006 · Cited by 51
Martínez-Agüero, S., Soguero-Ruíz, C., Alonso, J. M., Mora-Jiménez, I., Álvarez-Rodríguez, J., & Marques, A. G. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.02.021 · Cited by 48
Don’t pause giant AI for the wrong reasons
Ienca, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00649-x · Cited by 33
Placing Trust at the Heart of Health Policy and Systems
McKee, M., van Schalkwyk, M. C., Greenley, R., & Permanand, G. (2024) · International Journal of Health Policy and Management · doi:10.34172/ijhpm.2024.8410 · Cited by 24
Self-citationTrustworthy AI Explanations as an Interface in Medical Diagnostic Systems
Kaur, D., Uslu, S., & Durresi, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-14314-4_12 · Cited by 13
Research works printed but not linked: 4
- Seligman, AB, 2000, The Problem of Trust, 240
- Kahneman, D, 2011, Thinking, Fast and Slow
- Lewicki, RJ, 2011, Framing Trust: Trust as a Heuristic, 110
- MadsenMGregorSMeasuring human-computer trust2000Proceedings of the 11th Australasian Conference on Information SystemsJan 10, 2000Australia68
Cited by
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236
“The goal is not only to satisfy regulatory and ethical requirements but also to foster human–AI collaboration by improving the understanding and confidence of clinicians in AI-driven tools [15,16].”
Salimparsa, M., Sedig, K., Lizotte, D. J., Abdullah, S. S., Chalabianloo, N., & Muanda, F. T. (2025) · Informatics · doi:10.3390/informatics12040119 · Cited by 39
“Moreover, fostering trust is not an unqualified good–excessive or unfounded trust (over-trust) in AI is risky [59].”
InfluentialYu, Y., Gomez-Cabello, C. A., Haider, S. A., Genovese, A., Prabha, S., Trabilsy, M., Collaco, B. G., Wood, N. G., Bagaria, S., Tao, C., et al. (2025) · Diagnostics · doi:10.3390/diagnostics15172204 · Cited by 32
“Explainable AI (XAI) aims to clarify how AI systems arrive at decisions, thus helping clinicians feel more confident in AI-generated recommendations [4].”
Explainable artificial intelligence for multi-modal cancer analysis: from genomics to immunology
Zhang, M., & Yin, L. (2026) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2025.105040 · Cited by 5
Van Dort, B. A., Engelsma, T., Sneekes, P., Peute, L., & Medlock, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001417 · Cited by 9
“Previous reviews have explored XAI in healthcare, with a focus on explainability techniques [11,12], human computer interaction [13], and trust and intention to use systems [14].”
Metadata
- Authors
- Rosenbacke, R., Melhus, Å., McKee, M., & Stuckler, D.
- Affiliations
- Copenhagen Business School (DK)
- Uppsala University (SE)
- European Observatory on Health Systems and Policies (BE)
- Corresponding author
- Rosenbacke, Rikard
- Venue
- JMIR AI
- Publisher
- JMIR Publications Inc.
- Published
- 2024-10-30
- Volume
- 3
- Pages
- e53207
- ISSN
- 2817-1705 (ISSN-L)
- PMC
- PMC11561425
- OpenAlex
- W4403914562
- Cited by
- 305
- References
- 38
- Citations by year
- 2026: 223 · 2025: 81 · 2024: 1
- FWCI
- 10.33
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- JMIR AI (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Diva portal (Dalarna University Library) (submittedVersion, other-oa)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- CBS Research Portal (Copenhagen Business School) (submittedVersion, cc-by)
- Funders
- European Observatory on Health Systems and Policies
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- explainable AI, clinician trust in AI, clinical decision support, AI explanations, explanation quality, trust in AI recommendations, black box problem, clinical judgment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
