The role of domain expertise in trusting and following explainable AI decision support systems
Abstract
Although the roots of artificial intelligence (AI) stretch back some years, it currently flourishes in research and practice. However, AI deals with trust issues. One possible solution approach is making AI explain itself to its user, but it is still unclear how an AI can accomplish this in decision-making scenarios. This study focuses on how a user’s expertise influences trust in explainable AI (XAI) and how this influences behaviour. To test our theoretical assumptions, we develop an AI-based decision support system (DSS), observe user behaviour in an online experiment, complemented with survey data. The results show that domain-specific expertise negatively affects trust in AI-based DSS. We conclude that the focus on explanations might be overrated for users with low domain-specific expertise, whereas it is vital for users with high expertise. Investigating the influence of expertise on explanations of an AI-based DSS, this study contributes to research on XAI and DSS.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bayer et al. (2022) showed that domain expertise reduces trust in AI-based DSS.
“Similarly, Bayer et al. (2022) showed that domain expertise reduces trust in AI-based DSS.”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Bayer et al., 2022
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587 · see citing works
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · see citing works
Structural equation models with unobservable variables and measurement error: Algebra and statistics
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800313 · Cited by 13,287 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Predictive model assessment in PLS-SEM: guidelines for using PLSpredict
Shmueli, G., Sarstedt, M., Hair, J. F., Cheah, J.-H., Ting, H., Vaithilingam, S., & Ringle, C. M. (2019) · European Journal of Marketing · doi:10.1108/ejm-02-2019-0189 · Cited by 4,618 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Building Effective Online Marketplaces with Institution-Based Trust
Pavlou, P. A., & Gefen, D. (2004) · Information Systems Research · doi:10.1287/isre.1040.0015 · Cited by 2,516 · see citing works
Becker, J.-M., Klein, K., & Wetzels, M. (2012) · Long Range Planning · doi:10.1016/j.lrp.2012.10.001 · Cited by 2,279 · see citing works
Sheeran, P., & Webb, T. L. (2016) · Social and Personality Psychology Compass · doi:10.1111/spc3.12265 · Cited by 2,185 · cited 1× · see citing works
“It is worth noting, though, that there can be a huge discrepancy between the intention to do something and the actual behaviour (Sheeran & Webb, 2016).”
BackgroundRingle, C. M., Sarstedt, M., & Straub, D. W. (2012) · MIS Quarterly · doi:10.2307/41410402 · Cited by 2,184 · see citing works
Sniehotta, F. F., Scholz, U., & Schwarzer, R. (2005) · Psychology & Health · doi:10.1080/08870440512331317670 · Cited by 1,350 · see citing works
Roese, N. J., & Vohs, K. D. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612454303 · Cited by 457 · cited 1× · see citing works
“This is the technical term for the human tendency to perceive an event ‘more predictable after it becomes known than it was before it became known’ (Roese and Vohs, 2012, p.411).”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · see citing works
Wang, W., & Benbasat, I. (2008) · Journal of Management Information Systems · doi:10.2753/mis0742-1222240410 · Cited by 241 · cited 1× · see citing works
“In the area of e-commerce, it seems to hold true (Wang & Benbasat, 2008), yet for clinical DSS, explanations do not increase trust (Bussone et al., 2015).”
BackgroundArtificial intelligence for human flourishing – Beyond principles for machine learning
Stahl, B. C., Andreou, A., Brey, P., Hatzakis, T., Kirichenko, A., Macnish, K., Laulhé Shaelou, S., Patel, A., Ryan, M., & Wright, D. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.11.030 · Cited by 203 · see citing works
Human-Centric Justification of Machine Learning Predictions
Biran, O., & McKeown, K. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/202 · Cited by 84 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · see citing works
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130 · see citing works
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141 · see citing works
The Trust Recovery Journey. The Effect of Timing of Errors on the Willingness to Follow AI Advice.
Kahr, P. K., Rooks, G., Snijders, C., & Willemsen, M. C. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645167 · Cited by 21 · see citing works
Dogru, E. Ö., & Krämer, N. C. (2025) · Journal of Decision Systems · doi:10.1080/12460125.2025.2593251 · Cited by 15 · see citing works
Jung, W., Jeon, K., & Babon-Ayeng, P. (2026) · Proceedings of the 2026 ACM Sustainability Week · doi:10.1145/3765611.3815499
- Authors
- Bayer, S., Gimpel, H., & Markgraf, M.
- Affiliations
- University of Augsburg (DE)
- University of Hohenheim (DE)
- Corresponding author
- Markgraf, Moritz
- Venue
- Journal of Decision Systems
- Publisher
- Informa UK Limited
- Published
- 2021-08-11
- Volume
- 32
- Issue
- 1
- Pages
- 110-138
- ISSN
- 1246-0125 (ISSN-L)
- 2116-7052
- OpenAlex
- W3198062477
- Cited by
- 103
- References
- 18
- Citations by year
- 2026: 29 · 2025: 30 · 2024: 30 · 2023: 10 · 2022: 4
- FWCI
- 3.87
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Decision System (publishedVersion)
- ERef Bayreuth (University of Bayreuth) (submittedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- OPUS (Augsburg University) (submittedVersion)
- Funders
- funding agencies in the public, commercial, or not-for-profit sectors
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › Information Systems and Management › Technology Adoption and User Behaviour
- Keywords
- explainable AI, domain knowledge, trust in AI, decision support system, user behavior, AI explanations, human-AI interaction, decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
