The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Explainable Artificial Intelligence (XAI) from a user perspective: A synthesis of prior literature and problematizing avenues for future research
Open accessAbstract
The rapid growth and use of artificial intelligence (AI)-based systems have raised concerns regarding explainability. Recent studies have discussed the emerging demand for explainable AI (XAI); however, a systematic review of explainable artificial intelligence from an end user's perspective can provide a comprehensive understanding of the current situation and help close the research gap. The purpose of this study was to perform a systematic literature review of explainable AI from the end user's perspective and to synthesize the findings. To be precise, the objectives were to 1) identify the dimensions of end users' explanation needs; 2) investigate the effect of explanation on end user's perceptions, and 3) identify the research gaps and propose future research agendas for XAI, particularly from end users' perspectives based on current knowledge. The final search query for the Systematic Literature Review (SLR) was conducted on July 2022. Initially, we extracted 1707 journal and conference articles from the Scopus and Web of Science databases. Inclusion and exclusion criteria were then applied, and 58 articles were selected for the SLR. The findings show four dimensions that shape the AI explanation, which are format (explanation representation format), completeness (explanation should contain all required information, including the supplementary information), accuracy (information regarding the accuracy of the explanation), and currency (explanation should contain recent information). Moreover, along with the automatic representation of the explanation, the users can request additional information if needed. We have also described five dimensions of XAI effects: trust, transparency, understandability, usability, and fairness. We investigated current knowledge from selected articles to problematize future research agendas as research questions along with possible research paths. Consequently, a comprehensive framework of XAI and its possible effects on user behavior has been developed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709
From Intentions to Actions: A Theory of Planned Behavior
Ajzen, I. (1985) · Action Control · doi:10.1007/978-3-642-69746-3_2 · Cited by 19,057
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Information Systems Success: The Quest for the Dependent Variable
DeLone, W. H., & McLean, E. R. (1992) · Information Systems Research · doi:10.1287/isre.3.1.60 · Cited by 10,697
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Moore, G. C., & Benbasat, I. (1991) · Information Systems Research · doi:10.1287/isre.2.3.192 · Cited by 9,150
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Beyond Accuracy: What Data Quality Means to Data Consumers
Wang, R. Y., & Strong, D. M. (1996) · Journal of Management Information Systems · doi:10.1080/07421222.1996.11518099 · Cited by 4,672
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · cited 1×
“Linardatos et al., 2021 This study highlights the programming implementation in recent studies that contributes to increasing the interpretability of ML models from both theorist and practitioner perspectives.”
BackgroundA Theoretical Integration of User Satisfaction and Technology Acceptance
Wixom, B. H., & Todd, P. A. (2005) · Information Systems Research · doi:10.1287/isre.1050.0042 · Cited by 2,835
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 2×
“Furthermore, the General Data Protection Regulation (GDPR) has also emphasized the explainability of AI systems by introducing the “ right to explanation ” (Goodman and Flaxman, 2017).”
BackgroundCan we open the black box of AI?
Castelvecchi, D. (2016) · Nature · doi:10.1038/538020a · Cited by 1,773
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360
Fjeld, J., Achten, N., Hilligoss, H., Nagy, A., & Srikumar, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3518482 · Cited by 998 · cited 1×
“AI-based decision-making systems can be used for scalable and larger ecosystems; however, the systems need to be incorporated with some principles related to ethics and rights (Fjeld et al., 2020).”
BackgroundExplainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 10×
“Another use case related to the flight re-routing system offers the latest flight data to users so that travel is flexible and comfortable (Binns et al., 2018).”
BackgroundMethodInfluentialAntoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Why and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · cited 2×
“The hybrid explanation format is also required for other context-aware systems (Lim et al., 2009), general decisionmaking systems (Brennen, 2020; Schrills and Franke, 2020), travel guides (Lim and Dey, 2009), cooking recommendation systems (Broekens et al.”
BackgroundExpanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 8×
“Explanations in e-commerce (Ehsan et al., 2021; Eslami et al., 2018), education (Cheng et al., 2019; Conati et al., 2021; Mucha et al., 2021; Putnam and Conati, 2019), finance (Binns et al., 2018; Chromik et al., 2021; Cirqueira et al., 2020), law (Liu et al., 2021a, 2021b; G ´ orski and…”
BackgroundInfluentialUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448
Lauritsen, S. M., Kristensen, M., Olsen, M. V., Larsen, M. S., Lauritsen, K. M., Jørgensen, M. J., Lange, J., & Thiesson, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17431-x · Cited by 448
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441
Transparency and trust in artificial intelligence systems
Schmidt, P., Biessmann, F., & Teubner, T. (2020) · Journal of Decision Systems · doi:10.1080/12460125.2020.1819094 · Cited by 440
Transparent, explainable, and accountable AI for robotics
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · Science Robotics · doi:10.1126/scirobotics.aan6080 · Cited by 413
Oh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301
Responsible AI—two frameworks for ethical design practice
Peters, D., Vold, K., Robinson, D., & Calvo, R. A. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2974991 · Cited by 293 · cited 1×
“With our findings, we contribute to research conducted to design and govern responsible AI systems (Wearn et al., 2019; Maas, 2018; Peters et al., 2020; Rakova et al., 2021).”
MethodExplaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 9×
“Explanations in e-commerce (Ehsan et al., 2021; Eslami et al., 2018), education (Cheng et al., 2019; Conati et al., 2021; Mucha et al., 2021; Putnam and Conati, 2019), finance (Binns et al., 2018; Chromik et al., 2021; Cirqueira et al., 2020), law (Liu et al., 2021a, 2021b; G ´ orski and…”
BackgroundInfluentialExplaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286
Haque, A. K. M. B., Bhushan, B., & Dhiman, G. (2022) · Expert Systems · doi:10.1111/exsy.12753 · Cited by 271 · cited 1×
“…et al., 2012), news websites, social media (Feng et al., 2020), healthcare (Haque et al., 2020), and other public services (Hengstler et al., 2016; Haque et al., 2021; Du and Xie, 2021); however, the working principle of AI systems is unclear as the machine-learning models used in different AI…”
Self-citationBringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219
Automated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210
Artificial intelligence for human flourishing – Beyond principles for machine learning
Stahl, B. C., Andreou, A., Brey, P., Hatzakis, T., Kirichenko, A., Macnish, K., Laulhé Shaelou, S., Patel, A., Ryan, M., & Wright, D. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.11.030 · Cited by 203
Janssen, M., Hartog, M., Matheus, R., Yi Ding, A., & Kuk, G. (2022) · Social Science Computer Review · doi:10.1177/0894439320980118 · Cited by 192 · cited 9×
“To promote transparency, different attributes, such as age, gender, income, profession, and other related specifics, can be included in the explanation (Janssen et al., 2020).”
BackgroundMethodInfluentialCommunicating Algorithmic Process in Online Behavioral Advertising
Eslami, M., Krishna Kumaran, S. R., Sandvig, C., & Karahalios, K. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174006 · Cited by 171
How to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
The Impact of Placebic Explanations on Trust in Intelligent Systems
Eiband, M., Buschek, D., Kremer, A., & Hussmann, H. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312787 · Cited by 152
Lee, M. K., & Rich, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445570 · Cited by 145
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144
Weitz, K., Schiller, D., Schlagowski, R., Huber, T., & André, E. (2019) · Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents · doi:10.1145/3308532.3329441 · Cited by 136
The explainability paradox: Challenges for xAI in digital pathology
Evans, T., Retzlaff, C. O., Geißler, C., Kargl, M., Plass, M., Müller, H., Kiehl, T.-R., Zerbe, N., & Holzinger, A. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.03.009 · Cited by 135
Malgieri, G. (2019) · Computer Law & Security Review · doi:10.1016/j.clsr.2019.05.002 · Cited by 124 · cited 2×
“To be more precise, the data subject has the right to be informed about “ meaningful information about the logic involved ” if any decision related to the subject is “ based solely on automatic, automated processing ” (Malgieri, 2019).”
BackgroundResponsible AI: Requirements and challenges
Ghallab, M. (2019) · AI Perspectives · doi:10.1186/s42467-019-0003-z · Cited by 120
Responsible AI for conservation
Wearn, O. R., Freeman, R., & Jacoby, D. M. P. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0022-7 · Cited by 112
Experiences with Improving the Transparency of AI Models and Services
Hind, M., Houde, S., Martino, J., Mojsilovic, A., Piorkowski, D., Richards, J., & Varshney, K. R. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3383051 · Cited by 44
Explainable artificial intelligence, lawyer's perspective
Górski, Ł., & Ramakrishna, S. (2021) · Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law · doi:10.1145/3462757.3466145 · Cited by 40
Color for Characters - Effects of Visual Explanations of AI on Trust and Observability
Schrills, T., & Franke, T. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-50334-5_8 · Cited by 17 · cited 6×
“…explanation format is also required for other context-aware systems (Lim et al., 2009), general decision-making systems (Brennen, 2020; Schrills and Franke, 2020), travel guides (Lim and Dey, 2009), cooking recommendation systems (Broek-ens et al., 2010), and wearable systems (Danry et al., 2020).”
BackgroundInfluential
Cited by
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Explainable artificial intelligence for drug discovery and development: a comprehensive survey
Alizadehsani, R., Oyelere, S. S., Hussain, S., Jagatheesaperumal, S. K., Calixto, R. R., Rahouti, M., Roshanzamir, M., & De Albuquerque, V. H. C. (2024) · IEEE Access · doi:10.1109/access.2024.3373195 · Cited by 124
AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation
Shaer, O., Cooper, A., Mokryn, O., Kun, A. L., & Ben Shoshan, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642414 · Cited by 130
A Survey of Privacy Risks and Mitigation Strategies in the Artificial Intelligence Life Cycle
Shahriar, S., Allana, S., Hazratifard, S. M., & Dara, R. (2023) · IEEE Access · doi:10.1109/access.2023.3287195 · Cited by 114
Metadata
- Authors
- Haque, A. B., Islam, A. K. M. N., & Mikalef, P.
- Venue
- Technological Forecasting and Social Change
- Publisher
- Elsevier BV
- Published
- 2023-01
- Volume
- 186
- Pages
- 122120
- ISSN
- 0040-1625 (ISSN-L)
- OpenAlex
- W4308292292
- Cited by
- 240
- References
- 54
- FWCI
- 15.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.