Human-centered evaluation of explainable AI applications: a systematic review
Open accessThe DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Abstract
Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there's been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user's perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human-AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Previous reviews of XAI have also reported that explainability is sensitive to contextual factors and future research should investigate different groups of end users [15,16].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
User Acceptance of Information Technology: Toward A Unified View1
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003) · MIS Quarterly · doi:10.2307/30036540 · Cited by 45,035
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Technology Acceptance Model 3 and a Research Agenda on Interventions
Venkatesh, V., & Bala, H. (2008) · Decision Sciences · doi:10.1111/j.1540-5915.2008.00192.x · Cited by 8,159
Developing and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550
Cacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359
Explanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · cited 1×
“For more details about the dichotomies described above, the reader is referred to Linardatos et al. (2020).”
BackgroundInternational Conference on Artificial Neural Networks
(1990) · Neural Networks · doi:10.1016/0893-6080(90)90065-s · Cited by 2,088
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799
Ngai, E. W. T., Hu, Y., Wong, Y. H., Chen, Y., & Sun, X. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.08.006 · Cited by 1,361
Explainable AI: From black box to glass box
Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317 · cited 1×
“However, they pose challenges in terms of interpretability, accountability, and trust (Rai, 2020; Diakopoulos, 2014).”
BackgroundProceedings of the 2021 CHI Conference on Human Factors in Computing Systems
(2021) · doi:10.1145/3411764 · Cited by 1,240
Effect of Clinical Decision-Support Systems: A Systematic Review
Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., Samsa, G., Hasselblad, V., Williams, J. W., Musty, M. D., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-1-201207030-00450 · Cited by 1,200
O’Brien, H. L., Cairns, P., & Hall, M. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.004 · Cited by 1,014
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Trust In and Adoption of Online Recommendation Agents
University of British Columbia, C., Benbasat, I., Wang, W., & University of British Columbia, C. (2005) · Journal of the Association for Information Systems · doi:10.17705/1jais.00065 · Cited by 841
Explaining the user experience of recommender systems
Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9118-4 · Cited by 799
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Algorithmic Accountability: Journalistic investigation of computational power structures
Diakopoulos, N. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976411 · Cited by 733 · cited 1×
“However, they pose challenges in terms of interpretability, accountability, and trust (Rai, 2020; Diakopoulos, 2014).”
BackgroundProceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Gini, M. L., Shehory, O. M., Itō, T., & Jonker, C. M. (2013) · Cited by 703
Li, X., Hess, T. J., & Valacich, J. S. (2008) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2008.01.001 · Cited by 635
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606
Notions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
(2018) · doi:10.1145/3173574 · Cited by 569
Loyola-Gonzalez, O. (2019) · IEEE Access · doi:10.1109/access.2019.2949286 · Cited by 559 · cited 1×
“…is determined without involving the users of the AI system. research from a machine learning perspective focuses on the technical challenges of generating explanations of black-box models (or alternatively, designing high-performing inherently interpretable models) (e.g., Loyola-Gonzalez, 2019).”
BackgroundMetrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518
2014 Science and Information Conference
(2014) · doi:10.1109/sai33325.2014 · Cited by 493
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.011 · Cited by 483
Explainable artificial intelligence: Objectives, stakeholders, and future research opportunities
Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2022) · Information Systems Management · doi:10.1080/10580530.2020.1849465 · Cited by 365
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320
Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317 · cited 1×
“…2023; Ooge et al., 2022 Jian et al. (2000) Trust in automation scale Brdnik et al., 2023; Conijn et al., 2023; Kartikeya, 2022; Weitz et al., 2021 Körber (2019) Trust in automation scale Faulhaber et al., 2021; Ghai et al., 2021 Hart and Staveland (1988) Task load index (NASA-TLX) User…”
BackgroundProxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1096257 · Cited by 272
Haque, A. B., Islam, A. K. M. N., & Mikalef, P. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122120 · Cited by 240
Explainable Agents and Robots: Results from a Systematic Literature Review
Anjomshoae, S., Najjar, A., Calvaresi, D., & Främling, K. (2019) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/kczb5817 · Cited by 227
Toward personalized XAI: A case study in intelligent tutoring systems
Conati, C., Barral, O., Putnam, V., & Rieger, L. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103503 · Cited by 192
On quantitative aspects of model interpretabilityPreprint
Nguyen, A.-P., & Martínez, M. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07584 · Cited by 181
Interpretable dropout prediction: towards XAI-based personalized intervention
Nagy, M., & Molontay, R. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00331-8 · Cited by 179 · cited 1×
“…et al., 2023 3 La Gatta et al., 2021a,b; Nazaretsky et al., 2022 12 Brdnik et al., 2023; Förster et al., 2021; Kim et al., 2023; Meas et al., 2022; Nagy and Molontay, 2023; Polley et al., 2021; Scheers and De Laet, 2021; Schulze-Weddige and Zylowski, 2021; Swamy et al., 2023; van der Waa et al.,…”
BackgroundConsiderations for Evaluation and Generalization in Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_1 · Cited by 176
Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102941 · Cited by 170
Wysocki, O., Davies, J. K., Vigo, M., Armstrong, A. C., Landers, D., Lee, R., & Freitas, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2022.103839 · Cited by 161
How to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
Selten, F., Robeer, M., & Grimmelikhuijsen, S. (2023) · Public Administration Review · doi:10.1111/puar.13602 · Cited by 153
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129
Co-design of human-centered, explainable AI for clinical decision support
Panigutti, C., Beretta, A., Fadda, D., Giannotti, F., Pedreschi, D., Perotti, A., & Rinzivillo, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3587271 · Cited by 127
Panigutti, C., Beretta, A., Giannotti, F., & Pedreschi, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502104 · Cited by 126
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120
“Let me explain!”: exploring the potential of virtual agents in explainable AI interaction design
Weitz, K., Schiller, D., Schlagowski, R., Huber, T., & André, E. (2021) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-020-00332-0 · Cited by 119 · cited 2×
“…2021; Neves et al., 2021; Ooge et al., 2022; Panigutti et al., 2022, 2023; Schellingerhout et al., 2022; Veldhuis et al., 2022; Warren et al., 2022; Weitz et al., 2021; Zöller et al., 2023 7 Bansal et al., 2021; Buçinca et al., 2020; Confalonieri et al., 2021; Ibrahim et al., 2023; Jmoona et al.,…”
BackgroundPost-hoc explanation of black-box classifiers using confident itemsets
Moradi, M., & Samwald, M. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113941 · Cited by 118
Interpretable heartbeat classification using local model-agnostic explanations on ECGs
Neves, I., Folgado, D., Santos, S., Barandas, M., Campagner, A., Ronzio, L., Cabitza, F., & Gamboa, H. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104393 · Cited by 117
Confalonieri, R., Weyde, T., Besold, T. R., & Moscoso del Prado Martín, F. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103471 · Cited by 113 · cited 1×
“…et al., 2022; Veldhuis et al., 2022; Warren et al., 2022; Weitz et al., 2021; Zöller et al., 2023 7 Bansal et al., 2021; Buçinca et al., 2020; Confalonieri et al., 2021; Ibrahim et al., 2023; Jmoona et al., 2023; Raab et al., 2023; Schrills and Franke, 2023 our framework, this includes 11…”
BackgroundXAI Systems Evaluation: A Review of Human and Computer-Centred Methods
Lopes, P., Silva, E., Braga, C., Oliveira, T., & Rosado, L. (2022) · Applied Sciences · doi:10.3390/app12199423 · Cited by 112
Kim, D., Song, Y., Kim, S., Lee, S., Wu, Y., Shin, J., & Lee, D. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122343 · Cited by 111
Does Explainable Artificial Intelligence Improve Human Decision-Making?
Alufaisan, Y., Marusich, L. R., Bakdash, J. Z., Zhou, Y., & Kantarcioglu, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16819 · Cited by 106
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106
HealthXAI: Collaborative and explainable AI for supporting early diagnosis of cognitive decline
Khodabandehloo, E., Riboni, D., & Alimohammadi, A. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.030 · Cited by 103
The role of domain expertise in trusting and following explainable AI decision support systems
Bayer, S., Gimpel, H., & Markgraf, M. (2022) · Journal of Decision Systems · doi:10.1080/12460125.2021.1958505 · Cited by 103
Lundberg, H., Mowla, N. I., Abedin, S. F., Thar, K., Mahmood, A., Gidlund, M., & Raza, S. (2022) · IEEE Access · doi:10.1109/access.2022.3208573 · Cited by 102
An empirical evaluation of the User Engagement Scale (UES) in online news environments
O'Brien, H. L., & Cairns, P. A. (2015) · Information Processing & Management · doi:10.1016/j.ipm.2015.03.003 · Cited by 100
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations
Abdul, A., von der Weth, C., Kankanhalli, M., & Lim, B. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376615 · Cited by 96
Raab, D., Theissler, A., & Spiliopoulou, M. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07809-x · Cited by 94 · cited 1×
“…al., 2022; Weitz et al., 2021; Zöller et al., 2023 7 Bansal et al., 2021; Buçinca et al., 2020; Confalonieri et al., 2021; Ibrahim et al., 2023; Jmoona et al., 2023; Raab et al., 2023; Schrills and Franke, 2023 our framework, this includes 11 evaluated properties, as shown in Table 4 (blue cells).”
BackgroundA Taxonomy for Human Subject Evaluation of Black-Box Explanations in XAI.
Chromik, M., & Schuessler, M. (2020) · Cited by 91
Jung, J., Lee, H., Jung, H., & Kim, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16110 · Cited by 87
Avetisyan, L., Ayoub, J., & Zhou, F. (2022) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2022.07.010 · Cited by 84
Extending the Nested Model for User-Centric XAI: A Design Study on GNN-based Drug Repurposing
Wang, Q., Huang, K., Chandak, P., Zitnik, M., & Gehlenborg, N. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209435 · Cited by 82
Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design
Swamy, V., Du, S., Marras, M., & Käser, T. (2023) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3576050.3576147 · Cited by 81
Explainable AI and Adoption of Financial Algorithmic Advisors: An Experimental Study
David, D. B., Resheff, Y. S., & Tron, T. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462565 · Cited by 78
Interpretable confidence measures for decision support systems
Waa, J. V. D., Schoonderwoerd, T., Diggelen, J. V., & Neerincx, M. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102493 · Cited by 77
The Effects of Explanations in Automated Essay Scoring Systems on Student Trust and Motivation
Conijn, R., Kahr, P., & Snijders, C. (2023) · Journal of Learning Analytics · doi:10.18608/jla.2023.7801 · Cited by 76
GPT2SP: A Transformer-Based Agile Story Point Estimation Approach
Fu, M., & Tantithamthavorn, C. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3158252 · Cited by 71
Explainable Activity Recognition for Smart Home Systems
Das, D., Nishimura, Y., Vivek, R. P., Takeda, N., Fish, S. T., Plötz, T., & Chernova, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3561533 · Cited by 70
What Are People Doing About XAI User Experience? A Survey on AI Explainability Research and Practice
Ferreira, J. J., & Monteiro, M. S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-49760-6_4 · Cited by 68
Nazaretsky, T., Bar, C., Walter, M., & Alexandron, G. (2022) · LAK22: 12th International Learning Analytics and Knowledge Conference · doi:10.1145/3506860.3506861 · Cited by 67
Umbrello, S., & Yampolskiy, R. V. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-021-00790-w · Cited by 61
Bertrand, A., Eagan, J., & Maxwell, W. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594053 · Cited by 61
CASTLE: Cluster-aided space transformation for local explanations
Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115045 · Cited by 51
Explaining Recommendations in E-Learning: Effects on Adolescents' Trust
Ooge, J., Kato, S., & Verbert, K. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511140 · Cited by 51
XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning
Zöller, M.-A., Titov, W., Schlegel, T., & Huber, M. F. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3625240 · Cited by 51
XFlag: Explainable Fake News Detection Model on Social Media
Chien, S.-Y. J., Yang, C., & Yu, F. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2062113 · Cited by 50
Hernandez-Bocanegra, D. C., & Ziegler, J. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3579541 · Cited by 50
Bunde, E. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.154 · Cited by 48 · cited 2×
“In-context expl quality N Papers 14 Bayer et al., 2022; Branley-Bell et al., 2020; Bunde, 2021; Faulhaber et al., 2021; Fu and Tantithamthavorn, 2022; Kartikeya, 2022; Kühnlenz and Kühnlenz, 2023; Lundberg et al., 2022; Okumura and Nagao, 2023; Reeder et al., 2023; Selten et al., 2023; Upasane et…”
BackgroundTowards Trustworthiness in the Context of Explainable Search
Polley, S., Koparde, R. R., Gowri, A. B., Perera, M., & Nuernberger, A. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462799 · Cited by 48
Warren, G., Keane, M. T., & Byrne, R. M. J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.10152 · Cited by 48
Brdnik, S., Podgorelec, V., & Šumak, B. (2023) · Electronics · doi:10.3390/electronics12122594 · Cited by 48 · cited 3×
“…Bertrand et al., 2023; Ghai et al., 2021; Hernandez-Bocanegra and Ziegler, 2023; Ooge et al., 2022 Jian et al. (2000) Trust in automation scale Brdnik et al., 2023; Conijn et al., 2023; Kartikeya, 2022; Weitz et al., 2021 Körber (2019) Trust in automation scale Faulhaber et al., 2021; Ghai et…”
BackgroundInfluentialXAI tools in the public sector: a case study on predicting combined sewer overflows
Maltbie, N., Niu, N., Doren, M. V., & Johnson, R. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468547 · Cited by 44
User Trust and Understanding of Explainable AI: Exploring Algorithm Visualisations and User Biases
Branley-Bell, D., Whitworth, R., & Coventry, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49065-2_27 · Cited by 44
Building Trust in Interactive Machine Learning via User Contributed Interpretable Rules
Guo, L., Daly, E. M., Alkan, O., Mattetti, M., Cornec, O., & Knijnenburg, B. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511111 · Cited by 43
Veldhuis, M. S., Ariëns, S., Ypma, R. J. F., Abeel, T., & Benschop, C. C. G. (2021) · Forensic Science International Genetics · doi:10.1016/j.fsigen.2021.102632 · Cited by 42
Bhattacharya, A., Ooge, J., Stiglic, G., & Verbert, K. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584075 · Cited by 40
Meas, M., Machlev, R., Köse, A., Tepljakov, A., Loo, L., Levron, Y., Petlenkov, E., & Belikov, J. (2022) · Sensors · doi:10.3390/s22176338 · Cited by 39
Borrego-Díaz, J., & Galán-Páez, J. (2022) · Minds and Machines · doi:10.1007/s11023-022-09603-z · Cited by 34
Upasane, S. J., Hagras, H., Anisi, M. H., Savill, S., Taylor, I., & Manousakis, K. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3279808 · Cited by 34
Comparing User Perception of Explanations Developed with XAI Methods
Aechtner, J., Cabrera, L., Katwal, D., Onghena, P., Valenzuela, D. P., & Wilbik, A. (2022) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee55066.2022.9882743 · Cited by 32
Schrills, T., & Franke, T. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3588594 · Cited by 32
Supporting High-Uncertainty Decisions through AI and Logic-Style Explanations
Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584080 · Cited by 30
Towards XAI in the SOC – a user centric study of explainable alerts with SHAP and LIME
Eriksson, H. S., & Grov, G. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020248 · Cited by 29
Examining explainable clinical decision support systems with think aloud protocols
Anjara, S. G., Janik, A., Dunford-Stenger, A., Mc Kenzie, K., Collazo-Lorduy, A., Torrente, M., Costabello, L., & Provencio, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0291443 · Cited by 28
LEAFAGE: Example-based and Feature importance-based Explanations for Black-box ML models
Adhikari, A., Tax, D. M. J., Satta, R., & Faeth, M. (2019) · 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2019.8858846 · Cited by 27
Faulhaber, A. K., Ni, I., & Schmidt, L. (2021) · Mensch und Computer (MuC) · doi:10.1145/3473856.3473886 · Cited by 23
Interactive and Explainable Advising Dashboard Opens the Black Box of Student Success Prediction
Scheers, H., & Laet, T. D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86436-1_5 · Cited by 21
Souza, J., & Leung, C. K.-S. (2021) · Springer eBooks · doi:10.1007/978-3-030-76409-8_4 · Cited by 21
Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation
Wang, Y., Venkatesh, P., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517551 · Cited by 21
Evaluating Explainable AI (XAI) in Terms of User Gender and Educational Background
Reeder, S., Jensen, J., & Ball, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35891-3_18 · Cited by 21
Examining Correlation Between Trust and Transparency with Explainable Artificial Intelligence
Kartikeya, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-10464-0_23 · Cited by 19
Fernandes, G. J., Choi, A., Schauer, J. M., Pfammatter, A. F., Spring, B. J., Darwiche, A. Y., & Alshurafa, N. I. (2023) · Journal of Medical Internet Research · doi:10.2196/42047 · Cited by 17
Explainable AI for Breast Cancer Diagnosis: Application and User’s Understandability Perception
Larasati, R. A. (2022) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet55527.2022.9872950 · Cited by 15 · cited 1×
“…and Sontakke, 2021; Fernandes et al., 2023; Guo et al., 2022; Hernandez-Bocanegra and Ziegler, 2023; Jang et al., 2023; Khodabandehloo et al., 2021; Larasati, 2022; Moradi and Samwald, 2021; Neves et al., 2021; Ooge et al., 2022; Panigutti et al., 2022, 2023; Schellingerhout et al., 2022;…”
BackgroundUser Study on the Effects Explainable AI Visualizations on Non-experts
Schulze‐Weddige, S., & Zylowski, T. (2022) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-95531-1_31 · Cited by 14
PASTLE: Pivot-aided space transformation for local explanations
Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.05.018 · Cited by 13
Explaining the Unexplainable: Role of XAI for Flight Take-Off Time Delay Prediction
Jmoona, W., Ahmed, M. U., Islam, M. R., Barua, S., Begum, S., Ferreira, A. C. E., & Cavagnetto, N. (2023) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-34107-6_7 · Cited by 13
Žlahtič, B., Završnik, J., Vošner, H. B., Kokol, P., Šuran, D., & Završnik, T. (2023) · Applied Sciences · doi:10.3390/app13148329 · Cited by 13
User-Centric Explainability in Fintech Applications
Deo, S., & Sontakke, N. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-78642-7_64 · Cited by 8
Dialogue Explanations for Rule-Based AI Systems
Xu, Y., Collenette, J., Dennis, L. A., & Dixon, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40878-6_4 · Cited by 7
Jang, J., Kim, M., Bui, T.-C., & Li, W.-S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33377-4_13 · Cited by 5
Kühnlenz, K., & Kühnlenz, B. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309319 · Cited by 3 · cited 1×
“…14 Bayer et al., 2022; Branley-Bell et al., 2020; Bunde, 2021; Faulhaber et al., 2021; Fu and Tantithamthavorn, 2022; Kartikeya, 2022; Kühnlenz and Kühnlenz, 2023; Lundberg et al., 2022; Okumura and Nagao, 2023; Reeder et al., 2023; Selten et al., 2023; Upasane et al., 2024; Wang and Yin, 2021;…”
BackgroundOkumura, H., & Nagao, T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44223-0_19 · Cited by 3
Research works printed but not linked: 4
- Förster M. Hühn P. Klier M. Kluge K. (2021). “Capturing users' reality: a novel approach to generate coherent counterfactual explanations,” in Hawaii International Conference on System Sciences. Available at: https://api.semanticscholar.org/CorpusID:232412682
- Ibrahim L. Ghassemi M. M. Alhanai T. (2023). “Do explanations improve the quality of AI-assisted human decisions? An algorithm-in-the-loop analysis of factual and counterfactual explanations,” in Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London), 326– 334.
- Kim J. Maathuis H. van Montfort K. Sent D. (in press). “Identifying XAI user needs: gaps between literature use cases in the financial sector,” in Proceedings of the 2nd Workshop on Responsible Applied Artificial Intelligence (RAAIT), at HHAI 2024 (Malmö: CEUR Workshop Proceedings (CEUR-WS.org)). Accepted for publication.
- Schellingerhout R. Medentsiy V. Marx M. (2022). Explainable career path predictions using neural models.
Cited by
Advancing smart public administration: Challenges and benefits of artificial intelligence
Barsekh-Onji, A., Hernandez, Z. T., & Espinosa, E. O. C. (2025) · Urban Governance · doi:10.1016/j.ugj.2025.06.003 · Cited by 45
Wang, K., Hou, W., Hong, L., & Guo, J. (2025) · Electronics · doi:10.3390/electronics14030420 · Cited by 40
Bergomi, L., Nicora, G., Orlowska, M. A., Podrecca, C., Bellazzi, R., Fregosi, C., Salinaro, F., Bonzano, M., Crescenzi, G., Speciale, F., et al. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03045-0 · Cited by 34
“Like other previous similar studies [16, 18, 20, 38], we acknowledge the limited sample size (10 clinicians) as one of the main limitations of this work, and we recognize that the findings cannot be broadly generalized across all specialties or clinical settings.”
Chettri, K., Sen, B., & Ghosal, P. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-025-02625-w · Cited by 22
Kibria, M. R., Lafond, S., & Arslan, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104405 · Cited by 16
Metadata
- Authors
- Kim, J., Maathuis, H., & Sent, D.
- Affiliations
- University of Applied Sciences Utrecht (NL)
- Corresponding author
- Kim, Jenia
- Maathuis, Henry
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2024-10-17
- Volume
- 7
- Pages
- 1456486-1456486
- ISSN
- 2624-8212 (ISSN-L)
- PMC
- PMC11525002
- OpenAlex
- W4403501023
- Cited by
- 130
- References
- 118
- Citations by year
- 2026: 105 · 2025: 25
- FWCI
- 4.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Artificial Intelligence (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- explainable AI, human-centered evaluation, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.