Do stakeholder needs differ? - Designing stakeholder-tailored explainable artificial intelligence (XAI) interfaces
Open accessAbstract
Explainable AI (XAI) is increasingly being used in the healthcare domain. In health management, clinicians and patients are critical stakeholders, requiring tailored XAI explanations based on their unique needs. Our study investigates the differences in explanation needs between clinicians and patients and designs corresponding explanation interfaces for each group. Using a scenario-based approach, we assessed stakeholder-tailored needs, analyzed differences, and designed interfaces using theoretical frameworks. The results demonstrate diverse stakeholder motivations for seeking explanations, leading to varied requirements. The designed interfaces effectively address these requirements, as validated by the preference selection and qualitative feedback from clinicians and patients. Their suggestions provide design insights and highlight the divergent needs of these stakeholder groups. This study contributes practical and theoretical implications to XAI research, emphasizing the importance of understanding diverse stakeholder needs and incorporating relevant theoretical concepts into user-centered interface design.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The intuitiveness of model explanations varies substantially with respect to end users.
“the intuitiveness of model explanations varies substantially with respect to end users [35-37]”
From Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review (2026) · cites this paper as [37]
Cites
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · see citing works
The potential for artificial intelligence in healthcare
Davenport, T., & Kalakota, R. (2019) · Future Healthcare Journal · doi:10.7861/futurehosp.6-2-94 · Cited by 3,800 · see citing works
Deo, R. C. (2015) · Circulation · doi:10.1161/circulationaha.115.001593 · Cited by 3,466 · see citing works
Shared decision making — The pinnacle of patient-centered care
Barry, M. J., & Edgman-Levitan, S. (2012) · New England Journal of Medicine · doi:10.1056/nejmp1109283 · Cited by 3,433 · see citing works
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
Mark, G., Fussell, S. R., Lampe, C., Schraefel, M., Hourcade, J., Appert, C., & Wigdor, D. J. (2017) · International Conference on Human Factors in Computing Systems · doi:10.1145/3025453 · Cited by 2,222 · see citing works
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · see citing works
Health behavior : theory, research, and practice
Glanz, K., Rimer, B. K., & Viswanath, K. (2015) · Cited by 2,130 · see citing works
Big data and machine learning algorithms for health-care delivery
Ngiam, K. Y., & Khor, I. W. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30149-4 · Cited by 1,424 · see citing works
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
(2021) · doi:10.1145/3411764 · Cited by 1,240 · see citing works
Blanchard, C. M., Courneya, K. S., & Stein, K. (2008) · Journal of Clinical Oncology · doi:10.1200/jco.2007.14.6217 · Cited by 1,021 · see citing works
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Temporal self-regulation theory: A model for individual health behavior
Hall, P. A., & Fong, G. T. (2007) · Health Psychology Review · doi:10.1080/17437190701492437 · Cited by 679 · see citing works
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works
What do we need to build explainable AI systems for the medical domain?
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017) · arXiv · doi:10.48550/arxiv.1712.09923 · Cited by 636 · see citing works
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Maior, H. A., Wilson, M. L., Locke, C., & Swann, D. (2018) · doi:10.1145/3173574 · Cited by 570 · see citing works
Explainable Artificial Intelligence (XAI)
Hossain, M. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6371435 · Cited by 544 · see citing works
Artificial intelligence in healthcare: An essential guide for health leaders
Chen, M., & Decary, M. (2020) · Healthcare Management Forum · doi:10.1177/0840470419873123 · Cited by 538 · see citing works
Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities
Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2020) · Information Systems Management · doi:10.1080/10580530.2020.1849465 · Cited by 426 · see citing works
Lysaght, T., Lim, H. Y., Xafis, V., & Ngiam, K. Y. (2019) · Asian Bioethics Review · doi:10.1007/s41649-019-00096-0 · Cited by 332 · see citing works
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318 · see citing works
Recommender systems in the healthcare domain: state-of-the-art and research issues
Tran, T. N. T., Felfernig, A., Trattner, C., & Holzinger, A. (2020) · Journal of Intelligent Information Systems · doi:10.1007/s10844-020-00633-6 · Cited by 305 · see citing works
Self-regulation theory: Applications to medical education: AMEE Guide No. 58
Sandars, J. E., & Cleary, T. J. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.595434 · Cited by 298 · see citing works
Applications of Artificial Intelligence (AI) in healthcare: A reviewPreprint
Shaheen, M. Y. (2021) · doi:10.14293/s2199-1006.1.sor-.ppvry8k.v1 · Cited by 270 · see citing works
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
Kitamura, Y., Kitamura, Y., Quigley, A., Isbister, K., Igarashi, T., Japan, C. '. C. C. O. H. F. I. C. S. 2. Y., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer-Human Interaction, & SIGs, A. (2021) · doi:10.1145/3411763 · Cited by 245 · see citing works
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219 · see citing works
Holzinger, A., Dehmer, M., Emmert‐Streib, F., Cucchiara, R., Augenstein, I., Ser, J. D., Samek, W., Jurišica, I., & Díaz-Rodríguez, N. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.10.007 · Cited by 215 · see citing works
Human-centered XAI: Developing design patterns for explanations of clinical decision support systems
Schoonderwoerd, T. A. J., Jorritsma, W., Neerincx, M. A., & van den Bosch, K. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102684 · Cited by 210 · see citing works
Apps That Motivate: a Taxonomy of App Features Based on Self-Determination Theory
Villalobos-Zúñiga, G., & Cherubini, M. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102449 · Cited by 193 · see citing works
The role of patient, physician and systemic factors in the management of type 2 diabetes mellitus
Brown, J. B. (2002) · Family Practice · doi:10.1093/fampra/19.4.344 · Cited by 190 · see citing works
Scobbie, L., Dixon, D. L., & Wyke, S. (2010) · Clinical Rehabilitation · doi:10.1177/0269215510389198 · Cited by 165 · see citing works
Can we do better explanations? A proposal of user-centered explainable AI
Ribera, M., & Lapedriza, À. (2019) · IUI Workshops · Cited by 134 · see citing works
Coventry, P. A., Fisher, L., Kenning, C., Bee, P., & Bower, P. (2014) · BMC Health Services Research · doi:10.1186/s12913-014-0536-y · Cited by 130 · see citing works
Human-XAI Interaction: A Review and Design Principles for Explanation User Interfaces
Chromik, M., & Butz, A. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_36 · Cited by 128 · see citing works
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120 · see citing works
Human-Computer Interaction – INTERACT 2021
(2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85623-6 · Cited by 118 · see citing works
The Next Frontier: AI We Can Really Trust
Holzinger, A. (2021) · PKDD/ECML Workshops · doi:10.1007/978-3-030-93736-2_33 · Cited by 114 · see citing works
XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
Lopes, P., Silva, E., Braga, C., Oliveira, T., & Rosado, L. (2022) · Applied Sciences · doi:10.3390/app12199423 · Cited by 112 · see citing works
Personas for Artificial Intelligence (AI) an Open Source Toolbox
Holzinger, A., Kargl, M., Kipperer, B., Regitnig, P., Plass, M., & Müller, H. (2022) · IEEE Access · doi:10.1109/access.2022.3154776 · Cited by 99 · see citing works
Scobbie, L., McLean, D., Dixon, D. L., Duncan, E., & Wyke, S. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-190 · Cited by 96 · see citing works
Toward Human–AI Interfaces to Support Explainability and Causability in Medical AI
Holzinger, A., & Müller, H. (2021) · Computer · doi:10.1109/mc.2021.3092610 · Cited by 95 · see citing works
Cabitza, F., Campagner, A., Malgieri, G., Natali, C., Schneeberger, D., Stoeger, K., & Holzinger, A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118888 · Cited by 89 · see citing works
Explainable AI, But Explainable to Whom? An Exploratory Case Study of xAI in Healthcare
Gerlings, J., Jensen, M. S., & Shollo, A. (2021) · Intelligent systems reference library · doi:10.1007/978-3-030-83620-7_7 · Cited by 85 · see citing works
Examining the effect of explanation on satisfaction and trust in AI diagnostic systems
Alam, L., & Mueller, S. T. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01542-6 · Cited by 82 · see citing works
Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty
Jiang, J., Kahai, S. S., & Yang, M. H. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102839 · Cited by 82 · see citing works
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIPreprint
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07487 · Cited by 81 · see citing works
A Survey on Explainable Artificial Intelligence Techniques and Challenges
Hanif, A., Zhang, X., & Wood, S. (2021) · IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW) · doi:10.1109/edocw52865.2021.00036 · Cited by 69 · see citing works
Liao, Q. V., Zhang, Y., Luss, R., Doshi-Velez, F., & Dhurandhar, A. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21995 · Cited by 69 · see citing works
Explainable Artificial Intelligence (XAI): An Engineering PerspectivePreprint
Hussain, F. A., Hussain, R., & Hossain, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03613 · Cited by 66 · see citing works
26th International Conference on Intelligent User Interfaces
(2021) · doi:10.1145/3397482 · Cited by 65 · see citing works
Artificial Intelligence Applications in Tracking Health Behaviors During Disease Epidemics
Ganasegeran, K., & Abdulrahman, S. A. (2019) · Learning and analytics in intelligent systems · doi:10.1007/978-3-030-35139-7_7 · Cited by 62 · see citing works
The short- and long-term effectiveness of the LSVT®for dysarthria following TBI and stroke
Wenke, R., Theodoros, D., & Cornwell, P. (2008) · Brain Injury · doi:10.1080/02699050801960987 · Cited by 56 · see citing works
Saleem, F., Hassali, M. A., Shafie, A. A., Awad, G. A., Atif, M., Haq, N. U., Aljadhey, H. S., & Farooqui, M. (2012) · BMC Public Health · doi:10.1186/1471-2458-12-318 · Cited by 56 · see citing works
Nakao, Y., Strappelli, L., Stumpf, S., Naseer, A., Regoli, D., & Gamba, G. D. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2067936 · Cited by 55 · see citing works
He, X., Hong, Y., Xi, Z., & Zhang, Y. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2095093 · Cited by 48 · see citing works
Du, Y., Antoniadi, A. M., McNestry, C., McAuliffe, F. M., & Mooney, C. (2022) · Applied Sciences · doi:10.3390/app122010323 · Cited by 47 · see citing works
Polite or Direct? Conversation Design of a Smart Display for Older Adults Based on Politeness Theory
Hu, Y., Qu, Y., Maus, A., & Mutlu, B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517525 · Cited by 44 · see citing works
Towards a multi-stakeholder value-based assessment framework for algorithmic systems
Yurrita, M., Murray-Rust, D., Balayn, A., & Bozzon, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533118 · Cited by 42 · see citing works
Simkute, A., Luger, E. A., Jones, B., Evans, M., & Jones, R. (2021) · Journal of Responsible Technology · doi:10.1016/j.jrt.2021.100017 · Cited by 40 · see citing works
Artificial Intelligence and Machine Learning for Digital Pathology
(2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50402-1 · Cited by 36 · see citing works
Schmidt, M., & Tawfik, A. (2022) · Journal of Applied Instructional Design · doi:10.51869/111/msat · Cited by 35 · see citing works
Cock, E. D., Oostra, K. M., Bliki, L., Volkaerts, A., Hemelsoet, D., Herdt, V. D., & Batens, K. (2021) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12607 · Cited by 33 · see citing works
“Interfaces for explanations in human-AI interaction: proposing a design evaluation approach,”
Mucha, H., Robert, S., Breitschwerdt, R., & Fellmann, M. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451759 · Cited by 30 · see citing works
Handbook of Artificial Intelligence in Healthcare
(2021) · Intelligent systems reference library · doi:10.1007/978-3-030-83620-7 · Cited by 24 · see citing works
Kim, S., Choo, S., Park, D., Park, H., Nam, C. S., Jung, J., & Lee, S. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103009 · Cited by 21 · see citing works
How Can We Develop Explainable Systems? Insights from a Literature Review and an Interview Study
Chazette, L., Klünder, J., Balci, M., & Schneider, K. A. (2022) · doi:10.1145/3529320.3529321 · Cited by 21 · see citing works
XAI for learning: Narrowing down the digital divide between “new” and “old” experts
Simkute, A., Surana, A., Luger, E. A., Evans, M., & Jones, R. (2022) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3547522.3547678 · Cited by 17 · see citing works
Human Behaviour Analysis Using Intelligent Systems
(2019) · Learning and analytics in intelligent systems · doi:10.1007/978-3-030-35139-7 · Cited by 15 · see citing works
Kargl, M., Regitnig, P., Müller, H., & Holzinger, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50402-1_7 · Cited by 12 · see citing works
Alam, L., & Mueller, S. T. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221085114 · Cited by 11 · see citing works
Evaluation of a Person-Centered, Theory-Based Intervention to Promote Health Behaviors
Worawong, C., Borden, M. J., Cooper, K. M., Pérez, Ó. A., & Lauver, D. (2017) · Nursing Research · doi:10.1097/nnr.0000000000000254 · Cited by 9 · see citing works
Designing a Medical Crowdfunding Website from Sense of Community Theory
Kim, J. G., Kraut, R. E., & Karahalios, K. G. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555181 · Cited by 9 · see citing works
AI Healthcare System Interface: Explanation Design for Non-Expert User Trust.
Larasati, R. A., Liddo, A. D., & Motta, E. (2021) · Open Research Online - ORO (The Open University) · Cited by 8 · see citing works
MoReXAI - A Model to Reason About the eXplanation Design in AI Systems
Carvalho, N. D. O., Sampaio, A. L., & de Vasconcelos, D. R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05643-7_9 · Cited by 7 · see citing works
Creating personas for exposome research: the experience from the HEAP project
Coombs, H., Wootton, T., Dillner, J., Müller, H., Berger, A., & Kozlakidis, Z. (2023) · Open Research Europe · doi:10.12688/openreseurope.15474.1 · Cited by 2 · see citing works
Kleim (n.d.) · doi:10.1044/1092-4388(2008/018
Flagged first, then by citations
1 of the 88 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Patient Consent and The Right to Notice and Explanation of AI Systems Used in Health Care
Hurley, M. E., Lang, B. H., Kostick, K. M., Smith, J. N., & Blumenthal‐Barby, J. (2024) · The American Journal of Bioethics · doi:10.1080/15265161.2024.2399828 · Cited by 74 · see citing works
“…of algorithm visualizations and level of transparency into processes in the AI system) can contribute to proper trust calibration and higher levels of satisfaction (Kim et al. 2024; Starke et al. 2022; Zerilli, Bhatt, and Weller 2022; Starke 2021; Branley-Bell, Whitworth, and Coventry 2020).”
Herrera, F. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103133 · Cited by 61 · see citing works
Zhao, J., Lv, Z., Li, D., Feng, X., Wang, Z., Wu, Y., Shi, D., Fowler, M. W., & Burke, A. F. (2024) · eTransportation · doi:10.1016/j.etran.2024.100364 · Cited by 42 · see citing works
Miedema, E., Waschull, S., & Emmanouilidis, C. (2025) · Computers in Industry · doi:10.1016/j.compind.2025.104409 · Cited by 38 · see citing works
Why should I trust you? Influence of explanation design on consumer behavior in AI-based services
Nizette, F., Hammedi, W., van Riel, A. C. R., & Steils, N. (2024) · Journal of service management · doi:10.1108/josm-05-2024-0223 · Cited by 36 · see citing works
Artificial intelligence and corporate investment efficiency: Evidence from China
Wang, L., & Chen, Y. (2025) · Emerging Markets Review · doi:10.1016/j.ememar.2025.101314 · Cited by 31 · see citing works
Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI
Liu, H., Das, A., Boltz, A., Zhou, D., Pinaroc, D., Lease, M., & Lee, M. K. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686962 · Cited by 31 · see citing works
“"”
Herrera, F., & Calderón, R. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103302 · Cited by 28 · see citing works
Information that matters: Exploring information needs of people affected by algorithmic decisions
Schmude, T., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103380 · Cited by 13 · see citing works
“…First, the group of affected stakeholders is diverse and can include "laypeople" [84, 88, 92] or "AI novices" 1 , i.e., people with little technical literacy who have heterogeneous information needs, aims, and prior knowledge, necessitating accessible and adaptive explanations [25, 51, 90].”
BackgroundExplanation needs and ethical demands: unpacking the instrumental value of XAI
Alpsancar, S., Buhl, H. M., Matzner, T., & Scharlau, I. (2024) · AI and Ethics · doi:10.1007/s43681-024-00622-3 · Cited by 13 · see citing works
Deep learning models and the limits of explainable artificial intelligence
Bjerring, J. C., Mainz, J. T., & Munch, L. A. (2025) · Asian Journal of Philosophy · doi:10.1007/s44204-024-00238-8 · Cited by 11 · see citing works
“6 Insofar as transparency and understandability promote trust in medical algo-rithms—and there is evidence to suggest they often do (Kim et al., 2024; Sivara-man et al., 2023)—there is hence reason to seek transparent, non-opaque medical decision aids.”
Fang, P., Liu, Y., Bai, X., & Niu, Z. (2025) · Sustainability · doi:10.3390/su17052064 · Cited by 9 · see citing works
Human-Centric Digital Workplace
Sakshi, & Dhand, S. (2025) · Advances in human resources management and organizational development book series · doi:10.4018/979-8-3693-8181-6.ch006 · Cited by 7 · see citing works
Mohammed, I., & Guillet, B. D. (2025) · Tourism Economics · doi:10.1177/13548166251382289 · Cited by 7 · see citing works
“Although real-world applications of XAI remain limited, its potential has been demonstrated across sectors such as healthcare, fi nance, and autonomous vehicles (Weber et al., 2024; Gerlings et al., 2022), with calls to tailor its design to sector-speci fi c user needs (Kim et al., 2024).”
Füßl, A., & Nissen, V. (2025) · Requirements Engineering · doi:10.1007/s00766-025-00437-6 · Cited by 6 · see citing works
Ma, Y., Geng, H., Leng, A., & Zhuang, Y. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2480845 · Cited by 5 · see citing works
“However, existing studies have devoted limited empirical research to understanding public preferences for explainable AI (Kim et al., 2024).”
Towards a transparent and interpretable AI model for medical image classifications
Wen, B., Wu, Y., Daqqaq, T. S., & Chaddad, A. (2025) · Cognitive Neurodynamics · doi:10.1007/s11571-025-10343-w · Cited by 5 · see citing works
“4) The theoretical framework can guide the design of solutions to meet the specific needs and goals of users [62].”
Initial results on personalizing explanations of AI hints in an ITS
Bahel, V., Sriram, H., & Conati, C. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659566 · Cited by 5 · see citing works
“Kim et al. [9] studied personalized explanations for the behaviour of a (simulated) AI-driven system that helps patients and their clinicians manage a speech impairment.”
BackgroundInfluentialUnderstanding User Preferences in Explainable Artificial Intelligence: A Mapping Function Proposal
Hashemi, M., Darejeh, A., & Cruz, F. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3733837 · Cited by 4 · see citing works
“Furthermore, both groups can utilize XAI to foster trust in the AI systems they develop (AI experts) or utilize (non-AI domain experts) [19].”
Mill, E., Garn, W., & Turner, C. J. (2024) · Applied Artificial Intelligence · doi:10.1080/08839514.2024.2430867 · Cited by 4 · see citing works
“Second, Kim et al. (2024) investigates a scenario-based approach to XAI within a medical context, using scenarios based on symptoms presented by a fictitious patient.”
Towards Optimising EEG Decoding using Post-hoc Explanations and Domain Knowledge
Rajpura, P., & Meena, Y. K. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10781846 · Cited by 4 · see citing works
“A relevant direction for this vision is to understand the needs of the stakeholders for explanations that augment their decision-making [5].”
Haas, S., Hegestweiler, K., Rapp, M. T., Muschalik, M., & Hüllermeier, E. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1471208 · Cited by 3 · see citing works
AI Doctor for ASD: Physician Perceptions and Adoption Challenges in Autism Clinical Practice
Zhu, Y., Zhang, M., Fang, C., Fang, L., Liu, M. Q., Long, Y., Lee, K.-P., Zhang, L., & Wang, S. J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710925 · Cited by 4 · see citing works
Vo, V., Woode, M. E., Carter, S. M., Degeling, C., & Chen, G. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.013 · Cited by 5 · see citing works
Design Principles for Human-Centred Explainable AI: A Scoping Review
Hughes, N. G. J., Jia, Y., Sujan, M. A., Lawton, T., Habli, I., & McDermid, J. A. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3771720 · Cited by 3 · see citing works
“This may also involve considering user cognition to understand what should and should not be traded off (e.g., ‘balancing the tradeoff between coherence with the explainee’s mental model, novelty and overall plausibility’, Paper [47]).”
Baeyaert, J. (2025) · EthAIca · doi:10.56294/ai2025417 · Cited by 3 · see citing works
Jung, I.-C., Zerlik, M., Schuler, K., Sedlmayr, M., & Sedlmayr, B. (2026) · JMIR Formative Research · doi:10.2196/77481 · Cited by 3 · see citing works
“Current research supports this by showing that different health care provider groups can have different explanation preferences [15] or that different stakeholders have different explanation needs [47].”
Quantifying Spatial Domain Explanations in BCI using Earth Mover’s Distance
Rajpura, P., Cecotti, H., & Meena, Y. K. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650619 · Cited by 3 · see citing works
“Concurrent efforts to understand the needs and the nuances of explanations for relevant stakeholders involved in the domain applications of BCIs are also required [19].”
BackgroundTELL-ME: Toward Personalized Explanations of Large Language Models
Jeck, J., Leiser, F., Hüsges, A., & Sunyaev, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719982 · Cited by 2 · see citing works
“Research underscores the importance of incorporating user background for explanation interpretability, as no one-size-fits-all explanation exists [33, 58, 66].”
A review on empirical studies in explainable artificial intelligence
Jahn, T., Hühn, P., & Reichert, M. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11595-6 · Cited by 2 · see citing works
“Kim et al. ( 2024a ) review 73 papers describing 77 user studies on AI systems trained with tabular or textual data and propose a taxonomy of human-grounded XAI evaluation.”
InfluentialMaathuis, H., Stalenhoef, M., van Otterloo, S., Zwaal, R., van Montfort, K., & Sent, D. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1668029 · Cited by 2 · see citing works
Disentangling Stakeholder Role and Expertise in User-Centered Explainable AI
Szymanski, M., Abeele, V. V., & Verbert, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728351 · Cited by 1 · see citing works
Fairhurst, M., Kaesling, K., & Klös, V. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08330-2_17 · Cited by 1 · see citing works
Demystifying Explainable Artificial Intelligence (XAI) in Healthcare
Harshavardhan, K., Murugadoss, H., Priyankaa, V., & Rajanadh, M. G. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6400-1.ch001 · Cited by 1 · see citing works
It’s Not Just for Trust: Designing for Emerging Uses of Explainable AI in Clinical Decision-Making
Morrison, K., Li, Z., Kim, M. J., Hong, C. (., Kraisangka, J., Correa‐Jaque, P., Fauvel, C., Sahay, S., Vanderpool, R. R., Everett, A., et al. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3815175 · Cited by 1 · see citing works
Time for an Explanation: A Mini-Review of Explainable Physio-Behavioural Time-Series Classification
Schneider, J., Cheruvalath, S. S., & Hassan, T. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3679001 · Cited by 1 · see citing works
“The benefit of involving relevant stakeholders in the design of the system is that it provides an opportunity to obtain feedback that can be used to tailor the explanations and their presentation to the expectations of the recipients [16].”
Exploring Client and Vendor Perspective on Software-As-A-Service Adoption Decision
Lustofin, P., Kuciapski, M., & Soja, P. (2024) · Proceedings of the International Conference on Information Systems Development · doi:10.62036/isd.2024.51
Yokota, T., & Nakao, Y. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-82633-7_8
“As machine learning (ML) technologies have impacts on a diverse range of stake-holders, there have been ongoing efforts to improve the transparency of ML systems by including a variety of stakeholders in the evaluation of ML models [26,29,28,17,8].”
Shi, C., Shen, Y., Chen, J., Liu, F., Li, C., & Wang, C. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103781
Mechanistically Interpretable Wearable-Based Sleep Staging
Kadem, M., & Zheng, R. (2025) · 2025 IEEE Annual Congress on Artificial Intelligence of Things (AIoT) · doi:10.1109/aiot66900.2025.00055 · Cited by 2 · see citing works
“Clinicians need global explanations about system functionality and failure modes, while patients require simple, accessible explanations preserving autonomy [25].”
Lee, W., Hwang, K., Kim, S., Ban, G., Song, Y., & Park, W. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2638556 · Cited by 1 · see citing works
Kim, Y., Kim, M., Kim, S., Cho, S., & Kim, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790971
Islam, F., Islam, A., Amin, M. A., & Zaber, M. I. (2026) · Journal of Medical Internet Research · doi:10.2196/87158
Yu, D., & Knoble, C. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-27672-9.00007-0
XAI in cybersecurity: A survey on techniques, challenges, and future directions
Sani, S. D., Olapojoye, R., Salman, T., & Teixeira, M. A. (2026) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2026.104563
Bounia, L., & Ndour, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30849-8_2
Goyal, K., Zeiw, Z., Arora, S., Endlaw, S., Das, S., & Nizamani, S. B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30849-8_26
Wang, X., & Liu, W. (2026) · Science & Sports · doi:10.1016/j.scispo.2026.05.003
Gerlings, J., Keramidis, P., Shollo, A., & Constantiou, I. (2026) · Information Systems Journal · doi:10.1111/isj.70059
Reséndez, V., Kazokas, M., Dumlu, S. N., Borsci, S., & Yildirim, F. (2026) · Human Factors in Healthcare · doi:10.1016/j.hfh.2026.100158
Lamaazi, K., & Alaoui, F. Z. S. (2026) · International Conference on Circuit, Systems and Communication (ICCSC) · doi:10.1109/iccsc71566.2026.11650253
“This insufficient and inappropriate integration can lead to “unstable interaction patterns” and loss of user trust, particularly in critical situations such as healthcare, business, or social support [6] [8] [9].”
AI Explanations Through the IT Artifact Views
Vatn, D. M. K., & Mikalef, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38014-2_28
Meske, C., Brenne, J., Ünal, E., Ölçer, S., & Doğangün, A. (2026) · Behaviour and Information Technology · doi:10.2139/ssrn.6211215 · Cited by 5 · see citing works
Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review
Ebraheem, M., Toghranegar, J., Bridge2AI-Voice Consortium, Bensoussan, Y., & Templeton, J. M. (2026) · Journal of Medical Internet Research · doi:10.2196/83790
“Furthermore, the intuitiveness of model explanations varies substantially with respect to end users [35-37].”
- Authors
- Kim, M., Kim, S., Kim, J., Song, T.-J., & Kim, Y.
- Affiliations
- Yonsei University (KR)
- Corresponding author
- Kim, Yuyoung
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2024-01
- Volume
- 181
- Pages
- 103160
- ISSN
- 1071-5819 (ISSN-L)
- OpenAlex
- W4386989268
- Cited by
- 53
- References
- 88
- Citations by year
- 2026: 20 · 2025: 20 · 2024: 13
- FWCI
- 5.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Human-Computer Studies (publishedVersion, cc-by)
- Funders
- Institute for Information and Communications Technology Promotion
- Ministry of Science, ICT and Future Planning (2022-0-00621)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, stakeholder needs, clinicians, patients, health management, user-centered design, scenario-based assessment, clinical decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
