Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Open accessAbstract
Explainable AI (XAI) is widely viewed as a sine qua non for ever-expanding AI research. A better understanding of the needs of XAI users, as well as human-centered evaluations of explainable models are both a necessity and a challenge. In this paper, we explore how human-computer interaction (HCI) and AI researchers conduct user studies in XAI applications based on a systematic literature review. After identifying and thoroughly analyzing 97 core papers with human-based XAI evaluations over the past five years, we categorize them along the measured characteristics of explanatory methods, namely trust, understanding, usability, and human-AI collaboration performance. Our research shows that XAI is spreading more rapidly in certain application domains, such as recommender systems than in others, but that user evaluations are still rather sparse and incorporate hardly any insights from cognitive or social sciences. Based on a comprehensive discussion of best practices, i.e., common models, design choices, and measures in user studies, we propose practical guidelines on designing and conducting user studies for XAI researchers and practitioners. Lastly, this survey also highlights several open research directions, particularly linking psychological science and human-centered XAI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CAs offering technical explanations or counterfactual options, even if not fully comprehended, signal transparency.
“CAs offering technical explanations or counterfactual options, even if not fully comprehended, signal transparency (Liao et al., 2020;Rong et al., 2024).”
From Explainability in AI: Comparing Human-Like and System-Like Trust Repair Strategies (2026) · cites this paper as Rong et al., 2024
Section 4.2.2 (page unavailable) says users’ perceived understanding can differ from their actual comprehension. The Usability findings (page unavailable) report that counterfactual explanations can increase perceived fairness. Neither passage establishes that conversational agents’ technical explanations or counterfactual options signal transparency when users do not fully comprehend them. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · see citing works
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Expectancies, values and academic behaviors
Eccles, J. S. (1983) · Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) · Cited by 3,901 · see citing works
Protocol Analysis: Verbal Reports as Data
Ericsson, K. A., & Simon, H. A. (1993) · doi:10.7551/mitpress/5657.001.0001 · Cited by 3,537 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
The EU General Data Protection Regulation (GDPR)
Voigt, P., & von dem Bussche, A. (2017) · doi:10.1007/978-3-319-57959-7 · Cited by 3,001 · see citing works
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 4× · see citing works
“Understanding subjective [7, 12, 13, 14, 16, 17, 22, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44] objective [12, 13, 22, 32, 35, 39, 40, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60] explanation model [21, 46, 49, 61, 62, 63, 64, 65]”
BackgroundMethodInfluentialPaas, F. G. W. C. (1992) · Journal of Educational Psychology · doi:10.1037/0022-0663.84.4.429 · Cited by 2,150 · see citing works
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · see citing works
Machine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · see citing works
A survey of deep learning techniques for autonomous driving
Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020) · Journal of Field Robotics · doi:10.1002/rob.21918 · Cited by 1,809 · see citing works
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799 · see citing works
Can we open the black box of AI?
Castelvecchi, D. (2016) · Nature · doi:10.1038/538020a · Cited by 1,773 · see citing works
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works
Machine learning for medical imaging
Erickson, B. J., Korfiatis, P., Akkus, Z., & Kline, T. L. (2017) · RadioGraphics · doi:10.1148/rg.2017160130 · Cited by 1,672 · see citing works
Artificial Intelligence in Human Resources Management: Challenges and a Path Forward
Tambe, P., Cappelli, P., & Yakubovich, V. (2019) · California Management Review · doi:10.1177/0008125619867910 · Cited by 1,609 · see citing works
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526 · see citing works
Artificial Intelligence (AI) applications for COVID-19 pandemic
Vaishya, R., Javaid, M., Khan, I. H., & Haleem, A. (2020) · Diabetes & Metabolic Syndrome Clinical Research & Reviews · doi:10.1016/j.dsx.2020.04.012 · Cited by 1,502 · see citing works
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · see citing works
Machine learning in medicine: a practical introduction
Sidey-Gibbons, J. A. M., & Sidey-Gibbons, C. J. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0681-4 · Cited by 1,308 · see citing works
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240 · see citing works
Murphy, R. R. (2001) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir.2001.28.3.266.1 · Cited by 1,060 · see citing works
The misunderstood limits of folk science: An illusion of explanatory depth
Rozenblit, L., & Keil, F. (2002) · Cognitive Science · doi:10.1207/s15516709cog2605_1 · Cited by 1,032 · see citing works
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008 · see citing works
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894 · see citing works
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845 · see citing works
Explaining the user experience of recommender systems
Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9118-4 · Cited by 799 · see citing works
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 9× · see citing works
“Likewise, utilizing explanations has no significant impact on human performance in [89, 92], but simply showing model predictions has a positive effect in [92].”
BackgroundInfluentialAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011) · Educational Psychology Review · doi:10.1007/s10648-011-9174-7 · Cited by 715 · see citing works
Collaborative filtering and deep learning based recommendation system for cold start items
Jian, W., He, J., Chen, K., Zhou, Y., & Tang, Z. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.09.040 · Cited by 713 · see citing works
Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks
Cui, H., Radosavljević, V., Chou, F., Lin, T.-H., Nguyen, T., Huang, T.-K., Schneider, J., & Djuric, N. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793868 · Cited by 702 · see citing works
Self-citation'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 3× · see citing works
“One of the most critical design choices when assessing objective understanding is the selection of a suitable proxy task.”
BackgroundTowards Explainable Artificial Intelligence
Samek, W., & Müller, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_1 · Cited by 650 · see citing works
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590 · see citing works
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · see citing works
Towards Robust Interpretability with Self-Explaining Neural NetworksPreprint
Alvarez-Melis, D., & Alvarez-Melis, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07538 · Cited by 545 · see citing works
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 4× · see citing works
“Besides the medical domain, other works [3, 4, 19, 21] also invite subjects with particular professions such as engineers in a technology company.”
BackgroundMethodInfluentialUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1× · see citing works
“When deploying a proxy task, its difficulty should be gauged and monitored carefully.”
MethodMetrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504 · see citing works
Evaluating Fluency in Human–Robot Collaboration
Hoffman, G. (2019) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2019.2904558 · Cited by 462 · see citing works
Statistical and machine learning models in credit scoring: A systematic literature survey
Dastile, X., Çelik, T., & Potsane, M. M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106263 · Cited by 458 · see citing works
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
International standards for HCI and usability
Bevan, N. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2001.0483 · Cited by 430 · see citing works
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
Rajani, N. F., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1487 · Cited by 421 · see citing works
A systematic review and taxonomy of explanations in decision support and recommender systems
Nunes, I., & Jannach, D. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9195-0 · Cited by 416 · see citing works
Transparent, explainable, and accountable AI for robotics
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · Science Robotics · doi:10.1126/scirobotics.aan6080 · Cited by 413 · see citing works
Holzinger, A., Carrington, A., & Müller, H. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00636-z · Cited by 406 · see citing works
Evaluating XAI: A comparison of rule-based and example-based explanations
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393 · see citing works
Are Attention Check Questions a Threat to Scale Validity?
Kung, F. Y. H., Kwok, N., & Brown, D. J. (2017) · Applied Psychology · doi:10.1111/apps.12108 · Cited by 392 · see citing works
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388 · see citing works
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · see citing works
Making connections: Replicating and extending the utility value intervention in the classroom.
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2016) · Journal of Educational Psychology · doi:10.1037/edu0000146 · Cited by 370 · see citing works
Human-Centered Artificial Intelligence and Machine Learning
Riedl, M. O. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.117 · Cited by 352 · see citing works
Credit Risk Analysis Using Machine and Deep Learning Models
Addo, P. M., Guégan, D., & Hassani, B. K. (2018) · Risks · doi:10.3390/risks6020038 · Cited by 320 · see citing works
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · see citing works
Towards Conversational Search and Recommendation: System Ask, User Respond
Zhang, Y., Chen, X., Ai, Q., Yang, L., & Croft, W. B. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271776 · Cited by 319 · see citing works
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 10× · see citing works
“…of using explanations to using no explanations, or placebo (randomly generated) explanations [8, 25], about half of the papers validate that explanations have a positive impact on user trust [1, 8, 10, 13, 16, 25, 27, 28], while the other half cannot verify this hypothesis [3, 11, 12, 21, 22, 24].”
BackgroundMethodInfluentialExplaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286 · see citing works
Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274 · see citing works
Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
Hase, P., & Bansal, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.491 · Cited by 254 · see citing works
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · see citing works
Causal interpretability for machine learning-problems, methods and evaluation
Moraffah, R., Karami, M., Guo, R., Raglin, A., & Liu, H. (2020) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3400051.3400058 · Cited by 238 · see citing works
Hohman, F., Head, A., Caruana, R., DeLine, R. A., & Drucker, S. M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225 · see citing works
The effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1208 · Cited by 214 · see citing works
Automated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210 · see citing works
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207 · see citing works
Judge v Robot? Artificial Intelligence and Judicial Decision-Making
Sourdin, T. (2018) · University of New South Wales Law Journal · doi:10.53637/zgux2213 · Cited by 206 · see citing works
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204 · see citing works
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204 · see citing works
Bayesian Theory of Mind: Modeling Joint Belief-Desire Attribution
Baker, C. L., Saxe, R. R., & Tenenbaum, J. B. (2011) · Cited by 199 · see citing works
Towards a science of integrated AI and Robotics
Rajan, K., & Saffiotti, A. (2017) · Artificial Intelligence · doi:10.1016/j.artint.2017.03.003 · Cited by 180 · see citing works
Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2019) · arXiv · Cited by 176 · see citing works
Comparing Automatic and Human Evaluation of Local Explanations for Text Classification
Nguyen, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1097 · Cited by 175 · see citing works
Personalized explanations for hybrid recommender systems
Kouki, P., Schaffer, J., Pujara, J., O’Donovan, J., & Getoor, L. C. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302306 · Cited by 175 · see citing works
Evaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163 · see citing works
Panigutti, C., Perotti, A., & Pedreschi, D. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372855 · Cited by 162 · see citing works
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
Aher, G., Arriaga, R. I., & Kalai, A. T. (2022) · arXiv · doi:10.48550/arxiv.2208.10264 · Cited by 161 · see citing works
Artificial Intelligence in Human Resources Management: Challenges and a Path ForwardPreprint
Cappelli, P. H., Tambe, P. B., & Yakubovich, V. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3263878 · Cited by 160 · see citing works
Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300717 · Cited by 156 · see citing works
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302313 · Cited by 148 · see citing works
How model accuracy and explanation fidelity influence user trustPreprint
Papenmeier, A., Englebienne, G., & Seifert, C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.1907.12652 · Cited by 145 · see citing works
Panigutti, C., Beretta, A., Giannotti, F., & Pedreschi, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502104 · Cited by 145 · see citing works
Yang, S., Korayem, M., AlJadda, K., Grainger, T., & Natarajan, S. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.08.017 · Cited by 144 · see citing works
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144 · see citing works
A Primer for Conducting Experiments in Human–Robot Interaction
Hoffman, G., & Zhao, X. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3412374 · Cited by 138 · see citing works
Visual, textual or hybrid: the effect of user expertise on different explanations
Szymanski, M., Millecamp, M., & Verbert, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450662 · Cited by 131 · see citing works
Mapping Machine Learning Advances from HCI Research to Reveal Starting Places for Design Innovation
Yang, Q., Banović, N., & Zimmerman, J. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173704 · Cited by 130 · see citing works
Explainable Recommendation through Attentive Multi-View Learning
Gao, J., Wang, X., Wang, Y., & Xie, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013622 · Cited by 127 · see citing works
Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
Gajos, K. Z., & Mamykina, L. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511138 · Cited by 127 · see citing works
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123 · see citing works
Answering Questions about Charts and Generating Visual Explanations
Kim, D. H., Hoque, E., & Agrawala, M. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376467 · Cited by 122 · see citing works
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120 · see citing works
I can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119 · see citing works
Dhanorkar, S., Wolf, C. T., Qian, K., Xu, A., Popa, L., & Li, Y. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462131 · Cited by 119 · see citing works
Springer, A., & Whittaker, S. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302322 · Cited by 115 · cited 4× · see citing works
“Understanding subjective [7, 12, 13, 14, 16, 17, 22, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44] objective [12, 13, 22, 32, 35, 39, 40, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60] explanation model [21, 46, 49, 61, 62, 63, 64, 65]”
BackgroundInfluentialAn empirical study on the perceived fairness of realistic, imperfect machine learning models
Harrison, G., Hanson, J., Jacinto, C., Ramirez, J., & Ur, B. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372831 · Cited by 114 · see citing works
Smith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112 · see citing works
Colley, M., Eder, B., Rixen, J. O., & Rukzio, E. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445351 · Cited by 108 · see citing works
Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · cited 11× · see citing works
“[56] study the order of observing system weakness and strengths, which reveals that encountering weakness first results in a lower rate of us-”
BackgroundInfluentialDoes Explainable Artificial Intelligence Improve Human Decision-Making?
Alufaisan, Y., Marusich, L. R., Bakdash, J. Z., Zhou, Y., & Kantarcioglu, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16819 · Cited by 106 · see citing works
Enabling robots to communicate their objectives
Huang, S. H., Held, D., Abbeel, P., & Dragan, A. D. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9771-0 · Cited by 103 · see citing works
Anik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103 · cited 11× · see citing works
“Usability workload [3, 16, 21, 48, 66] helpfulness [13, 45, 46, 48, 56, 65, 67, 68] satisfaction [1, 6, 7, 16, 18, 19, 29, 47, 69, 70] undesired behavior detection [2, 24, 27, 38, 53, 57, 71, 72, 73, 74, 75, 76, 77, 78, 79] ease of use and others [1, 3, 13, 20, 21, 24, 30, 32, 37, 48, 65, 66, 71, 80, 81, 82, 83, 84, 8…”
BackgroundMethodInfluentialBell, A., Solano-Kamaiko, I., Nov, O., & Stoyanovich, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533090 · Cited by 103 · see citing works
Reframing Human-AI Collaboration for Generating Free-Text Explanations
Wiegreffe, S., Hessel, J., Swayamdipta, S., Riedl, M. O., & Choi, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.47 · Cited by 97 · see citing works
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations
Abdul, A., von der Weth, C., Kankanhalli, M., & Lim, B. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376615 · Cited by 96 · see citing works
Establishing Appropriate Trust via Critical States
Huang, S. H., Bhatia, K., Abbeel, P., & Dragan, A. D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593649 · Cited by 94 · see citing works
What can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92 · see citing works
Measuring Cognitive Load: Are There More Valid Alternatives to Likert Rating Scales?
Ouwehand, K., Kroef, A. V. D., Wong, J., & Paas, F. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.702616 · Cited by 90 · see citing works
Exploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers
Tsai, C.-H., You, Y., Gui, X., Kou, Y., & Carroll, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445101 · Cited by 86 · see citing works
What Are People Doing About XAI User Experience? A Survey on AI Explainability Research and Practice
Ferreira, J. J., & Monteiro, M. D. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49760-6_4 · Cited by 83 · see citing works
ExplAIn Yourself! Transparency for Positive UX in Autonomous Driving
Schneider, T., Hois, J., Rosenstein, A., Ghellal, S., Theofanou-Fülbier, D., & Gerlicher, A. R. S. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3446647 · Cited by 81 · see citing works
Grgić-Hlača, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · arXiv · doi:10.48550/arxiv.1802.09548 · Cited by 75 · see citing works
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., & Tan, C. (2021) · arXiv · doi:10.48550/arxiv.2112.11471 · Cited by 75 · see citing works
ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language ModelsPreprint
Wang, S., Zhao, Z., Ouyang, X., Wang, Q., & Shen, D. G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.07257 · Cited by 74 · see citing works
Crowdsourcing Perceptions of Fair Predictors for Machine Learning
van Berkel, N., Gonçalves, J., Hettiachchi, D., Wijenayake, S., Kelly, R., & Kostakos, V. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359130 · Cited by 73 · see citing works
HIVE: Evaluating the Human Interpretability of Visual Explanations
Kim, S. S. Y., Meister, N., Ramaswamy, V. V., Fong, R., & Russakovsky, O. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19775-8_17 · Cited by 72 · see citing works
Explaining A Black-box By Using A Deep Variational Information Bottleneck Approach
Bang, S., Xie, P., Lee, H., Wu, W., & Xing, E. P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17358 · Cited by 69 · see citing works
Bove, C., Aigrain, J., Lesot, M., Tijus, C., & Detyniecki, M. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511139 · Cited by 68 · see citing works
Towards relatable explainable AI with the perceptual process
Zhang, W., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501826 · Cited by 67 · see citing works
Explaining recommendations in an interactive hybrid social recommender
Tsai, C.-H., & Brusilovsky, P. L. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302318 · Cited by 60 · see citing works
Do explanations make VQA models more predictable to a human?
Chandrasekaran, A., Prabhu, V., Yadav, D., Chattopadhyay, P., & Parikh, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1128 · Cited by 59 · see citing works
Liao, M., Sundar, S. S., & Walther, J. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501936 · Cited by 59 · see citing works
Explaining Recommendations in E-Learning: Effects on Adolescents' Trust
Ooge, J., Kato, S., & Verbert, K. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511140 · Cited by 55 · see citing works
The effect of explanations and algorithmic accuracy on visual recommender systems of artistic images
Dominguez, V., Messina, P., Donoso-Guzmán, I., & Parra, D. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302274 · Cited by 53 · see citing works
Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human PerceptionsPreprint
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B. E., & Riedl, M. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.03729 · Cited by 52 · see citing works
Leveraging rationales to improve human task performance
Das, D., & Chernova, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377512 · Cited by 52 · see citing works
Interpretable machine learning
Molnar, C. (2020) · doi:10.58248/pn633 · Cited by 51 · see citing works
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., Drury, C. G., et al. (1998) · doi:10.21236/ada388787 · Cited by 49 · see citing works
Getting a CLUE: A Method for Explaining Uncertainty Estimates
Antorán, J., Bhatt, U., Adel, T., Weller, A., & Hernández-Lobato, J. M. (2021) · Apollo - University of Cambridge Repository · doi:10.17863/cam.69929 · Cited by 49 · see citing works
Regularizing Black-box Models for Improved InterpretabilityPreprint
Plumb, G., Al-Shedivat, M., Cabrera, Á. A., Perer, A., Xing, E. P., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06787 · Cited by 46 · see citing works
A deep learning model for behavioural credit scoring in banks
Alaraj, M., Abbod, M., Majdalawieh, M., & Jum’a, L. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06695-z · Cited by 46 · see citing works
Exploring Computational User Models for Agent Policy Summarization
Lage, I., Lifschitz, D., Doshi‐Velez, F., & Amir, O. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/194 · Cited by 45 · see citing works
Tsai, C.-H., & Brusilovsky, P. L. (2018) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3172944.3172959 · Cited by 44 · see citing works
Building Trust in Interactive Machine Learning via User Contributed Interpretable Rules
Guo, L., Daly, E., Alkan, Ö., Mattetti, M., Cornec, O., & Knijnenburg, B. P. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511111 · Cited by 44 · see citing works
Improving understandability of feature contributions in model-agnostic explainable AI tools
Hadash, S., Willemsen, M., Snijders, C., & Ijsselsteijn, W. (2022) · International Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517650 · Cited by 43 · see citing works
Machine learning in medical imaging
Suzuki, K., Zhou, L., & Wang, Q. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.10.020 · Cited by 42 · see citing works
Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User StudyPreprint
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Bianchi‐Berthouze, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00772 · Cited by 41 · see citing works
Evaluating the Interpretability of Generative Models by Interactive Reconstruction
Ross, A., Chen, N., Hang, E. Z., Glassman, E. L., & Doshi-Velez, F. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445296 · Cited by 40 · see citing works
A Taxonomy for Human Subject Evaluation of Black-Box Explanations in XAI.
Chromik, M., & Schuessler, M. (2020) · Cited by 39 · see citing works
Getting a CLUE: A Method for Explaining Uncertainty EstimatesPreprint
Antorán, J., Bhatt, U., Adel, T., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06848 · Cited by 38 · see citing works
Mitigating belief projection in explainable artificial intelligence via Bayesian teaching
Yang, S. C., Vong, W. K., Sojitra, R. B., Folke, T., & Shafto, P. (2021) · Scientific Reports · doi:10.1038/s41598-021-89267-4 · Cited by 37 · see citing works
Shang, R., Feng, K. J. K., & Shah, C. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533189 · Cited by 37 · see citing works
Liao, M., & Sundar, S. S. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445415 · Cited by 36 · see citing works
Explaining the News Feed Algorithm
Cotter, K., Cho, J., & Rader, E. J. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053114 · Cited by 34 · see citing works
Shen, H., & Huang, T.-H. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7477 · Cited by 32 · see citing works
Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable RecoursesPreprint
Rawal, K., & Lakkaraju, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07165 · Cited by 32 · see citing works
Nguyen, G., Kim, D., & Nguyen, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14944 · Cited by 32 · see citing works
A Consistent and Efficient Evaluation Strategy for Attribution MethodsPreprint
Rong, Y., Leemann, T., Borisov, V., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.00449 · Cited by 31 · see citing works
Self-citationExplain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations
Arora, S., Pruthi, D., Sadeh, N., Cohen, W. W., Lipton, Z. C., & Neubig, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20464 · Cited by 30 · see citing works
What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative PlayPreprint
Feng, S., & Boyd-Graber, J. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09648 · Cited by 28 · see citing works
Do Humans Prefer Debiased AI Algorithms? A Case Study in Career Recommendation
Wang, C., Wang, K., Bian, A., Islam, R., Keya, K. N., Foulds, J. R., & Pan, S. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511108 · Cited by 28 · see citing works
Exploring the Role of Local and Global Explanations in Recommender Systems
Radensky, M., Downey, D., Lo, K., Popović, Z., & Weld, D. S. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519795 · Cited by 27 · see citing works
Improving User Confidence in Concept Maps
Bras, P. L., Robb, D. A., Methven, T. S., Padilla, S., & Chantler, M. J. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173978 · Cited by 26 · see citing works
Towards Interpretable Natural Language Understanding with Explanations as Latent VariablesPreprint
Zhou, W., Hu, J., Zhang, H., Liang, X., Sun, M., Xiong, C., & Tang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.05268 · Cited by 26 · see citing works
Human Interpretation of Saliency-based Explanation Over Text
Schuff, H., Jacovi, A., Adel, H., Goldberg, Y., & Vu, N. T. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533127 · Cited by 26 · see citing works
Use-Case-Grounded Simulations for Explanation EvaluationPreprint
Chen, V., Johnson, N., Topin, N., Plumb, G., & Talwalkar, A. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2206.02256 · Cited by 26 · see citing works
Choi, S., Aizawa, K., & Sebe, N. (2018) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3172944.3173001 · Cited by 25 · see citing works
Explaining recommendations by means of aspect-based transparent memories
Donkers, T., Kleemann, T., & Ziegler, J. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377520 · Cited by 25 · see citing works
Arendt, D., Nur, N., Huang, Z., et al. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377514 · Cited by 24 · see citing works
Artificial Intelligence Methods in In-Cabin Use Cases: A Survey
Rong, Y., Han, C., Hellert, C., Loyal, A., & Kasneci, E. (2021) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2021.3050883 · Cited by 24 · see citing works
Self-citationSchoeffer, J., & Kuehl, N. (2021) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3462204.3481742 · Cited by 24 · see citing works
How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?
Balayn, A., Rikalo, N., Lofi, C., Yang, J., & Bozzon, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517474 · Cited by 24 · see citing works
Kuhl, U., Artelt, A., & Hammer, B. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3534630 · Cited by 24 · see citing works
Suresh, H., Lewis, K. M., Guttag, J. V., & Satyanarayan, A. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511160 · Cited by 22 · see citing works
Model Agnostic Multilevel ExplanationsPreprint
Ramamurthy, K. N., Vinzamuri, B., Zhang, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06005 · Cited by 21 · see citing works
Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation
Wang, Y., Venkatesh, P., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517551 · Cited by 21 · see citing works
XAlgo: a Design Probe of Explaining Algorithms’ Internal States via Question-Answering
Rebanal, J., Combitsis, J., Tang, Y.-Q., & Chen, X. '. (2021) · International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450676 · Cited by 20 · see citing works
Kuhl, U., Artelt, A., & Hammer, B. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1087929 · Cited by 20 · see citing works
Are Attention Check Questions a Threat to Scale Validity?Preprint
Kung, F. Y. H., Kwok, N., & Brown, D. (2017) · doi:10.31234/osf.io/9vrdn · Cited by 17 · see citing works
Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning
Zhang, W., Dimiccoli, M., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517522 · Cited by 15 · see citing works
Smith, A., Kumar, V. R., Boyd-Graber, J. L., Seppi, K. D., & Findlater, L. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377491 · Cited by 14 · see citing works
User Trust on an Explainable AI-based Medical Diagnosis Support SystemPreprint
Rong, Y., Castner, N., Bozkir, E., & Kasneci, E. (2022) · arXiv.org · doi:10.48550/arxiv.2204.12230 · Cited by 13 · see citing works
Self-citationPerception of Fairness in Group Music Recommender Systems
Htun, N. N., Lecluse, E., & Verbert, K. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450642 · Cited by 12 · see citing works
Li, T., Convertino, G., Tayi, R. K., et al. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302294 · Cited by 10 · see citing works
Inherently Explainable Reinforcement Learning in Natural LanguagePreprint
Peng, X., Riedl, M., & Ammanabrolu, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.08907 · Cited by 8 · see citing works
Visual correspondence-based explanations improve AI robustness and human-AI team accuracyPreprint
Nguyen, G., Taesiri, M. R., & Nguyen, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.00780 · Cited by 8 · see citing works
How Do People Rank Multiple Mutant Agents?
Dodge, J. E., Anderson, A. A., Olson, M., Dikkala, R., & Burnett, M. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511115 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- R. Paleja, M. Ghuy, N. Ranawaka Arachchige, R. Jensen, and M. Gombolay, "The utility of explain- able ai in ad hoc human-machine teaming," NeurIPS, 2021.
- J. Schoeffer, N. Kuehl, and Y. Machowski, ""there is not enough information": On the effects of ex- planations on perceptions of informational fairness and trustworthiness in automated decision-making," arXiv preprint arXiv:2205.05758, 2022. · arXiv
- K. Natesan Ramamurthy, B. Vinzamuri, Y. Zhang, and A. Dhurandhar, "Model agnostic multilevel explana- tions," NeurIPS, 2020.
- J. Borowski, R. S. Zimmermann, J. Schepers, R. Geirhos, T. S. A. Wallis, M. Bethge, and W. Brendel, "Exemplary natural images explain {cnn} activations better than state-of-the-art feature visualization," in ICLR, 2021.
- C.-K. Yeh, B. Kim, S. O. Arik, C.-L. Li, T. Pfister, and P. Ravikumar, "On completeness-aware concept- based explanations in deep neural networks," in NeurIPS, 2019.
- A. Ghorbani, J. Wexler, J. Y. Zou, and B. Kim, "Towards automatic concept-based explanations," in NeurIPS, 2019.
- F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker, "Gamut: A design probe to understand how data scientists understand machine learning models," in CHI, 2019.
- G. Nguyen, M. R. Taesiri, and A. Nguyen, "Vi- sual correspondence-based explanations improve ai robustness and human-ai team accuracy," NeurIPS, 2022.
- C. Panigutti, A. Perotti, and D. Pedreschi, "Doctor xai: an ontology-based approach to black-box sequential data classification explanations," in ACM FAccT, 2020.
- A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., "Language models are unsuper- vised multitask learners," OpenAI blog, vol. 1, no. 8, p. 9, 2019.
Cited by
Sorted by citations
What large language models know and what people think they know
Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L. W., & Smyth, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00976-7 · Cited by 209 · see citing works
On opportunities and challenges of large multimodal foundation models in education
Küchemann, S., Avila, K. E., Dinc, Y., Hortmann, C., Revenga, N., Ruf, V., Stausberg, N., Steinert, S., Fischer, F., Fischer, M., et al. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00301-w · Cited by 47 · see citing works
Self-citationSalimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41 · see citing works
“In addition to these measures of reliance, we also evaluated participants’ decision-making accuracy, demonstrating the human-AI team performance [11, 108].”
Building Appropriate Mental Models: What Users Know and Want to Know about an Agentic AI Chatbot
Brachman, M., Kunde, S., Miller, S., Fucs, A., Dempsey, S., Jabbour, J., & Geyer, W. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712071 · Cited by 18 · see citing works
“Our work builds upon and contributes to the fields of human-centered AI transparency and explainable AI [46, 47, 59, 80].”
Survey of Computerized Adaptive Testing: A Machine Learning Perspective
Zhuang, Y., Liu, Q., Bi, H., Huang, Z., Huang, W., Li, J., Yu, J., Liu, Z., Hu, Z., Hong, Y., et al. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3672850 · Cited by 9 · see citing works
“The assessment of intelligent agents, whether human or AI systems, is essential for ensuring that individuals are well-prepared to meet the demands of their respective roles [1], [2].”
Towards Explainable AI in Agentic Retrieval-Augmented Generation: A Systematic Review
Habib, A., Abdulmahmod, O. F., Raza, M., Gu, Y. H., Aydoğan, M., & Al-antari, M. A. (2025) · 2025 9th International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap68205.2025.11222281 · Cited by 3 · see citing works
Explainability in AI: Comparing Human-Like and System-Like Trust Repair Strategies
Konopka, B., & Wiesche, M. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10751-1 · Cited by 1 · see citing works
- Authors
- Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E.
- Affiliations
- Technical University of Munich (DE)
- University of Tübingen (DE)
- Rice University (US)
- ORCID
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2024-04
- Volume
- 46
- Issue
- 4
- Pages
- 2104-2122
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4388624604
- Cited by
- 278
- References
- 208
- Citations by year
- 2026: 120 · 2025: 98 · 2024: 57 · 2023: 3
- FWCI
- 5.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- explainable AI, model explanation, user study, human-centered AI, human-computer interaction, trust in AI, usability, human-AI collaboration, explanation evaluation, recommender systems
- MeSH terms
- Algorithms, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
