Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
Open accessAbstract
This is an integrative review that address the question, "What makes for a good explanation?" with reference to AI systems. Pertinent literatures are vast. Thus, this review is necessarily selective. That said, most of the key concepts and issues are expressed in this Report. The Report encapsulates the history of computer science efforts to create systems that explain and instruct (intelligent tutoring systems and expert systems). The Report expresses the explainability issues and challenges in modern AI, and presents capsule views of the leading psychological theories of explanation. Certain articles stand out by virtue of their particular relevance to XAI, and their methods, results, and key points are highlighted. It is recommended that AI/XAI researchers be encouraged to include in their research reports fuller details on their empirical or experimental methods, in the fashion of experimental psychology research reports: details on Participants, Instructions, Procedures, Tasks, Dependent Variables (operational definitions of the measures and metrics), Independent Variables (conditions), and Control Conditions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI explanation is important to communicate an AI output to a user and to realize human-AI collaboration for a decision-making task.
“researchers have discussed the importance of an AI explanation to communicate an AI output to a user [5, 20, 29] and realize human-AI collaboration [48] for a decision-making task [15, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [48]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287 · see citing works
Understanding Neural Networks Through Deep VisualizationPreprint
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,687 · see citing works
Opening the Black Box of Deep Neural Networks via InformationPreprint
Shwartz-Ziv, R., & Tishby, N. (2017) · arXiv.org · doi:10.48550/arxiv.1703.00810 · Cited by 1,674 · see citing works
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
A note on the evaluation of generative modelsPreprint
Theis, L., van den Oord, A., & Bethge, M. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.01844 · Cited by 1,345 · see citing works
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · see citing works
Deep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853 · see citing works
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845 · see citing works
Label-Embedding for Image Classification
Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2487986 · Cited by 785 · see citing works
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · see citing works
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699 · see citing works
Visualizing and Understanding Neural Models in NLP
Li, J., Chen, X., Hovy, E. H., & Jurafsky, D. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1082 · Cited by 673 · see citing works
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2015) · arXiv · doi:10.48550/arxiv.1502.05698 · Cited by 670 · see citing works
Learning to Reason: End-to-End Module Networks for Visual Question Answering
Hu, R., Andreas, J., Rohrbach, M., Darrell, T., & Saenko, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.93 · Cited by 523 · see citing works
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515 · see citing works
Accountability of AI Under the Law: The Role of Explanation
Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S. J., O'Brien, D., Scott, K., Shieber, S., Waldo, J., Weinberger, D., et al. (2017) · doi:10.2139/ssrn.3064761 · Cited by 457 · see citing works
What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
Doran, D., Schulz, S., & Besold, T. R. (2017) · arXiv · doi:10.48550/arxiv.1710.00794 · Cited by 333 · see citing works
A Survey on Bayesian Deep Learning
Wang, H., & Yeung, D.-Y. (2021) · ACM Computing Surveys · doi:10.1145/3409383 · Cited by 247 · see citing works
On the safety of machine learning: Cyber-physical systems, decision sciences, and data products
Varshney, K. R., & Alemzadeh, H. (2017) · Big Data · doi:10.1089/big.2016.0051 · Cited by 211 · see citing works
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153 · see citing works
Interpretable and Explorable Approximations of Black Box ModelsPreprint
Lakkaraju, H., Kamar, E. S., Caruana, R., & Leskovec, J. (2017) · arXiv (Cornell University) · Cited by 123 · see citing works
Usunier, N., Synnaeve, G., Lin, Z., & Chintala, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.02993 · Cited by 116 · see citing works
Hind, M., Wei, D., Campbell, M. S., Codella, N., Dhurandhar, A., Mojsilović, A., Ramamurthy, K. N., & Varshney, K. R. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314273 · Cited by 111 · see citing works
Kumar, D., Wong, A. K. C., & Taylor, G. W. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.215 · Cited by 76 · see citing works
Natural Language Generation enhances human decision-making with uncertain information
Gkatzia, D., Lemon, O., & Rieser, V. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2043 · Cited by 59 · see citing works
Rawlsian Fairness for Machine Learning.Preprint
Joseph, M., Kearns, M. J., Morgenstern, J., Neel, S., & Roth, A. (2016) · arXiv (Cornell University) · Cited by 52 · see citing works
On the Robustness of Most Probable ExplanationsPreprint
Chan, H., & Darwiche, A. Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6819 · Cited by 43 · see citing works
Explaining Classification Models Built on High-Dimensional Sparse DataPreprint
Moeyersoms, J., d'Alessandro, B., Provost, F. J., & Martens, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06280 · Cited by 15 · see citing works
Interpreting Visual Question Answering Models.Preprint
Goyal, Y., Mohapatra, A., Parikh, D., & Batra, D. (2016) · arXiv (Cornell University) · Cited by 14 · see citing works
Explainable artificial intelligence (XAI), the goodness criteria and the grasp-ability testPreprint
Kim, T. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09598 · Cited by 12 · see citing works
Toward a general, scaleable framework for Bayesian teaching with applications to topic modelsPreprint
Eaves, B., & Shafto, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07999 · Cited by 4 · see citing works
Explaining Violation Traces with Finite State Natural Language Generation Models
Pace, G. J., & Rosner, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10223-8_17 · Cited by 2 · see citing works
Flagged first, then by citations
3 of the 44 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [9]9. Bibliography 109 APPENDIX: Evaluations of XAI System Performance Using Human Participants 170 5 DARPA XAI Literature Review p.
Cited by
Sorted by citations
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · see citing works
Wang, B., Rau, P.-L. P., & Yuan, T. (2023) · Behaviour & Information Technology · doi:10.1080/0144929x.2022.2072768 · Cited by 1,036 · see citing works
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811 · see citing works
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783 · see citing works
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., van Gerven, M. A. J., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 486 · see citing works
A historical perspective of explainable Artificial Intelligence
Confalonieri, R., Çoba, L., Wagner, B., & Besold, T. R. (2020) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1391 · Cited by 385 · see citing works
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363 · see citing works
Bias and Discrimination in AI: A Cross-Disciplinary Perspective
Ferrer, X., Nuenen, T. V., Such, J. M., Cote, M., & Criado, N. (2021) · IEEE Technology and Society Magazine · doi:10.1109/mts.2021.3056293 · Cited by 308 · see citing works
“Procedural approaches will become more beneficial with further progress in explainable AI [6].”
BackgroundInfluentialHoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1096257 · Cited by 272 · see citing works
Self-citationFour principles of explainable artificial intelligence
Phillips, P. J., Hahn, C. A., Fontana, P. C., Yates, A. N., Greene, K., Broniatowski, D. A., & Przybocki, M. A. (2021) · doi:10.6028/nist.ir.8312 · Cited by 262 · see citing works
A Survey of Data-driven and Knowledge-aware eXplainable AI
Li, X.-H., Cao, C. C., Shi, Y., Bai, W., Gao, H., Qiu, L., Wang, C., Gao, Y., Zhang, S., Xue, X., et al. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2983930 · Cited by 224 · see citing works
Kenny, E. M., Ford, C., Quinn, M., & Keane, M. T. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103459 · Cited by 211 · see citing works
The role of shared mental models in human-AI teams: a theoretical review
Andrews, R. W., Lilly, J. M., Srivastava, D., & Feigh, K. M. (2022) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2022.2061080 · Cited by 199 · see citing works
Pataranutaporn, P., Liu, R., Finn, E., & Maes, P. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00720-7 · Cited by 208 · see citing works
Lukyanenko, R., Maaß, W., & Storey, V. C. (2022) · Electronic Markets · doi:10.1007/s12525-022-00605-4 · Cited by 194 · see citing works
Herrmann, T., & Pfeiffer, S. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01391-5 · Cited by 173 · see citing works
Explainable Artificial Intelligence by Genetic Programming: A Survey
Mei, Y., Chen, Q., Lensen, A., Xue, B., & Zhang, M. (2022) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3225509 · Cited by 170 · see citing works
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144 · see citing works
Four Principles of Explainable Artificial Intelligence
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020) · doi:10.6028/nist.ir.8312-draft · Cited by 136 · see citing works
Taylor, J. E. T., & Taylor, G. W. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01825-5 · Cited by 135 · see citing works
Levels of explainable artificial intelligence for human-aligned conversational explanations
Dazeley, R., Vamplew, P. W., Foale, C., Young, C., Aryal, S., & Cruz, F. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103525 · Cited by 122 · see citing works
A survey on artificial intelligence assurance
Batarseh, F. A., Freeman, L., & Huang, C. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00445-7 · Cited by 117 · see citing works
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · see citing works
Collaborating with technology-based autonomous agents
Seeber, I., Waizenegger, L., Seidel, S., Morana, S., Benbasat, I., & Lowry, P. (2020) · Internet Research · doi:10.1108/intr-12-2019-0503 · Cited by 68 · see citing works
Danry, V., Pataranutaporn, P., Mao, Y., & Maes, P. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580672 · Cited by 110 · see citing works
Explainable Artificial Intelligence Approaches: A SurveyPreprint
Islam, S. R., Eberle, W., Ghafoor, S., & Ahmed, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.09429 · Cited by 98 · see citing works
Brasse, J., Broder, H. R., Förster, M., Klier, M., & Sigler, I. (2023) · Electronic Markets · doi:10.1007/s12525-023-00644-5 · Cited by 101 · see citing works
Explainable Artificial Intelligence (XAI)
Ridley, M. (2022) · Information Technology and Libraries · doi:10.6017/ital.v41i2.14683 · Cited by 87 · see citing works
Examining the effect of explanation on satisfaction and trust in AI diagnostic systems
Alam, L., & Mueller, S. T. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01542-6 · Cited by 82 · see citing works
Self-citationRiveiro, M., & Thill, S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103507 · Cited by 81 · see citing works
Lee, M. H., & Chew, C. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610218 · Cited by 78 · see citing works
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 83 · see citing works
Barbado, A., Corcho, Ó., & Benjamins, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116100 · Cited by 77 · see citing works
Caldwell, S., Sweetser, P., O’Donnell, N., Knight, M. J., Aitchison, M., Gedeon, T., Johnson, D. M., Brereton, M. F., Gallagher, M. R., & Conroy, D. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3514257 · Cited by 82 · see citing works
The Effects of Explanations in Automated Essay Scoring Systems on Student Trust and Motivation
Conijn, R., Kahr, P. K., & Snijders, C. C. P. (2023) · Journal of Learning Analytics · doi:10.18608/jla.2023.7801 · Cited by 100 · see citing works
Scenario-Based Requirements Elicitation for User-Centric Explainable AI
Cirqueira, D., Nedbal, D., Helfert, M., & Bezbradica, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_18 · Cited by 75 · see citing works
Cultural Bias in Explainable AI Research: A Systematic Analysis
Peters, U., & Carman, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14888 · Cited by 73 · see citing works
March, C. (2021) · Journal of Economic Psychology · doi:10.1016/j.joep.2021.102426 · Cited by 72 · see citing works
Beyond explainability: justifiability and contestability of algorithmic decision systems
Henin, C., & Le Métayer, D. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01251-8 · Cited by 71 · see citing works
Interpretable machine learning for genomics
Watson, D. (2021) · Human Genetics · doi:10.1007/s00439-021-02387-9 · Cited by 78 · see citing works
Liao, Q. V., Zhang, Y., Luss, R., Doshi-Velez, F., & Dhurandhar, A. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21995 · Cited by 69 · see citing works
A Survey on Explainable Artificial Intelligence Techniques and Challenges
Hanif, A., Zhang, X., & Wood, S. (2021) · IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW) · doi:10.1109/edocw52865.2021.00036 · Cited by 69 · see citing works
Explainable AI In Education : Current Trends, Challenges, And Opportunities
Rachha, A., & Seyam, M. (2023) · SoutheastCon 2023 · doi:10.1109/southeastcon51012.2023.10115140 · Cited by 68 · see citing works
Toward Explainable Affective Computing: A Review
Cortiñas-Lorenzo, K., & Lacey, G. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3270027 · Cited by 78 · see citing works
Explainable Multivariate Time Series Classification
Hsieh, T.-Y., Wang, S., Sun, Y., & Honavar, V. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441815 · Cited by 62 · see citing works
An explanation framework and method for AI-based text emotion analysis and visualisation
Li, Y., Chan, J., Peko, G. M., & Sundaram, D. (2023) · Decision Support Systems · doi:10.1016/j.dss.2023.114121 · Cited by 58 · see citing works
Accessible Cultural Heritage through Explainable Artificial Intelligence
Díaz-Rodríguez, N., & Pisoni, G. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3386392.3399276 · Cited by 55 · see citing works
Li, T., Vorvoreanu, M., DeBellis, D., & Amershi, S. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511605 · Cited by 55 · see citing works
Survey of XAI in Digital Pathology
Pocevičiūtė, M., Eilertsen, G., & Lundström, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50402-1_4 · Cited by 51 · see citing works
Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble
Ali, W. B., Pesaranghader, A., Avram, R., Overtchouk, P., Perrin, N., Laffite, S., Cartier, R., Ibrahim, R., Modine, T., & Hussin, J. (2021) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.711401 · Cited by 56 · see citing works
An objective metric for Explainable AI: How and why to estimate the degree of explainability
Sovrano, F., & Vitali, F. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110866 · Cited by 54 · see citing works
Exploring Explainable AI in the Financial Sector: Perspectives of Banks and Supervisory Authorities
Kuiper, O. X., van den Berg, M., van der Burgt, J., & Leijnen, S. (2022) · Communications in computer and information science · doi:10.1007/978-3-030-93842-0_6 · Cited by 47 · see citing works
A Review of Machine Learning for Convective Weather
McGovern, A., Chase, R. J., Flora, M. L., Gagne, D. J., Lagerquist, R., Potvin, C. K., Snook, N., & Loken, E. D. (2023) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-22-0077.1 · Cited by 53 · see citing works
Explainable AI for Deep Learning Based Disease Detection
Kinger, S., & Kulkarni, V. Y. (2021) · International Conference on Contemporary Computing · doi:10.1145/3474124.3474154 · Cited by 44 · see citing works
Ebermann, C., Selisky, M., & Weibelzahl, S. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2126812 · Cited by 43 · see citing works
Explainable Student Agency Analytics
Saarela, M., Heilala, V., Jääskelä, P., Rantakaulio, A., & Kärkkäinen, T. (2021) · IEEE Access · doi:10.1109/access.2021.3116664 · Cited by 40 · see citing works
A Taxonomy for Human Subject Evaluation of Black-Box Explanations in XAI.
Chromik, M., & Schuessler, M. (2020) · Cited by 39 · see citing works
Towards Design Principles for User-Centric Explainable AI in Fraud Detection
Cirqueira, D., Helfert, M., & Bezbradica, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_2 · Cited by 39 · see citing works
Ridley, M. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24889 · Cited by 51 · see citing works
Explaining Naive Bayes and Other Linear Classifiers with Polynomial Time and DelayPreprint
Marques-Silva, J. P., Gerspacher, T., Cooper, M., Ignatiev, A., & Narodytska, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05803 · Cited by 37 · see citing works
Contrastive Explanations of Plans through Model Restrictions
Krarup, B., Krivić, S., Magazzeni, D., Long, D. P., Cashmore, M., & Smith, D. E. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12813 · Cited by 36 · see citing works
Bogatu, A., Wysocka, M., Wysocki, O., Butterworth, H., Pillai, M. R., Allison, J., Landers, D., Kilgour, E., Thistlethwaite, F. C., & Freitas, A. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104367 · Cited by 36 · see citing works
Tractable Explanations for d-DNNF Classifiers
Huang, X., Izza, Y., Ignatiev, A., Cooper, M., Asher, N. M., & Marques-Silva, J. P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20514 · Cited by 32 · see citing works
Cognitive Anthropomorphism of AI: How Humans and Computers Classify Images
Mueller, S. T. (2020) · Ergonomics in Design The Quarterly of Human Factors Applications · doi:10.1177/1064804620920870 · Cited by 32 · see citing works
Self-citationModel-based contrastive explanations for explainable planning
Krarup, B., Cashmore, M., Magazzeni, D., & Miller, T. (2019) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 29 · see citing works
Schulz, A., Hinder, F., & Hammer, B. (2020) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 27 · see citing works
Uslu, S., Kaur, D., Rivera, S. J., Arjan Durresi, A., Babbar‐Sebens, M., & Tilt, J. H. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106683 · Cited by 25 · see citing works
Perceived artificial intelligence literacy and employability of university students
Wut, T. M., Chan, E. A.-H., Wong, H. S.-M., & Chan, J. K. Y. (2025) · Education + Training · doi:10.1108/et-06-2024-0272 · Cited by 40 · see citing works
Barbado, A., & Oscar Corcho, B. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105222 · Cited by 30 · see citing works
Algorithmic fog of war: When lack of transparency violates the law of armed conflict
Kwik, J., & van Engers, T. (2021) · Journal of Future Robot Life · doi:10.3233/frl-200019 · Cited by 24 · see citing works
Increasing the Value of XAI for Users: A Psychological Perspective
Hoffman, R. R., Miller, T., Klein, G. A., Mueller, S. T., & Clancey, W. J. (2023) · Künstliche Intell. · doi:10.1007/s13218-023-00806-9 · Cited by 24 · see citing works
Self-citationA New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)
Speith, T., & Langer, M. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00061 · Cited by 23 · see citing works
Kuźba, M., & Biecek, P. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-65965-3_30 · Cited by 23 · see citing works
A Meta Survey of Quality Evaluation Criteria in Explanation Methods
Löfström, H., Hammar, K., & Johansson, U. (2022) · Lecture notes in business information processing · doi:10.1007/978-3-031-07481-3_7 · Cited by 23 · see citing works
A Theoretical Framework for AI Models Explainability with Application in Biomedicine
Rizzo, M., Veneri, A., Albarelli, A., Lucchese, C., Nobile, M. S., & Conati, C. (2023) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb56990.2023.10264877 · Cited by 23 · see citing works
Calibrated explanations: With uncertainty information and counterfactuals
Löfström, H., Löfström, T., Johansson, U., & Sönströd, C. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123154 · Cited by 21 · see citing works
Work with AI and Work for AI: Autonomous Vehicle Safety Drivers’ Lived Experiences
Chu, M., Zong, K., Shu, X., Gong, J., Lu, Z., Guo, K., Dai, X., & Zhou, G. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581564 · Cited by 21 · see citing works
New research paradigms and agenda of human factors science in the intelligence era
Wei, X. U., Gao, Z., & Ge, L. (2023) · Acta Psychologica Sinica · doi:10.3724/sp.j.1041.2024.00363 · Cited by 20 · see citing works
ContrXT: Generating contrastive explanations from any text classifier
Malandri, L., Mercorio, F., Mezzanzanica, M., Nobani, N., & Seveso, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.11.016 · Cited by 20 · see citing works
Defect Reduction Planning (Using TimeLIME)
Peng, K., & Menzies, T. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3062968 · Cited by 20 · see citing works
Schulz, A., Hinder, F., & Hammer, B. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/319 · Cited by 19 · see citing works
McGovern, A., Bostrom, A., Davis, P., Demuth, J. L., Ebert‐Uphoff, I., He, R., Hickey, J. J., Gagne, D. J., Snook, N., Stewart, J. Q., et al. (2022) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-21-0020.1 · Cited by 19 · see citing works
Psychology and AI at a Crossroads: How Might Complex Systems Explain Themselves?
Hoffman, R. R., Miller, T., & Clancey, W. J. (2022) · The American Journal of Psychology · doi:10.5406/19398298.135.4.01 · Cited by 19 · see citing works
Self-citationRoadmap of Designing Cognitive Metrics for Explainable Artificial Intelligence (XAI)
Hsiao, J. H.-W., Ngai, H. H. T., Qiu, L., Yang, Y., & Cao, C. C. (2021) · arXiv · doi:10.48550/arxiv.2108.01737
Keane, M. T., & Smyth, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58342-2_11 · Cited by 158 · see citing works
Introduction to Explainable AI and Intelligent Transportation
Gaur, L., & Sahoo, B. M. (2022) · Springer eBooks · doi:10.1007/978-3-031-09644-0_1 · Cited by 19 · see citing works
An Assurance Case Pattern for the Interpretability of Machine Learning in Safety-Critical Systems
Ward, F. R., & Habli, I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-55583-2_30 · Cited by 18 · see citing works
A call for more explainable AI in law enforcement
Matulionytė, R., & Hanif, A. (2021) · IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW) · doi:10.1109/edocw52865.2021.00035 · Cited by 18 · see citing works
Explainable AI in Finance: Addressing the Needs of Diverse Stakeholders
Wilson, C. H. (2025) · doi:10.56227/25.1.25 · Cited by 18 · see citing works
Color for Characters - Effects of Visual Explanations of AI on Trust and Observability
Schrills, T., & Franke, T. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-50334-5_8 · Cited by 17 · see citing works
Jung, J., Kang, S., Choi, J., El-Kareh, R. E., Lee, H., & Kim, H. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105943 · Cited by 19 · see citing works
Trustworthy AI: AI developers’ lens to implementation challenges and opportunities
Cousineau, C., Dara, R. A., & Chowdhury, A. H. (2024) · Data and Information Management · doi:10.1016/j.dim.2024.100082 · Cited by 17 · see citing works
Interpretable Diagnosis for Whole-Slide Melanoma Histology Images Using Convolutional Neural Network
Xie, P., Zuo, K., Liu, J., Chen, M., Zhao, S., Kang, W., & Li, F. (2021) · Journal of Healthcare Engineering · doi:10.1155/2021/8396438 · Cited by 17 · see citing works
RetractedExploring limits to tree planting as a natural climate solution
Rana, P., & Varshney, L. R. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.135566 · Cited by 16 · see citing works
Applications of explainable artificial intelligence in renewable energy research
Perr‐Sauer, J., Ugirumurera, J., Gafur, J., Bensen, E. A., Nguyen, T., Paul, S., Severino, J., Nag, A., Vijayshankar, S., Gasper, P. J., et al. (2025) · Energy Reports · doi:10.1016/j.egyr.2025.08.046 · Cited by 15 · see citing works
Yang, H., Xu, N., Lin, X., & Zhang, W. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100833 · Cited by 21 · see citing works
Evaluating machine-generated explanations: a “Scorecard” method for XAI measurement science
Hoffman, R. R., Jalaeian, M., Tate, C., Klein, G. A., & Mueller, S. T. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1114806 · Cited by 15 · see citing works
Self-citationA unified and practical user-centric framework for explainable artificial intelligence
Kaplan, S., Uusitalo, H. M. T., & Lensu, L. T. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111107 · Cited by 15 · see citing works
- Authors
- Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1902.01876
- OpenAlex
- W2913693384
- Cited by
- 215
- References
- 44
- Citations by year
- 2026: 4 · 2025: 13 · 2024: 25 · 2023: 43 · 2022: 35 · 2021: 57 · 2020: 32 · 2019: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- explainable AI, human-AI interaction, explanation quality, intelligent tutoring systems, expert systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
