A Survey of Data-driven and Knowledge-aware eXplainable AI
Abstract
We are witnessing a fast development of Artificial Intelligence (AI), but it becomes dramatically challenging to explain AI models in the past decade. “Explanation” has a flexible philosophical concept of “satisfying the subjective curiosity for causal information”, driving a wide spectrum of methods being invented and/or adapted from many aspects and communities, including machine learning, visual analytics, human-computer interaction and so on. Nevertheless, from the view-point of data and knowledge engineering (DKE), a best explaining practice that is cost-effective in terms of extra intelligence acquisition should exploit the causal information and explaining scenarios which is hidden richly in the data itself. In the past several years, there are plenty of works contributing in this line but there is a lack of a clear taxonomy and systematic review of the current effort. To this end, we propose this survey, reviewing and taxonomizing existing efforts from the view-point of DKE, summarizing their contribution, technical essence and comparative characteristics. Specifically, we categorize methods into data-driven methods where explanation comes from the task-related data, and knowledge-aware methods where extraneous knowledge is incorporated. Furthermore, in the light of practice, we provide survey of state-of-art evaluation metrics and deployed explanation applications in industrial practice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Explainability is a growing issue for financial institutions who seek to ensure that AI conclusions and procedures are easily comprehensible by bank officials.
“Furthermore, explainability is a growing issue for financial institutions who seek to ensure that AI conclusions and procedures are ''easily comprehensible'' by bank officials.”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
There are a myriad ways to investigate global interpretability.
“There are a myriad ways to investigate global interpretability [1], [54].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
AI technology has grown considerably over the years and has become a need for most humans in their everyday lives.
“Artificial Intelligence (AI) is a technology that has been growing considerably over the years. It has grown to the extent where it can beat humans in open challenges and has become a need for most humans in their everyday lives [1].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
There exist several surveys conducted that primarily focus on one or two aspects of TAI.
“There exist several surveys conducted that primarily focus on one or two aspects of TAI.”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
Post-hoc explainability methodologies establish lesser intricate surrogate models to emulate complicated black-box ML techniques.
“Post-hoc explainability methodologies establish lesser intricate surrogate models to emulate complicated black-box ML techniques [1], [50].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
Global model interpretability is the explanation through the architecture and characteristics of a network and illustrates a system using the most salient rules uncovered from the training set.
“Global model interpretability is the explanation through the architecture and characteristics of a network and illustrates a system using the most salient rules uncovered from the training set [1], [53].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
Extensive research can be impacted by explainability breakthroughs, for instance, by using them to develop and inject drugs or organize medical testing.
“Extensive research can indeed be impacted by these breakthroughs, for instance, by using them to develop and inject drugs or organize medical testing [1], [34].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
Feature extraction compresses the dimensionality of the problem and thereby undermines the veracity of the initial formulation.
“Nevertheless, such extraction puts emphasis on compressing the dimensionality of the problem, which undermines the veracity of the initial formulation [1], [57].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
The fundamental premise is to develop a simplified prototype system to emulate the complicated primary model by optimizing the loss function.
“By optimizing the loss function underneath, the fundamental premise is to develop a simplified prototype system to emulate the complicated primary model [1], [57]:”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [1]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Data mining: concepts and techniques
Sharma, R., Shah, K., Shirahatti, Y., Patel, S., Sharma, R., Shah, K., Patel, S., Sharma, R., & Han, J. (2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,655 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747 · see citing works
17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images
Boykov, Y. Y., & Jolly, M.-P. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937505 · Cited by 3,674 · see citing works
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326 · see citing works
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
Convolutional 2D Knowledge Graph Embeddings
Dettmers, T., Minervini, P., Stenetorp, P., & Riedel, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11573 · Cited by 2,486 · see citing works
Embedding Entities and Relations for Learning and Inference in Knowledge Bases
Yang, B., Yih, W.-T., He, X., Gao, J., & Deng, L. (2014) · arXiv · doi:10.48550/arxiv.1412.6575 · Cited by 2,252 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · see citing works
Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861 · see citing works
Adversarial Examples: Attacks and Defenses for Deep Learning
Yuan, X., He, P., Zhu, Q., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886017 · Cited by 1,733 · see citing works
Self-citationVisualizing the effects of predictor variables in black box supervised learning models
Apley, D. W., & Zhu, J. (2020) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12377 · Cited by 1,535 · see citing works
Network Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389 · see citing works
Techniques for interpretable machine learning
Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268 · see citing works
Survey and critique of techniques for extracting rules from trained artificial neural networks
Andrews, R., Diederich, J., & Tickle, A. B. (1995) · Knowledge-Based Systems · doi:10.1016/0950-7051(96)81920-4 · Cited by 1,225 · see citing works
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147 · see citing works
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · see citing works
GNNExplainer: Generating Explanations for Graph Neural Networks
Ying, R., Bourgeois, D., You, J., Zitnik, M., & Leskovec, J. (2019) · arXiv · doi:10.48550/arxiv.1903.03894 · Cited by 850 · see citing works
Interpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833 · see citing works
Explainable Reasoning over Knowledge Graphs for Recommendation
Wang, X., Wang, D., Xu, C., He, X., Cao, Y., & Chua, T.-S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015329 · Cited by 785 · see citing works
Interpretation of Neural Networks Is Fragile
Ghorbani, A., Abid, A., & Zou, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013681 · Cited by 765 · see citing works
How to Explain Individual Classification DecisionsPreprint
Baehrens, D., Schroeter, T. S., Stefan Harmeling, B., Kawanabe, M., Hansen, K., & Mueller, K.-R. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0912.1128 · Cited by 743 · see citing works
Hengstler, M., Enkel, E., & Duelli, S. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.014 · Cited by 661 · see citing works
The (Un)reliability of Saliency Methods
Kindermans, P.-J., Hooker, S., Adebayo, J., Alber, M., Schütt, K. T., Dähne, S., Erhan, D., & Kim, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_14 · Cited by 652 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Superhuman AI for multiplayer poker
Brown, N., & Sandholm, T. (2019) · Science · doi:10.1126/science.aay2400 · Cited by 592 · see citing works
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · see citing works
Inceptionism: Going Deeper into Neural Networks
Mordvintsev, A., Olah, C., & Tyka, M. (2015) · Cited by 569 · see citing works
Holographic embeddings of knowledge graphs
Nickel, M., Rosasco, L., & Poggio, T. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10314 · Cited by 559 · see citing works
<i>Why and why not</i> explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · see citing works
Li, O., Liu, H., Chen, C., & Rudin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11771 · Cited by 541 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation
Ai, Q., Azizi, V., Chen, X., & Zhang, Y. (2018) · Algorithms · doi:10.3390/a11090137 · Cited by 514 · see citing works
Visualizing Deep Convolutional Neural Networks Using Natural Pre-images
Mahendran, A., & Vedaldi, A. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0911-8 · Cited by 487 · see citing works
Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks
Huang, J., Zhao, W. X., Dou, H., Wen, J.-R., & Chang, E. Y. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210017 · Cited by 486 · see citing works
End-To-End Interpretable Neural Motion Planner
Zeng, W., Luo, W., Suo, S., Sadat, A., Yang, B., Casas, S., & Urtasun, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00886 · Cited by 406 · see citing works
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405 · see citing works
Interpretable Counterfactual Explanations Guided by Prototypes
Looveren, A. V., & Klaise, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86520-7_40 · Cited by 396 · see citing works
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · see citing works
Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention
Kim, J., & Canny, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.320 · Cited by 373 · see citing works
Leong, K. (2018) · International Journal of Innovation Management and Technology · doi:10.18178/ijimt.2018.9.2.791 · Cited by 360 · see citing works
Textual Explanations for Self-Driving Vehicles
Kim, J., Rohrbach, A., Darrell, T., Canny, J., & Akata, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_35 · Cited by 355 · see citing works
The Book of Why: The New Science of Cause and Effect†
Goldberg, L. R. (2019) · Quantitative finance (Print) · doi:10.1080/14697688.2019.1655928 · Cited by 354 · see citing works
Human-Centered Artificial Intelligence and Machine Learning
Riedl, M. O. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.117 · Cited by 352 · see citing works
Interpreting CNNs via Decision Trees
Zhang, Q., Yang, Y., Ma, H., & Wu, Y. N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00642 · Cited by 350 · see citing works
How should I explain? A comparison of different explanation types for recommender systems
Gedikli, F., Jannach, D., & Ge, M. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2013.12.007 · Cited by 337 · see citing works
Evaluating the effectiveness of explanations for recommender systems
Tintarev, N., & Masthoff, J. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9117-5 · Cited by 334 · see citing works
Using Sampling and Queries to Extract Rules from Trained Neural Networks
Craven, M. W., & SHAVLIK, J. W. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50013-1 · Cited by 312 · see citing works
Interpretable Machine Learning
Boehmke, B., & Greenwell, B. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9780367816377-16 · Cited by 271 · see citing works
Interpretable Basis Decomposition for Visual Explanation
Zhou, B., Sun, Y., Bau, D., & Torralba, A. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_8 · Cited by 270 · see citing works
Prototype selection for interpretable classification
Bien, J., & Tibshirani, R. J. (2011) · The Annals of Applied Statistics · doi:10.1214/11-aoas495 · Cited by 252 · see citing works
Explainable AI for Trees: From Local Explanations to Global UnderstandingPreprint
Lundberg, S., Erion, G., Chen, H., DeGrave, A. J., Prutkin, J. M., Nair, B. G., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04610 · Cited by 252 · see citing works
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · see citing works
Top-Down Neural Attention by Excitation Backprop
Zhang, J., Lin, Z., Brandt, J., Shen, X., & Sclaroff, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_33 · Cited by 246 · see citing works
DeepRED – Rule Extraction from Deep Neural Networks
Zilke, J. R., Mencía, E. L., & Janssen, F. W. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46307-0_29 · Cited by 236 · see citing works
Visualizing the Feature Importance for Black Box Models
Casalicchio, G., Molnar, C., & Bischl, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_40 · Cited by 236 · see citing works
Jointly Learning Explainable Rules for Recommendation with Knowledge Graph
Ma, W., Zhang, M., Cao, Y., Jin, W., Wang, C., Liu, Y., Ma, S., & Ren, X. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313607 · Cited by 231 · see citing works
Self-citationTell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225 · see citing works
Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations
Ehsan, U., Harrison, B., Chan, L., & Riedl, M. O. (2017) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278736 · Cited by 225 · see citing works
Toward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. J. (2008) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1378773.1378804 · Cited by 217 · see citing works
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215 · see citing works
Automated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210 · see citing works
Verifiable Reinforcement Learning via Policy ExtractionPreprint
Bastani, O., Pu, Y., & Solar-Lezama, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08328 · Cited by 188 · see citing works
Machine Learning Explainability in Finance: An Application to Default Risk AnalysisPreprint
Bracke, P., Datta, A., Jung, C., & Sen, S. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3435104 · Cited by 175 · see citing works
Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems
Augasta, M. G., & Kathirvalavakumar, T. (2011) · Neural Processing Letters · doi:10.1007/s11063-011-9207-8 · Cited by 174 · see citing works
Extracting symbolic rules from trained neural network ensembles
Zhou, Z., Jiang, Y. Y., & Chen, S. (2003) · doi:10.5555/859130.859132 · Cited by 169 · see citing works
Efficient Data Representation by Selecting Prototypes with Importance Weights
Gurumoorthy, K. S., Dhurandhar, A., Cecchi, G., & Aggarwal, C. C. (2019) · Proceedings · doi:10.1109/icdm.2019.00036 · Cited by 135 · see citing works
Extracting decision trees from trained neural networks
Krishnan, R., Sivakumar, G., & Bhattacharya, P. (1999) · Pattern Recognition · doi:10.1016/s0031-3203(98)00181-2 · Cited by 127 · see citing works
Interpretable and Steerable Sequence Learning via Prototypes
Ming, Y., Xu, P., Qu, H., & Ren, L. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330908 · Cited by 124 · see citing works
Rule extraction from neural networks via decision tree induction
Sato, M., & Tsukimoto, H. (2002) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2001.938448 · Cited by 120 · see citing works
Grounding Human-To-Vehicle Advice for Self-Driving Vehicles
Kim, J., Misu, T., Chen, Y.-T., Tawari, A., & Canny, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01084 · Cited by 117 · see citing works
Improving the Interpretability of Deep Neural Networks with Knowledge Distillation
Liu, X., Wang, X., & Matwin, S. (2018) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2018.00132 · Cited by 114 · see citing works
An Interpretable Model with Globally Consistent Explanations for Credit RiskPreprint
Chen, C.-F., Lin, K., Rudin, C., Shaposhnik, Y., Wang, S., & Wang, T. (2018) · arXiv.org · doi:10.48550/arxiv.1811.12615 · Cited by 108 · see citing works
Generative Counterfactual Introspection for Explainable Deep Learning
Liu, S., Kailkhura, B., Loveland, D., & Han, Y. (2019) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip45357.2019.8969491 · Cited by 106 · see citing works
Fisher, A., Rudin, C. D., & Dominici, F. (2018) · arXiv (Cornell University) · Cited by 86 · see citing works
Understanding Intra-Class Knowledge Inside CNNPreprint
Wei, D., Zhou, B., Torrabla, A., & Freeman, W. T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02379 · Cited by 79 · see citing works
Explanations and Expectations: Trust Building in Automated Vehicles
Haspiel, J., Du, N., Meyerson, J., Robert Jr., L. P., Tilbury, D., Yang, X. J., & Pradhan, A. K. (2018) · Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3173386.3177057 · Cited by 79 · see citing works
Robust Intelligence and Trust in Autonomous Systems
(2016) · doi:10.1007/978-1-4899-7668-0 · Cited by 78 · see citing works
Interpretable Credit Application Predictions With Counterfactual ExplanationsPreprint
McGrath, R., Costabello, L., Van, C. L., Sweeney, P., Kamiab, F., Shen, Z., & Lécué, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05245 · Cited by 78 · see citing works
Learning Global Additive Explanations for Neural Nets Using Model DistillationPreprint
Tan, S., Caruana, R., Hooker, G., Koch, P., & Gordo, A. (2018) · arXiv (Cornell University) · Cited by 77 · see citing works
Extracting decision trees from trained neural networks
Boz, O. (2002) · doi:10.1145/775047.775113 · Cited by 73 · see citing works
Evolving decision trees using oracle guides
Johansson, U., & Niklasson, L. (2009) · doi:10.1109/cidm.2009.4938655 · Cited by 63 · see citing works
Software transparency as a key requirement for self-driving cars
Cysneiros, L. M., Raffi, M., & Sampaio do Prado Leite, J. C. (2018) · 2018 IEEE 26th International Requirements Engineering Conference (RE) · doi:10.1109/re.2018.00-21 · Cited by 61 · see citing works
ProtoSteer: Steering Deep Sequence Model with Prototypes
Ming, Y., Xu, P., Cheng, F., Qu, H., & Ren, L. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934267 · Cited by 59 · see citing works
On Attribution of Recurrent Neural Network Predictions via Additive Decomposition
Du, M., Liu, N., Yang, F., Ji, S., & Hu, X. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313545 · Cited by 57 · see citing works
Measurable Counterfactual Local Explanations for Any ClassifierPreprint
White, A., & Garcez, A. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03020 · Cited by 53 · see citing works
Interpretable Credit Application Predictions With Counterfactual Explanations
McGrath, R., Costabello, L., Van, C. L., Sweeney, P., Kamiab, F., Shen, Z., & Lécué, F. (2018) · HAL · Cited by 49 · see citing works
Inverse Classification for Comparison-based Interpretability in Machine LearningPreprint
Laugel, T., Lesot, M., Marsala, C., Renard, X., & Detyniecki, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.08443 · Cited by 45 · see citing works
TreeView: Peeking into Deep Neural Networks Via Feature-Space PartitioningPreprint
Thiagarajan, J. J., Kailkhura, B., Sattigeri, P., & Ramamurthy, K. N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07429 · Cited by 44 · see citing works
Visual concepts and compositional voting
Wang, J., Zhang, Z., Xie, C., Zhou, Y., Premachandran, V., Zhu, J., Xie, L., & Yuille, A. (2018) · Annals of Mathematical Sciences and Applications · doi:10.4310/amsa.2018.v3.n1.a5 · Cited by 33 · see citing works
Efficient Concept Induction for Description Logics
Sarker, M. K., & Hitzler, P. (2019) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v33i01.33013036 · Cited by 32 · see citing works
EFFECTS OF AUGMENTED SITUATIONAL AWARENESS ON DRIVER TRUST IN SEMI-AUTONOMOUS VEHICLE OPERATION
Petersen, L., Tilbury, D., Robert, L., & Yang, X. J. (2017) · SAE Technical Paper Series · doi:10.4271/2024-01-3654 · Cited by 18 · see citing works
Interactively Transferring CNN Patterns for Part LocalizationPreprint
Zhang, Q., Cao, R., Zhang, S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01783 · Cited by 16 · see citing works
Self-citationTowards Interpretable R-CNN by Unfolding Latent StructuresPreprint
Wu, T., Sun, W., Li, X., Song, X., & Li, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05226 · Cited by 13 · see citing works
Self-citationExplaining and predicting abnormal expenses at large scale using knowledge graph based reasoning
Lecue, F., & Wu, J. (2017) · Journal of Web Semantics · doi:10.1016/j.websem.2017.05.003 · Cited by 9 · see citing works
Entity Suggestion with Conceptual Expanation
Zhang, Y., Xiao, Y., Hwang, S.-W., et al. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/593 · Cited by 8 · see citing works
Enabling Trust in Clinical Decision Support Recommendations through Semantics.
Seneviratne, O., Das, A. K., Chari, S., et al. (2019) · Cited by 4 · see citing works
Streaming Methods for Restricted Strongly Convex Functions with Applications to Prototype SelectionPreprint
Gurumoorthy, K. S., & Dhurandhar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08091 · Cited by 1 · see citing works
Visualization for explainable machine learning
Ming, Y. (2019) · doi:10.14711/thesis-991012786067703412 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Machlev, R., Heistrene, L., Perl, M., Levy, K. Y., Belikov, J., Mannor, S., & Levron, Y. (2022) · Energy and AI · doi:10.1016/j.egyai.2022.100169 · Cited by 552 · see citing works
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352 · see citing works
“become a need for most humans in their everyday lives [1].”
BackgroundMethodInfluentialKute, D. V., Pradhan, B., Shukla, N., & Alamri, A. (2021) · IEEE Access · doi:10.1109/access.2021.3086230 · Cited by 209 · see citing works
Nwakanma, C. I., Ahakonye, L. A. C., Njoku, J. N., Odirichukwu, J. C., Okolie, S. A., Uzondu, C., Ndubuisi Nweke, C. C., & Kim, D.-S. (2023) · Applied Sciences · doi:10.3390/app13031252 · Cited by 202 · see citing works
Enabling AI in Future Wireless Networks: A Data Life Cycle Perspective
Nguyen, D. C., Cheng, P., Ding, M., Lopez-Perez, D., Pathirana, P. N., Li, J., Seneviratne, A., Li, Y., & Poor, H. V. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3024783 · Cited by 160 · see citing works
A survey on deep transfer learning to edge computing for mitigating the COVID-19 pandemic
Sufian, A., Ghosh, A., Sadiq, A. S., & Smarandache, F. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2020.101830 · Cited by 156 · see citing works
A survey on explainable artificial intelligence for cybersecurity
Rjoub, G., Bentahar, J., Abdel Wahab, O., Mizouni, R., Song, A., Cohen, R., Otrok, H., & Mourad, A. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2023.3282740 · Cited by 158 · see citing works
Sheu, R.-K., & Pardeshi, M. S. (2022) · Sensors · doi:10.3390/s22208068 · Cited by 129 · see citing works
- Authors
- Li, X.-H., Cao, C. C., Shi, Y., Bai, W., Gao, H., Qiu, L., Wang, C., Gao, Y., Zhang, S., Xue, X., & Chen, L.
- Affiliations
- Huawei Technologies (China) (CN)
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2020
- Pages
- 1-1
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W3015177564
- Cited by
- 224
- References
- 120
- Citations by year
- 2026: 14 · 2025: 38 · 2024: 47 · 2023: 48 · 2022: 44 · 2021: 28 · 2020: 4 · 2019: 1
- FWCI
- 12.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Hong Kong RGC CRF (No. C6030-18F)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- explainable AI, causal information, explanation evaluation metrics, knowledge engineering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
