Explainable artificial intelligence: A survey
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
In the last decade, with availability of large datasets and more computing power, machine learning systems have achieved (super)human performance in a wide variety of tasks. Examples of this rapid development can be seen in image recognition, speech analysis, strategic game planning and many more. The problem with many state-of-the-art models is a lack of transparency and interpretability. The lack of thereof is a major drawback in many applications, e.g. healthcare and finance, where rationale for model's decision is a requirement for trust. In the light of these issues, explainable artificial intelligence (XAI) has become an area of interest in research community. This paper summarizes recent developments in XAI in supervised learning, starts a discussion on its connection with artificial general intelligence, and gives proposals for further research directions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Therefore, post-hoc explanation techniques are employed to explain existing models (Došilović et al., 2018; Lipton, 2018).”
From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Došilović et al., 2018
In the citing paper
“With the frequent utilization of complex Deep Learning (DL) architectures, it requires urgent attention to understand the inner workings and to get insights into the outcomes.This is a core motivation of Explainable AI (XAI) [15].”
From Explainable AI (XAI): Core Ideas, Techniques, and Solutions (2023) · cites this paper as [15]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2008) · Journal of Machine Learning Research · Cited by 33,051
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Explaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080
Deep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845
Datta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687
Visualizing and Understanding Neural Models in NLP
Li, J., Chen, X., Hovy, E., & Jurafsky, D. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1082 · Cited by 625
Comprehensible classification models: a position paper
Freitas, A. A. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2594473.2594475 · Cited by 572
Generating Visual Explanations
Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_1 · Cited by 534
Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies
Jin, Y., & Sendhoff, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.919172 · Cited by 455
Exploring the filter bubble: the effect of using recommender systems on content diversity
Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L., & Konstan, J. A. (2014) · Proceedings of the 23rd international conference on World wide web · doi:10.1145/2566486.2568012 · Cited by 452
Comprehensible credit scoring models using rule extraction from support vector machines
Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.04.051 · Cited by 448
Using sensitivity analysis and visualization techniques to open black box data mining models
Cortez, P., & Embrechts, M. J. (2013) · Information Sciences · doi:10.1016/j.ins.2012.10.039 · Cited by 443
Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.12.003 · Cited by 425
What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
Doran, D., Schulz, S., & Besold, T. R. (2017) · arXiv · doi:10.48550/arxiv.1710.00794 · Cited by 333
Auditing black-box models for indirect influence
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280
Prototype selection for interpretable classification
Bien, J., & Tibshirani, R. (2011) · The Annals of Applied Statistics · doi:10.1214/11-aoas495 · Cited by 245
Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
Besold, T. R., Garcez, A. D., Bader, S., Bowman, H., Domingos, P., Hitzler, P., Kühnberger, K., Lamb, L. C., Lowd, D., Lima, P. M. V., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03902 · Cited by 232
A critical review of multi-objective optimization in data mining: a position paper
Freitas, A. A. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1046456.1046467 · Cited by 193
Wolf, M. J., Miller, K., & Grodzinsky, F. S. (2017) · ACM SIGCAS Computers and Society · doi:10.1145/3144592.3144598 · Cited by 172
Study epidemiology of fake news
Kucharski, A. J. (2016) · Nature · doi:10.1038/540525a · Cited by 171
Performance of classification models from a user perspective
Martens, D., Vanthienen, J., Verbeke, W., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2011.01.013 · Cited by 153
Seeing the Forest Through the Trees: Learning a Comprehensible Model from an Ensemble
Assche, A. V., & Blockeel, H. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_39 · Cited by 94
An interpretable classification rule mining algorithm
Cano, A., Zafra, A., & Ventura, S. (2013) · Information Sciences · doi:10.1016/j.ins.2013.03.038 · Cited by 87
Linear and non-linear credit scoring by combining logistic regression and support vector machines
Gestel, T. V., Baesens, B., Dijcke, P. V., Suykens, J. A. K., & Garcia, J. P. S. (2005) · The Journal of Credit Risk · doi:10.21314/jcr.2005.025 · Cited by 77
ObamaNet: Photo-realistic lip-sync from text
Kumar, R., Sotelo, J., Kumar, K., de Brébisson, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01442 · Cited by 74
Israelsen, B. W., & Ahmed, N. R. (2019) · ACM Computing Surveys · doi:10.1145/3267338 · Cited by 74
Interpreting Black-Box Classifiers Using Instance-Level Visual Explanations
Tamagnini, P., Krause, J., Dasgupta, A., & Bertini, E. (2017) · Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics · doi:10.1145/3077257.3077260 · Cited by 69
User-oriented Assessment of Classification Model Understandability
Hiva, A., & Niklas, L. (2011) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/978-1-60750-754-3-11 · Cited by 52
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Schiele, B., Darrell, T. J., & Rohrbach, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.04757 · Cited by 51
Trading Interpretability for Accuracy: Oblique Treed Sparse Additive Models
Wang, J., Fujimaki, R., & Motohashi, Y. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783407 · Cited by 42
Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
Dong, Y., Su, H., Zhu, J. J., & Bao, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05493 · Cited by 39
Besold, T. R., & Kühnberger, K. (2015) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2015.09.003 · Cited by 34
Towards Interrogating Discriminative Machine Learning ModelsPreprint
Guo, W., Zhang, K., Lin, L., Huang, S., & Xing, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08564 · Cited by 10
Extracting symbolic rules from trained neural network ensembles
ZhouZhi-Hua, JiangYuan, & ChenShi-Fu (2003) · AI Communications · doi:10.5555/859130.859132 · Cited by 5
Research works printed but not linked: 5
- piltaver, 2014, Multi-objective learning of accurate and comprehensible classifiers – a case study
- edwards, 2017, Slave to the Algorithm? Why a ‘Right to an Explanation’ Is Probably Not the Remedy You Are Looking For, Social Science Research Network
- goodman, 2016, European ‘Union regulations on algorithmic decision-making and a ‘right to explanation ’
- lipton, 2016, The mythos of model interpretability
- crawford, 2016, Opinion | Artificial Intelligence's White Guy Problem, The New York Times
Cited by
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
“[64] proposed a general overview of the topic in their conference paper.”
BackgroundA Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
“Dosilovic et al. [70]. provided a general review of XAI in their paper’s closing remarks.”
Background
Metadata
- Authors
- Dosilovic, F. K., Brcic, M., & Hlupic, N.
- Venue
- 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 0210-0215
- OpenAlex
- W2809925683
- Cited by
- 1,147
- References
- 52
- Citations by year
- 2026: 48 · 2025: 128 · 2024: 186 · 2023: 249 · 2022: 209 · 2021: 179 · 2020: 108 · 2019: 39 · 2018: 1
- FWCI
- 33.39
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Convention on Information and Communication Technology, Electronics and Microelectronics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, model interpretability, model transparency, trust in machine learning, supervised learning, artificial general intelligence, medical applications, financial applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.