EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Open accessAbstract
Intelligent applications supported by Machine Learning have achieved remarkable performance rates for a wide range of tasks in many domains. However, understanding why a trained algorithm makes a particular decision remains problematic. Given the growing interest in the application of learning-based models, some concerns arise in the dealing with sensible environments, which may impact users’ lives. The complex nature of those models’ decision mechanisms makes them the so-called “black boxes,” in which the understanding of the logic behind automated decision-making processes by humans is not trivial. Furthermore, the reasoning that leads a model to provide a specific prediction can be more important than performance metrics, which introduces a trade-off between interpretability and model accuracy. Explaining intelligent computer decisions can be regarded as a way to justify their reliability and establish trust. In this sense, explanations are critical tools that verify predictions to discover errors and biases previously hidden within the models’ complex structures, opening up vast possibilities for more responsible applications. In this review, we provide theoretical foundations of Explainable Artificial Intelligence (XAI), clarifying diffuse definitions and identifying research objectives, challenges, and future research lines related to turning opaque machine learning outputs into more transparent decisions. We also present a careful overview of the state-of-the-art explainability approaches, with a particular analysis of methods based on feature importance, such as the well-known LIME and SHAP. As a result, we highlight practical applications of the successful use of XAI.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ortigossa et al. (2024) emphasize that incorporating practitioner perspectives is vital to ensure that explanations align with real-world needs and cognitive workflows.
“Ortigossa et al. (2024) emphasize that incorporating practitioner perspectives is vital to ensure that explanations align with real-world needs and cognitive workflows.”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Ortigossa et al. (2024)
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Artificial intelligence: a modern approach
Kumar, D. A., Balasubramanian, S., Byeon, D. H., & Manerkar, P. G. V. (1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 22,182 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101 · see citing works
17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces
Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aace8c · Cited by 5,115 · cited 1× · see citing works
“Lawhern et al. [254] developed a Convolutional Neural Network to classify electroencephalogram signals using DeepLIFT to provide feature importance and support the predictions’ confidence.”
MethodOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · see citing works
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443 · see citing works
A study of the behavior of several methods for balancing machine learning training data
Batista, G. E. A. P. A., Prati, R., & Monard, M. (2004) · SKDD · doi:10.1145/1007730.1007735 · Cited by 4,271 · see citing works
Harnad, S. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90087-6 · Cited by 4,235 · cited 1× · see citing works
“Furthermore, representational elements must be formalized manually instead of being acquired from data [58].”
BackgroundGrad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639 · see citing works
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · see citing works
Temporal Fusion Transformers for interpretable multi-horizon time series forecasting
Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2021) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.03.012 · Cited by 3,291 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“Despite the high levels of accuracy, the complex nature in which learning models operate reduces the transparency of their decision processes, turning them into so-called ‘‘black boxes’’ [18].”
BackgroundTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · see citing works
An Introduction to Statistical Learning: with Applications in R
James, G. M., Witten, D., Hastie, T. J., & Tibshirani, R. J. (2018) · doi:10.25334/q4ht55 · Cited by 2,913 · see citing works
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665 · see citing works
Automated detection of COVID-19 cases using deep neural networks with X-ray images
Öztürk, T., Talo, M., Yildirim, E. A., Baloğlu, U. B., Yıldırım, Ö., & Acharya, U. R. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103792 · Cited by 2,607 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values
Huang, J. Z. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009769707641 · Cited by 2,513 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
One Pixel Attack for Fooling Deep Neural Networks
Su, J., Vargas, D. V., & Sakurai, K. (2019) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2890858 · Cited by 2,406 · see citing works
Understanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400 · see citing works
Explaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286 · see citing works
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1× · see citing works
“The authors also developed two LIME extensions [208], [209], including an improved version with provides clearer textual explanations based on rules [210].”
BackgroundExplainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967 · see citing works
Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861 · see citing works
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · cited 1× · see citing works
“Some studies have contested the real need for explanations [250], [251].”
BackgroundChklovski, A., Parks, D. H., Woodcroft, B. J., & Tyson, G. W. (2023) · Nature Methods · doi:10.1038/s41592-023-01940-w · Cited by 1,705 · see citing works
Computational Complexity: A Modern Approach
Arora, S., & Barak, B. (2009) · doi:10.1017/cbo9780511804090 · Cited by 1,690 · see citing works
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328 · cited 1× · see citing works
“…several recent studies have reviewed XAI for applications in different domains where machine learning has been used, highlighting medicine, specifically cardiology [43], breast cancer diagnosis and surgery [44], medical image analysis [45], radiology [46], and healthcare [28], [47], [48].”
BackgroundEffective gene expression prediction from sequence by integrating long-range interactions
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y., Jumper, J., Kohli, P., & Kelley, D. R. (2021) · Nature Methods · doi:10.1038/s41592-021-01252-x · Cited by 1,326 · see citing works
Predictive learning via rule ensembles
Friedman, J. H., & Popescu, B. E. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas148 · Cited by 1,315 · see citing works
Deep learning: new computational modelling techniques for genomics
Eraslan, G., Avsec, Ž., Gagneur, J., & Theis, F. J. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0122-6 · Cited by 1,229 · see citing works
Exploring the Limits of Language ModelingPreprint
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y.-H. (2016) · arXiv.org · doi:10.48550/arxiv.1602.02410 · Cited by 1,202 · see citing works
Deep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,182 · see citing works
Aas, K., Jullum, M., & Løland, A. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103502 · Cited by 1,159 · see citing works
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147 · see citing works
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · see citing works
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 1,080 · see citing works
Parsa, A. B., Movahedi, A., Taghipour, H., Derrible, S., & Mohammadian, A. (2019) · Accident Analysis & Prevention · doi:10.1016/j.aap.2019.105405 · Cited by 1,065 · see citing works
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063 · see citing works
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047 · see citing works
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008 · see citing works
Visual interpretability for deep learning: a survey
Zhang, Q., & Zhu, S. C. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700808 · Cited by 975 · see citing works
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966 · see citing works
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · see citing works
Transformer Interpretability Beyond Attention Visualization
Chefer, H., Gur, S., & Wolf, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00084 · Cited by 903 · see citing works
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375830 · Cited by 900 · see citing works
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization
Jiang, P.-T., Zhang, C., Hou, Q., Cheng, M., & Wei, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3089943 · Cited by 899 · see citing works
Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets
Oh, Y., Park, S. J., & Ye, J. C. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2993291 · Cited by 897 · see citing works
Ahmed, I., Jeon, G., & Piccialli, F. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3146552 · Cited by 853 · see citing works
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works
Noise : a flaw in human judgment
Kahneman, D. J., Sibony, O., & Sunstein, C. R. (2021) · Cited by 749 · see citing works
Loh, H. W., Ooi, C. P., Seoni, S., Barua, P. D., Molinari, F., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107161 · Cited by 734 · see citing works
The structure and function of explanations.
Lombrozo, T. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.08.004 · Cited by 729 · see citing works
Chen, R. J., Lu, M. Y., Wang, J., Williamson, D. F. K., Rodig, S. J., Lindeman, N. I., & Mahmood, F. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3021387 · Cited by 729 · see citing works
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Computational intelligence: a logical approach
(1998) · Choice Reviews Online · doi:10.5860/choice.35-5701 · Cited by 726 · see citing works
iml: An R package for Interpretable Machine Learning
Molnar, C. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00786 · Cited by 720 · see citing works
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Not Just a Black Box: Learning Important Features Through Propagating Activation DifferencesPreprint
Shrikumar, A., Greenside, P., Shcherbina, A., & Kundaje, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01713 · Cited by 704 · see citing works
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699 · see citing works
Controlling the false discovery rate via knockoffs
Barber, R. F., & Candès, E. J. (2015) · The Annals of Statistics · doi:10.1214/15-aos1337 · Cited by 698 · see citing works
Analysis of regression in game theory approach
Lipovetsky, S., & Conklin, M. (2001) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.446 · Cited by 677 · see citing works
Jiang, D., Wu, Z., Hsieh, C.-Y., Chen, G., Liao, B., Wang, Z., Shen, C., Cao, D., Wu, J., & Hou, T. (2021) · Journal of Cheminformatics · doi:10.1186/s13321-020-00479-8 · Cited by 664 · see citing works
Captum: A unified and generic model interpretability library for PyTorch
Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.07896 · Cited by 658 · see citing works
Explainability in Graph Neural Networks: A Taxonomic Survey
Yuan, H., Yu, H., Gui, S., & Ji, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204236 · Cited by 638 · see citing works
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-rays
Brunese, L., Mercaldo, F., Reginelli, A., & Santone, A. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105608 · Cited by 623 · see citing works
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606 · see citing works
A Multiscale Visualization of Attention in the Transformer Model
Vig, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3007 · Cited by 600 · see citing works
Notions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598 · see citing works
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
Mathew, B., Saha, P., Yimam, S. M., Biemann, C., Goyal, P., & Mukherjee, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17745 · Cited by 585 · see citing works
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582 · see citing works
An Efficient Explanation of Individual Classifications using Game Theory
Štrumbelj, E., & Kononenko, I. (2010) · Repository of the University of Ljubljana (University of Ljubljana) · doi:10.5555/1756006.1756007 · Cited by 570 · see citing works
Accurate intelligible models with pairwise interactions
Lou, Y., Caruana, R., Gehrke, J., & Hooker, G. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487579 · Cited by 564 · see citing works
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · see citing works
ERASER: A Benchmark to Evaluate Rationalized NLP Models
DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., & Wallace, B. C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.408 · Cited by 561 · see citing works
Benign overfitting in linear regression
Bartlett, P. L., Long, P. M., Lugosi, G., & Tsigler, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907378117 · Cited by 554 · see citing works
Opening the black box: the promise and limitations of explainable machine learning in cardiology
Petch, J., Di, S., & Nelson, W. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.09.004 · Cited by 552 · see citing works
Intelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547 · see citing works
DARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546 · see citing works
Shefrin, H. (2002) · doi:10.1093/0195161211.001.0001 · Cited by 540 · see citing works
Machine learning-based prediction of COVID-19 diagnosis based on symptoms
Zoabi, Y., Deri-Rozov, S., & Shomron, N. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00372-6 · Cited by 530 · see citing works
Explainable machine learning in materials science
Zhong, X., Gallagher, B. J., Liu, S., Kailkhura, B., Hiszpanski, A. M., & Han, T. Y.-J. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00884-7 · Cited by 526 · see citing works
Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition
Singh, G., Memoli, F., & Carlsson, G. (2007) · The Eurographics Association · doi:10.2312/spbg/spbg07/091-100 · Cited by 514 · see citing works
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504 · see citing works
On the interpretability of artificial intelligence in radiology: challenges and opportunities
Reyes, M., Meier, R., Pereira, S., Silva, C. A., Dahlweid, F.-M., Tengg-Kobligk, H. V., Summers, R. M., & Wiest, R. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020190043 · Cited by 484 · see citing works
An Overview of Machine Learning
Carbonell, J., Michalski, R. S., & Mitchell, T. M. (1983) · Springer eBooks · doi:10.1007/978-3-662-12405-5_1 · Cited by 474 · see citing works
Interpretability of deep learning models: A survey of results
Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., Srivastava, M. B., Preece, A. D., Julier, S., Rao, R., et al. (2017) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397411 · Cited by 469 · see citing works
Attention is not ExplanationPreprint
Jain, S., & Wallace, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10186 · Cited by 468 · see citing works
Causal Interpretations of Black-Box Models
Zhao, Q., & Hastie, T. J. (2019) · Journal of Business and Economic Statistics · doi:10.1080/07350015.2019.1624293 · Cited by 465 · see citing works
Hastie, T. J., & Tibshirani, R. J. (2017) · Routledge eBooks · doi:10.1201/9780203753781 · Cited by 461 · see citing works
Biologically informed deep neural network for prostate cancer discovery
Elmarakeby, H. A., Hwang, J., Arafeh, R., Crowdis, J., Gang, S., Liu, D., AlDubayan, S. H., Salari, K., Kregel, S., Richter, C., et al. (2021) · Nature · doi:10.1038/s41586-021-03922-4 · Cited by 456 · see citing works
Advancing mathematics by guiding human intuition with AI
Davies, A., Veličković, P., Buesing, L., Blackwell, S., Zheng, D., Tomašev, N., Tanburn, R., Battaglia, P., Blundell, C., Juhász, A., et al. (2021) · Nature · doi:10.1038/s41586-021-04086-x · Cited by 454 · see citing works
A reference map of potential determinants for the human serum metabolome
Bar, N., Korem, T., Weissbrod, O., Zeevi, D., Rothschild, D., Leviatan, S., Kosower, N., Lotan‐Pompan, M., Weinberger, A., Roy, C. I. L., et al. (2020) · Nature · doi:10.1038/s41586-020-2896-2 · Cited by 440 · see citing works
A Benchmark for Interpretability Methods in Deep Neural NetworksPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10758 · Cited by 438 · see citing works
Multimodal deep learning for Alzheimer's disease dementia assessment
Qiu, S., Miller, M. I., Joshi, P. S., Lee, J. C., Xue, C., Ni, Y., Wang, Y., De Anda-Duran, I., Hwang, P. H., Cramer, J. A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-31037-5 · Cited by 420 · see citing works
Shifting machine learning for healthcare from development to deployment and from models to data
Zhang, A., Xing, L., Zou, J., & Wu, J. C. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00898-y · Cited by 420 · see citing works
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419 · see citing works
Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability
Zafar, M. R., & Khan, N. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030027 · Cited by 408 · see citing works
Metabolomic profiles predict individual multidisease outcomes
Buergel, T., Steinfeldt, J., Ruyoga, G., Pietzner, M., Bizzarri, D., Vojinovic, D., Upmeier zu Belzen, J., Loock, L., Kittner, P., Christmann, L., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01980-3 · Cited by 406 · see citing works
Interpretable Counterfactual Explanations Guided by Prototypes
Looveren, A. V., & Klaise, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86520-7_40 · Cited by 396 · see citing works
Weakly Supervised Deep Learning for COVID-19 Infection Detection and Classification From CT Images
Hu, S., Gao, Y., Niu, Z., Jiang, Y., Li, L., Xiao, X., Wang, M., Fang, E. F., Menpes-Smith, W., Xia, J., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3005510 · Cited by 387 · see citing works
Obtaining genetics insights from deep learning via explainable artificial intelligence
Novakovsky, G., Dexter, N., Libbrecht, M. W., Wasserman, W. W., & Mostafavi, S. (2022) · Nature Reviews Genetics · doi:10.1038/s41576-022-00532-2 · Cited by 387 · see citing works
Kim, Y., Denton, C., Hoang, L., & Rush, A. M. (2017) · arXiv · doi:10.48550/arxiv.1702.00887 · Cited by 381 · see citing works
A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods
Speith, T. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3534639 · Cited by 377 · see citing works
Stepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376 · see citing works
Applications of Explainable Artificial Intelligence in Diagnosis and Surgery
Zhang, Y., Weng, Y., & Lund, J. (2022) · Diagnostics · doi:10.3390/diagnostics12020237 · Cited by 368 · see citing works
Explanations in Autonomous Driving: A Survey
Omeiza, D., Webb, H., Jirotka, M., & Kunze, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3122865 · Cited by 355 · see citing works
Brito, L. C., Susto, G. A., Brito, J. N., & Duarte, M. A. V. (2021) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108105 · Cited by 355 · see citing works
Parameterized Explainer for Graph Neural Network
Luo, D., Cheng, W., Xu, D., Yu, W., Zong, B., Chen, H., & Zhang, X. (2020) · arXiv · doi:10.48550/arxiv.2011.04573 · Cited by 355 · see citing works
Benchmarking and survey of explanation methods for black box models
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00933-9 · Cited by 353 · see citing works
XGNN: Towards Model-Level Explanations of Graph Neural Networks
Yuan, H., Tang, J., Hu, X., & Ji, S. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403085 · Cited by 343 · see citing works
ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
Kahng, M., Andrews, P., Kalro, A., & Chau, D. H. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744718 · Cited by 336 · see citing works
Detecting formal thought disorder by deep contextualized word representations
Sarzyńska‐Wawer, J., Wawer, A., Pawlak, A., Szymanowska, J., Stefaniak, I., Jarkiewicz, M., & Okruszek, Ł. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.114135 · Cited by 336 · see citing works
What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
Doran, D., Schulz, S., & Besold, T. R. (2017) · arXiv · doi:10.48550/arxiv.1710.00794 · Cited by 333 · see citing works
Optimized CRISPR guide RNA design for two high-fidelity Cas9 variants by deep learning
Wang, D., Zhang, C., Wang, B., Li, B., Wang, Q., Liu, D., Wang, H., Zhou, Y., Shi, L., Lan, F., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-12281-8 · Cited by 331 · see citing works
InterpretML: A Unified Framework for Machine Learning Interpretability
Nori, H., Jenkins, S., Koch, P., & Caruana, R. (2019) · arXiv · doi:10.48550/arxiv.1909.09223 · Cited by 322 · see citing works
FACE: Feasible and Actionable Counterfactual Explanations
Poyiadzi, R., Sokol, K., Santos-Rodriguez, R., De Bie, T., & Flach, P. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375850 · Cited by 317 · see citing works
Extraction of organic chemistry grammar from unsupervised learning of chemical reactions
Schwaller, P., Hoover, B., Reymond, J., Strobelt, H., & Laino, T. (2021) · Science Advances · doi:10.1126/sciadv.abe4166 · Cited by 299 · see citing works
Explaining a series of models by propagating Shapley values
Chen, H., Lundberg, S., & Lee, S. (2022) · Nature Communications · doi:10.1038/s41467-022-31384-3 · Cited by 289 · see citing works
Mapping forest change using stacked generalization: An ensemble approach
Healey, S. P., Cohen, W. B., Yang, Z., Brewer, C. K., Brooks, E. B., Gorelick, N., Hernández, A. J., Huang, C., Hughes, M. J., Kennedy, R. E., et al. (2017) · Remote Sensing of Environment · doi:10.1016/j.rse.2017.09.029 · Cited by 288 · see citing works
Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning
Byrne, R. M. J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/876 · Cited by 285 · see citing works
Visualizing the Hidden Activity of Artificial Neural Networks
Rauber, P. E., Fadel, S. G., Falcao, A. X., & Telea, A. C. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598838 · Cited by 280 · see citing works
The Disagreement Problem in Explainable Machine Learning: A Practitioner's PerspectivePreprint
Krishna, S., Han, T., Gu, A., Jabbari, S., Wu, Z. S., & Lakkaraju, H. (2023) · Research Square · doi:10.21203/rs.3.rs-2963888/v1 · Cited by 279 · see citing works
The hidden assumptions behind counterfactual explanations and principal reasons
Barocas, S., Selbst, A., & Raghavan, M. (2019) · FAT* · doi:10.1145/3351095.3372830 · Cited by 273 · see citing works
(2018) · Project Muse (Johns Hopkins University) · Cited by 272 · see citing works
Interpretable Machine Learning
Boehmke, B., & Greenwell, B. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9780367816377-16 · Cited by 271 · see citing works
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Arya, V., Bellamy, R. K. E., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2019) · arXiv · doi:10.48550/arxiv.1909.03012 · Cited by 268 · see citing works
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K. E., Dickerson, J. P., & Shah, C. (2020) · arXiv · doi:10.48550/arxiv.2010.10596 · Cited by 267 · see citing works
Hooker, G., Mentch, L., & Zhou, S. (2021) · Statistics and Computing · doi:10.1007/s11222-021-10057-z · Cited by 264 · see citing works
Nonato, L. G., & Aupetit, M. J. M. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2846735 · Cited by 261 · see citing works
Self-citationExplanation Methods in Deep Learning: Users, Values, Concerns and Challenges
Ras, G., van Gerven, M., & Haselager, P. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_2 · Cited by 255 · see citing works
Reconciling deep learning with symbolic artificial intelligence: representing objects and relations
Garnelo, M., & Shanahan, M. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.12.010 · Cited by 255 · see citing works
Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
Hase, P., & Bansal, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.491 · Cited by 254 · see citing works
Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000016 · Cited by 251 · cited 10× · see citing works
“Amann et al. [16] claimed such an interpretability reduction might be related to the overhead induced by the large amount of information users are exposed to during the understanding process, even in transparent models.”
BackgroundMethodInfluentialMachine learning enables interpretable discovery of innovative polymers for gas separation membranes
Yang, J., Tao, L., He, J., McCutcheon, J. R., & Li, Y. (2022) · Science Advances · doi:10.1126/sciadv.abn9545 · Cited by 244 · see citing works
What does a platypus look like? Generating customized prompts for zero-shot image classification
Pratt, S., Covert, I., Liu, R., & Farhadi, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01438 · Cited by 240 · see citing works
Visualizing the Feature Importance for Black Box Models
Casalicchio, G., Molnar, C., & Bischl, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_40 · Cited by 236 · see citing works
Using neuroscience to develop artificial intelligence
Ullman, S. (2019) · Science · doi:10.1126/science.aau6595 · Cited by 231 · see citing works
Problems with Shapley-value-based explanations as feature importance measures
Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., & Friedler, S. (2020) · arXiv · doi:10.48550/arxiv.2002.11097 · Cited by 228 · cited 9× · see citing works
“…study, we consider the questions about the ‘‘human-centric’’ lack of significance in explainability and the impacts on comprehension’s mental models of users very pertinent – both as a consequence of the lack of explanations completeness and the information overload generated by XAI methods [218].”
BackgroundMethodInfluentialExplainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges
Zablocki, É., Ben-Younes, H., Pérez, P., & Cord, M. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01657-x · Cited by 227 · see citing works
Passi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225 · see citing works
Identifying who has long COVID in the USA: a machine learning approach using N3C data
Pfaff, E., Girvin, A. T., Bennett, T. D., Bhatia, A., Brooks, I. M., Deer, R., Dekermanjian, J., Jolley, S. E., Kahn, M. G., Kostka, K., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00048-6 · Cited by 223 · see citing works
OpenXAI: Towards a Transparent Evaluation of Model ExplanationsPreprint
Agarwal, C., Ley, D., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Puri, I., Zitnik, M., & Lakkaraju, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.11104 · Cited by 223 · see citing works
Feature relevance quantification in explainable AI: A causal problem
Janzing, D., Minorics, L., & Blöbaum, P. (2019) · arXiv · doi:10.48550/arxiv.1910.13413 · Cited by 207 · see citing works
Visualizing the Impact of Feature Attribution Baselines
Sturmfels, P., Lundberg, S., & Lee, S. (2020) · Distill · doi:10.23915/distill.00022 · Cited by 206 · see citing works
Emergent and Predictable Memorization in Large Language ModelsPreprint
Biderman, S., Prashanth, U. S., Sutawika, L., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2304.11158 · Cited by 205 · see citing works
Conceptual complexity and the bias/variance tradeoff
Briscoe, E., & Feldman, J. (2010) · Cognition · doi:10.1016/j.cognition.2010.10.004 · Cited by 169 · see citing works
Study epidemiology of fake news
Kucharski, A. J. (2016) · Nature · doi:10.1038/540525a · Cited by 161 · see citing works
Towards Best Practice in Explaining Neural Network Decisions with LRP
Kohlbrenner, M., Bauer, A., Nakajima, S., Binder, A., Samek, W., & Lapuschkin, S. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206975 · Cited by 159 · see citing works
Designing and implementing transparency for real time inspection of autonomous robots
Theodorou, A. A., Wortham, R. H., & Bryson, J. J. (2017) · Connection Science · doi:10.1080/09540091.2017.1310182 · Cited by 154 · see citing works
Explaining Models by Propagating Shapley Values of Local Components
Chen, H., Lundberg, S., & Lee, S. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-53352-6_24 · Cited by 139 · see citing works
Machine Learning for Scent: Learning Generalizable Perceptual Representations of Small MoleculesPreprint
Sánchez-Lengeling, B., Wei, J. N., Lee, B. K., Gerkin, R. C., Aspuru‐Guzik, A., & Wiltschko, A. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10685 · Cited by 133 · see citing works
Interpretable Deep Learning in Drug Discovery
Preuer, K., Klambauer, G., Rippmann, F., Hochreiter, S., & Unterthiner, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_18 · Cited by 131 · see citing works
Interpretability of Fuzzy Systems: Current Research Trends and Prospects
Alonso, J. M., Castiello, C., & Mencar, C. (2015) · Springer eBooks · doi:10.1007/978-3-662-43505-2_14 · Cited by 126 · see citing works
Hedström, A., Weber, L., Bareeva, D., Krakowczyk, D., Motzkus, F., Samek, W., Lapuschkin, S., & Höhne, M. M. C. (2022) · arXiv · doi:10.48550/arxiv.2202.06861 · Cited by 116 · see citing works
Three Methods to Share Joint Costs or Surplus
Friedman, E., & Moulin, H. J. (1999) · Journal of Economic Theory · doi:10.1006/jeth.1999.2534 · Cited by 110 · see citing works
Benchmarking Attribution Methods with Relative Feature ImportancePreprint
Yang, M., & Kim, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09701 · Cited by 108 · see citing works
A Survey of the State of Explainable AI for Natural Language ProcessingPreprint
Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00711 · Cited by 108 · see citing works
Are explanations always important?: a study of deployed, low-cost intelligent interactive systems
Bunt, A., Lount, M., & Lauzon, C. (2012) · Proceedings of the 2012 ACM international conference on Intelligent User Interfaces · doi:10.1145/2166966.2166996 · Cited by 106 · see citing works
Using attribution to decode binding mechanism in neural network models for chemistry
McCloskey, K., Taly, A., Monti, F., Brenner, M. P., & Colwell, L. J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1820657116 · Cited by 105 · see citing works
Machine Learning in Transportation Data Analytics
Bhavsar, P., Safro, I., Bouaynaya, N. C., et al. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809715-1.00012-2 · Cited by 101 · see citing works
Meske, C., & Bunde, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_4 · Cited by 99 · see citing works
To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
Amparore, E. G., Perotti, A., & Bajardi, P. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.479 · Cited by 92 · see citing works
Simple Rules for Complex DecisionsPreprint
Jung, J., Concannon, C., Shroff, R., Goel, S., & Goldstein, D. G. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2919024 · Cited by 91 · see citing works
Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets
Sridhar, C. (2011) · Journal of Cancer Research and Therapeutics · doi:10.4103/0973-1482.80442 · Cited by 85 · see citing works
Contrastive Explanations for Model Interpretability
Jacovi, A., Swayamdipta, S., Ravfogel, S., Elazar, Y., Choi, Y., & Goldberg, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.120 · Cited by 83 · see citing works
Alibi Explain: Algorithms for Explaining Machine Learning Models
Klaise, J., Looveren, A. V., Vacanti, G., & Coca, A. (2021) · Journal of Machine Learning Research · Cited by 78 · see citing works
Turner, R. (2016) · Machine learning for signal processing ... · doi:10.1109/mlsp.2016.7738872 · Cited by 77 · see citing works
Understanding the bias-variance tradeoff
Fortmann‐Roe, S. (2012) · Cited by 63 · see citing works
A Framework for Considering Comprehensibility in Modeling
Gleicher, M. L. (2016) · Big Data · doi:10.1089/big.2016.0007 · Cited by 63 · see citing works
Model combination for credit risk assessment: A stacked generalization approach
Doumpos, M., & Zopounidis, C. (2006) · Annals of Operations Research · doi:10.1007/s10479-006-0120-x · Cited by 61 · see citing works
Goldreich, O. (2008) · ACM SIGACT News · doi:10.1145/1412700.1412710 · Cited by 61 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Democratization of Artificial Intelligence: Theoretical Framework
Costa, C. J., Aparicio, M., Aparicio, S., & Aparicio, J. T. (2024) · Applied Sciences · doi:10.3390/app14188236 · Cited by 44 · see citing works
Explainable AI: enhancing decision-making in the detection of cyber threats
Prasad, P. W. C., Sayeed, M. S., Nguyen, D.-M., Hutabarat, D. P., & Mohiuddin, G. M. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1762332 · Cited by 16 · see citing works
- Authors
- Ortigossa, E. S., Gonçalves, T., & Nonato, L. G.
- Affiliations
- Universidade de São Paulo (BR)
- Institute of Mathematics and Computer Science, University of São Paulo (ICMC-USP), São Carlos, Brazil
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 80799-80846
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4399449888
- Cited by
- 124
- References
- 245
- Citations by year
- 2026: 66 · 2025: 52 · 2024: 6
- FWCI
- 18.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior–Brasil (CAPES)–Finance Code 001
- São Paulo Research Foundation
- National Council for Scientific and Technological Development
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Fundação de Amparo à Pesquisa do Estado de São Paulo (2018/24516-0, 2022/09091-8)
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (307184/2021-8)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, model interpretability, black-box models, automated decision-making, trust in machine learning, feature importance, lime, SHAP, model transparency, algorithmic bias detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review