The DOI matches its Crossref record; 245 of the 245 references it cites were found in a registry or an index.
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Open accessAbstract
Intelligent applications supported by Machine Learning have achieved remarkable performance rates for a wide range of tasks in many domains. However, understanding why a trained algorithm makes a particular decision remains problematic. Given the growing interest in the application of learning-based models, some concerns arise in the dealing with sensible environments, which may impact users’ lives. The complex nature of those models’ decision mechanisms makes them the so-called “black boxes,” in which the understanding of the logic behind automated decision-making processes by humans is not trivial. Furthermore, the reasoning that leads a model to provide a specific prediction can be more important than performance metrics, which introduces a trade-off between interpretability and model accuracy. Explaining intelligent computer decisions can be regarded as a way to justify their reliability and establish trust. In this sense, explanations are critical tools that verify predictions to discover errors and biases previously hidden within the models’ complex structures, opening up vast possibilities for more responsible applications. In this review, we provide theoretical foundations of Explainable Artificial Intelligence (XAI), clarifying diffuse definitions and identifying research objectives, challenges, and future research lines related to turning opaque machine learning outputs into more transparent decisions. We also present a careful overview of the state-of-the-art explainability approaches, with a particular analysis of methods based on feature importance, such as the well-known LIME and SHAP. As a result, we highlight practical applications of the successful use of XAI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Ortigossa et al. (2024) emphasize that incorporating practitioner perspectives is vital to ensure that explanations align with real-world needs and cognitive workflows.”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Ortigossa et al. (2024)
Citations
Cites
Items 201–245 of 245
Model extraction from counterfactual explanationsPreprint
Aïvodji, U., Bolot, A., & Gambs, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01884 · Cited by 61
Goldreich, O. (2008) · ACM SIGACT News · doi:10.1145/1412700.1412710 · Cited by 61
Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction InvariancePreprint
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05817 · Cited by 60
A generalized Mahalanobis distance for mixed data
de Leon, A. R., & Carriere, K. C. (2003) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2003.08.006 · Cited by 47
Characterizing the Decision Boundary of Deep Neural Networks
Karimi, H., Derr, T., & Tang, J. (2019) · arXiv · doi:10.48550/arxiv.1912.11460 · Cited by 46
TreeView: Peeking into Deep Neural Networks Via Feature-Space PartitioningPreprint
Thiagarajan, J. J., Kailkhura, B., Sattigeri, P., & Ramamurthy, K. N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07429 · Cited by 44
When Comparing to Ground Truth is Wrong
Faber, L., Moghaddam, A. K., & Wattenhofer, R. P. (2021) · doi:10.1145/3447548.3467283 · Cited by 41
Arya, V., Bellamy, R., Chen, P., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2020) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3430984.3430987 · Cited by 37
On the Decision Boundary of Deep Neural Networks
Li, Y., Ding, L., & Gao, X. (2018) · arXiv · doi:10.48550/arxiv.1808.05385 · Cited by 31
Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary
Guan, S., & Loew, M. H. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00025 · Cited by 31
Neuron to Graph: Interpreting Language Model Neurons at ScalePreprint
Foote, A., Nanda, N., Kran, E., Konstas, I., Cohen, S. B., & Barez, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.19911 · Cited by 31
Explainable Artificial Intelligence for the Remaining Useful Life Prognosis of the Turbofan Engines
Hong, C. W., Lee, C., Lee, K., Ko, M.-S., & Hur, K. (2020) · IEEE International Conference on Knowledge Innovation and Invention (ICKII) · doi:10.1109/ickii50300.2020.9318912 · Cited by 29
OmniXAI: A Library for Explainable AIPreprint
Yang, W., Le, H., Laud, T., Savarese, S., & Hoi, S. C. H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.01612 · Cited by 29
Learning to Explain: Generating Stable Explanations Fast
Situ, X., Ingrid Zukerman, A., Paris, C. L., Maruf, S., & Haffari, G. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.415 · Cited by 27
A survey on methods and metrics for the assessment of explainability under the proposed AI Act
Sovrano, F., Sapienza, S., Palmirani, M., & Vitali, F. (2021) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia210342 · Cited by 27
Reconstructing Training Data from Trained Neural NetworksPreprint
Haim, N., Vardi, G., Yehudai, G., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.07758 · Cited by 27
Synthetic Benchmarks for Scientific Research in Explainable Machine LearningPreprint
Liu, Y., Khandagale, S., White, C., & Neiswanger, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12543 · Cited by 25
GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural NetworksPreprint
Amara, K., Ying, R., Zhang, Z., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.09677 · Cited by 25
A Network Science Perspective of Graph Convolutional Networks: A Survey
Jia, M., Gabryś, B., & Musiał, K. (2023) · IEEE Access · doi:10.1109/access.2023.3268797 · Cited by 24
mTSeer: Interactive Visual Exploration of Models on Multivariate Time-series Forecast
Xu, K., Yuan, J., Wang, Y., Silva, C. T., & Bertini, E. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445083 · Cited by 20
Decision Boundary Visualization for Counterfactual Reasoning
Sohns, J., Garth, C., & Leitte, H. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14650 · Cited by 17
Legal Knowledge and Information Systems
Saarbrücken, J. (. 2., Francesconi, E., Borges, G., & Sorge, C. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia362 · Cited by 16
Predicting Stroke Lesion and Clinical Outcome with Random Forests
Maier, O., & Heinz Handels, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-55524-9_21 · Cited by 16
Chan, G. Y.-Y., Bertini, E., Nonato, L. G., Barr, B., & Silva, C. T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10614 · Cited by 16
Self-citationOn Baselines for Local Feature AttributionsPreprint
Haug, J., Zürn, S., El-Jiz, P., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00905 · Cited by 16
Getting over High-Dimensionality: How Multidimensional Projection Methods Can Assist Data Science
Ortigossa, E. S., Dias, F. F., & do Nascimento, D. C. (2022) · Applied Sciences · doi:10.3390/app12136799 · Cited by 16
Self-citationMapping the Potential of Explainable AI for Fairness Along the AI LifecyclePreprint
Deck, L., Schomäcker, A., Speith, T., Schöffer, J., Kästner, L., & Kühl, N. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.18736 · Cited by 16
Exploring Neural Network Hidden Layer Activity Using Vector Fields
Cantareira, G. D., Etemad, E., & Paulovich, F. V. (2020) · Information · doi:10.3390/info11090426 · Cited by 15
FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., & Chau, D. H. (2019) · 2019 IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast47406.2019.8986948 · Cited by 14
Yuan, J., Chan, G. Y.-Y., Barr, B., Overton, K., Rees, K., Nonato, L. G., Bertini, E., & Silva, C. T. (2022) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2022.3199727 · Cited by 14
Self-citationBASED-XAI: Breaking Ablation Studies Down for Explainable Artificial IntelligencePreprint
Hameed, I., Sharpe, S., Barcklow, D., Au-Yeung, J. C., Verma, S., Huang, J., Barr, B., & Bruss, C. B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.05566 · Cited by 12
DeepDecipher: Accessing and Investigating Neuron Activation in Large Language ModelsPreprint
Garde, A., Kran, E., & Barez, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.01870 · Cited by 10
Accurately measuring model prediction error
Fortmann‐Roe, S. (2012) · Cited by 9
SUBPLEX: Towards a Better Understanding of Black Box Model Explanations at the Subpopulation LevelPreprint
Yuan, J., Chan, G. Y.-Y., Barr, B., Overton, K., Rees, K., Nonato, L. G., Bertini, E., & Silva, C. T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10609 · Cited by 9
Self-citationMining Pareto-optimal counterfactual antecedents with a branch-and-bound model-agnostic algorithm
Raimundo, M. M., Nonato, L. G., & Poco, J. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00906-4 · Cited by 9
Self-citationAlexandrina, E. C., Ortigossa, E. S., Lui, E. S., et al. (2019) · Atmospheric Pollution Research · doi:10.1016/j.apr.2019.03.001 · Cited by 8
Self-citationIntegrating Prior Knowledge in Post-hoc Explanations
Jeyasothy, A., Laugel, T., Lesot, M., et al. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-08974-9_56 · Cited by 8
T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients
Ortigossa, E. S., Dias, F. F., Barr, B., Silva, C. T., & Nonato, L. G. (2025) · IEEE Intelligent Systems · doi:10.1109/mis.2025.3564330 · Cited by 8
Self-citationEfficient SVDD sampling with approximation guarantees for the decision boundary
Englhardt, A., Trittenbach, H., Kottke, D., Sick, B., & Böhm, K. (2022) · Machine Learning · doi:10.1007/s10994-022-06149-0 · Cited by 5
Neural Greedy Pursuit for Feature Selection
Das, S., Javid, A. M., Gohain, P. B., Eldar, Y. C., & Chatterjee, S. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892946 · Cited by 4
A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and CountermeasuresPreprint
Nguyên, T. T., Huynh, T. T., Ren, Z., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.00673 · Cited by 4
ATICVis: A Visual Analytics System for Asymmetric Transformer Models Interpretation and Comparison
Wu, J.-L., Chang, P.-C., Wang, C., & Wang, K.-C. (2023) · Applied Sciences · doi:10.3390/app13031595 · Cited by 2
Understanding the Inner-workings of Language Models Through Representation Dissimilarity
Brown, D., Godfrey, C., Konz, N., Tu, J. H., & Kvinge, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.403 · Cited by 2
Model-agnostic interpretation by visualization of feature perturbationsPreprint
Marc'ilio-Jr, W. E., Eler, D. M., & Breve, F. (2021) · arXiv.org · doi:10.48550/arxiv.2101.10502 · Cited by 1
Model Reconstruction Using Counterfactual Explanations: A Perspective From Polytope TheoryPreprint
Dissanayake, P., & Dutta, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.05369 · Cited by 1
Flagged first, then by citations
Research works printed but not linked: 52
- 2023, 2016/679 of the European Parliament and of the council of 27 April 2016 on the General Data Protection Regulation
- 2024, Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)—Discussion Paper and Request for Feedback
- 10.1016/j.media.2021.102306 · DOI
- 2021, Ethics and Governance of Artificial Intelligence for Health: WHO Guidance
- 2021, California Consumer Privacy Act (CCPA)
- 10.3389/fdata.2021.688969 · DOI · Paper page
- 2016, The Neural Network Zoo—The Asimov Institute
- Ghojogh, 2019, The theory behind overfitting, cross validation, regularization, bagging, and boosting: Tutorial, arXiv:1905.12787 · arXiv
- 10.48550/ARXIV.1706.03762 · DOI · Paper page
- Neal, 2018, A modern take on the bias-variance tradeoff in neural networks, arXiv:1810.08591 · arXiv
- Goodfellow, 2016, Deep Learning
- Wang, Benign overfitting in multiclass classification: All roads lead to interpolation, 34, 24164
- 10.2307/2699986 · DOI
- Kumar, Shapley residuals: Quantifying the limits of the Shapley value for explanations, 34, 1
- Vashishth, 2019, Attention interpretability across NLP tasks, arXiv:1909.11218 · arXiv
- Munroe, 2023, XKCD—A Webcomic of Romance, Sarcasm, Math, and Language
- Yousefzadeh, 2019, Interpreting neural networks using flip points, arXiv:1903.08789 · arXiv
- Ba, Do deep nets really need to be deep?, 27, 1
- Kass, 1988, The need for user models in generating expert system explanations, Int. J. Exp. Syst., 1, 1
- Kumar, 2021, Is Shapley explanation for a model unique?, arXiv:2111.11946 · arXiv
- Wojtas, Feature importance ranking for deep learning, 33, 5105
- Alvarez-Melis, 2018, On the robustness of interpretability methods, arXiv:1806.08049 · arXiv
- Mishra, 2021, A survey on the robustness of feature importance and counterfactual explanations, arXiv:2111.00358 · arXiv
- Guidotti, 2018, Local rule-based explanations of black box decision systems, arXiv:1805.10820 · arXiv
- Xenopoulos, GALE: Globally assessing local explanations, 322
- 10.2139/ssrn.3063289 · DOI · Paper page
- Ying, GNNexplainer: Generating explanations for graph neural networks, 32, 1
- Vu, PGM-Explainer: Probabilistic graphical model explanations for graph neural networks, 33, 12225
- Yuan, On explainability of graph neural networks via subgraph explorations, 12241
- Baldassarre, 2019, Explainability techniques for graph convolutional networks, arXiv:1905.13686 · arXiv
- Biderman, Pythia: A suite for analyzing large language models across training and scaling, 2397
- Vig, BertViz: A tool for visualizing multihead self-attention in the BERT model, 23, 1
- Sundararajan, Axiomatic attribution for deep networks, 70, 3319
- Hamilton, 2021, Axiomatic explanations for visual search, retrieval, and similarity learning, arXiv:2103.00370 · arXiv
- Smilkov, 2017, SmoothGrad: Removing noise by adding noise, arXiv:1706.03825 · arXiv
- Shrikumar, Learning important features through propagating activation differences, 3145
- Sundararajan, The many Shapley values for model explanation, 9269
- Yousefzadeh, 2019, Investigating decision boundaries of trained neural networks, arXiv:1908.02802 · arXiv
- Lakkaraju, Robust and stable black box explanations, 5628
- Lundberg, 2018, Consistent individualized feature attribution for tree ensembles, arXiv:1802.03888 · arXiv
- 10.1007/s11222-021-10057-z · DOI · Paper page
- Barr, The disagreement problem in faithfulness metrics, 1
- Ye, Variable selection via penalized neural network: A drop-out-one loss approach, 5620
- Weerts, 2019, A human-grounded evaluation of SHAP for alert processing, arXiv:1907.03324 · arXiv
- Tsirtsis, Decisions, counterfactual explanations and strategic behavior, 33, 16749
- Alvarez-Melis, 2018, Towards robust interpretability with self-explaining neural networks, arXiv:1806.07538 · arXiv
- Petsiuk, 2018, RISE: Randomized input sampling for explanation of black-box models, arXiv:1806.07421 · arXiv
- Agarwal, 2022, Rethinking stability for attribution-based explanations, arXiv:2203.06877 · arXiv
- Han, Which explanation should I choose? A function approximation perspective to characterizing post hoc explanations, 35, 5256
- Banachewicz, 2022, The Kaggle Book: Data Analysis and Machine Learning for Competitive Data Science
- Alber, 2019, iNNvestigate neural networks!, J. Mach. Learn. Res., 20, 1
- Preece, 2018, Stakeholders in explainable AI, arXiv:1810.00184 · arXiv
Cited by
Sorted by citations
The Democratization of Artificial Intelligence: Theoretical Framework
Costa, C. J., Aparicio, M., Aparicio, S., & Aparicio, J. T. (2024) · Applied Sciences · doi:10.3390/app14188236 · Cited by 44
Explainable AI: enhancing decision-making in the detection of cyber threats
Prasad, P. W. C., Sayeed, M. S., Nguyen, D.-M., Hutabarat, D. P., & Mohiuddin, G. M. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1762332 · Cited by 16
Metadata
- Authors
- Ortigossa, E. S., Gonçalves, T., & Nonato, L. G.
- Affiliations
- Universidade de São Paulo (BR)
- Institute of Mathematics and Computer Science, University of São Paulo (ICMC-USP), São Carlos, Brazil
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 80799-80846
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4399449888
- Cited by
- 124
- References
- 245
- Citations by year
- 2026: 66 · 2025: 52 · 2024: 6
- FWCI
- 18.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior–Brasil (CAPES)–Finance Code 001
- São Paulo Research Foundation
- National Council for Scientific and Technological Development
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Fundação de Amparo à Pesquisa do Estado de São Paulo (2018/24516-0, 2022/09091-8)
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (307184/2021-8)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, model interpretability, black-box models, automated decision-making, trust in machine learning, feature importance, lime, SHAP, model transparency, algorithmic bias detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
