The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Explainable AI is a field of study that produces AI models whose reasoning is transparent to outside scrutiny.
“Explainable AI is a field of study that produces AI models whose reasoning is transparent to outside scrutiny [69].”
From Efficiency vs. Equity: A Structured Interdisciplinary Review of AI in Criminal Justice Risk Assessments (2026) · cites this paper as [69]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 4×
“In other words, despite the broad interest in the design of XAI methods [18,31,48–50], it is still unclear how to compare the results of different evaluations and establish a common understanding of how to evaluate explanations.”
BackgroundInfluentialSelf-citationZawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 1×
“Another area of application of AI that has benefited from adopting XAI methods is the intelligent analysis, modelling, and management of agricultural and forest ecosystems, an essential task for securing our planet for future generations.”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Self-citationTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
VanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · cited 1×
“The effectiveness and adequacy of explainability as a tool for AI safety may be limited in specific scenarios [269].”
BackgroundChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · cited 1×
“Likewise, plagiarism detection is becoming central in LLMs that excel across different domains, for example, with ChatGPT [277].”
BackgroundMachine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Self-citationThe false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · cited 2×
“Then, one possible course of action would be not to provide users with any XAI support if this is deemed detrimental or irrelevant in a given setting, for instance, in radiological settings (see [255,257]).”
BackgroundDiffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · cited 1×
“A lot of work exists on explaining the decisions of a classifier with attribution methods [44].”
BackgroundSelf-citationEthics of AI in Education: Towards a Community-Wide Framework
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., et al. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00239-1 · Cited by 1,437
ChatGPT is fun, but not an author
Thorp, H. H. (2023) · Science · doi:10.1126/science.adg7879 · Cited by 1,437
Quantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.385 · Cited by 995
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811
Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · cited 1×
“While the interest in the machine unlearning paradigm [280] has been on the rise [281], it is unclear how to efficiently ensure that data owned by a particular user is unlearned by a given large-scale generative model, so that it can be ensured that no instance like that of the user claiming their…”
BackgroundDíaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
Self-citationDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 1×
“Thus, whether information biases the generative process, as, for example, with the prompt in a language-to-image stable diffusion model, is sufficiently original for intellectual property and author attribution claims remains an open question.”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606
Self-citationNotions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598 · cited 5×
“Similarly, Vilone and Longo [27] aggregate evaluation approaches for XAI methods from several scientific studies via a hierarchical system.”
BackgroundInfluentialSelf-citationNauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590
Tschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R. P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., et al. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30333-x · Cited by 555
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419
Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C., & Jorge, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.003 · Cited by 302
Rodríguez-Barroso, N., Jiménez-López, D., Luzón, M. V., Herrera, F., & Martínez-Cámara, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.011 · Cited by 296
Self-citationZoom In: An Introduction to Circuits
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020) · Distill · doi:10.23915/distill.00024.001 · Cited by 288
Neural Prototype Trees for Interpretable Fine-grained Image Recognition
Nauta, M., van Bree, R., & Seifert, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01469 · Cited by 269
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Arya, V., Bellamy, R. K. E., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2019) · arXiv · doi:10.48550/arxiv.1909.03012 · Cited by 268 · cited 2×
“Some researchers use terms like explainability and interpretability syn-onymously [23,26], while others draw significant distinctions between them [64].”
BackgroundHuman Mental Workload: A Survey and a Novel Inclusive Definition
Longo, L., Wickens, C. D., Hancock, G., & Hancock, P. A. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.883321 · Cited by 222
Self-citationArtificial Intelligence as an Effective Classroom Assistant
du Boulay, B. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.93 · Cited by 213 · cited 1×
“On the other hand, for instructors, AIED tools can act as intelligent teaching assistants [114], help them orchestrate the classroom [115], grade assessments [116], and answer student queries [117], minimizing students dropout [118].”
BackgroundA Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193 · cited 1×
“While the interest in the machine unlearning paradigm [280] has been on the rise [281], it is unclear how to efficiently ensure that data owned by a particular user is unlearned by a given large-scale generative model, so that it can be ensured that no instance like that of the user claiming their…”
BackgroundMiller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172
Black Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security
Kuppa, A., & Le-Khac, N.-A. (2020) · 2020 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206780 · Cited by 123 · cited 1×
“Several works have advocated the idea of exploiting explanations beyond just explaining decisions [30,167], for instance, also to improve models.”
BackgroundOn generating trustworthy counterfactual explanations
Del Ser, J., Barredo-Arrieta, A., Díaz-Rodríguez, N., Herrera, F., Saranti, A., & Holzinger, A. (2024) · Information Sciences · doi:10.1016/j.ins.2023.119898 · Cited by 122
Self-citationRams, hounds and white boxes: investigating human–AI collaboration protocols in medical diagnosis
Cabitza, F., Campagner, A., Ronzio, L., Cameli, M., Mandoli, G. E., Pastore, M. C., Sconfienza, L. M., Folgado, D., Barandas, M., & Gamboa, H. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102506 · Cited by 117
Self-citationFrom Responsibility to Reason-Giving Explainable Artificial Intelligence
Baum, K., Mantel, S., Schmidt, E., & Speith, T. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00510-w · Cited by 103 · cited 1×
“Explanations are often requested to clarify issues such as accountability [203,235,263,264].”
BackgroundSelf-citationBeyond explainability: justifiability and contestability of algorithmic decision systems
Henin, C., & Le Métayer, D. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01251-8 · Cited by 71
The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions
Bruckert, S., Finzel, B., & Schmid, U. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.507973 · Cited by 66
Dissecting scientific explanation in AI (sXAI): A case for medicine and healthcare
Durán, J. M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103498 · Cited by 62
Progress measures for grokking via mechanistic interpretability
Nanda, N., Chan, L., Lieberum, T., Smith, J., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2301.05217 · Cited by 51
Contesting automated decisions
Bayamlioglu, E. (2018) · European Data Protection Law Review · doi:10.21552/edpl/2018/4/6 · Cited by 21
A framework to contest and justify algorithmic decisions
Henin, C., & Le Métayer, D. (2021) · AI and Ethics · doi:10.1007/s43681-021-00054-3 · Cited by 14
The clock and the pizza: Two stories in mechanistic explanation of neural networks
Zhong, Z., Liu, Z., Tegmark, M., & Andreas, J. (2023) · arXiv · doi:10.48550/arxiv.2306.17844 · Cited by 14
Flagged first, then by citations
Research works printed but not linked: 140
- Swartout, 1991, Explanations in knowledge systems: Design for explainable expert systems, IEEE Expert, 6, 58, 10.1109/64.87686 · DOI
- Confalonieri, 2021, A historical perspective of explainable Artificial Intelligence, WIREs Data Min. Knowl. Discov., 11
- Speith, 2022, A review of taxonomies of explainable artificial intelligence (XAI) methods, 2239
- Schwalbe, 2023, A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts, Data Min. Knowl. Discov., 1
- Langer, 2021, What do we want from Explainable Artificial Intelligence (XAI)?–A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research, Artificial Intelligence, 296, 10.1016/j.artint.2021.103473 · DOI · Paper page
- Cao, 2022, Ai in finance: challenges, techniques, and opportunities, ACM Comput. Surv., 55, 1, 10.1145/3502289 · DOI
- Haresamudram, 2023, Three levels of AI transparency, Computer, 56, 93, 10.1109/MC.2022.3213181 · DOI · Paper page
- Zerilli, 2022, Explaining machine learning decisions, Philos. Sci., 89, 1, 10.1017/psa.2021.13 · DOI
- Chazette, 2020, Explainability as a non-functional requirement: challenges and recommendations, Requir. Eng., 25, 493, 10.1007/s00766-020-00333-1 · DOI
- Páez, 2019, The pragmatic turn in explainable artificial intelligence (XAI), Minds Mach., 29, 441, 10.1007/s11023-019-09502-w · DOI
- Weber, 2023, Beyond explaining: Opportunities and challenges of XAI-based model improvement, Inf. Fusion, 92, 154, 10.1016/j.inffus.2022.11.013 · DOI
- Bodria, 2023, Benchmarking and survey of explanation methods for black box models, Data Min. Knowl. Discov., 1
- Guidotti, 2022, Counterfactual explanations and how to find them: literature review and benchmarking, Data Min. Knowl. Discov., 1
- Machlev, 2022, Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities, Energy AI, 9, 10.1016/j.egyai.2022.100169 · DOI · Paper page
- Mei, 2023, Explainable artificial intelligence by genetic programming: A survey, IEEE Trans. Evol. Comput., 27, 621, 10.1109/TEVC.2022.3225509 · DOI
- Minh, 2022, Explainable artificial intelligence: a comprehensive review, Artif. Intell. Rev., 1
- Theissler, 2022, Explainable AI for time series classification: A review, taxonomy and research directions, IEEE Access, 10, 100700, 10.1109/ACCESS.2022.3207765 · DOI
- Yang, 2022, Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond, Inf. Fusion, 77, 29, 10.1016/j.inffus.2021.07.016 · DOI · Paper page
- Zini, 2022, On the explainability of natural language processing deep models, ACM Comput. Surv., 55, 1, 10.1145/3529755 · DOI
- Heuillet, 2021, Explainability in deep reinforcement learning, Knowl.-Based Syst., 214, 10.1016/j.knosys.2020.106685 · DOI
- Mohseni, 2021, A multidisciplinary survey and framework for design and evaluation of explainable AI systems, ACM Trans. Interact. Intell. Syst. (TiiS), 11, 1
- Vilone, 2021, Classification of explainable artificial intelligence methods through their output formats, Mach. Learn. Knowl. Extr., 3, 615, 10.3390/make3030032 · DOI
- Guidotti, 2018, A survey of methods for explaining black box models, ACM Comput. Surv., 51, 1, 10.1145/3236009 · DOI · Paper page
- Rawal, 2021, Recent advances in trustworthy explainable artificial intelligence: Status, challenges, and perspectives, IEEE Trans. Artif. Intell., 3, 852, 10.1109/TAI.2021.3133846 · DOI
- Yeom, 2021, Pruning by explaining: A novel criterion for deep neural network pruning, Pattern Recognit., 115, 10.1016/j.patcog.2021.107899 · DOI
- D. Slack, S. Hilgard, E. Jia, S. Singh, H. Lakkaraju, Fooling lime and shap: Adversarial attacks on post hoc explanation methods, in: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 2020, pp. 180–186. · DOI · Paper page
- Lapuschkin, 2019, Unmasking Clever Hans predictors and assessing what machines really learn, Nature Commun., 10, 1096, 10.1038/s41467-019-08987-4 · DOI · Paper page
- S. Lapuschkin, A. Binder, K.-R. Müller, W. Samek, Understanding and comparing deep neural networks for age and gender classification, in: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 1629–1638. · DOI
- Grinsztajn, 2022, Why do tree-based models still outperform deep learning on typical tabular data?, vol. 35, 507
- Rokach, 2016, Decision forest: Twenty years of research, Inf. Fusion, 27, 111, 10.1016/j.inffus.2015.06.005 · DOI
- Hatwell, 2020, CHIRPS: Explaining random forest classification, Artif. Intell. Rev., 53, 5747, 10.1007/s10462-020-09833-6 · DOI
- Fürnkranz, 2020, On cognitive preferences and the plausibility of rule-based models, Mach. Learn., 109, 853, 10.1007/s10994-019-05856-5 · DOI
- Krakauer, 2023, Unifying complexity science and machine learning, Front. Complex Syst., 1, 10.3389/fcpxs.2023.1235202 · DOI
- Fernandez, 2019, Evolutionary fuzzy systems for explainable artificial intelligence: Why, when, what for, and where to?, IEEE Comput. Intell. Mag., 14, 69, 10.1109/MCI.2018.2881645 · DOI
- Huang, 2023, From decision trees to explained decision sets, Vol. 372, 1100
- Gorishniy, 2021, Revisiting deep learning models for tabular data, Adv. Neural Inf. Process. Syst., 34, 18932
- S.Ö. Arik, T. Pfister, Tabnet: Attentive interpretable tabular learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 6679–6687, (8). · DOI
- Reduan Achtibat, 2024, Attnlrp: attention-aware layer-wise relevance propagation for transformers, arXiv:2402.05602 · arXiv
- Lécué, 2020, On the role of knowledge graphs in explainable AI, Semant. Web, 11, 41
- Vassiliades, 2021, Argumentation and explainable artificial intelligence: a survey, Knowl. Eng. Rev., 36, 10.1017/S0269888921000011 · DOI
- Baroni, 2011, An introduction to argumentation semantics, Knowl. Eng. Rev., 26, 365, 10.1017/S0269888911000166 · DOI
- L. Rizzo, L. Longo, Inferential Models of Mental Workload with Defeasible Argumentation and Non-monotonic Fuzzy Reasoning: a Comparative Study, in: Proceedings of the 2nd Workshop on Advances in Argumentation in Artificial Intelligence, Co-Located with XVII International Conference of the Italian Association for Artificial Intelligence, AI3@AI*IA 2018, 20-23 November 2018, Trento, Italy, 2018, pp. 11–26.
- Longo, 2021, Examining the modelling capabilities of defeasible argumentation and non-monotonic fuzzy reasoning, Knowl.-Based Syst., 211, 10.1016/j.knosys.2020.106514 · DOI
- S Band, 2023, Application of explainable artificial intelligence in medical health: A systematic review of interpretability methods, Inform. Med. Unlocked, 40, 10.1016/j.imu.2023.101286 · DOI
- Han, 2020, Which explanation should I choose? A function approximation perspective to characterizing post hoc explanations, Adv. Neural Inf. Process. Syst., 35
- Agarwal, 2022, OpenXAI: Towards a transparent evaluation of model explanations, vol. 35, 15784
- Bussmann, 2021, Explainable machine learning in credit risk management, Comput. Econ., 57, 203, 10.1007/s10614-020-10042-0 · DOI
- Sachan, 2020, An explainable AI decision-support-system to automate loan underwriting, Expert Syst. Appl., 144, 10.1016/j.eswa.2019.113100 · DOI
- Rudin, 2019, Why are we using black box models in AI when we don’t need to? A lesson from an explainable AI competition, Harv. Data Sci. Rev., 1, 10
- Sharma, 2022, Drones and machine learning for estimating forest carbon storage, Carbon Res., 1, 21, 10.1007/s44246-022-00021-5 · DOI
- Möllmann, 2017, A practical way to integrate risk in forest management decisions, Ann. For. Sci., 74, 1, 10.1007/s13595-017-0670-x · DOI
- Gollob, 2020, Forest inventory with long range and high-speed personal laser scanning (PLS) and simultaneous localization and mapping (SLAM) technology, Remote Sens., 12, 1509, 10.3390/rs12091509 · DOI
- Holzinger, 2022, Digital transformation in smart farm and forest operations needs human-centered AI: challenges and future directions, Sensors, 22, 3043, 10.3390/s22083043 · DOI
- Holzinger, 2022, Information fusion as an integrative cross-cutting enabler to achieve robust, explainable, and trustworthy medical artificial intelligence, Inf. Fusion, 79, 263, 10.1016/j.inffus.2021.10.007 · DOI
- Desmarais, 2012, A review of recent advances in learner and skill modeling in intelligent learning environments, User Model. User-Adapt. Interact., 22, 9, 10.1007/s11257-011-9106-8 · DOI
- Bull, 2020, There are open learner models about!, IEEE Trans. Learn. Technol., 13, 425, 10.1109/TLT.2020.2978473 · DOI
- Holstein, 2019, Co-designing a real-time classroom orchestration tool to support teacher-AI complementarity, Grantee Submiss.
- A. Singh, S. Karayev, K. Gutowski, P. Abbeel, Gradescope: a fast, flexible, and fair system for scalable assessment of handwritten work, in: Proceedings of the Fourth (2017) Acm Conference on Learning@ Scale, 2017, pp. 81–88. · DOI · Paper page
- Hiremath, 2018, Chatbot for education system, Int. J. Adv. Res. Ideas Innov. Technol., 4, 37
- Liz-Domínguez, 2019, Systematic literature review of predictive analysis tools in higher education, Appl. Sci., 9, 5569, 10.3390/app9245569 · DOI
- Khosravi, 2019, RiPPLE: A crowdsourced adaptive platform for recommendation of learning activities, J. Learn. Anal., 6, 91
- S. Abdi, H. Khosravi, S. Sadiq, D. Gasevic, Complementing educational recommender systems with open learner models, in: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 2020, pp. 360–365. · DOI · Paper page
- Croitoru, 2023, Diffusion models in vision: A survey, IEEE Trans. Pattern Anal. Mach. Intell., 45, 10850, 10.1109/TPAMI.2023.3261988 · DOI
- Cammarata, 2020, Thread: Circuits, Distill, 10.23915/distill.00024 · DOI
- Brcic, 2023, Impossibility results in AI: A survey, ACM Comput. Surv., 56, 8:1
- Bárcena, 2023, Enabling federated learning of explainable AI models within beyond-5G/6G networks, Comput. Commun., 210, 356, 10.1016/j.comcom.2023.07.039 · DOI
- W. Du, M.J. Atallah, Secure multi-party computation problems and their applications: a review and open problems, in: Proceedings of the 2001 Workshop on New Security Paradigms, 2001, pp. 13–22. · DOI
- R.R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization, in: IEEE International Conference on Computer Vision, ICCV, 2017, pp. 618–626. · DOI · Paper page
- Chen, 2019, This looks like that: deep learning for interpretable image recognition, Adv. Neural Inf. Process. Syst., 32
- D. Rymarczyk, Ł. Struski, J. Tabor, B. Zieliński, Protopshare: Prototypical parts sharing for similarity discovery in interpretable image classification, in: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 1420–1430. · DOI
- Achtibat, 2023, From attribution maps to human-understandable explanations through Concept Relevance Propagation, Nat. Mach. Intell., 5, 1006, 10.1038/s42256-023-00711-8 · DOI · Paper page
- Sarker, 2021, Neuro-symbolic artificial intelligence, AI Commun., 34, 197, 10.3233/AIC-210084 · DOI
- Hamilton, 2022, Is neuro-symbolic AI meeting its promises in natural language processing? A structured review, Semant. Web, 15, 1
- Tiddi, 2022, Knowledge graphs as tools for explainable machine learning: A survey, Artificial Intelligence, 302, 10.1016/j.artint.2021.103627 · DOI
- J. Johnson, B. Hariharan, L. Van Der Maaten, L. Fei-Fei, C. Lawrence Zitnick, R. Girshick, Clevr: A diagnostic dataset for compositional language and elementary visual reasoning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2901–2910. · DOI · Paper page
- K. Yi, C. Gan, Y. Li, P. Kohli, J. Wu, A. Torralba, J.B. Tenenbaum, CLEVRER: Collision Events for Video Representation and Reasoning, in: ICLR, 2020.
- Müller, 2021, Kandinsky patterns, Artificial Intelligence, 300, 10.1016/j.artint.2021.103546 · DOI · Paper page
- Schneider, 2022, Explaining classifiers by constructing familiar concepts, Mach. Learn., 1
- Yeh, 2019, On the (in) fidelity and sensitivity of explanations, Adv. Neural Inf. Process. Syst., 32
- Ferrario, 2022, The robustness of counterfactual explanations over time, IEEE Access, 10, 82736, 10.1109/ACCESS.2022.3196917 · DOI
- L. Qiu, Y. Yang, C.C. Cao, Y. Zheng, H. Ngai, J. Hsiao, L. Chen, Generating perturbation-based explanations with robustness to out-of-distribution data, in: Proceedings of the ACM Web Conference 2022, 2022, pp. 3594–3605. · DOI
- Krishnan, 2020, Against interpretability: A critical examination of the interpretability problem in machine learning, Philos. Technol., 33, 487, 10.1007/s13347-019-00372-9 · DOI
- Graziani, 2022, A global taxonomy of interpretable AI: Unifying the terminology for the technical and social sciences, Artif. Intell. Rev., 1
- Robbins, 2019, A misdirected principle with a catch: Explicability for AI, Minds Mach., 29, 495, 10.1007/s11023-019-09509-3 · DOI
- Palladino, 2023, A ‘biased’ emerging governance regime for artificial intelligence? How AI ethics get skewed moving from principles to practices, Telecommun. Policy, 47, 10.1016/j.telpol.2022.102479 · DOI
- Khalifa, 2012, Inaugurating understanding or repackaging explanation?, Philos. Sci., 79, 15, 10.1086/663235 · DOI
- Strevens, 2013, No understanding without explanation, Stud. Hist. Philos. Sci. A, 44, 510, 10.1016/j.shpsa.2012.12.005 · DOI
- P. Lipton, Understanding without explanation, in: Scientific Understanding: Philosophical Perspectives, 2009, pp. 43–63. · DOI
- Mizrahi, 2012, Idealizations and scientific understanding, Philos. Stud., 160, 237, 10.1007/s11098-011-9716-3 · DOI
- Carter, 2016, Objectual understanding, factivity and belief, Vol. 423
- Erasmus, 2021, What is interpretability?, Philos. Technol., 34, 833, 10.1007/s13347-020-00435-2 · DOI
- Sullivan, 2022, Understanding from machine learning models, British J. Philos. Sci.
- Creel, 2020, Transparency in complex computational systems, Philos. Sci., 87, 568, 10.1086/709729 · DOI
- Fleisher, 2022, Understanding, idealization, and explainable AI, Episteme, 19, 534, 10.1017/epi.2022.39 · DOI
- Pirozelli, 2022, Sources of understanding in supervised machine learning models, Philos. Technol., 35, 23, 10.1007/s13347-022-00524-4 · DOI
- M.M. De Graaf, B.F. Malle, How people explain action (and autonomous intelligent systems should too), in: 2017 AAAI Fall Symposium Series, 2017, pp. 19–26.
- B. Mittelstadt, C. Russell, S. Wachter, Explaining explanations in AI, in: Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019, pp. 279–288. · DOI · Paper page
- Guidotti, 2021, Evaluating local explanation methods on ground truth, Artificial Intelligence, 291, 10.1016/j.artint.2020.103428 · DOI
- Sevillano-García, 2023, REVEL framework to measure local linear explanations for black-box models: Deep learning image classification case study, Int. J. Intell. Syst., 2023, 1, 10.1155/2023/8068569 · DOI · Paper page
- M.T. Keane, E.M. Kenny, E. Delaney, B. Smyth, If only we had better counterfactual explanations: Five key deficits to rectify in the evaluation of counterfactual xai techniques, in: Proceedings of the 30th International Joint Conference on Artificial Intelligence, IJCAI-21, 2021, pp. 4466–4474. · DOI
- Metta, 2022, Exemplars and counterexemplars explanations for skin lesion classifiers, vol. 354, 258
- van der Lee, 2021, Human evaluation of automatically generated text: Current trends and best practice guidelines, Comput. Speech Lang., 67, 10.1016/j.csl.2020.101151 · DOI
- Confalonieri, 2023, An operational framework for guiding human evaluation in Explainable and Trustworthy AI, IEEE Intell. Syst., 1
- Hedström, 2023, Quantus: An explainable AI toolkit for responsible evaluation of neural network explanation, J. Mach. Learn. Res., 24, 1
- Arras, 2022, CLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanations, Inf. Fusion, 81, 14, 10.1016/j.inffus.2021.11.008 · DOI
- Confalonieri, 2021, Using ontologies to enhance human understandability of global post-hoc explanations of Black-box models, Artificial Intelligence, 296, 10.1016/j.artint.2021.103471 · DOI
- Nielsen, 2022, Robust explainability: A tutorial on gradient-based attribution methods for deep neural networks, IEEE Signal Process. Mag., 39, 73, 10.1109/MSP.2022.3142719 · DOI
- Yuan, 2023, Compositional scene representation learning via reconstruction: A survey, IEEE Trans. Pattern Anal. Mach. Intell., 45, 11540, 10.1109/TPAMI.2023.3286184 · DOI
- Rizzo, 2020, An empirical evaluation of the inferential capacity of defeasible argumentation, non-monotonic fuzzy reasoning and expert systems, Expert Syst. Appl., 147, 10.1016/j.eswa.2020.113220 · DOI
- L. Rizzo, L. Longo, A Qualitative Investigation of the Explainability of Defeasible Argumentation and Non-Monotonic Fuzzy Reasoning, in: Proceedings for the 26th AIAI Irish Conference on Artificial Intelligence and Cognitive Science Trinity College Dublin, Dublin, Ireland, December 6-7th, 2018, 2018, pp. 138–149.
- Vilone, 2022, An XAI method for the automatic formation of an abstract argumentation framework from a neural network and its objective evaluation, vol. 3209
- Vielhaben, 2024, Explainable ai for time series via virtual inspection layers, Pattern Recognition, 150, 110309, 10.1016/j.patcog.2024.110309 · DOI · Paper page
- Ahmed, 2023, Interpreting disentangled representations of person-specific convolutional variational autoencoders of spatially preserving EEG topographic maps via clustering and visual plausibility, Information, 14, 10.3390/info14090489 · DOI
- Krakauer, 2014, The computational systems of the world, BioScience, 64, 351, 10.1093/biosci/biu024 · DOI
- Badreddine, 2022, Logic tensor networks, Artificial Intelligence, 303, 10.1016/j.artint.2021.103649 · DOI
- Meske, 2020, Explainable artificial intelligence: Objectives, stakeholders and future research opportunities, Inf. Syst. Manage.
- R. Hamon, H. Junklewitz, G. Malgieri, P.D. Hert, L. Beslay, I. Sanchez, Impossible explanations? Beyond explainable AI in the GDPR from a COVID-19 use case scenario, in: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 549–559. · DOI
- Flack, 2012, Multiple time-scales and the developmental dynamics of social systems, Philos. Trans. R. Soc. B, 367, 1802, 10.1098/rstb.2011.0214 · DOI
- P. Sanchez, S.A. Tsaftaris, Diffusion causal models for counterfactual estimation, in: Conference on Causal Learning and Reasoning, CLeaR, 2022.
- M. Augustin, V. Boreiko, F. Croce, M. Hein, Diffusion Visual Counterfactual Explanations, in: NeurIPS, 2022. · DOI
- J. Schneider, J.P. Handali, Personalized Explanation for Machine Learning: a Conceptualization, in: European Conference on Information Systems, ECIS, 2019.
- Zhu, 2023, Principled reinforcement learning with human feedback from pairwise or K-wise comparisons, vol. 202, 43037
- Rudin, 2019, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nat. Mach. Intell., 1, 206, 10.1038/s42256-019-0048-x · DOI · Paper page
- Hamon, 2022, Bridging the gap between AI and explainability in the GDPR: Towards trustworthiness-by-design in automated decision-makin, IEEE Comput. Intell. Mag., 17, 72, 10.1109/MCI.2021.3129960 · DOI
- Gunning, 2019, XAI—Explainable artificial intelligence, Science Robotics, 4, 10.1126/scirobotics.aay7120 · DOI · Paper page
- Cabitza, 2021, The need to move away from agential-AI: Empirical investigations, useful concepts and open issues, Int. J. Hum.-Comput. Stud., 155, 10.1016/j.ijhcs.2021.102696 · DOI · Paper page
- M.L. Leavitt, A. Morcos, Towards falsifiable interpretability research, in: NeurIPS 2020 Workshop: ML Retrospectives, Surveys and Meta-Analyses, ML-RSA, 2020.
- A. Krajna, M. Brcic, M. Kovac, A. Sarcevic, Explainable Artificial Intelligence: An Updated Perspective, in: Proceedings of 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO) 2022, Opatija, Croatia, 2022, pp. 859–864. · DOI
- J. Schneider, C. Meske, M. Vlachos, Deceptive AI Explanations: Creation and Detection, in: International Conference on Agents and Artificial Intelligence, ICAART, 2022, pp. 44–55. · DOI
- Schneider, 2023, Reflective-net: Learning from explanations, Data Min. Knowl. Discov.
- Yampolskiy, 2020, Unexplainability and incomprehensibility of AI, J. Artif. Intell. Conscious., 07, 277, 10.1142/S2705078520500150 · DOI
- Yampolskiy, 2017, What are the ultimate limits to computational techniques: verifier theory and unverifiability, Phys. Scr., 92, 10.1088/1402-4896/aa7ca8 · DOI
- Boenisch, 2021, A systematic review on model watermarking for neural networks, Front. Big Data, 4, 10.3389/fdata.2021.729663 · DOI
- J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, T. Goldstein, A Watermark for Large Language Models, in: Proceedings of the 40th International Conference on Machine Learning, 2023, pp. 202:17061–17084.
- Cabitza, 2023, Quod erat demonstrandum?-Towards a typology of the concept of explanation for the design of explainable AI, Expert Syst. Appl., 213, 10.1016/j.eswa.2022.118888 · DOI · Paper page
- Malgieri, 2021, “Just” algorithms: justification (beyond explanation) of automated decisions under the general data protection regulation, Law Bus., 1, 16, 10.2478/law-2021-0003 · DOI · Paper page
- L.M. Austin, Enough about me: why privacy is about power, not consent (or harm), in: A. Sarat (Ed.), A World Without Privacy: What Law Can and Should Do?, 2014, pp. 131–189. · DOI
- Kaminski, 2020, Algorithmic impact assessments under the GDPR: producing multi-layered explanations, Int. Data Priv. Law, 19
- Gregory, 2003, Scandinavian approaches to participatory design, Int. J. Eng. Educ., 19, 62
- Malgieri, 2023, In/acceptable marketing and consumers’ privacy expectations: Four tests from EU data protection law, J. Consum. Mark., 40, 209, 10.1108/JCM-03-2021-4571 · DOI
Cited by
Sorted by citations
Ennab, M., & Mcheick, H. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1444763 · Cited by 238
Mienye, I. D., Obaido, G., Jere, N., Mienye, E., Aruleba, K., Emmanuel, I. D., & Ogbuokiri, B. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101587 · Cited by 212
Ukwaththa, J., Herath, S., & Meddage, D. P. P. (2024) · Materials Today Communications · doi:10.1016/j.mtcomm.2024.110294 · Cited by 118
Retrieval-Augmented Generation (RAG)
Klesel, M., & Wittmann, H. F. (2025) · Business & Information Systems Engineering · doi:10.1007/s12599-025-00945-3 · Cited by 92
“Providing references to the contextual data is closely related to the concept of Explainable AI (XAI) (Schneider 2024; Longo et al. 2024), because users get additional information about the results of an LLM.”
Exploring creativity in human-AI co-creation: a comparative study across design experience
Wang, N., Kim, H., Peng, J., & Wang, J. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1672735 · Cited by 59
Large language models meet user interfaces: The case of provisioning feedback
Pozdniakov, S., Brazil, J., Abdi, S., Bakharia, A., Sadiq, S., Gašević, D., Denny, P., & Khosravi, H. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100289 · Cited by 43
Self-citationBrankovic, A., Cook, D., Rahman, J., Delaforce, A., Li, J., Magrabi, F., Cabitza, F., Coiera, E., & Bradford, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01764-2 · Cited by 37
Self-citationKabir, S., Hossain, M. S., & Andersson, K. (2025) · Algorithms · doi:10.3390/a18090556 · Cited by 44
Practitioners’ expectations on log anomaly detection
Ma, X., Li, Y., Keung, J., Yu, X., Zou, H., Yang, Z., Sarro, F., & Barr, E. T. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3586700 · Cited by 21
“These techniques aim to provide feature-level explanations by approximating the behavior of the original model with an interpretable model [77], [78], which are also valuable for log anomaly detection.”
Paolini, L., Vahdati, S., Di Iorio, A., Wardenga, R., Heibi, I., & Peroni, S. (2025) · Scientometrics · doi:10.1007/s11192-025-05418-8 · Cited by 8
“However, the complexity of ES and the opacity of PLMs working dynamics (e.g., latent representations in trans-former layers) challenge interpretability (Longo et al., 2024).”
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
Nomological Deductive Reasoning for Trustworthy, Human-Readable, and Actionable AI Outputs
Hakizimana, G., & Ledezma Espino, A. (2025) · Algorithms · doi:10.3390/a18060306 · Cited by 2
Atkinson, G., & Casagrande, K. (2026) · Information · doi:10.3390/info17060574
Metadata
- Authors
- Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., Jiang, R., Khosravi, H., Lecue, F., Malgieri, G., Páez, A., Samek, W., Schneider, J., Speith, T., & Stumpf, S.
- ORCID
- Longo, Luca
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2024-06
- Volume
- 106
- Pages
- 102301
- ISSN
- 1566-2535
- Cited by
- 626
- References
- 57
- Language
- en
- Licence
- Funders
- Ministry of Science, ICT and Future Planning (2022-0-00984, ERC-2018-ADG G.A. 834756, 871042)
- European Commission (AI4ES, CER-20211030)
- Deutsche Forschungsgemeinschaft (389792660, KI-FOR 5363)
- Volkswagen Foundation (AZ 98509, AZ 9B830, AZ 98514)
- Ministero dell’Istruzione, dell’Università e della Ricerca
- Institute for Information and Communications Technology Promotion
- Eusko Jaurlaritza (PID2020-119478GB-I00, BIRD231830, IT1456-22)
- Ministerio de Ciencia e Innovación
- Centro para el Desarrollo Tecnológico Industrial
- Agencia Estatal de Investigación
- Austrian Science Fund (PRIN 2022 PNRR 1409/22, P-32554)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review