Quod erat demonstrandum? - Towards a typology of the concept of explanation for the design of explainable AI
Open accessAbstract
In this paper, we present a fundamental framework for defining different types of explanations of AI systems and the criteria for evaluating their quality. Starting from a structural view of how explanations can be constructed, i.e., in terms of an explanandum (what needs to be explained), multiple explanantia (explanations, clues, or parts of information that explain), and a relationship linking explanandum and explanantia, we propose an explanandum-based typology and point to other possible typologies based on how explanantia are presented and how they relate to explanandia. We also highlight two broad and complementary perspectives for defining possible quality criteria for assessing explainability: epistemological and psychological (cognitive). These definition attempts aim to support the three main functions that we believe should attract the interest and further research of XAI scholars: clear inventories, clear verification criteria, and clear validation methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Validation (model, solution checking) is critical in CSC with embedded AI.
“Validation: Model, solution checking [65] (critical in CSC with embedded AI).”
From A requirements model for AI algorithms in functional safety-critical systems with an explainable self-enforcing network from a developer perspective (2024) · cites this paper as [65]
Cites
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · see citing works
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · see citing works
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308 · see citing works
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
Explainable artificial intelligence: an analytical review
Angelov, P. P., Soares, E. A., Jiang, R., Arnold, N. I., & Atkinson, P. M. (2021) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1424 · Cited by 843 · see citing works
The seven tools of causal inference, with reflections on machine learning
Pearl, J. (2019) · Communications of the ACM · doi:10.1145/3241036 · Cited by 659 · see citing works
Notions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598 · see citing works
Loyola-Gonzalez, O. (2019) · IEEE Access · doi:10.1109/access.2019.2949286 · Cited by 559 · see citing works
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419 · see citing works
Stepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376 · see citing works
de Fine Licht, K., & de Fine Licht, J. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00960-w · Cited by 334 · see citing works
Cabitza, F., Campagner, A., Soares, F., García de Guadiana-Romualdo, L., Challa, F., Sulejmani, A., Seghezzi, M., & Carobene, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106288 · Cited by 284 · see citing works
Self-citationActivity Theory and Distributed Cognition: Or What Does CSCW Need to DO with Theories?
Halverson, C. A. (2002) · Computer Supported Cooperative Work (CSCW) · doi:10.1023/a:1015298005381 · Cited by 283 · see citing works
The perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making
de Bruijn, H., Warnier, M., & Janssen, M. (2022) · Government Information Quarterly · doi:10.1016/j.giq.2021.101666 · Cited by 264 · see citing works
Privacy as Protection of the Incomputable Self: From Agnostic to Agonistic Machine Learning
Hildebrandt, M. (2019) · Theoretical Inquiries in Law · doi:10.1515/til-2019-0004 · Cited by 153 · see citing works
Malgieri, G. (2019) · Computer Law & Security Review · doi:10.1016/j.clsr.2019.05.002 · Cited by 124 · see citing works
Self-citationMetrics, Explainability and the European AI Act Proposal
Sovrano, F., Sapienza, S., Palmirani, M., & Vitali, F. (2022) · J · doi:10.3390/j5010010 · Cited by 71 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare
Hulsen, T. (2023) · AI · doi:10.3390/ai4030034 · Cited by 305 · see citing works
“[28] discuss this approach and conclude that existing XAI methods fail to be bona fide explanations, which is why their framework cannot be applied to current XAI work.”
BackgroundThe EU Artificial Intelligence Act (2024): Implications for healthcare
van Kolfschooten, H., & van Oirschot, J. (2024) · Health Policy · doi:10.1016/j.healthpol.2024.105152 · Cited by 250 · see citing works
Kim, M., Kim, S., Kim, J., Song, T.-J., & Kim, Y. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103160 · Cited by 53 · see citing works
Sáez, C., Ferri, P., & García-Gómez, J. M. (2024) · Journal of Medical Internet Research · doi:10.2196/50295 · Cited by 36 · see citing works
Liability Regimes in the Age of AI: a Use-Case Driven Analysis of the Burden of Proof
Fernández Llorca, D., Charisi, V., Hamon, R., Sánchez, I., & Gómez, E. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14565 · Cited by 20 · see citing works
Study on the Helpfulness of Explainable Artificial Intelligence
Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J. (2024) · Communications in Computer and Information Science · doi:10.1007/978-3-031-63803-9_16 · Cited by 8 · see citing works
“The typology of an explanation is given by an explanandum, the object (thing or phenomenon) to be explained, the explanans (the actual explanation that is perceived by the users), and the relationship between both [5].”
BackgroundKlüver, C., Greisbach, A., Kindermann, M., & Püttmann, B. (2024) · Security and Safety · doi:10.1051/sands/2024020 · Cited by 8 · see citing works
“• Validation: Model, solution checking [65] (critical in CSC with embedded AI).”
- Authors
- Cabitza, F., Campagner, A., Malgieri, G., Natali, C., Schneeberger, D., Stoeger, K., & Holzinger, A.
- Affiliations
- IRCCS Ospedale Galeazzi - Sant'Ambrogio (IT)
- Corresponding author
- Cabitza, Federico
- ORCID
- Venue
- Expert Systems with Applications
- Publisher
- Elsevier BV
- Published
- 2023-03
- Volume
- 213
- Pages
- 118888
- ISSN
- 0957-4174 (ISSN-L)
- OpenAlex
- W4296982335
- Cited by
- 89
- References
- 23
- Citations by year
- 2026: 9 · 2025: 24 · 2024: 26 · 2023: 30
- FWCI
- 6.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Expert Systems with Applications (publishedVersion, cc-by)
- BOA (University of Milano-Bicocca) (submittedVersion)
- Funders
- Università degli Studi di Milano-Bicocca
- Austrian Science Fund (P-32554)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- explainable AI, concept of explanation, cognitive assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
