The DOI matches its Crossref record; 96 of the 100 references it cites were found in a registry or an index; none of the 58 sources its claims cite is checked for support yet.
Explaining Explanations in AI
Open accessAbstract
Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions. These models are a useful pedagogical device for teaching trained professionals how to predict what decisions will be made by the complex system, and most importantly how the system might break. However, when considering any such model it's important to remember Box's maxim that "All models are wrong but some are useful." We focus on the distinction between these models and explanations in philosophy and sociology. These models can be understood as a "do it yourself kit" for explanations, allowing a practitioner to directly answer "what if questions" or generate contrastive explanations without external assistance. Although a valuable ability, giving these models as explanations appears more difficult than necessary, and other forms of explanation may not have the same trade-offs. We contrast the different schools of thought on what makes an explanation, and suggest that machine learning might benefit from viewing the problem more broadly.
Overview
CoChat Summary
TL;DR
The paper argues that most explainable AI (xAI) methods produce simplified approximations of black-box models, which function more like scientific models than explanations as studied in philosophy, cognitive science, and social science. It calls for contrastive, selective, and socially interactive explanations that support contestability and accountability.
Key findings
- xAI’s dominant approach retro-fits local or global simplified models—linear/gradient-based approximations or decision-tree methods—over complex algorithms; these are useful for experts and debugging but can be misleading to non-experts when their domain and limitations are undocumented.
- Work in the “explanation sciences” indicates that everyday explanations are contrastive, selective, and social: people ask why P rather than Q, select one or two causes from many, and explanations involve interactive dialogue and argumentation.
- Contrastive explanation methods (e.g., Martens and Provost 2013; Wachter et al. 2018) directly return alternative data points and avoid some approximation/domain problems, but raise relevance concerns if the counterfactual does not correspond to a user’s interests.
- The paper argues xAI should develop interactive post-hoc methods that make algorithmic decisions contestable and facilitate dialogue among users, developers, systems, and stakeholders; it states the field largely fails this task at present.
Methods
- Conceptual/position paper; no empirical study, dataset, or quantitative analysis is reported.
- Reviews xAI interpretability approaches—transparency versus post-hoc interpretation, local/global approximations, and linear/gradient versus tree-based methods—and contrasts them with philosophical, cognitive science, and social science literature on explanation.
- Uses argumentation and synthesis, with illustrative examples such as Newtonian physics, a bee/spider example, and image binarization choices.
Limitations
- Stated: a complete overview of the philosophy of explanation is unfeasible; the paper focuses solely on ways of providing explanations; contrastive theories remain debated; and choosing relevant contrast cases is not straightforward.
- Evident: no empirical validation, user study, or comparative evaluation of explanation methods is presented; claims about misleading non-experts and the need for interactive methods are argued rather than tested.
- Evident: the analysis relies on secondary literature and selected examples, with no new data or formal metrics.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 8 Not valid · 42 Not checked · 64 Own results and interpretations
None of the sources cited for this claim have been checked yet.
In the paper
“Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions.”
- Also stated in: § 3 EXPLAINABLE AI · p. 3 · plain
“These simplified models approximate the true criteria used to make decisions (e.g. Baehrens et al. (2010); Ribeiro et al. (2016); Selvaraju et al. (2016); Simonyan et al. (2013)).”
What the sources say
- Not checked yetcites Baehrens et al., 2010
- Not checked yetcites Ribeiro et al., 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 3 EXPLAINABLE AI · p. 3 · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We then examine research on explanations in philosophy, cognitive science, and the social sciences which suggests that 'why-questions' (e.g. 'why did the model exhibit that behaviour?') require explanations that are contrastive, selective, and socially interactive.”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We argue that these approximations function more like scientific models than the types of scientific and 'everyday' explanations considered in philosophy, cognitive science, and psychology.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If this comparison holds, it follows that the majority of currently available methods will, at best, produce locally reliable but globally misleading explanations of model functionality.”
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, local approximations are accurate representations only of a specific domain or 'slice' of a model.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
50 more claims
Create a free account to see all 114 claims and to verify the 58 references still open.
- Probably not a substantial claim. 42
- It may state two separate claims as one. 28
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There can be multiple ways of making data and code accessible, and there can be multiple ways to justify a decision and different degrees of transparency in the decision (Miller 2019; Mittelstadt et al. 2019).”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Mittelstadt et al. 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For users to trust a system, they need to know over which domain a model is reliable, where it is uncertain, and where it is likely to break down86.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [86]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Critics of the current principles-based approach to AI governance have emphasized a number of limits to operationalizing fairness, the practical limits of providing algorithmic explainability or transparency, and the lack of professional accountability mechanisms needed to ensure their consistent implementation [119 -121].”
From Artificial intelligence, systemic risks, and sustainability (2021) · cites this paper as [121]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Explainability tools can (at least currently) provide only an approximation of the decision-making functions in black-box systems (Mittelstadt et al., 2019)”
From Hidden Risks: Artificial Intelligence and Hermeneutic Harm (2025) · cites this paper as Mittelstadt et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Citizens want to receive clear reasons for a (negative) decision to assess the fairness of a decision (De Fine Licht and De Fine Licht 2020; Tyler 2006) or to be able to contest an algorithmic decision (Mittelstadt et al. 2019).”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Mittelstadt et al. 2019
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 12×
“However, very few stop to check what they are agreeing to, and to find out what explainable AI means to other people involved in the discussion (Lipton, 2016).”
BackgroundInfluentialDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 4×
“These simplified models approximate the true criteria used to make decisions (e.g. Baehrens et al. (2010); Ribeiro et al. (2016); Selvaraju et al. (2016); Simonyan et al. (2013)).”
BackgroundMethodInfluentialExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
Research works printed but not linked: 16
- Biran Or, 2014, Proceedings of the AutoML workshop at ICML, 2014
- Craven Mark, 1996, Shavlik
- Ruth, 2017, Fong and Andrea Vedaldi
- Habermas J., The Theory of Communicative Action: Volume 1: Reason and the Rationalization of Society
- Hilton Denis J., 1986, Slugoski
- Kim Been, 2014, Shah
- Kroll Joshua A., 2016, Accountable algorithms, U. Pa. L. Rev., 165, 633
- Lepri Bruno, 2017, The Premise, the Proposed Solutions, and the Open Challenges. Philosophy & Technology (15 8
- David Lewis. 1973. Counterfactuals. Blackwell Oxford. David Lewis. 1973. Counterfactuals. Blackwell Oxford.
- Lipton Zachary C., 2016, The Mythos of Model Interpretability. arXiv:1606.03490 {cs, stat} (10 6 · arXiv
- Lisboa Paulo JG, 2015, Fuzzy Logic and Applications
- Martens David, 2007, Tony Van Gestel, and Jan Vanthienen
- McClure John L., 2003, Hilton
- Sandvig Christian, 2014, Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry
- Andrew, 2018, Selbst and Solon Barocas
- Wachter Sandra, 2018, Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology forthcoming
Cited by
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
“Enormous technical efforts are undertaken to meet ethical targets in the fields of accountability and explainable AI (Mittelstadt et al. 2019), fairness and discrimination aware data mining (Gebru et al.”
BackgroundThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
“Enormous technical efforts are undertaken to meet ethical targets in the fields of accountability and explainable AI (Mittelstadt et al. 2019), fairness and discrimination aware data mining (Gebru et al.”
BackgroundBias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043
“Given that a decision will often not be reasoned (hence the reasons will be unknown), it will be difficult to establish that it was made on the basis of a biased decision-making process (Mittelstadt et al., 2019).”
Explainable machine learning for scientific insights and discoveries
Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020) · IEEE Access · doi:10.1109/access.2020.2976199 · Cited by 1,040
“Concerning explanations, it has also been argued that there is a gap of expectations between ML and so-called explanation sciences such as law, cognitive science, philosophy, and the social sciences [28].”
Background
Metadata
- Authors
- Mittelstadt, B., Russell, C., & Wachter, S.
- Affiliations
- University of Oxford (GB)
- University of Surrey (GB)
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 279-288
- OpenAlex
- W2898694742
- Semantic Scholar
- cf4bbe1e39c6f0f88aa69da2a95125b47de0ec7b
- Cited by
- 794
- References
- 100
- Citations by year
- 2026: 79 · 2025: 99 · 2024: 110 · 2023: 117 · 2022: 115 · 2021: 125 · 2020: 107 · 2019: 40
- FWCI
- 37.01
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- SSRN Electronic Journal (submittedVersion)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/N510129/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- explainable AI, interpretability, philosophy of explanation, contrastive explanations, what-if questions, teaching model
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Explaining Explanations in AI. https://app.cochat.ai/papers/10.1145/3287560.3287574
- The paper
- Mittelstadt, B., Russell, C., & Wachter, S. (2019). Explaining Explanations in AI. Proceedings of the Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3287560.3287574
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.