Explainable AI and Causal Understanding: Counterfactual Approaches Considered
Open accessAbstract
The counterfactual approach to explainable AI (XAI) seeks to provide understanding of AI systems through the provision of counterfactual explanations. In a recent systematic review, Chou et al. (Inform Fus 81:59–83, 2022) argue that the counterfactual approach does not clearly provide causal understanding . They diagnose the problem in terms of the underlying framework within which the counterfactual approach has been developed. To date, the counterfactual approach has not been developed in concert with the approach for specifying causes developed by Pearl (Causality: Models, reasoning, and inference. Cambridge University Press, 2000) and Woodward (Making things happen: A theory of causal explanation. Oxford University Press, 2003). In this paper, I build on Chou et al.’s work by applying the Pearl-Woodward approach. I argue that the standard counterfactual approach to XAI is capable of delivering causal understanding, but that there are limitations on its capacity to do so. I suggest a way to overcome these limitations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Too great an emphasis on making explanations psychologically fitting risks making it easier for bad actors to deceive and manipulate end-users.
“Too great an emphasis on making explanations psychologically fitting risks making it easier for bad actors to deceive and manipulate end-users (Baron, 2023;Lipton, 2018, pp. 21-22).”
From Hidden Risks: Artificial Intelligence and Hermeneutic Harm (2025) · cites this paper as Baron, 2023
Section 2, “Causal Understanding” (page not supplied), warns that an explanation users find satisfying may give them only perceived understanding rather than true beliefs about causes. This supports the concern about prioritizing psychological fit, but the passages do not say that doing so makes it easier for bad actors to deceive or manipulate end-users. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C., & Jorge, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.003 · Cited by 302 · see citing works
Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning
Byrne, R. M. J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/876 · Cited by 285 · see citing works
Reutlinger, A. (2016) · Philosophy of Science · doi:10.1086/687859 · Cited by 116 · see citing works
Flagged first, then by citations
Research works printed but not linked: 36
- Amir-Hossein, K., Schölkopf, B., & Valera, I. (2021), Algorithmic recourse: From counterfactual explanations to interventions. FAccT '21: Proceedings of the 2021 ACM Conference on Fair- ness, Accountability, and Transparency, 353-362.
- Baumgartner, M., & Gebharter, A. (2016). Constitutive relevance, mutual manipulability and fat- handedness. British Journal for the Philosophy of Science, 67, 731-756.
- Beckers, S. (2022). Causal explanations and xai. Proceedings of Machine Learning Research, 140, 1-20.
- Been, K., Khanna, R., & Koyejo, O. (2016), Examples are not enough, learn to criticize! criticism for interpretability. Proceedings of the 30th Conference on Neural Information Processing Systems (NIPS 2016).
- Chris, R., Mc Grath, R., & Costabello, L. (2020), Learning relevant explanations. 2020 ICML Workshop on Human Interpretability in Machine Learning (WHI 2020).
- Courtney, F., & Keane, M. T. (2022), Explaining classifications to non-experts: An xai user study of post- hoc explanations for a classifier when people lack expertise. https://arxiv.org/abs/2212.09342. · arXiv
- Dandl, S., Molnar, C., Binder, M., & Bischl, B. (2020). Multi-objective counterfactual explanations. Par- allel problem solving from nature. (pp. 448-469).
- Dhurandhar, A., Chen, P.-Y., Luss, R., Tu, C.-C., Ting, P., Shanmugam, K., & Das, P. (2018). Explana- tions based on the missing: Towards contrastive explanations with pertinent negatives. Advances in Neural Information Processing Systems, 31, 592-603.
- Divyat, M., Tan, C., & Sharma, A. (2019), Preserving causal constraints in counterfactual explanations for machine learning classifiers. CausalML: Machine Learning and Causal Inference for Improved Decision Making Workshop, 33rd Conference on Neural PRocessing Systems (NeurIPS2019), https://arxiv.org/abs/1912.03277. · arXiv
- Eoin, D., Pakrashi, A., Greene, D., & Keane, M. T. (2022), Counterfactual explanations for misclassified images: How human and machine explanations differ. https://arxiv.org/abs/2212.08733. · arXiv
- Fine, K. (1975). Review of "counterfactuals''. Mind, 84, 451-458.
- Greta, W., Keane, M. T., & Byrne, R. M. J. (2022), Features of explainability: How users understand counterfactual and causal explanations for categorical and continuous features in xai. https:// arxiv.org/abs/2204.10152. · arXiv
- Kacper, S., & Flach, P. (2019), Counterfactual explanations of machine learning predictions: Opportu- nities and challenges for ai safety. SafeAI@ AAAI (2019).
- Kaivalya, R., & Lakkaraju, H. (2020), Beyond individualized recorse: Interpretable and interactive summaries of actionable resources. Proceedings of the 34th Conference on Neural Information Processing Systems, 1-12.
- Kasirzadeh, A., & Smart, A. (2021), The use and misue of counterfactuals in ethical machine learn- ing. FAccT '21: Proceedings of the 2021 ACM Conference on fairness, accountability, and trans- parency, 228-236.
- Keane Mark T., Kenny, E. M., Delaney, E., & Smyth, B. (2021), If only we had better counterfactual explanations: Five key deficits to rectify in the evaluation of counterfactual xai techniques. Pro- ceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI-21), https:// arxiv.org/abs/2103.01035. · arXiv
- Lara, K., & Liefgreen, A. (2021), What if (and how...)? -actionability shapes people's perceptions of counterfactual explanations in automated decision-making. ICML (International Conference on Machine Learning) Workshop on Algorithmic Recourse, 1-5.
- Lewis, D. (1973). Causation. Journal of Philosophy, 70, 556-567.
- Mark, K., & Smyth, B. (2020), Good counterfactuals and where to find them: A case-based technique for generating counterfactuals for explainable ai (xai). International Conference on Case-Based Reasoning, 163-178.
- Maximilian, F., Klier, M., Kluge, K., & Sigler, I. (2020a), Evaluating explainable artificial intelli- gence: What users really appreciate. Twenty-Eigth European Conference on Information Systems (ECIS2020).
- Maximilian, F., Klier, M., Kluge, K., & Sigler, I. (2020b), Fostering human agency: A process for the design of user-centric xai systems. ICIS 2020 Proceedings.
- Mothilal, R. K., Sharma, A., & Tan, C. (2020), Explaining machine learning classifiers through diverse counterfactual explanations. Proceedings of the 2020 Conference on Fairness, Account- ability, and Transparency, 607-617.
- Oscar, L., Liu, H., Chen, C., & Rudin, C. (2018), Deep learning for case-based reasoning through pro- totypes: a neural network that explains its predictions. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, 3530-3537.
- Pearl, J. (2000). Causality: Models, reasoning, and inference. Cambridge University Press.
- Pritchard, D. (2014). Knowledge and understanding. In A. Fairweather (Ed.), Epistemology natural- ized. Synthese Library.
- Rafael, P., Sokol, K., Santos-Rodriguez, R., De Bie, T., & Flach, P. (2020), Face: Feasible and actionable counterfactual explanations. AIES'20: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 344-359.
- Räz, T., & Beisbart, C. (2022). The importance of understanding deep learning. Erkenntnis. https:// doi. org/ 10. 1007/ s10670-022-00605-y · DOI
- Rory, M., Costabello, L., Le Van, C., Sweeney, P., Kamiab, F., Shen, Z., & Lecue, F. (2018), Interpretable credit application predictions with counterfactual explanations. NIPS 2018 Workshop on challenges and opportunities for AI in financial services: The impact of fairness, explainability, accuracy, and privacy.
- Sahil, V., Dickerson, J., & Hines, K. (2020), Counterfactual explanations for machine learning: A review. https://arxiv.org/abs/2010.10596. · arXiv
- Schaffer, J. (2000). Trumping preemption. Journal of Philosophy, 9, 165-181.
- Shin, D. (2021). The effects of explainability and causability on perception, trust, and acceptance: Impli- cations for explainable ai. International Journal of Human-Computer Studies, 146, 1-10.
- Solon, B., Selbst, A. D., Raghavan, M (2020), The hidden assumptions behind counterfactual explana- tions and principal reasons. FAT* '20: Proceedings of the 2020 Conference on Fairness, Account- ability, and Transparency, 80-89.
- Sullivan, E. (2022). Understanding from machine learning models. British Journal for the Philosophy of Science, 73, 109-133.
- Thibault, L., Lesot, M-J., Marsala, C., Renard, X., & Detyniecki, M. (2019), The dangers of post-hoc interpretability: Unjustified counterfactual explanations. Proceedings of the Twenty-Eighth Interna- tional Joint Conference on Artificial Intelligence, ICJAI-19, 2801-2807.
- van Looveren, A., & Klaise, J. (2021). Interpretable counterfactual explanations guided by prototypes. Lecture Notes in Computer Science. In: Oliver, N, Pérez-Cruz, F., Kramer, S., Read, J., & Lozano, J. A. (Eds.), Machine learning and knowledge discovery in databases. Research track. EXCML PKDD 2021. Springer, Cham (pp. 650-665)
- Wachter, S., Mittelstadt, B., & Russell, C. (2018). Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harvard Journal of Law and Technology, 31, 841-887.
Cited by
Sorted by citations
Hidden Risks: Artificial Intelligence and Hermeneutic Harm
Rebera, A. P., Lauwaert, L., & Oimann, A.-K. (2025) · Minds and Machines · doi:10.1007/s11023-025-09733-0 · Cited by 6 · see citing works
“Too great an emphasis on making explanations psychologically fit - ting risks making it easier for bad actors to deceive and manipulate end-users (Baron, 2023 ; Lipton, 2018, pp. 21–22).”
- Authors
- Baron, S.
- Affiliations
- Australian Catholic University (AU)
- Corresponding author
- Baron, Sam
- ORCID
- Baron, Sam
- Venue
- Minds and Machines
- Published
- 2023-06
- Volume
- 33
- Issue
- 2
- Pages
- 347-377
- ISSN
- 0924-6495 (ISSN-L)
- 1572-8641
- OpenAlex
- W4380085287
- Cited by
- 58
- References
- 4
- Citations by year
- 2026: 15 · 2025: 22 · 2024: 18 · 2023: 3
- FWCI
- 5.13
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Minds and Machines (publishedVersion, cc-by)
- Funders
- Australian Catholic University Limited
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- explainable AI, counterfactual explanations, causal knowledge, causal explanation, counterfactual analysis, causal models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
