CoChat
Sign Up
Cited by 11Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIPrivacy-Preserving Technologies in Data

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Towards Causal Algorithmic Recourse

Open access
Karimi, Amir-Hossein, von Kügelgen, Julius, Schölkopf, Bernhard, Valera, Isabel·2022·Lecture Notes in Computer Science
DOI 10.1007/978-3-031-04083-2_8
Open this paper in CoChat — free accountOpen PDF

Abstract

Algorithmic recourse is concerned with aiding individuals who are unfavorably treated by automated decision-making systems to overcome their hardship, by offering recommendations that would result in a more favorable prediction when acted upon. Such recourse actions are typically obtained through solving an optimization problem that minimizes changes to the individual’s feature vector, subject to various plausibility, diversity, and sparsity constraints. Whereas previous works offer solutions to the optimization problem in a variety of settings, they critically overlook real-world considerations pertaining to the environment in which recourse actions are performed. The present work emphasizes that changes to a subset of the individual’s attributes may have consequential down-stream effects on other attributes, thus making recourse a fundamcausal problem. Here, we model such considerations using the framework of structural causal models, and highlight pitfalls of not considering causal relations through examples and theory. Such insights allow us to reformulate the optimization problem to directly optimize for minimally-costly recourse over a space of feasible actions (in the form of causal interventions) rather than optimizing for minimally-distant “counterfactual explanations”. We offer both the optimization formulations and solutions to deterministic and probabilistic recourse, on an individualized and sub-population level, overcoming the steep assumptive requirements of offering recourse in general settings. Finally, using synthetic and semi-synthetic experiments based on the German Credit dataset, we demonstrate how such methods can be applied in practice under minimal causal assumptions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-031-04083-2_8

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Accountability is a key requirement to be able to recourse when an AI model contributes to making a proven wrong decision, issuing explanations and recommendations to cases that are unfavorably treated by such decision.

    “On the other hand, accountability is a key requirement to be able to recourse [116] when an AI model contributes to making a proven wrong decision, issuing explanations and recommendations to cases that are unfavorably treated by such decision.”

    From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [116]

    The Introduction (page unavailable in our copy) describes recourse as reversing unfavorable algorithmic decisions and discusses explanations and recommended actions for affected individuals. It does not say that accountability is a prerequisite for recourse or that the decision must have been proven wrong; its examples include ordinary unfavorable decisions, such as a denied loan. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 72

  1. Abrevaya, J., Hsu, Y.C., Lieli, R.P.: Estimating conditional average treatment effects. J. Bus. Econ. Stat. 33(4), 485–505 (2015) · DOI
  2. Alaa, A.M., van der Schaar, M.: Bayesian inference of individualized treatment effects using multi-task Gaussian processes. In: Advances in Neural Information Processing Systems, pp. 3424–3432 (2017)
  3. Angrist, J.D., Imbens, G.W., Rubin, D.B.: Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91(434), 444–455 (1996) · DOI · Paper page
  4. Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine Bias. ProPublica, New York (2016)
  5. Balke, A., Pearl, J.: Counterfactual probabilities: computational methods, bounds and applications. In:: Uncertainty Proceedings 1994, pp. 46–54. Elsevier (1994) · DOI
  6. Barocas, S., Selbst, A.D., Raghavan, M.: The hidden assumptions behind counterfactual explanations and principal reasons. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 80–89 (2020) · DOI
  7. Bashtannyk, D.M., Hyndman, R.J.: Bandwidth selection for kernel conditional density estimation. Comput. Stat. Data Anal. 36(3), 279–298 (2001) · DOI
  8. Bastani, O., Kim, C., Bastani, H.: Interpretability via model extraction. arXiv preprint arXiv:1706.09773 (2017) · arXiv
  9. Bottou, L., Bousquet, O.: The tradeoffs of large scale learning. In: Advances in Neural Information Processing Systems, pp. 161–168 (2008)
  10. Cohen, L., Lipton, Z.C., Mansour, Y.: Efficient candidate screening under multiple tests and implications for fairness. arXiv preprint arXiv:1905.11361 (2019) · arXiv
  11. Cooper, G.F., Yoo, C.: Causal discovery from a mixture of experimental and observational data. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 116–125 (1999)
  12. Dhurandhar, A., et al.: Explanations based on the missing: towards contrastive explanations with pertinent negatives. In: Advances in Neural Information Processing Systems, pp. 592–603 (2018)
  13. Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017) · arXiv
  14. Friedman, N., Nachman, I.: Gaussian process networks. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 211–219 (2000)
  15. Grote, T., Berens, P.: On the ethics of algorithmic decision-making in healthcare. J. Med. Ethics 46(3), 205–211 (2020) · DOI · Paper page
  16. Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., Giannotti, F.: Local rule-based explanations of black box decision systems. arXiv preprint arXiv:1805.10820 (2018) · arXiv
  17. Gunning, D.: DARPA’S explainable artificial intelligence (XAI) program. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, p. ii. ACM (2019) · DOI · Paper page
  18. Gupta, V., Nokhiz, P., Roy, C.D., Venkatasubramanian, S.: Equalizing recourse across groups. arXiv preprint arXiv:1909.03166 (2019) · arXiv
  19. Hancox-Li, L.: Robustness in machine learning explanations: does it matter? In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 640–647 (2020) · DOI
  20. Holzinger, A., Malle, B., Saranti, A., Pfeifer, B.: Towards multi-modal causability with graph neural networks enabling information fusion for explainable AI. Inf. Fusion 71, 28–37 (2021) · DOI · Paper page
  21. Hoyer, P., Janzing, D., Mooij, J.M., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. In: Advances in Neural Information Processing Systems, pp. 689–696 (2009)
  22. ShJoshi, S., Koyejo, O., Vijitbenjaronk, W., Kim, B., Ghosh, J.: REVISE: towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615 (2019) · arXiv
  23. Karimi, A.H., Barthe, G., Balle, B., Valera, I.: Model-agnostic counterfactual explanations for consequential decisions. In: International Conference on Artificial Intelligence and Statistics, pp. 895–905 (2020)
  24. Karimi, A.-H., Barthe, G., Schölkopf, B., Valera, I.: A survey of algorithmic recourse: contrastive explanations and consequential recommendations. arXiv preprint arXiv:2010.04050 (2020) · arXiv
  25. Karimi, A.-H., von Kügelgen, J., Schölkopf, B., Valera, I.: Algorithmic recourse under imperfect causal knowledge: a probabilistic approach. In: Advances in Neural Information Processing Systems, pp. 265–277 (2020)
  26. Karimi, A.-H., Schölkopf, B., Valera, I.: Algorithmic recourse: from counterfactual explanations to interventions. In: 4th Conference on Fairness, Accountability, and Transparency (FAccT 2021), pp. 353–362 (2021) · DOI · Paper page
  27. Karush, W.: Minima of functions of several variables with inequalities as side conditions. Master’s thesis, Department of Mathematics, University of Chicago (1939)
  28. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference for Learning Representations (2015)
  29. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: 2nd International Conference on Learning Representations (2014)
  30. Kodratoff, Y.: The comprehensibility manifesto. KDD Nugget Newsl. 94(9) (1994)
  31. Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In: Neyman, J. (ed.) Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, Berkeley (1951) · DOI
  32. Lage, I.: An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006 (2019) · arXiv
  33. Laugel, T., Lesot, M.J., Marsala, C., Renard, X., Detyniecki, M.: Inverse classification for comparison-based interpretability in machine learning. arXiv preprint arXiv:1712.08443 (2017) · DOI
  34. Lewis, D.K.: Counterfactuals. Harvard University Press, Cambridge (1973)
  35. Lipton, Z.C.: The mythos of model interpretability. Queue 16(3), 31–57 (2018) · DOI · Paper page
  36. Louizos, C., Shalit, U., Mooij, J.M., Sontag, D., Zemel, R., Welling, M.: Causal effect inference with deep latent-variable models. In: Advances in Neural Information Processing Systems, pp. 6446–6456 (2017)
  37. Mahajan, D., Tan, C., Sharma, A.: Preserving causal constraints in counterfactual explanations for machine learning classifiers. arXiv preprint arXiv:1912.03277 (2019) · arXiv
  38. Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1–38 (2019) · DOI · Paper page
  39. Mothilal, R.K., Sharma, A., Tan, C.: DiCE: explaining machine learning classifiers through diverse counterfactual explanations. arXiv preprint arXiv:1905.07697 (2019) · DOI · Paper page
  40. Nabi, R., Shpitser, I.: Fair inference on outcomes. In: Proceedings of the... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence, vol. 2018, p. 1931. NIH Public Access (2018) · DOI · Paper page
  41. Pearl, J.: Causality. Cambridge University Press, Cambridge (2009) · DOI · Paper page
  42. Pearl, J.: Structural counterfactuals: a brief introduction. Cogn. Sci. 37(6), 977–985 (2013) · DOI
  43. Peters, J., Bühlmann, P.: Identifiability of Gaussian structural equation models with equal error variances. Biometrika 101(1), 219–228 (2014) · DOI
  44. Peters, J., Janzing, D., Schölkopf, B.: Elements of Causal Inference. The MIT Press, Cambridge (2017)
  45. Poyiadzi, R., Sokol, K., Santos-Rodriguez, R., De Bie, T., Flach, P.: FACE: feasible and actionable counterfactual explanations. arXiv preprint arXiv:1909.09369 (2019) · DOI · Paper page
  46. Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models. In: International Conference on Machine Learning, pp. 1278–1286 (2014)
  47. Rieckmann, A., et al.: Causes of outcome learning: a causal inference-inspired machine learning approach to disentangling common combinations of potential causes of a health outcome. medRxiv (2020) · DOI
  48. Ruben, D.-H.: Explaining Explanation. Routledge, London (2015) · DOI
  49. Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1(5), 206–215 (2019) · DOI · Paper page
  50. Rüping, S.: Learning interpretable models. Ph.D. dissertation, Technical University of Dortmund (2006)
  51. Russell, C.: Efficient search for diverse coherent explanations. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, pp. 20–28. ACM (2019) · DOI · Paper page
  52. Schulam, P., Saria, S.: Reliable decision support using counterfactual models. In: Advances in Neural Information Processing Systems, pp. 1697–1708 (2017)
  53. Schumann, C., Foster, J.S., Mattei, N., Dickerson, J.P.: We need fairness and explainability in algorithmic hiring. In: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp. 1716–1720 (2020)
  54. Shpitser, I., Pearl, J.: Identification of conditional interventional distributions. In: 22nd Conference on Uncertainty in Artificial Intelligence, UAI 2006, pp. 437–444 (2006)
  55. Shpitser, I., Pearl, J.: Complete identification methods for the causal hierarchy. J. Mach. Learn. Res. 9(Sep), 1941–1979 (2008)
  56. Silva, R., Gramacy, R.B.: Gaussian process structural equation models with latent variables. In: Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, pp. 537–545 (2010)
  57. Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: Advances in Neural Information Processing Systems, pp. 3483–3491 (2015)
  58. Starr, W.: Counterfactuals. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, fall 2019 edition (2019)
  59. Stöger, K., Schneeberger, D., Holzinger, A.: Medical artificial intelligence: the European legal perspective. Commun. ACM 64(11), 34–36 (2021) · DOI · Paper page
  60. Tian, J., Pearl, J.: Causal discovery from changes. In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 512–521 (2001)
  61. Tian, J., Pearl, J.: A general identification condition for causal effects. In: Eighteenth national conference on Artificial intelligence, pp. 567–573 (2002)
  62. Trippe, B.L., Turner, R.E.: Conditional density estimation with Bayesian normalising flows. arXiv preprint arXiv:1802.04908 (2018) · arXiv
  63. Ustun, B., Spangher, A., Liu, Y.: Actionable recourse in linear classification. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 10–19. ACM (2019) · DOI · Paper page
  64. Venkatasubramanian, S. , Alfano, M.: The philosophical basis of algorithmic recourse. In: Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM (2020) · DOI · Paper page
  65. Verma, S., Dickerson, J., Hines, K.: Counterfactual explanations for machine learning: a review. arXiv preprint arXiv:2010.10596 (2020) · arXiv
  66. von Kügelgen, J., Rubenstein, P.K., Schölkopf, B., Weller, A.: Optimal experimental design via Bayesian optimization: active causal structure learning for Gaussian process networks. In: NeurIPS Workshop “Do the right thing”: machine learning and causal inference for improved decision making (2019)
  67. von Kügelgen, J., Agarwal, N., Zeitler, J., Mastouri, A., Schölkopf, B.: Algorithmic recourse in partially and fully confounded settings through bounding counterfactual effects. In: ICML Workshop on Algorithmic Recourse (2021)
  68. von Kügelgen, J., Karimi, A.-H., Bhatt, U., Valera, I., Weller, A., Schölkopf, B.: On the fairness of causal algorithmic recourse. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence (2022) · DOI · Paper page
  69. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harvard J. Law Technol. 31(2) (2017) · DOI · Paper page
  70. Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends® Mach. Learn. 1(1–2), 1–305 (2008) · DOI
  71. Wang, Y., Blei, D.M.: The blessings of multiple causes. J. Am. Stat. Assoc. 114, 1–71 (2019) · DOI
  72. Williams, C.K.I., Rasmussen, C.E.: Gaussian Processes for Machine Learning, vol. 2. MIT Press, Cambridge (2006)

Cited by

Sorted by citations

Metadata

Authors
Karimi, A.-H., von Kügelgen, J., Schölkopf, B., & Valera, I.
Affiliations
  • ETH Zurich (CH)
  • University of Cambridge (GB)
  • Max Planck Institute for Intelligent Systems (DE)
Corresponding author
Karimi, Amir-Hossein
ORCID
Venue
Lecture Notes in Computer Science
Publisher
Springer International Publishing
Published
2022
Pages
139-166
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-031-04083-2_8
OpenAlex
W4226318288
Cited by
11
References
5
Citations by year
2026: 1 · 2025: 2 · 2024: 3 · 2023: 5
FWCI
1.38
Type
conference paper
Language
en
Open access
Licence
Locations
Lecture notes in computer science (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
algorithmic recourse, automated decision-making, structural causal models, causal intervention, counterfactual explanations, algorithmic fairness, German credit dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review