CoChat
Sign Up
Cited by 301Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIPrivacy-Preserving Technologies in Data

A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations

Open access
Karimi, Amir-Hossein, Barthe, Gilles, Schölkopf, Bernhard, Valera, Isabel·2023·ACM Computing Surveys
DOI 10.1145/3527848
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning is increasingly used to inform decision making in sensitive situations where decisions have consequential effects on individuals’ lives. In these settings, in addition to requiring models to be accurate and robust, socially relevant values such as fairness, privacy, accountability, and explainability play an important role in the adoption and impact of said technologies. In this work, we focus on algorithmic recourse , which is concerned with providing explanations and recommendations to individuals who are unfavorably treated by automated decision-making systems. We first perform an extensive literature review, and align the efforts of many authors by presenting unified definitions, formulations, and solutions to recourse. Then, we provide an overview of the prospective research directions toward which the community may engage, challenging existing assumptions and making explicit connections to other ethical challenges such as security, privacy, and fairness.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3527848

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The recourse literature has evolved significantly, with work investigating robustness, fairness, and privacy risks of recourse.

    “The recourse literature has evolved significantly over the years (Karimi et al. 2022), with work investigating the robustness (Upadhyay, Joshi, and Lakkaraju 2021;König et al. 2025), fairness (Von Kügelgen et al. 2022), and privacy risks (Pawelczyk, Lakkaraju, and Neel 2023) of recourse.”

    From Explainable AI Isn't Enough! Rethinking Algorithmic Contestability (2026) · cites this paper as Karimi et al. 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 172

  1. Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y Lim, and Mohan Kankan- halli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1-18.
  2. Charu C Aggarwal, Chen Chen, and Jiawei Han. 2010. The inverse classification problem. Journal of Computer Science and Technology 25, 3 (2010), 458-468.
  3. Carlos Aguilar-Palacios, Sergio Muñoz-Romero, and José Luis Rojo-Álvarez. 2020. Cold-Start Promotional Sales Forecasting through Gradient Boosted-based Contrastive Explanations. IEEE Access (2020).
  4. Ulrich Aïvodji, Hiromi Arai, Olivier Fortineau, Sébastien Gambs, Satoshi Hara, and Alain Tapp. 2019. Fairwashing: the risk of rationalization. arXiv preprint arXiv:1901.09749 (2019). · arXiv
  5. Ulrich Aïvodji, Alexandre Bolot, and Sébastien Gambs. 2020. Model extraction from counterfactual explanations. arXiv preprint arXiv:2009.01884 (2020). · arXiv
  6. David Alvarez-Melis and Tommi S Jaakkola. 2018. On the robustness of inter- pretability methods. arXiv preprint arXiv:1806.08049 (2018). · arXiv
  7. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica, May 23 (2016), 2016.
  8. André Artelt and Barbara Hammer. 2019. Efficient computation of counterfactual explanations of LVQ models. arXiv preprint arXiv:1908.00735 (2019). · arXiv
  9. André Artelt and Barbara Hammer. 2019. On the computation of counterfactual explanations-A survey. arXiv preprint arXiv:1911.07749 (2019). · arXiv
  10. André Artelt and Barbara Hammer. 2020. Convex Density Constraints for Com- puting Plausible Counterfactual Explanations. arXiv preprint arXiv:2002.04862 (2020). · arXiv
  11. Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. 2017. Latent space oddity: on the curvature of deep generative models. arXiv preprint arXiv:1710.11379 (2017). · arXiv
  12. Georgios Arvanitidis, Søren Hauberg, and Bernhard Schölkopf. 2020. Geometri- cally Enriched Latent Spaces. arXiv preprint arXiv:2008.00565 (2020). · arXiv
  13. Vincent Ballet, Xavier Renard, Jonathan Aigrain, Thibault Laugel, Pascal Frossard, and Marcin Detyniecki. 2019. Imperceptible Adversarial Attacks on Tabular Data. arXiv preprint arXiv:1911.03274 (2019). · arXiv
  14. E Bareinboim, JD Correa, D Ibeling, and T Icard. 2020. On Pearl's hierarchy and the foundations of causal inference. ACM Special Volume in Honor of Judea Pearl (provisional title) (2020).
  15. Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2017. Fairness in machine learning. NIPS Tutorial 1 (2017).
  16. Solon Barocas, Andrew D Selbst, and Manish Raghavan. 2020. The hidden assumptions behind counterfactual explanations and principal reasons. In Pro- ceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 80-89.
  17. Alejandro Barredo-Arrieta and Javier Del Ser. 2020. Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples. arXiv preprint arXiv:2003.11323 (2020). · arXiv
  18. Clark Barrett, Christopher L Conway, Morgan Deters, Liana Hadarean, Dejan Jovanović, Tim King, Andrew Reynolds, and Cesare Tinelli. 2011. Cvc4. In International Conference on Computer Aided Verification. Springer, 171-177.
  19. Osbert Bastani, Carolyn Kim, and Hamsa Bastani. 2017. Interpretability via model extraction. arXiv preprint arXiv:1706.09773 (2017). · arXiv
  20. Yahav Bechavod, Katrina Ligett, Aaron Roth, Bo Waggoner, and Steven Z Wu. 2019. Equal opportunity in online classification with partial feedback. In Ad- vances in Neural Information Processing Systems. 8974-8984.
  21. Amir Beck and Marc Teboulle. 2009. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences 2, 1 (2009), 183-202.
  22. Leopoldo Bertossi. 2020. An ASP-Based Approach to Counterfactual Explana- tions for Classification. arXiv preprint arXiv:2004.13237 (2020). · arXiv
  23. Umang Bhatt, McKane Andrus, Adrian Weller, and Alice Xiang. 2020. Ma- chine Learning Explainability for External Stakeholders. arXiv preprint arXiv:2007.05408 (2020). · arXiv
  24. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yun- han Jia, Joydeep Ghosh, Ruchir Puri, José MF Moura, and Peter Eckersley. 2020. Explainable machine learning in deployment. In Proceedings of the 2020 Confer- ence on Fairness, Accountability, and Transparency. 648-657.
  25. Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe. 2004. Convex opti- mization. Cambridge university press.
  26. Leo Breiman et al. 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3 (2001), 199-231.
  27. Longbing Cao, Dan Luo, and Chengqi Zhang. 2007. Knowledge actionability: sat- isfying technical and business interestingness. International Journal of Business Intelligence and Data Mining 2, 4 (2007), 496-514.
  28. Longbing Cao and Chengqi Zhang. 2006. Domain-driven actionable knowledge discovery in the real world. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 821-830.
  29. Longbing Cao, Yanchang Zhao, Huaifeng Zhang, Dan Luo, Chengqi Zhang, and Eun Kyo Park. 2009. Flexible frameworks for actionable knowledge discovery. IEEE Transactions on Knowledge and Data Engineering 22, 9 (2009), 1299-1312.
  30. Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. 2019. Unlabeled data improves adversarial robustness. arXiv preprint arXiv:1905.13736 (2019). · arXiv
  31. Matt Chapman-Rounds, Marc-Andre Schulz, Erik Pazos, and Konstantinos Geor- gatzis. 2019. EMAP: Explanation by Minimal Adversarial Perturbation. arXiv preprint arXiv:1912.00872 (2019). · arXiv
  32. Weilun Chen, Zhaoxiang Zhang, Xiaolin Hu, and Baoyuan Wu. 2020. Boosting decision-based black-box adversarial attacks with random sign flip. In Proceed- ings of the European Conference on Computer Vision.
  33. Yatong Chen, Jialu Wang, and Yang Liu. 2020. Strategic Recourse in Linear Classification. arXiv preprint arXiv:2011.00355 (2020). · arXiv
  34. Furui Cheng, Yao Ming, and Huamin Qu. 2020. DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models. arXiv preprint arXiv:2008.08353 (2020). · arXiv
  35. Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. 2019. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning. PMLR, 1310-1320.
  36. Lee Cohen, Zachary C Lipton, and Yishay Mansour. 2019. Efficient candidate screening under multiple tests and implications for fairness. arXiv preprint arXiv:1905.11361 (2019). · arXiv
  37. Gregory F Cooper and Changwon Yoo. 1999. Causal discovery from a mixture of experimental and observational data. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence. 116-125.
  38. Sam Corbett-Davies and Sharad Goel. 2018. The measure and mismeasure of fair- ness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018). · arXiv
  39. Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. 2009. Introduction to algorithms. MIT press.
  40. IBM ILOG Cplex. 2009. V12. 1: User's Manual for CPLEX. International Business Machines Corporation 46, 53 (2009), 157.
  41. Zhicheng Cui, Wenlin Chen, Yujie He, and Yixin Chen. 2015. Optimal action extraction for random forests and boosted trees. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 179-188.
  42. Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. 2020. Multi- Objective Counterfactual Explanations. arXiv preprint arXiv:2004.11165 (2020). · arXiv
  43. Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337-340.
  44. Sarah Dean, Sarah Rich, and Benjamin Recht. 2020. Recommendations and user agency: the reachability of collaboratively-filtered information. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 436-445.
  45. Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592-603.
  46. Amit Dhurandhar, Tejaswini Pedapati, Avinash Balakrishnan, Pin-Yu Chen, Karthikeyan Shanmugam, and Ruchir Puri. 2019. Model agnostic contrastive explanations for structured data. arXiv preprint arXiv:1906.00117 (2019). · arXiv
  47. Jinshuo Dong, Aaron Roth, Zachary Schutzman, Bo Waggoner, and Zhi- wei Steven Wu. 2018. Strategic classification from revealed preferences. In Proceedings of the 2018 ACM Conference on Economics and Computation. 55-70.
  48. Jianfeng Du, Yong Hu, Charles X Ling, Ming Fan, and Mei Liu. 2011. Efficient action extraction with many-to-many relationship between actions and features. In International Workshop on Logic, Rationality and Interaction. Springer, 384- 385.
  49. Cynthia Dwork and Vitaly Feldman. 2018. Privacy-preserving prediction. arXiv preprint arXiv:1803.10266 (2018). · arXiv
  50. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference. ACM, 214-226.
  51. Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2015. Fundamental limits on adversarial robustness. In Proc. ICML, Workshop on Deep Learning.
  52. Carlos Fernandez, Foster Provost, and Xintian Han. 2020. Explaining data- driven decisions made by ai systems: The counterfactual approach. arXiv preprint arXiv:2001.07417 (2020). · arXiv
  53. Alex A Freitas. 2014. Comprehensible classification models: a position paper. ACM SIGKDD explorations newsletter 15, 1 (2014), 1-10.
  54. Marco Gario and Andrea Micheli. 2015. PySMT: a solver-agnostic library for fast prototyping of SMT-based algorithms. In SMT workshop, Vol. 2015.
  55. Tobias Gerstenberg, Matthew F Peterson, Noah D Goodman, David A Lagnado, and Joshua B Tenenbaum. 2017. Eye-tracking causality. Psychological science 28, 12 (2017), 1731-1744.
  56. Azin Ghazimatin, Oana Balalau, Rishiraj Saha Roy, and Gerhard Weikum. 2020. PRINCE: Provider-side Interpretability with Counterfactual Explanations in Recommender Systems. In Proceedings of the 13th International Conference on Web Search and Data Mining. 196-204.
  57. Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 80-89.
  58. Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Counterfactual visual explanations. arXiv preprint arXiv:1904.07451 (2019). · arXiv
  59. Rory Mc Grath, Luca Costabello, Chan Le Van, Paul Sweeney, Farbod Kamiab, Zhao Shen, and Freddy Lecue. 2018. Interpretable credit application predictions with counterfactual explanations. arXiv preprint arXiv:1811.05245 (2018). · arXiv
  60. Thomas Grote and Philipp Berens. 2020. On the ethics of algorithmic decision- making in healthcare. Journal of medical ethics 46, 3 (2020), 205-211.
  61. Riccardo Guidotti, Anna Monreale, Stan Matwin, and Dino Pedreschi. 2019. Black Box Explanation by Learning Image Exemplars in the Latent Feature Space. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 189-205.
  62. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. 2018. Local rule-based explanations of black box decision systems. arXiv preprint arXiv:1805.10820 (2018). · arXiv
  63. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 51, 5 (2018), 1-42.
  64. Vivek Gupta, Pegah Nokhiz, Chitradeep Dutta Roy, and Suresh Venkatasubrama- nian. 2019. Equalizing Recourse across Groups. arXiv preprint arXiv:1909.03166 (2019). · arXiv
  65. Joseph Y Halpern and Judea Pearl. 2005. Causes and explanations: A structural- model approach. Part I: Causes. The British journal for the philosophy of science 56, 4 (2005), 843-887.
  66. Leif Hancox-Li. 2020. Robustness in machine learning explanations: does it matter?. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 640-647.
  67. Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters. 2016. Strategic classification. In Proceedings of the 2016 ACM conference on innovations in theoretical computer science. 111-122.
  68. Masoud Hashemi and Ali Fathi. 2020. PermuteAttack: Counterfactual Expla- nation of Machine Learning Credit Scorecards. arXiv preprint arXiv:2008.10138 (2020). · arXiv
  69. Denis J Hilton. 1990. Conversational processes and causal explanation. Psycho- logical Bulletin 107, 1 (1990), 65.
  70. Denis J Hilton and Ben R Slugoski. 1986. Knowledge-based causal attribution: The abnormal conditions focus model. Psychological review 93, 1 (1986), 75.
  71. Lily Hu, Nicole Immorlica, and Jennifer Wortman Vaughan. 2019. The disparate effects of strategic manipulation. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 259-268.
  72. Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. 2018. Multimodal explanations: Justifying decisions and pointing to the evidence. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8779-8788.
  73. Christina Ilvento. 2019. Metric Learning for Individual Fairness. arXiv preprint arXiv:1906.00250 (2019). · arXiv
  74. Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black- box adversarial attacks with limited queries and information. arXiv preprint arXiv:1804.08598 (2018). · arXiv
  75. Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joy- deep Ghosh. 2019. REVISE: Towards Realistic Individual Recourse and Ac- tionable Explanations in Black-Box Decision Making Systems. arXiv preprint arXiv:1907.09615 (2019). · arXiv
  76. Sin-Han Kang, Hong-Gyu Jung, Dong-Ok Won, and Seong-Whan Lee. 2020. Counterfactual Explanation Based on Gradual Construction for Deep Networks. arXiv preprint arXiv:2008.01897 (2020). · arXiv
  77. Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. 2020. Model- agnostic counterfactual explanations for consequential decisions. In Interna- tional Conference on Artificial Intelligence and Statistics. 895-905.
  78. Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. 2020. Algorithmic Recourse: from Counterfactual Explanations to Interventions. arXiv preprint arXiv:2002.06278 (2020). · arXiv
  79. Amir-Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. 2020. Algorithmic recourse under imperfect causal knowledge: a proba- bilistic approach. arXiv preprint arXiv:2006.06831 (2020). · arXiv
  80. Mark T Keane and Barry Smyth. 2020. Good Counterfactuals and Where to Find Them: A Case-Based Technique for Generating Counterfactuals for Explainable AI (XAI). arXiv preprint arXiv:2005.13997 (2020). · arXiv
  81. Niki Kilbertus, Manuel Gomez Rodriguez, Bernhard Schölkopf, Krikamol Muan- det, and Isabel Valera. 2020. Fair decisions despite imperfect predictions. In International Conference on Artificial Intelligence and Statistics. 277-287.
  82. Jon Kleinberg and Manish Raghavan. 2020. How Do Classifiers Induce Agents to Invest Effort Strategically? ACM Transactions on Economics and Computation (TEAC) 8, 4 (2020), 1-23.
  83. Maxim S Kovalev and Lev V Utkin. 2020. Counterfactual explanation of machine learning survival models. arXiv preprint arXiv:2006.16793 (2020). · arXiv
  84. Himabindu Lakkaraju and Osbert Bastani. 2020. "How do I fool you?" Manipu- lating User Trust via Misleading Black Box Explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 79-85.
  85. Michael T Lash, Qihang Lin, Nick Street, Jennifer G Robinson, and Jeffrey Ohlmann. 2017. Generalized inverse classification. In Proceedings of the 2017 SIAM International Conference on Data Mining. SIAM, 162-170.
  86. Michael T Lash, Qihang Lin, and W Nick Street. 2018. Prophit: Causal inverse classification for multiple continuously valued treatment policies. arXiv preprint arXiv:1802.04918 (2018). · arXiv
  87. Michael T Lash, Qihang Lin, W Nick Street, and Jennifer G Robinson. 2017. A budget-constrained inverse classification framework for smooth classifiers. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 1184-1193.
  88. Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, and Marcin De- tyniecki. 2019. Issues with post-hoc counterfactual explanations: a discussion. arXiv preprint arXiv:1906.04774 (2019). · arXiv
  89. Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. Inverse Classification for Comparison-based Inter- pretability in Machine Learning. arXiv preprint arXiv:1712.08443 (2017). · arXiv
  90. Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2019. The dangers of post-hoc interpretability: Unjustified counterfactual explanations. arXiv preprint arXiv:1907.09294 (2019). · arXiv
  91. Eugene L Lawler and David E Wood. 1966. Branch-and-bound methods: A survey. Operations research 14, 4 (1966), 699-719.
  92. David K Lewis. 1973. Counterfactuals. Harvard University Press.
  93. Lydia T Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer Chayes. 2020. The disparate equilibria of algorithmic de- cision making when individuals invest rationally. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 381-391.
  94. Daniel Lowd and Christopher Meek. 2005. Adversarial learning. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining. ACM, 641-647.
  95. Ana Lucic, Hinda Haned, and Maarten de Rijke. 2020. Why does my model fail? contrastive local explanations for retail forecasting. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 90-98.
  96. Ana Lucic, Harrie Oosterhuis, Hinda Haned, and Maarten de Rijke. 2019. Ac- tionable Interpretability through Optimizable Counterfactual Explanations for Tree Ensembles. arXiv preprint arXiv:1911.12199 (2019). · arXiv
  97. Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2020. Distal Explanations for Explainable Reinforcement Learning Agents. arXiv preprint arXiv:2001.10284 (2020). · arXiv
  98. Divyat Mahajan, Chenhao Tan, and Amit Sharma. 2019. Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers. arXiv preprint arXiv:1912.03277 (2019). · arXiv
  99. Michael V Mannino and Murlidhar V Koushik. 2000. The cost-minimizing inverse classification problem: a genetic algorithm approach. Decision Support Systems 29, 3 (2000), 283-300.
  100. Ann L McGill and Jill G Klein. 1993. Contrastive and counterfactual reasoning in causal judgment. Journal of Personality and Social Psychology 64, 6 (1993), 897.
  101. David Alvarez Melis and Tommi Jaakkola. 2018. Towards robust interpretabil- ity with self-explaining neural networks. In Advances in Neural Information Processing Systems. 7775-7784.
  102. John Miller, Smitha Milli, and Moritz Hardt. 2020. Strategic classification is causal modeling in disguise. In International Conference on Machine Learning. PMLR, 6917-6926.
  103. Tim Miller. 2018. Contrastive explanation: A structural-model approach. arXiv preprint arXiv:1811.03163 (2018). · arXiv
  104. Smitha Milli, John Miller, Anca D Dragan, and Moritz Hardt. 2019. The social cost of strategic classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 230-239.
  105. Smitha Milli, Ludwig Schmidt, Anca D Dragan, and Moritz Hardt. 2019. Model reconstruction from model explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 1-9.
  106. Brent Mittelstadt, Chris Russell, and Sandra Wachter. 2019. Explaining expla- nations in AI. In Proceedings of the conference on fairness, accountability, and transparency. 279-288.
  107. Kiarash Mohammadi, Amir-Hossein Karimi, Gilles Barthe, and Isabel Valera. 2020. Scaling Guarantees for Nearest Counterfactual Explanations. arXiv preprint arXiv:2010.04965 (2020). · arXiv
  108. Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. 2018. Meth- ods for interpreting and understanding deep neural networks. Digital Signal Processing 73 (2018), 1-15.
  109. Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Proceed- ings of the IEEE conference on computer vision and pattern recognition. 2574-2582.
  110. Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, and Huan Liu. 2020. Causal Interpretability for Machine Learning-Problems, Methods and Evaluation. ACM SIGKDD Explorations Newsletter 22, 1 (2020), 18-33.
  111. Ramaravind Kommiya Mothilal, Amit Sharma, and Chenhao Tan. 2019. DiCE: Explaining Machine Learning Classifiers through Diverse Counterfactual Ex- planations. arXiv preprint arXiv:1905.07697 (2019). · arXiv
  112. Amitabha Mukerjee, Rita Biswas, Kalyanmoy Deb, and Amrit P Mathur. 2002. Multi-objective evolutionary algorithms for the risk-return trade-off in bank loan management. International Transactions in operational research 9, 5 (2002), 583-597.
  113. Razieh Nabi and Ilya Shpitser. 2018. Fair inference on outcomes. In Proceedings of the... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence, Vol. 2018. NIH Public Access, 1931.
  114. Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition. 427-436.
  115. Jorge Nocedal and Stephen Wright. 2006. Numerical optimization. Springer Science & Business Media.
  116. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security. 506-519.
  117. Jaehyun Park and Stephen Boyd. 2017. General heuristics for nonconvex quadrat- ically constrained quadratic programming. arXiv preprint arXiv:1703.07870 (2017). · arXiv
  118. Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. 2020. On Counterfac- tual Explanations under Predictive Multiplicity. arXiv preprint arXiv:2006.13132 (2020). · arXiv
  119. Martin Pawelczyk, Johannes Haug, Klaus Broelemann, and Gjergji Kasneci. 2019. Towards User Empowerment. arXiv preprint arXiv:1910.09398 (2019). · arXiv
  120. Judea Pearl. 2000. Causality: models, reasoning and inference. Vol. 29. Springer.
  121. Judea Pearl. 2010. The foundations of causal inference. Sociological Methodology 40, 1 (2010), 75-149.
  122. Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. 2016. Causal inference in statistics: A primer. John Wiley & Sons.
  123. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532-1543.
  124. Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. 2017. Elements of causal inference. The MIT Press.
  125. Rafael Poyiadzi, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2019. FACE: Feasible and Actionable Counterfactual Explanations. arXiv preprint arXiv:1909.09369 (2019). · arXiv
  126. Goutham Ramakrishnan, Yun Chan Lee, and Aws Albargouthi. 2019. Syn- thesizing Action Sequences for Modifying Model Decisions. arXiv preprint arXiv:1910.00057 (2019). · arXiv
  127. Yanou Ramon, David Martens, Foster Provost, and Theodoros Evgeniou. 2019. Counterfactual explanation algorithms for behavioral and textual data. arXiv preprint arXiv:1912.01819 (2019). · arXiv
  128. Shubham Rathi. 2019. Generating counterfactual and contrastive explanations using SHAP. arXiv preprint arXiv:1906.09293 (2019). · arXiv
  129. Kaivalya Rawal, Ece Kamar, and Himabindu Lakkaraju. 2020. Can I Still Trust You?: Understanding the Impact of Distribution Shifts on Algorithmic Recourses. arXiv preprint arXiv:2012.11788 (2020). · arXiv
  130. Kaivalya Rawal and Himabindu Lakkaraju. 2020. Beyond Individualized Re- course: Interpretable and Interactive Summaries of Actionable Recourses. Ad- vances in Neural Information Processing Systems 33 (2020).
  131. Robert Nikolai Reith, Thomas Schneider, and Oleksandr Tkachenko. 2019. Effi- ciently stealing your machine learning models. In Proceedings of the 18th ACM Workshop on Privacy in the Electronic Society. 198-210.
  132. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should I trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135-1144.
  133. Marcel Jurriaan Robeer. 2018. Contrastive explanation for machine learning. Master's thesis.
  134. Nir Rosenfeld, Anna Hilgard, Sai Srivatsa Ravindranath, and David C Parkes. 2020. From Predictions to Decisions: Using Lookahead Regularization. Advances in Neural Information Processing Systems 33 (2020).
  135. Alexis Ross, Himabindu Lakkaraju, and Osbert Bastani. 2020. Ensuring Ac- tionable Recourse via Adversarial Training. arXiv preprint arXiv:2011.06146 (2020). · arXiv
  136. David-Hillel Ruben. 2015. Explaining explanation. Routledge.
  137. Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (2019), 206-215.
  138. Chris Russell. 2019. Efficient Search for Diverse Coherent Explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19). ACM, 20-28. https://doi.org/10.1145/3287560.3287569 · DOI · Paper page
  139. Bernhard Schölkopf. 2019. Causality for machine learning. arXiv preprint arXiv:1911.10500 (2019). · arXiv
  140. Candice Schumann, Jeffrey S Foster, Nicholas Mattei, and John P Dickerson. 2020. We Need Fairness and Explainability in Algorithmic Hiring. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems. 1716-1720.
  141. Andrew D Selbst and Solon Barocas. 2018. The intuitive appeal of explainable machines. Fordham L. Rev. 87 (2018), 1085.
  142. Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2019. CERTIFAI: Coun- terfactual Explanations for Robustness, Transparency, Interpretability, and Fair- ness of Artificial Intelligence models. arXiv preprint arXiv:1905.07857 (2019). · arXiv
  143. Reza Shokri, Martin Strobel, and Yair Zick. 2019. Privacy risks of explaining machine learning models. arXiv preprint arXiv:1907.00164 (2019). · arXiv
  144. Kacper Sokol and Peter A Flach. 2018. Conversational Explanations of Machine Learning Predictions Through Class-contrastive Counterfactual Statements.. In IJCAI. 5785-5786.
  145. Kacper Sokol and Peter A Flach. 2019. Counterfactual explanations of machine learning predictions: opportunities and challenges for AI safety. In SafeAI@ AAAI.
  146. Suvrit Sra, Sebastian Nowozin, and Stephen J Wright. 2012. Optimization for machine learning. Mit Press.
  147. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013). · arXiv
  148. Jin Tian and Judea Pearl. 2001. Causal discovery from changes. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence. 512-521.
  149. Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. Interpretable predictions of tree-based ensembles via actionable feature tweak- ing. In Proceedings of the 23rd ACM SIGKDD international conference on knowl- edge discovery and data mining. 465-474.
  150. Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction apis. In 25th {USENIX} Security Symposium ({USENIX} Security 16). 601-618.
  151. Stratis Tsirtsis and Manuel Gomez-Rodriguez. 2020. Decisions, Counterfactual Explanations and Strategic Behavior. arXiv preprint arXiv:2002.04333 (2020). · arXiv
  152. Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 10-19.
  153. Jasper van der Waa, Marcel Robeer, Jurriaan van Diggelen, Matthieu Brinkhuis, and Mark Neerincx. 2018. Contrastive explanations with local foil trees. arXiv preprint arXiv:1806.07470 (2018). · arXiv
  154. Arnaud Van Looveren and Janis Klaise. 2019. Interpretable Counterfactual Explanations Guided by Prototypes. arXiv preprint arXiv:1907.02584 (2019). · arXiv
  155. Suresh Venkatasubramanian and Mark Alfano. 2020. The philosophical basis of algorithmic recourse. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM.
  156. Tom Vermeire and David Martens. 2020. Explainable Image Classification with Evidence Counterfactual. arXiv preprint arXiv:2004.07511 (2020). · arXiv
  157. Julius von Kügelgen, Amir-Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, and Bernhard Schölkopf. 2020. On the Fairness of Causal Algorithmic Recourse. arXiv preprint arXiv:2010.06529 (2020). · arXiv
  158. Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology 31, 2 (2017).
  159. David E Wallin. 1992. Legal recourse and the demand for auditing. Accounting Review (1992), 121-147.
  160. Binghui Wang and Neil Zhenqiang Gong. 2018. Stealing hyperparameters in machine learning. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 36-52.
  161. Pei Wang and Nuno Vasconcelos. 2020. SCOUT: Self-aware Discriminant Coun- terfactual Explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8981-8990.
  162. Adrian Weller. 2017. Challenges for transparency. arXiv preprint arXiv:1708.01870 (2017). · arXiv
  163. James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fer- nanda Viégas, and Jimbo Wilson. 2019. The What-If Tool: Interactive Probing of Machine Learning Models. IEEE transactions on visualization and computer graphics 26, 1 (2019), 56-65.
  164. Adam White and Artur d'Avila Garcez. 2019. Measurable counterfactual local explanations for any classifier. arXiv preprint arXiv:1908.03020 (2019). · arXiv
  165. Darrell Whitley. 1994. A genetic algorithm tutorial. Statistics and computing 4, 2 (1994), 65-85.
  166. James Woodward. 2005. Making things happen: A theory of causal explanation. Oxford university press.
  167. James Woodward. 2016. Causation and Manipulability. In The Stanford Ency- clopedia of Philosophy (winter 2016 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University.
  168. Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. 2019. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 501-509.
  169. Qiang Yang, Jie Yin, Charles Ling, and Rong Pan. 2006. Extracting actionable knowledge from decision trees. IEEE Transactions on Knowledge and data Engineering 19, 1 (2006), 43-56.
  170. I-Cheng Yeh and Che-hui Lien. 2009. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Systems with Applications 36, 2 (2009), 2473-2480.
  171. Xin Zhang, Armando Solar-Lezama, and Rishabh Singh. 2018. Interpreting neural network judgments via minimal, stable, and symbolic corrections. In Advances in Neural Information Processing Systems. 4874-4885.
  172. Yunxia Zhao. 2020. Fast Real-time Counterfactual Explanations. arXiv preprint arXiv:2007.05684 (2020). · arXiv

Cited by

Sorted by citations

Authors
Karimi, A.-H., Barthe, G., Schölkopf, B., & Valera, I.
Affiliations
  • ETH Zurich (CH)
  • Max Planck Institute for Security and Privacy (DE)
  • MPI for Intelligent Systems, Germany
  • Saarland University (DE)
ORCID
Venue
ACM Computing Surveys
Publisher
Association for Computing Machinery (ACM)
Published
2022-12-03
Volume
55
Issue
5
Pages
1-29
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
DOI
10.1145/3527848
OpenAlex
W4226290207
Cited by
301
References
28
Citations by year
2026: 71 · 2025: 44 · 2024: 60 · 2023: 60 · 2022: 27 · 2021: 36 · 2020: 3
FWCI
14.90
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Computing Surveys (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by-nc)
  • arXiv (Cornell University)
  • Publications of the UdS (Saarland University)
  • ETH Zürich Research Collection
Funders
Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
algorithmic recourse, contrastive explanations, automated decision systems, unfair treatment, explainability, fairness, privacy, accountability, security (computing)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review