CoChat
Sign Up
PreprintCited by 8Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AISafety Systems Engineering in Autonomy

Explainable AI Systems Must Be Contestable: Here's How to Make It Happen

Open access
Moreira, Catarina, Palatkina, Anna, Braca, Dacia, Walsh, Dylan M., Leihn, Peter J., Chen, Fang, Hubig, Nina C.·2025·arXiv
DOI 10.48550/arxiv.2506.01662arXiv 2506.01662
Open this paper in CoChat — free accountOpen PDF

Abstract

As AI regulations around the world intensify their focus on system safety, contestability has become a mandatory, yet ill-defined, safeguard. In XAI, "contestability" remains an empty promise: no formal definition exists, no algorithm guarantees it, and practitioners lack concrete guidance to satisfy regulatory requirements. Grounded in a systematic literature review, this paper presents the first rigorous formal definition of contestability in explainable AI, directly aligned with stakeholder requirements and regulatory mandates. We introduce a modular framework of by-design and post-hoc mechanisms spanning human-centered interfaces, technical architectures, legal processes, and organizational workflows. To operationalize our framework, we propose the Contestability Assessment Scale, a composite metric built on more than twenty quantitative criteria. Through multiple case studies across diverse application domains, we reveal where state-of-the-art systems fall short and show how our framework drives targeted improvements. By converting contestability from regulatory theory into a practical framework, our work equips practitioners with the tools to embed genuine recourse and accountability into AI systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2506.01662

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Moreira et al. (2025) attempted to formalize contestability in a broad sense, encompassing risk analysis, accountability, and decision reversal.

    “More recently, Moreira et al. (2025) attempted to formalize contestability in a comparatively broad sense, encompassing the facilitation of risk analysis, the assurance of accountability, and the enablement of decision reversal.”

    From Explainable AI Isn't Enough! Rethinking Algorithmic Contestability (2026) · cites this paper as Moreira et al. 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 39

  1. Sarath Sreedharan, Utkarsh Soni, Mudit Verma, Siddharth Srivastava, and Subbarao Kambhampati. Bridging the gap: Providing post-hoc symbolic explanations for se- quential decision-making problems with inscrutable representations. arXiv preprint arXiv:2002.01080, 2020. · arXiv
  2. Swati Srivastava and Justin Bullock. Ai, global governance, and digital sovereignty. arXiv preprint arXiv:2410.17481, 2024. · arXiv
  3. Kars Alfrink, Ianus Keller, Neelke Doorn, and Gerd Kortuem. Contestable camera cars: a speculative design exploration of public ai that is open and responsive to dispute. In Proceedings of the 2023 CHI conference on human factors in computing systems, pages 1-16, 2023.
  4. Henrietta Lyons, Eduardo Velloso, and Tim Miller. Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proceedings of the ACM on Human- Computer Interaction, 5(CSCW1):1-25, 2021.
  5. Joel Walmsley. Artificial intelligence and the value of transparency. AI & society, 36(2): 585-595, 2021.
  6. Kars Alfrink, Ianus Keller, Gerd Kortuem, and Neelke Doorn. Contestable ai by design: Towards a framework. Minds and Machines, 33(4):613-639, 2023.
  7. Antonio Aloisi. Regulating algorithmic management at work in the european union: Data protection, non-discrimination and collective rights. International Journal of Comparative Labour Law and Industrial Relations, 40(1), 2024.
  8. Rahaf Alharbi, Pa Lor, Jaylin Herskovitz, Sarita Schoenebeck, and Robin N Brewer. Misfitting with ai: How blind people verify and contest ai errors. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility, pages 1-17, 2024.
  9. Gabriel Freedman, Adam Dejl, Deniz Gorur, Xiang Yin, Antonio Rago, and Francesca Toni. Argumentative large language models for explainable and contestable claim verification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 14930-14939, 2025.
  10. Winston Maxwell and Bruno Dumas. Meaningful xai based on user-centric design methodology: Combining legal and human-computer interaction (hci) approaches to achieve meaningful algorithmic explainability. Available at SSRN 4520754, 2023.
  11. Marco Almada. Human intervention in automated decision-making: Toward the construc- tion of contestable systems. In Proceedings of the Seventeenth International Conference on artificial intelligence and law, pages 2-11, 2019.
  12. Rosanna Fanni, Valerie Eveline Steinkogler, Giulia Zampedri, and Jo Pierson. Enhancing human agency through redress in artificial intelligence systems. AI & society, 38(2): 537-547, 2023.
  13. Isil Guzey, Ozlem Ucar, Nukhet Aladag Ciftdemir, and Betul Acunas. Context-dependent explainability and contestability for trustworthy medical artificial intelligence: Misclassi- fication identification of morbidity recognition models in preterm infants. arXiv preprint arXiv:2212.08821, 2022. · arXiv
  14. Sukumar Ganapati and Kevin Desouza. Public value principles for secure and trusted ai. In Proceedings of the 25th Annual International Conference on Digital Government Research, pages 251-257, 2024.
  15. Falaah Arif Khan, Eleni Manis, and Julia Stoyanovich. Towards substantive conceptions of algorithmic fairness: Normative guidance from equal opportunity doctrines. In Pro- ceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pages 1-10, 2022.
  16. Sundaraparipurnan Narayanan and Mark Potkewitz. A risk-based approach to assessing liability risk for ai-driven harms considering eu liability directive. arXiv preprint arXiv:2401.11697, 2023. · arXiv
  17. Deirdre K Mulligan, Daniel Kluttz, and Nitin Kohli. Shaping our tools: Contestability as a means to promote responsible algorithmic decision making in the professions. Available at SSRN 3311894, 2019.
  18. Tad Hirsch, Kritzia Merced, Shrikanth Narayanan, Zac E Imel, and David C Atkins. Designing contestability: Interaction design, machine learning, and mental health. In Proceedings of the 2017 Conference on Designing Interactive Systems, pages 95-99, 2017.
  19. Andrea Aler Tubella, Andreas Theodorou, Virginia Dignum, and Loizos Michael. Con- testable black boxes. In Rules and Reasoning: 4th International Joint Conference, RuleML+ RR 2020, Oslo, Norway, June 29-July 1, 2020, Proceedings 4, pages 159-167. Springer, 2020.
  20. Francesco Leofante, Hamed Ayoobi, Adam Dejl, Gabriel Freedman, Deniz Gorur, Junqi Jiang, Guilherme Paulino-Passos, Antonio Rago, Anna Rapberger, Fabrizio Russo, et al. Contestable ai needs computational argumentation. arXiv preprint arXiv:2405.10729, 2024. · arXiv
  21. Muhammad Raees, Inge Meijerink, Ioanna Lykourentzou, Vassilis-Javed Khan, and Konstantinos Papangelis. From explainable to interactive ai: A literature review on current trends in human-ai interaction. International Journal of Human-Computer Studies, page 103301, 2024.
  22. Mireia Yurrita, Agathe Balayn, and Ujwal Gadiraju. Generating process-centric ex- planations to enable contestability in algorithmic decision-making: Challenges and opportunities. arXiv preprint arXiv:2305.00739, 2023. · arXiv
  23. Naveena Karusala, Sohini Upadhyay, Rajesh Veeraraghavan, and Krzysztof Z Gajos. Understanding contestability on the margins: Implications for the design of algorithmic decision-making in public services. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1-16, 2024.
  24. Rowena Rodrigues. Legal and human rights issues of ai: Gaps, challenges and vulnera- bilities. Journal of Responsible Technology, 4:100005, 2020.
  25. Virginia Eubanks. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin's Press, 2018.
  26. Ruha Benjamin. Assessing risk, automating racism. Science, 366(6464):421-422, 2019.
  27. Alexandra Chouldechova and Aaron Roth. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810, 2018. · arXiv
  28. Sarah Myers West. Censored, suspended, shadowbanned: User interpretations of content moderation on social media platforms. New Media & Society, 20(11):4366-4383, 2018.
  29. Robert Gorwa, Reuben Binns, and Christian Katzenbach. Algorithmic content modera- tion: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1):2053951719897945, 2020.
  30. Isak Mendoza and Lee A Bygrave. The right not to be subject to automated decisions based on profiling. EU internet law: Regulation and enforcement, pages 77-98, 2017.
  31. Yulu Pi. Empowering individuals in automated decision-making: Explainability, con- testability and beyond. In Companion Publication of the 2024 Conference on Computer- Supported Cooperative Work and Social Computing, pages 1-4, 2024.
  32. Conrad Sanderson, Qinghua Lu, David Douglas, Xiwei Xu, Liming Zhu, and Jon Whittle. Towards implementing responsible ai. In 2022 IEEE International Conference on Big Data (Big Data), pages 5076-5081. IEEE, 2022.
  33. Berk Ustun, Alexander Spangher, and Yang Liu. Actionable recourse in linear classifi- cation. In Proceedings of the conference on fairness, accountability, and transparency, pages 10-19, 2019.
  34. Suresh Venkatasubramanian and Mark Alfano. The philosophical basis of algorith- mic recourse. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 284-293, 2020.
  35. Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. Interpretable to whom? a role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552, 2018. · arXiv
  36. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9459-9474.
  37. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/ 2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf.
  38. Andreas Holzinger, André Carrington, and Heimo Müller. Measuring the quality of explanations: the system causability scale (scs) comparing human and machine explanations. KI-Künstliche Intelligenz, 34(2):193-198, 2020.
  39. Thomas B Sheridan. Telerobotics, automation, and human supervisory control. MIT press, 1992.

Cited by

Sorted by citations

Authors
Moreira, C., Palatkina, A., Braca, D., Walsh, D. M., Leihn, P. J., Chen, F., & Hubig, N. C.
Affiliations
  • University of Technology Sydney (AU)
  • Interdisciplinary Transformation University Austria (AT)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2506.01662
arXiv
2506.01662
OpenAlex
W4414897597
Cited by
8
References
0
Citations by year
2026: 7 · 2025: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Keywords
contestability, explainable AI, AI regulation, accountability, recourse, AI safety, regulatory compliance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review