CoChat
Sign Up
Cited by 23Ethics and Social Impacts of AIDigitalization, Law, and RegulationArtificial Intelligence in Law

Taking Algorithms to Courts: A Relational Approach to Algorithmic Accountability

Open access
Metcalf, Jacob, Singh, Ranjit, Moss, Emanuel, Tafesse, Emnet, Watkins, Elizabeth Anne·2023·2023 ACM Conference on Fairness, Accountability, and Transparency
DOI 10.1145/3593013.3594092
Open this paper in CoChat — free accountOpen PDF

Abstract

In widely used sociological descriptions of how accountability is structured through institutions, an “actor” (e.g., the developer) is accountable to a “forum” (e.g., regulatory agencies) empowered to pass judgements on and demand changes from the actor or enforce sanctions. However, questions about structuring accountability persist: why and how is a forum compelled to keep making demands of the actor when such demands are called for? To whom is a forum accountable in the performance of its responsibilities, and how can its practices and decisions be contested? In the context of algorithmic accountability, we contend that a robust accountability regime requires a triadic relationship, wherein the forum is also accountable to another entity: the public(s). Typically, as is the case with environmental impact assessments, public(s) make demands upon the forum's judgements and procedures through the courts, thereby establishing a minimum standard of due diligence. However, core challenges relating to: (1) lack of documentation, (2) difficulties in claiming standing, and (3) struggles around admissibility of expert evidence on and achieving consensus over the workings of algorithmic systems in adversarial proceedings prevent the public from approaching the courts when faced with algorithmic harms. In this paper, we demonstrate that the courts are the primary route—and the primary roadblock—in the pursuit of redress for algorithmic harms. Courts often find algorithmic harms non-cognizable and rarely require developers to address material claims of harm. To address the core challenges of taking algorithms to court, we develop a relational approach to algorithmic accountability that emphasizes not what the actors do nor the results of their actions, but rather how interlocking relationships of accountability are constituted in a triadic relationship between actors, forums, and public(s). As is the case in other regulatory domains, we believe that impact assessments (and similar accountability documentation) can provide the grounds for contestation between these parties, but only when that triad is structured such that the public(s) are able to cohere around shared experiences and interests, contest the outcomes of algorithmic systems that affect their lives, and make demands upon the other parties. Where courts now find algorithmic harms non-cognizable, an impact assessment regime can potentially create procedural rights to protect substantive rights of the public(s). This would require algorithmic accountability policies currently under consideration to provide the public(s) with adequate standing in courts, and opportunities to access and contest the actor's documentation and the forum's judgments.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3593013.3594092

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Policy experts have argued that the chief outcomes of disclosure-based regulations are the capacities they build to achieve at least some compliance and facilitate grass-roots contestation.

    “Policy experts have argued that the chief outcomes of disclosure-based regulations are the capacities they build to achieve at least some compliance [58] and facilitate grass-roots contestation [44].”

    From Null Compliance: NYC Local Law 144 and the challenges of algorithm accountability (2024) · cites this paper as [44]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 68 references couldn’t be matched to a known paper.

Research works printed but not linked: 17

  1. ACLU. 2021. Michigan Father Sues Detroit Police Department for Wrongful Arrest Based on Faulty Facial Recognition Technology. Retrieved January 2, 2022 from https://www.aclu.org/press-releases/michigan-father-sues-detroit- police-department-wrongful-arrest-based-faulty-facial
  2. Kenneth A. Bamberger and Deirdre K. Mulligan. 2012. PIA requirements and privacy decision-making in US government agencies. In Privacy Impact Assess- ment. Springer, Dordrecht, 225-250. Retrieved from https://link.springer.com/ chapter/10.1007/978-94-007-2543-0_10 · DOI
  3. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Lan- guage (Technology) is Power: A Critical Survey of "Bias" in NLP. arXiv:2005. 14050 [cs]. Retrieved November 12, 2021 from http://arxiv.org/abs/2005.14050 · arXiv
  4. Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O'Keefe, Mark Koren, Théo Ryffel, J. B. Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh, Frens Kroeger, Girish Sastry, Re- becca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, and Markus An- derljung. 2020. Toward Trustworthy AI Development: Mechanisms for Support- ing Verifiable Claims. arXiv:2004.07213 [cs]. Retrieved December 29, 2020 from http://arxiv.org/abs/2004.07213 · arXiv
  5. Robert D. Bullard. 1999. Anatomy of Environmental Racism and the Environ- mental Justice Movement. In Confronting Environmental Racism: Voices From the Grassroots, Robert D. Bullard (ed.). South End Press.
  6. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accu- racy Disparities in Commercial Gender Classification. In Proceedings of Ma- chine Learning Research, 1-15. Retrieved from http://proceedings.mlr.press/v81/ buolamwini18a.html
  7. Joel Feinberg. 1984. Harm to Others. Oxford University Press.
  8. W. B. Gallie. 1955. Essentially Contested Concepts. Proceedings of the Aristotelian Society 56: 167-198.
  9. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2018. Datasheets for Datasets. In Proceedings of the 5th Workshop on Fairness, Accountabil- ity, and Transparency in Machine Learning. Retrieved July 6, 2020 from http: //arxiv.org/abs/1803.09010
  10. Margot E Kaminski and Jennifer M Urban. 2021. The Right to Contest AI. Columbia Law Review 121, 7: 1957-2048.
  11. Ida Koivisto. 2022. The Transparency Paradox. Oxford University Press, Oxford, United Kingdom.
  12. Joshua A Kroll, Joanna Huey, Solon Barocas, Edward W Felten, Joel R Reidenberg, David G Robinson, and Harlan Yu. 2017. Accountable Algorithms. University of Pennsylvania Law Review 165: 633-705.
  13. Mutale Nkonde. 2019. Automated Anti-Blackness: Facial Recognition in Brook- lyn, New York. Harvard Journal of African American Public Policy 20: 30-36.
  14. Sara Pirk. 2002. Expanding Public Participation in Environmental Justice: Meth- ods, Legislation, Litigation and Beyond. Journal of Environmental Law and Litigation 17, 1: 35.
  15. Robert Proctor and Londa L. Schiebinger (eds.). 2008. Agnotology: the making and unmaking of ignorance. Stanford University Press, Stanford, CA.
  16. John W. Wade. 1973. On the Nature of Strict Tort Liability for Products. Missis- sippi Law Journal 44, 5: 825-851.
  17. 1362019. Protect Our Communities Foundation v. LaCounte. Retrieved from https: //scholar.google.com/scholar_case?case$=$17922905489759569030

Cited by

Sorted by citations

Related

Authors
Metcalf, J., Singh, R., Moss, E., Tafesse, E., & Watkins, E. A.
Affiliations
  • Data & Society Research Institute (US)
  • Intel (United States) (US)
ORCID
Venue
2023 ACM Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2023-06-12
Pages
1450-1462
DOI
10.1145/3593013.3594092
OpenAlex
W4380318701
Cited by
21
References
68
Citations by year
2026: 7 · 2025: 7 · 2024: 6 · 2023: 1
FWCI
5.26
Type
conference paper
Language
en
Open access
Licence
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc-nd)
Funders
  • National Science Foundation (1704425)
  • Siegel Family Foundation
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Keywords
algorithmic accountability, algorithmic harms, courts, relational accountability, standing, admissibility of expert evidence, environmental impact assessment, procedural rights, citizen participation, due diligence, algorithmic transparency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review