CoChat
Sign Up
Cited by 67Ethics and Social Impacts of AIPolitical Philosophy and EthicsPhilosophical Ethics and Theory

The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic Decision-Making Systems

Open access
Creel, Kathleen, Hellman, Deborah·2022·Canadian Journal of Philosophy
DOI 10.1017/can.2022.3
Open this paper in CoChat — free accountOpen PDF

Abstract

This article examines the complaint that arbitrary algorithmic decisions wrong those whom they affect. It makes three contributions. First, it provides an analysis of what arbitrariness means in this context. Second, it argues that arbitrariness is not of moral concern except when special circumstances apply. However, when the same algorithm or different algorithms based on the same data are used in multiple contexts, a person may be arbitrarily excluded from a broad range of opportunities. The third contribution is to explain why this systemic exclusion is of moral concern and to offer a solution to address it.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1017/can.2022.3

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Increased reliance on LLMs has the potential to produce a kind of legal monoculture, with users being fed information from only a limited subset of judicial sources that elide many of the deeper nuances of the law.

    “increased reliance on LLMs also has the potential to produce a kind of legal monoculture (Creel and Hellman 2022; Kleinberg and Raghavan 2021), with users being fed information from only a limited subset of judicial sources that elide many of the deeper nuances of the law.”

    From Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models (2024) · cites this paper as Creel and Hellman 2022

    Section 3.a, “Why arbitrariness at scale arises” (page unavailable), says that reliance on similar datasets can standardize different algorithms and produce uniform outcomes. That supports the general concern about algorithmic monoculture, but the paper does not address LLMs feeding users a limited subset of judicial sources or eliding nuances of law. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. Anderson, Elizabeth S. 1999. "What Is the Point of Equality?" Ethics 109 (2): 287-337.
  2. Barocas, Solon, and Karen Levy. 2020. "Privacy Dependencies." Washington Law Review 95 (2): 555.
  3. Breiman, Leo. 2001. "Statistical Modeling: The Two Cultures." Statistical Science 16 (3): 199-231. https://doi.org/10.1214/ss/ 1009213726. · DOI · Paper page
  4. Bright, Liam Kofi, Daniel Malinsky, and Morgan Thompson. 2016. "Causally Interpreting Intersectionality Theory." Philosophy of Science 83 (1): 60-81. https://doi.org/10.1086/684173. · DOI
  5. Carbado, Devon W. 2013. "Colorblind Intersectionality." Signs: Journal of Women in Culture and Society 38 (4): 811-45. https:// doi.org/10.1086/669666. · DOI · Paper page
  6. Chemerinsky, Erwin. 1983. "In Defense of Equality: A Reply to Professor Westen." Michigan Law Review 81 (3): 575. https:// doi.org/10.2307/1288510. · DOI
  7. Citron, Danielle Keats, and Frank Pasquale. 2014. "The Scored Society: Due Process for Automated Predictions." Washington Law Review 89: 1.
  8. Corbett-Davies, Sam, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. "Algorithmic Decision Making and the Cost of Fairness." ArXiv:1701.08230. · arXiv
  9. Creel, Kathleen A. 2020. "Transparency in Complex Computational Systems." Philosophy of Science 87 (4): 568-89.
  10. Eubanks, Virginia. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York: St. Martin's Press.
  11. Fishkin, Joseph. 2014. Bottlenecks: A New Theory of Equal Opportunity. Oxford: Oxford University Press.
  12. Fourcade, Marion, and Kieran Healy. 2013. "Classification Situations: Life-Chances in the Neoliberal Era." Accounting, Organizations and Society 38 (8): 559-72. https://doi.org/10.1016/j.aos.2013.11.002. · DOI · Paper page
  13. Gandy, Jr., Oscar H. 2021. The Panoptic Sort: A Political Economy of Personal Information. New York: Oxford University Press.
  14. Gilpin, Leilani H., David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. "Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning." ArXiv:1806.00069. · arXiv
  15. Goh, Gabriel. 2019. "A Discussion of 'Adversarial Examples Are Not Bugs, They Are Features': Two Examples of Useful, Non- Robust Features." Distill 4 (8). e00019.3. https://doi.org/10.23915/distill.00019. · DOI
  16. Green, Anthony. 2021. "Want a Job? The AI Will See You Now." July 7, 2021. MIT Technology Review (podcast). https:// www.technologyreview.com/2021/07/07/1043089/podcast-want-a-job-the-ai-will-see-you-now-2/. · DOI
  17. Grgić-Hlača, Nina, Muhammad Bilal Zafar, Krishna P. Gummadi, and Adrian Weller. 2017. "On Fairness, Diversity and Randomness in Algorithmic Decision Making." ArXiv:1706.10208. · arXiv
  18. Hardt, Moritz, Eric Price, and Nathan Srebro. 2016. "Equality of Opportunity in Supervised Learning." ArXiv:1610.02413. · arXiv
  19. Hellman, Deborah. 2008. When Is Discrimination Wrong? Cambridge, MA: Harvard University Press.
  20. Hellman, Deborah. 2020. "Measuring Algorithmic Fairness." Virginia Law Review 106 (4). https://www.virginialawreview.org/ articles/measuring-algorithmic-fairness/.
  21. Hendrycks, Dan, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. 2020. "Natural Adversarial Examples." ArXiv: 1907.07174. · arXiv
  22. Hu, Lily, and Issa Kohler-Hausmann. 2020. "What's Sex Got to Do with Machine Learning?" In FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 513. https://doi.org/10.1145/3351095.3375674. · DOI · Paper page
  23. Ilyas, Andrew, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. 2019. "Adversarial Examples Are Not Bugs, They Are Features." ArXiv:1905.02175. · arXiv
  24. Kaushal, Amit, Russ Altman, and Curt Langlotz. 2020. "Geographic Distribution of US Cohorts Used to Train Deep Learning Algorithms." Journal of American Medical Association 324 (12): 1212-13. https://doi.org/10.1001/jama.2020.12067. · DOI · Paper page
  25. König, Pascal D. 2020. "Dissecting the Algorithmic Leviathan: On the Socio-Political Anatomy of Algorithmic Governance." Philosophy & Technology 33 (3): 467-85. https://doi.org/10.1007/s13347-019-00363-w. · DOI
  26. Marx, Charles T., Flavio du Pin Calmon, and Berk Ustun. 2020. "Predictive Multiplicity in Classification." ArXiv:1909.06677. · arXiv
  27. Mehrabi, Ninareh, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. "A Survey on Bias and Fairness in Machine Learning." ArXiv:1908.09635. · arXiv
  28. Mulgan, Tim. 2001. The Demands of Consequentialism. New York: Oxford University Press.
  29. Nussbaum, Martha. 2011. Creating Capabilities. Cambridge, MA: Harvard University Press.
  30. Raji, Inioluwa Deborah, and Joy Buolamwini. 2019. "Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products." In AIES: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 429-35. https://doi.org/10.1145/3306618.3314244. · DOI · Paper page
  31. Rawls, John. 1971. A Theory of Justice. Cambridge, MA: Harvard University Press.
  32. Raz, Joseph. 1988. The Morality of Freedom. The Morality of Freedom. Oxford: Oxford University Press.
  33. Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. "'Why Should I Trust You?': Explaining the Predictions of Any Classifier." In KDD: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-44. https://doi.org/10.1145/2939672.2939778. · DOI · Paper page
  34. Safiya, Noble. 2018. Algorithms of Oppression. New York: New York University Press.
  35. Sánchez-Monedero, Javier, Lina Dencik, and Lilian Edwards. 2020. "What Does It Mean to 'Solve' the Problem of Discrim- ination in Hiring? Social, Technical and Legal Perspectives from the UK on Automated Hiring Systems." In FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 458-68. https://doi.org/10.1145/ 3351095.3372849. · DOI · Paper page
  36. Schauer, Frederick. 2003. Profiles Probabilities and Stereotypes. Cambridge, MA: The Belknap Press of Harvard University Press.
  37. Sen, Amartya. 1980. "Equality of What?" In Tanner Lectures on Human Values, 197-220. Cambridge: Cambridge University Press.
  38. Sen, Amartya. 1993. "Capability and Well-Being." In The Quality of Life, edited by Martha Nussbaum and Amartya Sen, 30-53. Oxford: Clarendon Press.
  39. Vredenburgh, Kate. 2021. "The Right to Explanation." The Journal of Political Philosophy 0 (0): 1-21. https://doi.org/10.1111/ jopp.12262. · DOI
  40. Vredenburgh, Kate. Forthcoming. "Freedom at Work: Understanding, Alienation, and the AI-Driven Workplace." Canadian Journal of Philosophy.
  41. Waldron, Jeremy, and Peter Westen. 1991. "The Substance of Equality." Michigan Law Review 89 (6): 1350. https://doi.org/ 10.2307/1289475. · DOI
  42. Westen, Peter. 1982. "The Empty Idea of Equality." Harvard Law Review 95 (3): 537-96. https://doi.org/10.2307/1340593. · DOI · Paper page
  43. Zollman, Kevin J. S. 2019. "On the Normative Status of Mixed Strategies." Preprint. http://philsci-archive.pitt.edu/17979/.

Cited by

Sorted by citations

Authors
Creel, K., & Hellman, D.
Affiliations
  • Stanford University (US)
  • University of Virginia (US)
Corresponding author
Creel, Kathleen
ORCID
Venue
Canadian Journal of Philosophy
Publisher
Cambridge University Press (CUP)
Published
2022-01
Volume
52
Issue
1
Pages
26-43
ISSN
  • 0045-5091 (ISSN-L)
  • 1911-0820
DOI
10.1017/can.2022.3
OpenAlex
W3132748670
Cited by
67
References
8
Citations by year
2026: 14 · 2025: 26 · 2024: 14 · 2023: 6 · 2022: 6 · 2021: 1
FWCI
15.06
Type
journal article
Language
en
Open access
Licence
Locations
  • Canadian Journal of Philosophy (publishedVersion, cc-by)
  • PhilPapers (PhilPapers Foundation) (submittedVersion)
Funders
  • University of Pennsylvania
  • University of Virginia
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Political Science and International Relations › Political Philosophy and Ethics
  • Social Sciences › Arts and Humanities › Philosophy › Philosophical Ethics and Theory
Keywords
algorithmic decision-making, fairness, opportunity, systemic exclusion, moral concern, algorithmic justice, algorithmic bias, algorithmic transparency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review