CoChat
Sign Up
PreprintCited by 35Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Artificial Intelligence in Law

The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness

Open access
Watkins, Elizabeth Anne, McKenna, Michael, Chen, Jiahao·2022·arXiv
DOI 10.48550/arxiv.2202.09519arXiv 2202.09519
Open this paper in CoChat — free accountOpen PDF

Abstract

Computer scientists are trained to create abstractions that simplify and generalize. However, a premature abstraction that omits crucial contextual details creates the risk of epistemic trespassing, by falsely asserting its relevance into other contexts. We study how the field of responsible AI has created an imperfect synecdoche by abstracting the four-fifths rule (a.k.a. the 4/5 rule or 80% rule), a single part of disparate impact discrimination law, into the disparate impact metric. This metric incorrectly introduces a new deontic nuance and new potentials for ethical harms that were absent in the original 4/5 rule. We also survey how the field has amplified the potential for harm in codifying the 4/5 rule into popular AI fairness software toolkits. The harmful erasure of legal nuances is a wake-up call for computer scientists to self-critically re-evaluate the abstractions they create and use, particularly in the interdisciplinary field of AI ethics.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2202.09519

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The four-fifths rule is a bureaucratic compromise that is fundamentally arbitrary and not grounded in empirical evidence, and so may not be useful for judging algorithmic bias.

    “As critics point out, the four-fifths rule is a bureaucratic compromise that is fundamentally arbitrary and not grounded in empirical evidence, and so may not be useful for judging algorithmic bias [15, 49].”

    From Auditing Work: Exploring the New York City algorithmic bias audit regime (2024) · cites this paper as [49]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 51

  1. Nathan Ballantyne. 2019. Epistemic trespassing. Mind 128, 510 (2019), 367-395. https://doi.org/10.1093/mind/fzx042 · DOI
  2. Michelle Bao, Angela Zhou, Samantha A Zottola, Brian Brubach, Sarah Desmarais, Aaron Seth Horowitz, Kristian Lum, and Suresh Venkatasubramanian. 2021. It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) (NeurIPS '21). OpenReview, OpenReview.net, 18 pages. https://openreview.net/forum?id=qeM58whnpXM
  3. R. K. E. Bellamy, K. Dey, M. Hind, S. C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, A. Mojsilović, S. Nagar, K. Natesan Ramamurthy, J. Richards, D. Saha, P. Sattigeri, M. Singh, K. R. Varshney, and Y. Zhang. 2019. AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development 63, 4/5 (2019), 4:1-15. https://doi.org/10. 1147/JRD.2019.2942287 · DOI · Paper page
  4. Dan Biddle. 2006. Adverse Impact and Test Validation: A Practitioner's Guide to Valid and Defensible Employment Testing (2 ed.). Gower, Aldershot. https: //doi.org/10.4324/9781315263298 · DOI
  5. George E. P. Box. 1976. Science and Statistics. J. Amer. Statist. Assoc. 71, 356 (1976), 791-799. https://doi.org/10.1080/01621459.1976.10480949 · DOI
  6. Felice Cardone. 2021. Games, Full Abstraction and Full Completeness. In The Stanford Encyclopedia of Philosophy (Spring 2021 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University, Stanford, CA.
  7. Giuseppe Castagna. 1997. Object-Oriented Programming: A Unified Foundation. Birkhäuser, Boston. https://doi.org/10.1007/978-1-4612-4138-6 · DOI
  8. Brendan Churchill and Chabel Khan. 2021. Youth underemployment: A review of research on young people and the problems of less(er) employment in an era of mass education. Sociology Compass 15, 10 (2021), e12921. https://doi.org/10. 1111/soc4.12921 · DOI
  9. Samuel Deng and Achille Varzi. 2019. Methodological blind spots in machine learning fairness: Lessons from the philosophy of science and computer science. In Workshop on Human-Centric Machine Learning at the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). arXiv, Vancouver, Canada, 5 pages. arXiv:1910.14210 · arXiv
  10. Michael Etter, Christian Fieseler, and Glen Whelan. 2019. Sharing Economy, Shar- ing Responsibility? Corporate Social Responsibility in the Digital Age. Journal of Business Ethics 159, 4 (2019), 935-942.
  11. Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, 259-268. https://doi.org/ 10.1145/2783258.2783311 arXiv:1412.3756 · DOI · Paper page
  12. W. C. Fletcher. 1940. Premature Abstraction. The Mathematical Gazette 24, 259 (1940), 73-85. http://www.jstor.org/stable/3606739
  13. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets. Commun. ACM 64, 12 (2021), 86-92.
  14. Navdeep Gill, Patrick Hall, Kim Montgomery, and Nicholas Schmidt. 2020. A Responsible Machine Learning Workflow with Focus on Interpretable Models, Post-hoc Explanation, and Discrimination Testing. Information 11, 3 (29 Feb. 2020), 137. https://doi.org/10.3390/info11030137 · DOI
  15. Alan H. Goldman. 2015. Justice and Reverse Discrimination. Princeton University Press, Princeton, NJ. https://doi.org/10.1515/9781400868605 · DOI
  16. Abigail Z. Jacobs and Hanna Wallach. 2021. Measurement and Fairness. In Pro- ceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT '21). Association for Computing Machinery, New York, NY, USA, 375-385. https://doi.org/10.1145/3442188.3445901 · DOI · Paper page
  17. Monique Janneck. 2010. Challenges of Software Recontextualization: Lessons Learned. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI EA '10). Association for Computing Machinery, New York, NY, USA, 4613-4628. https://doi.org/10.1145/1753846.1754202 · DOI
  18. Jeff Kramer. 2007. Is Abstraction the Key to Computing? Commun. ACM 50, 4 (April 2007), 36-42. https://doi.org/10.1145/1232743.1232745 · DOI
  19. Michelle Seng Ah Lee and Jat Singh. 2021. The Landscape and Gaps in Open Source Fairness Toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Associa- tion for Computing Machinery, New York, NY, USA, Article 699, 13 pages. https://doi.org/10.1145/3411764.3445261 · DOI · Paper page
  20. Paul London. 1978. A Conversation with Eleanor Holmes Norton. Employee Relations Law Journal 3 (1978), 314-326.
  21. Michael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-Designing Checklists to Understand Organizational Challenges and Op- portunities around Fairness in AI. Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3313831.3376445 · DOI · Paper page
  22. David Manheim and Scott Garrabrant. 2018. Categorizing variants of Goodhart's Law. arXiv:1803.04585 · arXiv
  23. Donald Martin Jr, Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William S Isaac. 2020. Extending the machine learning abstraction boundary: A Complex systems approach to incorporate societal context. arXiv:2006.09663 · arXiv
  24. Abraham Harold Maslow. 1966. The Psychology of Science: A Reconnaissance. John Dewey Society Lectureship Series, Vol. 8. Harper & Row, New York, 15.
  25. Nick Merrill, Michael Madaio, and Richmond Wong. 2022. Seeking Like a Toolkit: How Toolkits Envision the Work of AI Ethics. arXiv:2202.08792 · arXiv
  26. Boris Miethlich and Anett G. Oldenburg. 2019. Social Inclusion Drives Business Sales: A Literature Review on the Case of the Employment of Persons With Disabilities. In 33nd International Business Information Management Association Conference (IBIMA), Education Excellence and Innovation Management through Vision 2020, Granada, Spain, 10-11.04.2019. IBIMA Publishing, King of Prussia, PA; King of Prussia, PA, 6253-6267. https://doi.org/10.33543/16002.62536267 · DOI
  27. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* '19). Association for Computing Machinery, New York, NY, USA, 220-229. https://doi.org/10.1145/ 3287560.3287596 · DOI · Paper page
  28. Shira Mitchell, Eric Potash, Solon Barocas, Alexander D'Amour, and Kristian Lum. 2021. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application 8 (2021), 141-163.
  29. Gina Neff. 2020. From Bad Users and Failed Uses to Responsible Technologies: A Call to Expand the AI Ethics Toolkit. Association for Computing Machinery, New York, NY, USA, 5-6. https://doi.org/10.1145/3375627.3377141 · DOI
  30. Jennifer L. Nelson and Steven P. Vallas. 2021. Race and inequality at work: An occupational perspective. Sociology Compass 15, 10 (2021), e12926. https: //doi.org/10.1111/soc4.12926 · DOI
  31. Liesbet Okkerse. 2008. How to measure labour market effects of immigration: a review. Journal of Economic Surveys 22, 1 (2008), 1-30. https://doi.org/10.1111/j. 1467-6419.2007.00533.x · DOI
  32. Chris Olah and Shan Carter. 2017. Research Debt. https://doi.org/10.23915/ distill.00005 · DOI
  33. Orlando Patterson and Xiaolin Zhuo. 2018. Modern Trafficking, Slavery, and Other Forms of Servitude. Annual Review of Sociology 44, 1 (2018), 407-439. https://doi.org/10.1146/annurev-soc-073117-041147 · DOI
  34. Amir Radfar, Seyed Ahmad Ahmadi Asgharzadeh, Fernando Quesada, and Irina Filip. 2018. Challenges and perspectives of child labor. Industrial psychiatry journal 27, 1 (Jan-Jun 2018), 17-20.
  35. Brianna Richardson, Jean Garcia-Gathright, Samuel F. Way, Jennifer Thom, and Henriette Cramer. 2021. Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 236, 13 pages. https: //doi.org/10.1145/3411764.3445604 · DOI
  36. John E. Roemer. 1982. A General Theory of Exploitation and Class. Harvard Uni- versity Press, Cambridge, MA. https://doi.org/10.4159/harvard.9780674435865 · DOI
  37. Benjamin I. Sachs. 2007-2008. Employment Law as Labor Law. Cardozo Law Re- view 29 (2007-2008), 2685-2748. https://heinonline.org/HOL/P?h=hein.journals/ cdozo29&i=2707
  38. Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby Stevens, Ari Anisfeld, Kit T. Rodolfa, and Rayid Ghani. 2019. Aequitas: A Bias and Fairness Audit Toolkit. arXiv:1811.05577 [cs.LG] · arXiv
  39. Javier Sánchez-Monedero, Lina Dencik, and Lilian Edwards. 2020. What Does It Mean to 'solve' the Problem of Discrimination in Hiring? Social, Technical and Legal Perspectives from the UK on Automated Hiring Systems. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 458-468. https://doi.org/10.1145/3351095.3372849 · DOI · Paper page
  40. Angela L. Sauer, Andra Parks, and Patricia C. Heyn. 2010. Assistive technology effects on the employment outcomes for people with cognitive disabilities: A systematic review. Disability and Rehabilitation: Assistive Technology 5, 6 (2010), 377-391. https://doi.org/10.3109/17483101003746360 · DOI
  41. Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* '19). Association for Computing Machinery, New York, NY, USA, 59-68. https://doi.org/10.1145/3287560.3287598 · DOI · Paper page
  42. Elaine W. Shoben. 1978. Differential Pass-Fail Rates in Employment Testing: Statistical Proof under Title VII. Harvard Law Review 91, 4 (1978), 793-813. http://www.jstor.org/stable/1340356
  43. Edouard J. Simon, Monique Janneck, and Dorina Gumm. 2006. Understanding Socio-Technical Change: Towards a Multidisciplinary Approach. In Social In- formatics: An Information Society for all? In Remembrance of Rob Kling, Jacques Berleur, Markku I. Nurminen, and John Impagliazzo (Eds.). Springer US, Boston, MA, 469-479.
  44. Jeremy Snyder. 2010. Exploitation and Sweatshop Labor: Perspectives and Issues. Business Ethics Quarterly 20, 2 (2010), 187-213. http://www.jstor.org/stable/ 25702393
  45. Christopher Strachey. 2000. Fundamental Concepts in Programming Languages. Higher-Order and Symbolic Computation 13, 1/2 (2000), 11-49. https://doi.org/10. 1023/A:1010000313106 · DOI
  46. Will Sutherland, Mohammad Hossein Jarrahi, Michael Dunn, and Sarah Beth Nel- son. 2020. Work Precarity and Gig Literacies in Online Freelancing. Work, Employ- ment and Society 34, 3 (2020), 457-475. https://doi.org/10.1177/0950017019886511 · DOI · Paper page
  47. Philip Wadler. 2015. Propositions as Types. Commun. ACM 58, 12 (nov 2015), 75-84. https://doi.org/10.1145/2699407 · DOI
  48. Ronald Weitzer. 2015. Human Trafficking and Contemporary Slavery. Annual Review of Sociology 41, 1 (2015), 223-242. https://doi.org/10.1146/annurev-soc- 073014-112506 · DOI
  49. Johanna Weststar. 2011. A Review of Women's Experiences of Three Dimensions of Underemployment. Springer New York, New York, NY, 105-125. https://doi.org/ 10.1007/978-1-4419-9413-4_6 · DOI
  50. Jeannette M. Wing. 2006. Computational Thinking. Commun. ACM 49, 3 (March 2006), 33-35. https://doi.org/10.1145/1118178.1118215 · DOI · Paper page
  51. Alice Xiang and Inioluwa Deborah Raji. 2019. On the legal compatibility of fairness definitions. In Workshop on Human-Centric Machine Learning at the 33rd Conference on Neural Information Processing Systems (NeurIPS '19). arXiv, Vancouver, Canada, 6 pages. arXiv:1912.00761 · arXiv

Cited by

Sorted by citations

Authors
Watkins, E. A., McKenna, M., & Chen, J.
Affiliations
Intel (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Pages
764-775
DOI
10.48550/arxiv.2202.09519
arXiv
2202.09519
OpenAlex
W4399364234
Cited by
35
References
0
Citations by year
2026: 5 · 2025: 9 · 2024: 10 · 2023: 6 · 2022: 5
FWCI
8.75
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Keywords
indirect discrimination, algorithmic fairness, responsible AI, legal abstraction, algorithmic discrimination, AI ethics, fairness metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review