Fairness, Equality, and Power in Algorithmic Decision-Making
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Much of the debate on the impact of algorithms is concerned with fairness, defined as the absence of discrimination for individuals with the same "merit." Drawing on the theory of justice, we argue that leading notions of fairness suffer from three key limitations: they legitimize inequalities justified by "merit;" they are narrowly bracketed, considering only differences of treatment within the algorithm; and they consider between-group and not within-group differences. We contrast this fairness-based perspective with two alternate perspectives: the first focuses on inequality and the causal impact of algorithms and the second on the distribution of power. We formalize these perspectives drawing on techniques from causal inference and empirical economics, and characterize when they give divergent evaluations. We present theoretical results and empirical examples which demonstrate this tension. We further use these insights to present a guide for algorithmic auditing and discuss the importance of inequality- and power-centered frameworks in algorithmic decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mapping the Margins: Intersectionality, Identity Politics, and Violence against Women of Color
Crenshaw, K. (1991) · Stanford Law Review · doi:10.2307/1229039 · Cited by 32,792
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“[4] provide a framework for understanding the negative consequences of such automated decision-making systems.”
BackgroundEstimation and Inference of Heterogeneous Treatment Effects using Random Forests
Wager, S., & Athey, S. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1319839 · Cited by 2,917 · cited 2×
“Many estimators are available to estimate n(x) under the assumption of conditional independence; in our application we have used the causal forest approach of [65].”
MethodAlgorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 2×
“We direct the reader to [10, 17, 47, 27, 33, 37, 50] for an overview and extensive discussions around various definitions of fairness as well as their relationship with other algorithmically-defined desiderata.”
BackgroundFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“We direct the reader to [10, 17, 47, 27, 33, 37, 50] for an overview and extensive discussions around various definitions of fairness as well as their relationship with other algorithmically-defined desiderata.”
BackgroundFairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124 · cited 1×
“We direct the reader to [10, 17, 47, 27, 33, 37, 50] for an overview and extensive discussions around various definitions of fairness as well as their relationship with other algorithmically-defined desiderata.”
BackgroundUsing Elasticities to Derive Optimal Income Tax Rates
Saez, E. (2001) · The Review of Economic Studies · doi:10.1111/1467-937x.00166 · Cited by 947
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Discrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“Many now-classic bodies of work study discrimination and harms caused by machine learning systems on historically disadvantaged groups in settings ranging from ad delivery [48] to facial analysis [8] to search engine bias [40] and provision of public services [13].”
BackgroundAlgorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
Racial Bias in Motor Vehicle Searches: Theory and Evidence
Knowles, J., Persico, N., & Todd, P. (2001) · Journal of Political Economy · doi:10.1086/318603 · Cited by 608
Sweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469
Generalized Social Marginal Welfare Weights for Optimal Tax Theory
Saez, E., & Stantcheva, S. (2016) · American Economic Review · doi:10.1257/aer.20141362 · Cited by 406
The cost of color: Skin color, discrimination, and health among African-Americans
Monk, E. P. (2015) · American Journal of Sociology · doi:10.1086/682162 · Cited by 386
50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · cited 1×
“Fairness Definitions Numerous definitions of fairness have been proposed in the literature [23, 37].”
BackgroundWho Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice
Kitagawa, T., & Tetenov, A. (2018) · Econometrica · doi:10.3982/ecta13288 · Cited by 277
Gebru, T. (2020) · The Oxford Handbook of Ethics of AI · doi:10.1093/oxfordhb/9780190067397.013.16 · Cited by 111
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82 · cited 1×
“We direct the reader to [10, 17, 47, 27, 33, 37, 50] for an overview and extensive discussions around various definitions of fairness as well as their relationship with other algorithmically-defined desiderata.”
BackgroundGupta, M., Cotter, A., Fard, M. M., & Wang, S. (2018) · arXiv · doi:10.48550/arxiv.1806.11212 · Cited by 16
Cited by
Regulating ChatGPT and other Large Generative AI Models
Hacker, P., Engel, A., & Mauer, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594067 · Cited by 466
“Due to space constraints, we cannot address all social and regulatory concerns regarding LGAIMs and have to bracket, for example, questions of IP law, power dynamics [35, 36], a deeper exploration of the comparative advantages of technology-neutral and technology-specific regulation [37], or the use of LGAIMs in milit…”
BackgroundLarge image datasets: A pyrrhic win for computer vision?
Birhane, A., & Prabhu, V. U. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00158 · Cited by 257
“Similarly, [46] contend that perspectives that acknowledge existing inequality and aim to redistribute power are pertinent as opposed to fairness-based perspectives.”
BackgroundHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“Collective self-governance is about power, which is a core theme in FATE work [Abdalla and Abdalla 2021; Barabas et al. 2020; Birhane et al. 2022; Boag et al. 2022; Kasy and Abebe 2021; Young et al. 2022].”
BackgroundRethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
Weinberg, L. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13196 · Cited by 127
Large image datasets: A pyrrhic win for computer vision?
Prabhu, V. U., & Birhane, A. (2020) · arXiv · doi:10.48550/arxiv.2006.16923 · Cited by 91
Metadata
- Authors
- Kasy, M., & Abebe, R.
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 576-586
- OpenAlex
- W3133953502
- Cited by
- 144
- References
- 23
- FWCI
- 24.35
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.