Inherent Trade-Offs in Algorithmic Fairness
Abstract
Recent discussion in both the academic literature and the public sphere about classification by algorithms has involved tension between competing notions of what it means for such a classification to be fair to different groups. We consider several of the key fairness conditions that lie at the heart of these debates, and discuss recent research establishing inherent trade-offs between these conditions. We also consider a variety of methods for promoting fairness and related notions for classification and selection problems that involve sets rather than just individuals.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Mathematical definitions of fairness are incompatible with one another and cannot hold simultaneously.
“While each of these notions is appealing on its own, it has been shown that they are incompatible with one another and cannot hold simultaneously [6,19].”
From Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning (2019) · cites this paper as [19]
There is no consensus on a precise definition of (un)fairness.
“Despite the recent surge of interest in Fair ML, there is no consensus on a precise definition of (un)fairness.”
From Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning (2019) · cites this paper as [19]
Explainable AI methods can help address concerns about bias and fairness in AI-based approaches and provide a better understanding of the underlying mechanisms and assumptions behind predictions.
“This can help to address concerns about bias and fairness in AI-based approaches [43] and provide a better understanding of the underlying mechanisms and assumptions behind the predictions [49].”
From The role of AI in drug discovery: challenges, opportunities, and strategies (2023) · cites this paper as [43]
Ethical considerations are another challenge because AI-based approaches may raise concerns about fairness and bias.
“Another challenge is presented by ethical considerations [42] since AI-based approaches may raise concerns about fairness and bias (see next section) [43].”
From The role of AI in drug discovery: challenges, opportunities, and strategies (2023) · cites this paper as [43]
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
The difference: how the power of diversity creates better groups, firms, schools, and societies
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1534 · Cited by 2,280 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Groups of diverse problem solvers can outperform groups of high-ability problem solvers
Hong, L., & Page, S. E. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0403723101 · Cited by 2,103 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 1× · see citing works
“We consider several of the key fairness conditions that lie at the heart of these debates, and discuss recent research establishing inherent trade-offs between these conditions [3, 4, 8, 10, 14].”
BackgroundSelf-citationOn Fairness and CalibrationPreprint
Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J. M., & Weinberger, K. Q. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.02012 · Cited by 509 · see citing works
Self-citationFlores, A. W., Bechtel, K., & Lowenkamp, C. T. (2016) · Federal probation · Cited by 240 · see citing works
Selection Problems in the Presence of Implicit Bias
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603209 · Cited by 21 · see citing works
Team Performance with Test Scores
Kleinberg, J. M., & Raghu, M. (2015) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2764468.2764496 · Cited by 16 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · see citing works
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124 · see citing works
Racial disparities in automated speech recognition
Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915768117 · Cited by 765 · see citing works
Algorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750 · see citing works
The role of AI in drug discovery: challenges, opportunities, and strategies
Blanco-González, A., Cabezón, A., Seco-González, A., Conde-Torres, D., Antelo-Riveiro, P., Piñeiro, Á., & Garcia-Fandino, R. (2023) · Pharmaceuticals · doi:10.3390/ph16060891 · Cited by 690 · see citing works
Measuring discrimination in algorithmic decision making
Žliobaitė, I. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-017-0506-1 · Cited by 272 · see citing works
Auditing black-box models for indirect influence
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280 · see citing works
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207 · see citing works
Srivastava, M., Heidari, H., & Krause, A. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330664 · Cited by 179 · see citing works
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113 · see citing works
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Deng, W. H., Nagireddy, M., Lee, M. S. A., Singh, J., Wu, Z. S., Holstein, K., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533113 · Cited by 99 · see citing works
FairSquare: probabilistic verification of program fairness
Albarghouthi, A., D'Antoni, L., Drews, S., & Nori, A. V. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133904 · Cited by 94 · see citing works
Perrone, V., Donini, M., Zafar, M. B., Schmucker, R., Kenthapadi, K., & Archambeau, C. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462629 · Cited by 66 · see citing works
Theory In, Theory Out: The Uses of Social Theory in Machine Learning for Social Science
Radford, J., & Joseph, K. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00018 · Cited by 58 · see citing works
Probabilistic verification of fairness properties via concentration
Bastani, O., Zhang, X., & Solar-Lezama, A. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360544 · Cited by 57 · see citing works
Kaur, D., Hughes, J. W., Rogers, A. J., Kang, G., Narayan, S. M., Ashley, E. A., & Perez, M. V. (2024) · Circulation: Heart Failure · doi:10.1161/circheartfailure.123.010879 · Cited by 58 · see citing works
Meritocratic Fairness for Infinite and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278764 · Cited by 24 · see citing works
- Authors
- Kleinberg, J.
- Affiliations
- Cornell University (US)
- Corresponding author
- Kleinberg, Jon
- ORCID
- Kleinberg, Jon
- Venue
- Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems
- Publisher
- ACM
- Published
- 2018-06-12
- Pages
- 40-40
- OpenAlex
- W2808105152
- Cited by
- 125
- References
- 13
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 22 · 2023: 12 · 2022: 19 · 2021: 20 · 2020: 15 · 2019: 18 · 2018: 7 · 2017: 3 · 2016: 1
- Type
- conference abstract
- Language
- en
- Open access
- closed
- Locations
- ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) (publishedVersion)
- Funders
- Simons Foundation (Simons Investigator Award)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic fairness, fairness trade-offs, group fairness, individual fairness, fairness in selection, set selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
