The DOI matches its DataCite record; 21 of the 35 references it cites were found in a registry or an index.
Inherent Trade-Offs in the Fair Determination of Risk Scores
Open accessAbstract
Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“each technical definition carry strong values assumptions which can often be in tension with each other (Friedler et al., 2016; Kleinberg et al., 2017).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Kleinberg et al., 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature often treats fairness and performance as a trade-off problem”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Kleinberg et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Many fairness metrics have been proposed, and the most widely used criteria were shown to be incompatible ([20], [41]), and to present risks for the populations they were meant to protect ([24]).”
From Fairness and explainability in automatic decision-making systems. A challenge for computer science and law (2023) · cites this paper as [41]
The cited source says what the claim attributes to it.
In the citing paper
“commonly used statistical fairness criteria can be mutually incompatible except under restrictive conditions”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Kleinberg et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“Kleinberg, Mullainathan and Raghavan (2017) shows theoretically that these notions of fairness are usually incompatible.”
From An algorithm for removing sensitive information: Application to race-independent recidivism prediction (2019) · cites this paper as Kleinberg, Mullainathan and Raghavan (2017)
The cited source says what the claim attributes to it.
In the citing paper
“These two notions of fairness are indeed distinct.”
From An algorithm for removing sensitive information: Application to race-independent recidivism prediction (2019) · cites this paper as Kleinberg, Mullainathan and Raghavan (2017)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“propose a framework based on a task-specific externally defined similarity metric between individuals, seeking to achieve fairness through the goal that “similar people [be] treated similarly” [11].”
BackgroundDiscrimination and racial disparities in health: evidence and needed research
Williams, D. R., & Mohammed, S. A. (2009) · Journal of Behavioral Medicine · doi:10.1007/s10865-008-9185-0 · Cited by 3,371 · cited 1×
“Here too we can ask whether such decision-making is being applied uniformly across different groups of patients [16, 27], and in particular how medical tests may play a differential role for conditions that vary widely in frequency between these groups.”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“Barocas and Selbst survey a range of ways in which data-analysis algorithms can lead to discriminatory outcomes [3], and review articles by Romei and Ruggieri [25] and Zliobaite [30] survey data-analytic and algorithmic methods for measuring discrimination.”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“consider the natural analogues of our conditions (B) and (C), balance for the negative and positive classes, in the case of classifiers that output binary “yes/no” predictions rather than real-valued scores as in our case [19].”
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 3×
“Also concurrent with our work and that of Hardt et al., Chouldechova [5] and Corbett-Davies et al. [6] (and see also [7]) consider binary prediction subject to these same analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction (requiri…”
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1×
“This latter global requirement is a version of statistical parity [12, 4, 21, 22].”
BackgroundAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. Z. (2017) · Knowledge Discovery and Data Mining · doi:10.1145/3097983.309809 · Cited by 1,464 · cited 1×
“[7] (and see also [6]) consider binary prediction subject to analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction.”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”
BackgroundThree naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · cited 3×
“Work in these directions has developed learning algorithms that penalize violations of statistical parity [4, 22].”
BackgroundAutomated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676 · cited 1×
“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”
Background
Flagged first, then by citations
14 of the 35 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [9][9] Amit Datta, Michael Carl Tschantz, and Anupam Datta. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies , 2015(1):92-112, 2015. -
- [14][14] Dean P. Foster and Rakesh V. Vohra. Asymptotic calibration. Biometrika , 85(2):379-390, 1998. -
- [29][29] Richard S Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning Fair Representations. Proceedings of the 30th International Conference on Machine Learning , 28:325-333, 2013. -
Cited by
Sorted by citations
Items 401–500 of 945
Fairness and Discrimination in Information Access SystemsPreprint
Ekstrand, M. D., Das, A., Burke, R., & Díaz, F. (2021) · arXiv (Cornell University) · Cited by 13
A Near-Optimal Algorithm for Debiasing Trained Machine Learning ModelsPreprint
Alabdulmohsin, I., & Lučić, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12887 · Cited by 17
“…have been introduced, such as statistical parity [Dwork et al., 2012, Zafar et al., 2017a], equality of opportunity [Hardt et al., 2016], and equalized odds [Hardt et al., 2016, Kleinberg et al., 2016], and these are not generally mutually compatible [Chouldechova, 2017, Kleinberg et al., 2016].”
BackgroundDbias: Detecting biases and ensuring Fairness in news articlesPreprint
Raza, S., Reji, D. J., & Ding, C. (2022) · Research Square · doi:10.21203/rs.3.rs-1356281/v1 · Cited by 15
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Dalrymple, D. "., Skalse, J., Bengio, Y., Russell, S., Tegmark, M., Seshia, S., Omohundro, S., Szegedy, C., Goldhaber, B., Ammann, N., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.06624 · Cited by 12
“AI decision making may also unfairly and systematically disadvantage certain groups, even when unintended by the system’s creators (e.g., Kleinberg et al., 2016; Buolamwini & Gebru, 2018; Das et al., 2023; Wang et al., 2022).”
BackgroundBerk, R. A., Kuchibhotla, A. K., & Tchetgen, E. T. (2023) · Sociological Methods & Research · doi:10.1177/00491241231155883 · Cited by 24
A Human-Centric Perspective on Model Monitoring
Shergadwala, M., Lakkaraju, H., & Kenthapadi, K. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21997 · Cited by 19
Algorithmic Fairness and Statistical Discrimination
Patty, J. W., & Penn, E. M. (2022) · Philosophy Compass · doi:10.1111/phc3.12891 · Cited by 14
Blinding to Circumvent Human Biases: Deliberate Ignorance in Humans, Institutions, and Machines
Hertwig, R., Herzog, S. M., & Kozyreva, A. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231188052 · Cited by 20
“Third, no single benchmark exists for evaluating algorithmic fairness, and frameworks may conflict or misalign with legal systems (Chouldechova, 2017; Heidari et al., 2018; Kleinberg et al., 2016; Lee et al., 2021; Mitchell et al., 2021; Wachter et al., 2021).”
BackgroundFair equality of chances for prediction-based decisions
Loi, M., Herlitz, A., & Heidari, H. (2023) · Economics and Philosophy · doi:10.1017/s0266267123000342 · Cited by 14
“In all other situations, a decision process can meet sufficiency or separation, but not both (Chouldechova 2017; Kleinberg et al .”
MLHOps: Machine Learning Health Operations
Khattak, F. K., Subasri, V., Krishnan, A., Pou-Prom, C., Koçak, S. A., Dolatabadi, E., Pandya, D., Seyyed-Kalantari, L., & Rudzicz, F. (2024) · IEEE Access · doi:10.1109/access.2024.3521279 · Cited by 31
“Jon Kleinberg et al., [145] showed that three fairness conditions evaluated could not be simultaneously satisfied.”
Navigating AI conformity: A design framework to assess fairness, explainability, and performance
von Zahn, M., Zacharias, J., Lowin, M., Chen, J., & Hinz, O. F. (2025) · Electronic Markets · doi:10.1007/s12525-025-00770-2 · Cited by 14
“For example, definitions of AI fairness are numbering in the double digits (c.f. Barocas et al., 2019), and some definitions are mathematically impossible to fulfill at the same time (Kleinberg et al., 2017).”
On the independence of association bias and empirical fairness in language models
Cabello, L., Jørgensen, A. K., & Søgaard, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594004 · Cited by 14
“The fairness metric literature is also full of controversies [24, 30, 36, 44], but there is a broad consensus that performance disparity or outcome disparity is a real challenge for responsible NLP research and development.”
BackgroundBipartite Ranking Fairness Through a Model Agnostic Ordering Adjustment
Cui, S., Pan, W., Zhang, C., & Wang, F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3290949 · Cited by 13
“Recent studies have shown that it is impossible to satisfy both error rate fairness and calibration simultaneously when the prevalence of positive instances are different across groups [30], [31].”
BackgroundGender Bias in AI Recruitment Systems: A Sociological-and Data Science-based Case Study
Njoto, S., Cheong, M., Lederman, R., McLoughney, A. J., Ruppanner, L., & Wirth, A. I. (2022) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas55053.2022.10227106 · Cited by 22
Understanding and mitigating multi-sided exposure bias in recommender systems
Mansoury, M. (2022) · ACM SIGWEB Newsletter · doi:10.1145/3566100.3566103 · Cited by 13
Zhang, B., Yu, R., Sun, H., Li, Y., Xu, J., & Wang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08855 · Cited by 15
“Standard choices for these statistics include positive classification rates [8], false positive or false negative rates [28, 39] and positive predictive value [11].”
BackgroundMetric-Free Individual Fairness in Online LearningPreprint
Bechavod, Y., Jung, C., & Wu, Z. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05474 · Cited by 19
“…attributes (e.g., race or gender) and then asks for approximate parity of some statistic of the predictor, such as positive classifi-cation rate or false positive rate, across these groups (see, e.g., Hardt et al. [2016], Chouldechova [2017], Kleinberg et al. [2017], Agarwal et al. [2018]).”
BackgroundDebiasing Classifiers: Is Reality at Variance With Expectation?Preprint
Agrawal, A., Pfisterer, F., Bischl, B., Chen, J., Sood, S., Shah, S., Buet-Golfouse, F., Mateen, B. A., & Vollmer, S. J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3711681 · Cited by 17
“Such fairness–fairness trade-offs are implied in situations where impossibility theorems apply [15, 27, 28], and are further compounded by considerations of out-of-sample generalizability that is crucial to the practical utility of learned classifiers.”
A Philosophical Theory of Fairness for Prediction-Based DecisionsPreprint
Loi, M., Herlitz, A., & Heidari, H. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3450300 · Cited by 12
Data Science: What the Educated Citizen Needs to Know
Garber, A. M. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.88ba42cb · Cited by 13
Bias and fairness in machine learning and artificial intelligence
Cirillo, D., & Rementeria, M. J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-821392-6.00006-6 · Cited by 14
Social Bias Meets Data Bias: The Impacts of Labeling and Measurement Errors on Fairness Criteria
Liao, Y., & Naghizadeh, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26054 · Cited by 16
Shapley Value-Based Approaches to Explain the Quality of Predictions by Classifiers
Pelegrina, G. D., & Siraj, S. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3365082 · Cited by 16
“Indeed, besides the performance, aspects such as fairness and explainability (among others) are also important when deciding which machine learning (ML) model to adopt [13], [14], [15], [16].”
BackgroundTen simple rules for building and maintaining a responsible data science workflow
Stoudt, S., Jernite, Y., Marshall, B., Marwick, B., Sharan, M., Whitaker, K., & Danchev, V. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1012232 · Cited by 11
“This is understandable given that fairness criteria have been found to be incompatible and no method or metric can satisfy desirable fairness criteria simultaneously [40–42].”
BackgroundNon-empirical problems in fair machine learning
Scantamburlo, T. (2021) · Ethics and Information Technology · doi:10.1007/s10676-021-09608-9 · Cited by 13
Certification and Trade-off of Multiple Fairness Criteria in Graph-based Spam Detection
Burkholder, K., Kwock, K., Xu, Y., Liu, J., Chen, C., & Xie, S. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482325 · Cited by 11
“The above question has been studied in [24, 26], but their statements are not about the representation learning on graphs.”
BackgroundMethodGrgić-Hlača, N., Castelluccia, C., & Gummadi, K. P. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21989 · Cited by 13
Group Fairness in Prediction-Based Decision Making: From Moral Assessment to Implementation
Baumann, J., & Heitz, C. (2022) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds54800.2022.00011 · Cited by 13
“In most cases, these criteria are mathematically incompatible [3], [4].”
BackgroundDavies, B., & Douglas, T. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780197539538.003.0006 · Cited by 11
TestSGD : Interpretable Testing of Neural Networks against Subtle Group Discrimination
Zhang, M., Sun, J., Wang, J., & Sun, B. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3591869 · Cited by 12
“There are multiple definitions of fairness [12, 14, 16, 23, 25, 47].”
InfluentialOn the site of predictive justice
Lazar, S., & Stone, J. O. H. (2023) · Noûs · doi:10.1111/nous.12477 · Cited by 14
On coresets for fair clustering in metric and Euclidean spaces and their applications
Bandyapadhyay, S., Fomin, F. V., & Simonov, K. (2024) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2024.103506 · Cited by 21
Seamless equal accuracy ratio for inclusive CTC speech recognition
Gao, H., Wang, X., Kang, S., Mina, R. J. L., Issa, D., Harvill, J. B., Sarı, L., Hasegawa‐Johnson, M., & Yoo, C. D. (2021) · Speech Communication · doi:10.1016/j.specom.2021.11.004 · Cited by 11
EADTC: An Approach to Interpretable and Accurate Crime Prediction
Ma, Y., Nakamura, K., Lee, E.-J., & Bhattacharyya, S. S. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945130 · Cited by 13
“As seen in the recent literature on fairness and recidivism prediction, it is not possible to maximize accuracy and all types of fairness at the same time [27] (see also [28] [29]).”
BackgroundHiring as ExplorationPreprint
Bergman, P., Li, D., & Raymond, L. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3630630 · Cited by 12
Top-label calibration and multiclass-to-binary reductionsPreprint
Gupta, C., & Ramdas, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08353 · Cited by 11
“A well calibrated classifier is not necessarily an accurate or a fair one, and vice versa [Kleinberg et al., 2017].”
MethodRodolfa, K. T., Lamba, H., & Ghani, R. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00396-x · Cited by 10
Robust reputation independence in ranking systems for multiple sensitive attributes
Ramos, G., Boratto, L., & Marras, M. (2022) · Machine Learning · doi:10.1007/s10994-022-06173-0 · Cited by 10
Disentangling and Operationalizing AI Fairness at LinkedIn
Candela, J. Q., Wu, Y., Hsu, B., Jain, S., Ramos, J., Adams, J., Hallman, R., & Basu, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594075 · Cited by 14
“measures are at odds with one another [19, 39, 44, 51].”
BackgroundOnline Fairness Auditing through Iterative Refinement
Maneriker, P., Burley, C., & Parthasarathy, S. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599454 · Cited by 21
Algorithmic fairness and resentment
Babic, B., & King, Z. J. (2023) · Philosophical Studies · doi:10.1007/s11098-023-02006-5 · Cited by 10
“A now-well-known impossibility result (Kleinberg & Mullainathan, 2016) shows that a decision-making algorithm cannot both be calibrated and display predictive parity when the base rates of the relevant trait genuinely do vary across social groups.”
Zezulka, S., & Genin, K. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659020 · Cited by 15
Soni, V. (2024) · International Conference on Knowledge Engineering and Communication Systems (ICKECS) · doi:10.1109/ickecs61492.2024.10617325 · Cited by 12
“The study advocates for a paradigm shift toward explainable AI in recruitment, wherein the decision-making processes of algorithms are made transparent, enabling organizations to identify and rectify biases effectively [10][11].”
BackgroundWhat’s Impossible about Algorithmic Fairness?
Sahlgren, O. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00814-z · Cited by 11
“…a fallible predictive model cannot simultaneously satisfy two popular notions of fairness, Sufficiency and Separation , expect in exceptional circumstances, where the comparison classes exhibit equal base rates (Kleinberg et al., 2017; Choul-dechova, 2017; see also Barocas et al., 2023, Ch. 3).”
Machine Learning and Law and Economics: A Preliminary Overview
Park, S.-C., & Ko, H. (2020) · Asian Journal of Law and Economics · doi:10.1515/ajle-2020-0030 · Cited by 12
How Colonial Continuities Underlie Algorithmic Injustices in Education
Karumbaiah, S., & Brooks, J. (2021) · Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT) · doi:10.1109/respect51740.2021.9620605 · Cited by 10
“It has also been shown that it is mathematically impossible to achieve multiple fairness considerations within the boundaries of standard ML principles [18].”
BackgroundSuk, Y., & Han, K. T. (2023) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986231171711 · Cited by 19
“First, it is impossible to satisfy all the fairness notions simultaneously Differential Algorithmic Functioning because some are inherently in conflict; see Chouldechova (2017), Berk et al. (2021), and Kleinberg et al. (2017).”
BackgroundStoyanovich, J., Howe, B., Jagadish, H. V., & Miklau, G. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3229863.3240494 · Cited by 9
“This portion of the panel will be informed by work on bias in computer systems and on algorithmic fairness, including [5, 6, 7, 11, 12, 14, 16, 17, 22, 25, 27].”
BackgroundEliminating Latent Discrimination: Train Then Mask
Ghili, S., Kazemi, E. M., & Karbasi, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013672 · Cited by 10
Values Embedded in Legal Artificial Intelligence
Surden, H. (2022) · IEEE Technology and Society Magazine · doi:10.1109/mts.2022.3147542 · Cited by 26
Inherent Trade-Offs in Algorithmic Fairness
Kleinberg, J. M. (2019) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3308809.3308832 · Cited by 12
Self-citationIs calibration a fairness requirement?
Loi, M., & Heitz, C. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533245 · Cited by 12
Thomas, C., & Pontón-Núñez, A. (2022) · Minnesota journal of law & inequality · doi:10.24926/25730037.649 · Cited by 10
Making recruitment more inclusive
Delecraz, S., Eltarr, L., Becuwe, M., Bouxin, H., Boutin, N., & Oullier, O. (2022) · 2022 IEEE/ACM International Workshop on Equitable Data & Technology (FairWare) · doi:10.1145/3524491.3527309 · Cited by 18
“[23] who provided statistical evidence for their mutual exclusion in most cases.”
BackgroundCounterfactual Fairness Is Basically Demographic Parity
Rosenblatt, L., & Witter, R. T. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26691 · Cited by 17
Advancing Subgroup Fairness via Sleeping ExpertsPreprint
Blum, A. L., & Lykouris, T. (2019) · arXiv (Cornell University) · doi:10.4230/lipics.itcs.2020.55 · Cited by 22
“There is a growing literature aiming to identify natural fairness notions and understand the limitations they impose; see [14, 22, 31, 12, 27, 1] for a non-exhaustive list.”
BackgroundContrastive Examples for Addressing the Tyranny of the MajorityPreprint
Sharmanska, V., Hendricks, L. A., Darrell, T., & Quadrianto, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.06524 · Cited by 26
“We report accuracy of predicting seven classes that correspond to age groups: [0-3], [4-12], [13-19], [20-30], [31-45], [46-60], [61+].”
BackgroundMethodInfluentialCalibrating Predictions to Decisions: A Novel Approach to Multi-Class CalibrationPreprint
Zhao, S.-J., Kim, M. P., Sahoo, R., Ma, T., & Ermon, S. (2021) · Neural Information Processing Systems · doi:10.48550/arxiv.2107.05719 · Cited by 103
“Calibration have been the focus of many works on fairness, starting with (Kleinberg et al., 2016; Pleiss et al., 2017).”
BackgroundArtificial Intelligence and Debiasing in Hiring: Impact on Applicant Quality and Gender DiversityPreprint
Awad, E., Balafoutas, L., Chen, L., Ip, E., & Vecci, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4626059 · Cited by 12
Sun, Y., Fung, B. C. M., & Haghighat, F. (2022) · Energy and Buildings · doi:10.1016/j.enbuild.2022.112565 · Cited by 9
“Note that achieving different types of fairness at the same time is almost impossible [13] because of their distinct evaluation criteria.”
BackgroundThe hard proxy problem: proxies aren’t intentional; they’re intentional
Johnson, G. M. (2025) · Philosophical Studies · doi:10.1007/s11098-025-02333-9 · Cited by 11
“These lematic results such as these are only possible under extremely unusual circumstances (Chouldechova, 2016; Kleinberg et al., 2016).”
Ethics, Artificial Intelligence, and Risk Assessment.
Spivak, B. L., & Shepherd, S. M. (2021) · PubMed · doi:10.29158/jaapl.210066-21 · Cited by 12
Chakraborty, J., Majumder, S., & Tu, H. (2022) · doi:10.1145/3524491.3527305 · Cited by 9
“All the prior supervised works (except Fair-SMOTE [35]) damage performance to achieve fairness [30, 51, 57].”
BackgroundThomsen, F. K. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780197539538.003.0013 · Cited by 10
Detection and Mitigation of Algorithmic Bias via Predictive Parity
DiCiccio, C. J., Hsu, B., Yu, Y., Nandy, P., & Basu, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594117 · Cited by 12
“In fairness, lack of equal calibration among groups is an issue in many contexts including criminal justice [23] and healthcare [32].”
BackgroundKoenecke, A., Giannella, E., Willer, R., & Goel, S. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22163 · Cited by 11
Ruggieri, S., Álvarez, J. M., Pugnana, A., State, L., & Turini, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26798 · Cited by 11
Bansak, K. C. (2018) · Political Analysis · doi:10.1017/pan.2018.55 · Cited by 9
Tracking and Improving Information in the Service of Fairness
Garg, S., Kim, M. P., & Reingold, O. (2019) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3328526.3329624 · Cited by 11
Leveraging Semi-Supervised Learning for Fairness using Neural Networks
Noroozi, V., Bahaadini, S., Sheikhi, S., Mojab, N., & Yu, P. S. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00017 · Cited by 8
“The age feature is bucketized at the boundaries [18, 25, 30, 35, 40, 45, 50, 55, 60, 65].”
BackgroundAgnostic learning with multiple objectives
Cortes, C., Mohri, M., Gonzalvo, J., & Storcheus, D. (2020) · neural information processing systems · Cited by 9
Fairness-Driven Private Collaborative Machine Learning
Pessach, D., Tassa, T., & Shmueli, E. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639368 · Cited by 10
“Furthermore, we show that the runtime of the proposed method is feasible, especially considering that it is executed once as a pre-process procedure.”
BackgroundMethodProbabilistic Permutation Graph Search
Vardasbi, A., Sarvi, F., & de Rijke, M. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532045 · Cited by 10
“Neither do we want to unify different fairness measures, since they deal with different aspects of fairness, and more importantly, it has been shown that there is an inherent trade-off between some fairness conditions [21], which makes it impossible to have a unified fairness measure.”
BackgroundCastro, C. (2022) · Public Affairs Quarterly · doi:10.5406/21520542.36.2.04 · Cited by 9
“As Kleinberg et al. (2016), Chouldechova (2017), and Corbett-Davies et al. (2017) have since shown, in many contexts—including the one COMPAS operates in—parity in overall error rates and parity in accuracy rates within risk groups are mutually exclusive options (this result is described in detail…”
BackgroundLocal Justice and the Algorithmic Allocation of Scarce Societal Resources
Das, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21486 · Cited by 11
“Notably there have been some impossibility results, showing that several different fairness criteria that all seem intuitively reasonable cannot be satisfied simultaneously (Kleinberg, Mullainathan, and Raghavan 2017; Pleiss et al. 2017; Feller et al. 2016).”
Value Alignment for Advanced Artificial Judicial Intelligence
Winter, C. K., Hollman, N., & Manheim, D. (2023) · American Philosophical Quarterly · doi:10.5406/21521123.60.2.06 · Cited by 12
“Meanwhile, literature on algorithmic decision-making identifies several possible conceptions of fairness that may be sufficiently specified, yet some are incompatible with one another (see Berk et al. 2021; Binns 2018; Clouser and Gert 1990; Kleinberg et al. 2017; Ruf and Detyniecki 2021).”
Access to Population-Level Signaling as a Source of Inequality
Immorlica, N., Ligett, K., & Ziani, J. (2019) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3287560.3287579 · Cited by 8
“At a high level, these objectives tend to focus either on ensuring grouplevel fairness [8, 17, 19, 23, 24, 29, 31, 32, 40] or individual-level fairness [15, 27, 30].”
BackgroundFair Decisions Despite Imperfect PredictionsPreprint
Kilbertus, N., Gomez-Rodriguez, M., Schölkopf, B., Muandet, K., & Valera, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02979 · Cited by 16
“Only recently has the distinction been made explicit (Corbett-Davies et al., 2017; Kleinberg et al., 2017a; Mitchell et al., 2018; Valera et al., 2018).”
BackgroundRolf, E., Simchowitz, M., Dean, S., Liu, L. T., Björkegren, D., Hardt, M., & Blumenstock, J. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06740 · Cited by 10
“Machine learning systems in general cannot satisfy multiple definitions of group fairness (Chouldechova, 2017; Kleinberg et al., 2017), and there are inherent limitations to using observational criteria (Kilbertus et al., 2017).”
BackgroundChen, S., Koehler, F., Moitra, A., & Yau, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04787 · Cited by 14
Individual Explanations in Machine Learning Models: A Survey for PractitionersPreprint
Carrillo, A., Cantú, L. F., & Noriega-Campero, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04144 · Cited by 18
“In the past two decades, algorithmic decision-making has spread increasingly to many relevant societal contexts, including the provision of health services [1], counter-terrorism [2], criminal justice [3], and assessment of credit and insurance risk [4], to name a few.”
FADE: FAir Double Ensemble Learning for Observable and Counterfactual Outcomes
Mishler, A., & Kennedy, E. H. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533167 · Cited by 9
Gradual (In)Compatibility of Fairness Criteria
Hertweck, C., & Räz, T. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21450 · Cited by 9
“…some combinations of these fairness measures is not possible; impossibility theorems establish that satisfaction of one fairness measure may prevent satisfaction of another under certain conditions (Kleinberg, Mullainathan, and Raghavan 2016; Chouldechova 2017; Barocas, Hardt, and Narayanan 2019).”
BackgroundACROCPoLis: A Descriptive Framework for Making Sense of Fairness
Aler Tubella, A., Coelho Mollo, D., Dahlgren Lindström, A., Devinney, H., Dignum, V., Ericson, P., Jonsson, A., Kampik, T., Lenaerts, T., Mendez, J. A., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594059 · Cited by 8
“Further complicating the issue, analyses have shown that most of such operationalizations are mutually incompatible, and thereby cannot be simply added to improve algorithmic fairness [36, 43].”
BackgroundProvable Detection of Propagating Sampling Bias in Prediction Models
Ravishankar, P., Mo, Q., McFowland, E., & Neill, D. B. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26144 · Cited by 8
Conditional Supervised Contrastive Learning for Fair Text Classification
Chi, J., Shand, W., Yu, Y., Chang, K.-W., Zhao, H., & Tian, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.199 · Cited by 10
Cooperative Multi-Agent Fairness and Equivariant Policies
Grupen, N. A., Selman, B., & Lee, D. D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21166 · Cited by 12
“Existing work has shown both theoretically (Calders, Kamiran, and Pechenizkiy 2009; Kleinberg, Mullainathan, and Raghavan 2016; Zhao and Gordon 2019) and empirically (Dwork et al. 2012; Feldman et al. 2015; Kamiran and Calders 2009; Lahoti, Gummadi, and Weikum 2019; Pannekoek and Spigler 2021) that…”
MethodFairness in machine learning from the perspective of sociology of statistics
Benbouzid, B. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593974 · Cited by 15
“even been formalized as a theorem [43].”
Fairness in Contextual Resource Allocation Systems: Metrics and Incompatibility Results
Jo, N., Tang, B., Dullerud, K., Aghaei, S., Rice, E. R. W., & Vayanos, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i10.26397 · Cited by 9
Speed And Accuracy Are Not Enough! Trustworthy Machine Learning
Kaul, S. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278796 · Cited by 8
“Many of these tradeoffs are surprisingly harsh, suggesting that simple notions of fairness cannot coincide [1, 7].”
BackgroundThe Invisible Power of Fairness. How Machine Learning Shapes Democracy
Beretta, E., Santangelo, A. D. M., Lepri, B., Vetrò, A., & Martin, J. C. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-18305-9_19 · Cited by 11
“Considering that more than one definition of fairness cannot be achieved simultaneously [18], design choices have a relevant impact on the effect that the algorithm’s outcomes will have on society.”
BackgroundReichman, A., & Giovanni Sartor, B. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108914857.009 · Cited by 8
Addressing multiple metrics of group fairness in data-driven decision makingPreprint
Miron, M., Tolan, S., Gómez, E., & Castillo, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04794 · Cited by 13
“Furthermore, there are impossibility theorems [20,10] that mathematically prove the impossibility of reconciling different fairness notions if the prevalence of the outcome, i.”
BackgroundInfluentialYu, B., Yuan, Y., Terveen, L., Wu, Z. S., Forlizzi, J., & Zhu, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03061 · Cited by 8
“Recent studies indicate that different desirable notions of fairness are not only incompatible with each other [43], but often mutually exclusive [33, 9].”
BackgroundInterpretable and Fair Boolean Rule Sets via Column GenerationPreprint
Lawless, C., Dash, S., Günlük, O., & Wei, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08466 · Cited by 13
“In fact, in most non-trivial applications strong adherence to fairness criteria comes at a large cost to accuracy (Kleinberg et al., 2017) and therefore one needs to consider how much these conditions are violated as a measure of fairness.”
BackgroundResultWhat is fair? Proxy discrimination vs. demographic disparities in insurance pricingPreprint
Lindholm, M., Richman, R., Tsanakas, A., & Wüthrich, M. V. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4436409 · Cited by 8
AI for the Common Good?! Pitfalls, challenges, and ethics pen-testing
Berendt, B. (2019) · Paladyn, Journal of Behavioral Robotics · doi:10.1515/pjbr-2019-0004 · Cited by 8
Comparing ‘fair’ machine learning models for detecting at-risk online gamblers
Murch, W. S., Kairouz, S., & French, M. A. (2024) · International Gambling Studies · doi:10.1080/14459795.2024.2412051 · Cited by 8
“Nor can an equalized system satisfy the standard of outcome calibration when the outcome base rate differs between groups (Kleinberg et al., 2017).”
Mahajan, S., Agarwal, R., & Gupta, M. (2025) · Risks · doi:10.3390/risks13090160 · Cited by 7
“Kleinberg et al.’s impossibility theorem shows that competing group notions cannot, in general, be satisfied simultaneously (Kleinberg et al. 2016), and Hardt et al. propose equal opportunity as a tractable compromise (Hardt et al. 2016).”
Algorithmic fairness: challenges to building an effective regulatory regime
Demirchyan, G. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1637134 · Cited by 11
Metadata
- Authors
- Kleinberg, J., Mullainathan, S., & Raghavan, M.
- Affiliations
- Cornell University (US)
- Harvard University (US)
- University of California, Berkeley (US)
- Venue
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Publisher
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W2962922665
- Cited by
- 943
- References
- 35
- Citations by year
- 2026: 49 · 2025: 74 · 2024: 112 · 2023: 139 · 2022: 102 · 2021: 153 · 2020: 129 · 2019: 107 · 2018: 54 · 2017: 21
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Leibniz international proceedings in informatics (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, risk score, probabilistic classification, fairness conditions, impossibility results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
