The DOI matches its DataCite record; 21 of the 35 references it cites were found in a registry or an index.
Inherent Trade-Offs in the Fair Determination of Risk Scores
Open accessAbstract
Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“each technical definition carry strong values assumptions which can often be in tension with each other (Friedler et al., 2016; Kleinberg et al., 2017).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Kleinberg et al., 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature often treats fairness and performance as a trade-off problem”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Kleinberg et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Many fairness metrics have been proposed, and the most widely used criteria were shown to be incompatible ([20], [41]), and to present risks for the populations they were meant to protect ([24]).”
From Fairness and explainability in automatic decision-making systems. A challenge for computer science and law (2023) · cites this paper as [41]
The cited source says what the claim attributes to it.
In the citing paper
“commonly used statistical fairness criteria can be mutually incompatible except under restrictive conditions”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Kleinberg et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“Kleinberg, Mullainathan and Raghavan (2017) shows theoretically that these notions of fairness are usually incompatible.”
From An algorithm for removing sensitive information: Application to race-independent recidivism prediction (2019) · cites this paper as Kleinberg, Mullainathan and Raghavan (2017)
The cited source says what the claim attributes to it.
In the citing paper
“These two notions of fairness are indeed distinct.”
From An algorithm for removing sensitive information: Application to race-independent recidivism prediction (2019) · cites this paper as Kleinberg, Mullainathan and Raghavan (2017)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“propose a framework based on a task-specific externally defined similarity metric between individuals, seeking to achieve fairness through the goal that “similar people [be] treated similarly” [11].”
BackgroundDiscrimination and racial disparities in health: evidence and needed research
Williams, D. R., & Mohammed, S. A. (2009) · Journal of Behavioral Medicine · doi:10.1007/s10865-008-9185-0 · Cited by 3,371 · cited 1×
“Here too we can ask whether such decision-making is being applied uniformly across different groups of patients [16, 27], and in particular how medical tests may play a differential role for conditions that vary widely in frequency between these groups.”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“Barocas and Selbst survey a range of ways in which data-analysis algorithms can lead to discriminatory outcomes [3], and review articles by Romei and Ruggieri [25] and Zliobaite [30] survey data-analytic and algorithmic methods for measuring discrimination.”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“consider the natural analogues of our conditions (B) and (C), balance for the negative and positive classes, in the case of classifiers that output binary “yes/no” predictions rather than real-valued scores as in our case [19].”
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 3×
“Also concurrent with our work and that of Hardt et al., Chouldechova [5] and Corbett-Davies et al. [6] (and see also [7]) consider binary prediction subject to these same analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction (requiri…”
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1×
“This latter global requirement is a version of statistical parity [12, 4, 21, 22].”
BackgroundAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. Z. (2017) · Knowledge Discovery and Data Mining · doi:10.1145/3097983.309809 · Cited by 1,464 · cited 1×
“[7] (and see also [6]) consider binary prediction subject to analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction.”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”
BackgroundThree naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · cited 3×
“Work in these directions has developed learning algorithms that penalize violations of statistical parity [4, 22].”
BackgroundAutomated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676 · cited 1×
“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”
Background
Flagged first, then by citations
14 of the 35 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [9][9] Amit Datta, Michael Carl Tschantz, and Anupam Datta. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies , 2015(1):92-112, 2015. -
- [14][14] Dean P. Foster and Rakesh V. Vohra. Asymptotic calibration. Biometrika , 85(2):379-390, 1998. -
- [29][29] Richard S Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning Fair Representations. Proceedings of the 30th International Conference on Machine Learning , 28:325-333, 2013. -
Cited by
Sorted by citations
Items 601–700 of 945
ARTIFICIAL INTELLIGENCE-BASED AUDIT SOFTWARE: TODAY'S REALITIES AND FUTURE VISION
Altundağ, S. (2024) · Denetişim · doi:10.58348/denetisim.1512650 · Cited by 3
Preference eigensystems for fair ranking
Melucci, M. (2025) · Expert systems with applications · doi:10.1016/j.eswa.2024.126324 · Cited by 2
Cherry on the cake: fairness is NOT an optimization problem
Favier, M., & Calders, T. (2025) · Machine Learning · doi:10.1007/s10994-025-06792-3 · Cited by 5
“Critically, we must be aware that some of the choices of µ ∆ may lead to measures that are not atomless, hence by Proposition 1, the space ∆ n × [0 , 1) provides a universal cake where the players µ i are defined by for some probability measure µ ∆ on ∆ n .”
Kanepajs, A., Basu, A., Ghose, S., Li, C. H., Mehta, A. J., Mehta, R., Tucker-Davis, S. D., Fischer, B., & Anthis, J. R. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732094 · Cited by 7
“Researchers have developed a number of bias metrics based on associations between sensitive attributes and model output [32], but the general-purpose applications of LLMs raise fundamental challenges for bias measurement given their wide-spread use and massive amounts of training data [5, 55].”
AI biases as asymmetries: a review to guide practice
Waters, G., & Honenberger, P. (2025) · Frontiers in Big Data · doi:10.3389/fdata.2025.1532397 · Cited by 7
Fairness analysis of machine learning predictions of aggression in acute psychiatric care
Wang, Y., Sikstrom, L., Xiao, R., Findlay, Z., Zaheer, J., Hill, S., & Maslej, M. M. (2026) · npj Mental Health Research · doi:10.1038/s44184-026-00194-6 · Cited by 4
Bayesian Sampler: Fairness in Sampling
Chakraborty, I. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00048 · Cited by 3
“The machine learning community has proposed different frameworks that attempt to define fairness in classification [20], [16], [21], [27], [22], [23], Equalized Odds [21] and Disparate Mistreatment [23].”
BackgroundConstantin, R., Duck, M., Alexandrov, A., Matosevic, P., Keidar, D., & El-Assady, M. (2022) · 2022 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX) · doi:10.1109/trex57753.2022.00005 · Cited by 3
“However, these definitions may contradict each other [18], and it is challenging to determine which fairness definition is the appropriate one for a given context.”
BackgroundOn the Impact of Data Quality on Image Classification Fairness
Barry, A., Han, L., & Demartini, G. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592031 · Cited by 5
“Then, the only way that calibration, FPR, and NPR can all be upheld at once is if the predictor is perfect, or, the base rates of the protected attributes are the same [21].”
BackgroundBanda, J. M., Shah, N. H., & Periyakoil, V. S. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad043 · Cited by 4
Declarative Encoding of Fairness in Logic Tensor Networks
Greco, G., Alberici, F., Palmonari, M., & Cosentini, A. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230360 · Cited by 4
Garip, F., & Macy, M. W. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197653609.013.11 · Cited by 4
Fairness in Predictive Learning Analytics: A Case Study in Online STEM Education
Al-Zawqari, A., & Vandersteen, G. (2023) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10343059 · Cited by 6
“Two common explanations for models’ poor performance are having less number of samples and historical biases in the data [3], [19].”
BackgroundFairness in Ranking under Disparate Uncertainty
Rastogi, R., & Joachims, T. (2024) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3689904.3694703 · Cited by 4
Strasser, K., & Niedermayer, B. (2021) · Management-Reihe Corporate Social Responsibility · doi:10.1007/978-3-662-63223-9_6 · Cited by 3
Correspondences between Privacy and Nondiscrimination: Why They Should Be Studied Together
Datta, A., Sen, S., & Tschantz, M. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.01735 · Cited by 4
“Recently a number of richer associative notions of nondiscrimination have been proposed [40, 49, 50] that take into account three variables, group membership, predicted outcome, and true outcome, in various combinations.”
BackgroundEliminating Latent Discrimination: Train Then MaskPreprint
Ghili, S., Kazemi, E. M., & Karbasi, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3309776 · Cited by 3
Fairness criteria through the lens of directed acyclic graphical modelsPreprint
Baer, B. R., Gilbert, D. E., & Wells, M. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.11333 · Cited by 3
“(See in particular Scenario 1: The Red Car in Kusner et al. [2017].) Specifically, suppose that the prediction R(X) = βX and the prediction S(X) = β AX, where each coefficient βA differs based on the sensitive characteristic A.”
BackgroundInsurance Discrimination and Fairness in Machine Learning: An Ethical AnalysisPreprint
Loi, M., & Christen, M. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3438823 · Cited by 3
Impact of Bias on School Admissions and Targeted Interventions.Preprint
Faenza, Y., Gupta, S., & Zhang, X. (2020) · arXiv (Cornell University) · Cited by 6
Fairness-Aware Online PersonalizationPreprint
Lal, G., Geyik, S. C., & Kenthapadi, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.15270 · Cited by 3
Where Is the Normative Proof? Assumptions and Contradictions in ML Fairness ResearchPreprint
Cooper, A. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10407 · Cited by 3
Towards Fairness Certification in Artificial IntelligencePreprint
Tommasi, T., Bucci, S., Barbara Caputo, B., & Asinari, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02498 · Cited by 3
“Indeed it might be infeasible to satisfy all the criteria at once except in highly constrained cases [7].”
BackgroundOn Fairness and Stability in Two-Sided MatchingsPreprint
Karni, G., Rothblum, G. N., & Yona, G. (2021) · arXiv (Cornell University) · doi:10.4230/lipics.itcs.2022.92 · Cited by 6
Wenn Algorithmen für uns entscheiden: Chancen und Risiken der künstlichen Intelligenz
Christen, M., Mader, C., Čas, J., Abou‐Chadi, T., Bernstein, A., Binder, N. B., Dell’Aglio, D., Fabbian, L., George, D., Gohdes, A., et al. (2020) · vdf Hochschulverlag AG an der ETH Zürich eBooks · doi:10.1553/10.3218/4002-9 · Cited by 13
An Anti-Subordination Approach to Fair ClassificationPreprint
Keswani, V., & Celis, L. E. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4116835 · Cited by 3
Sargeant, H. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4513990 · Cited by 3
Discretion Over Information: Evidence from pretrial decisionsPreprint
Davenport, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4580390 · Cited by 3
Practices for Engineering Trustworthy Machine Learning Applications
Serban, A., van der Blom, K., Hoos, H. H., & Visser, J. (2021) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/wain52551.2021.00021 · Cited by 24
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0005 · Cited by 2
Justicia algorítmica: un enfoque sociotécnico
Bolado, A. B. (2023) · Estudios Penales y Criminológicos · doi:10.15304/epc.44.8838 · Cited by 7
Improving Group Fairness Assessments with Proxies
Harvey, E., Lee, M. S. A., & Singh, J. P. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3677175 · Cited by 2
“In response, Northpointe, the creator of COMPAS, argued that their tool was well calibrated, a different group fairness metric that has been shown to be mathematically incompatible with the equality of error rates metric used by ProPublica [19, 28, 53].”
Finding representative group fairness metrics using correlation estimations
Anahideh, H., Nezami, N., & Asudeh, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125652 · Cited by 3
“Building on previous works [13, 41], [14] provides a comparative using mathematical representations to discuss the trade-off between some of the common notions of fairness.”
BackgroundSpanning in and Spacing out? A Reply to Eva
Nielsen, M., & Stewart, R. T. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00829-6 · Cited by 2
“Even in the presence of Equalized Odds, Spanning and Spacing are consistent.”
InfluentialFairness Challenges in the Design of Machine Learning Applications for Healthcare
Ryan, S., Cai, W., Bowman, R., & Doherty, G. J. (2025) · ACM Transactions on Computing for Healthcare · doi:10.1145/3728368 · Cited by 5
“This is further complicated by the fact that some definitions are incompatible with each other, and all of the definitions have their own set of assumptions that they make about the decision being made [27].”
Robustness and trustworthiness in AI: a no-go result from formal epistemology
Hornischer, L. (2026) · Synthese · doi:10.1007/s11229-025-05272-4 · Cited by 3
“We may similarly consider further concepts like bias or fairness (e.g., express modally the various defini - tions of fairness (Shmueli et al., 2023) and their joint inconsistency (Chouldechova, 2017; Kleinberg et al., 2017)).”
Who Invests, Who Gets Funded: Gender and Racial Bias in LLM-Generated Investment Advice
Wang, Y. E., & Gu, K. (2026) · Journal of Business Ethics · doi:10.1007/s10551-026-06251-6 · Cited by 2
“Kleinberg et al. (2018) also demonstrate that algorithmic risk assessments can achieve more equitable outcomes than human judges in judicial and financial contexts, especially when fairness metrics are explicitly encoded into the model’s objective function.”
Understanding AI risks from its characteristics and NMPA regulation perspectives
Liu, Y., & Yu, W. (2026) · Insights into Imaging · doi:10.1186/s13244-026-02283-8 · Cited by 2
“According to the well-established impossibility theorem [22], no single model can satisfy all fairness criteria when base rates differ across groups.”
The dark side of public values in algorithmic systems
Moats, D., Ruckenstein, M., Kristensen, D. B., Eidenskog, M., Elhadj, E., Lehtiniemi, T., Møhl, P., Žagar, A. P., Tanninen, M., & Velkova, J. (2026) · Dialogues on Digital Society · doi:10.1177/29768640261457507 · Cited by 5
“…within the FAccT community, 3 that it was dif fi cult to implement ethical principles algorithmically: fi rstly, because it was impossible to agree on what is meant by them (Narayanan, 2018) and secondly, because ful fi lling one principle often meant contra-vening another (Kleinberg et al., 2016).”
Increasing Fairness in Predictions Using Bias Parity Score Based Loss Function Regularization
Jain, B., Huber, M., & Elmasri, R. A. (2023) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.36.133311 · Cited by 9
In Defense of Sociotechnical Pragmatism
Watson, D., & Mökander, J. (2023) · Digital ethics lab yearbook · doi:10.1007/978-3-031-28678-0_10 · Cited by 3
The Equity Framework: Fairness Beyond Equalized Predictive Outcomes
Naggita, K., & Aguma, J. C. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230083 · Cited by 6
LightMIRM: Light Meta-learned Invariant Risk Minimization for Trustworthy Loan Default Prediction
Jiang, M., Zhang, Y., Gao, Y., Wang, Y., Feng, F., & He, X. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00268 · Cited by 2
“The measurement of fairness falls under several categories [20–22], while in our paper, we mainly focus on Calibration [23, 24] which means that false positive rates across groups should be similar.”
BackgroundAnalysis and Mitigation of Religion Bias in Indonesian Natural Language Processing Datasets
Fauzan, M. A., & Saptawijaya, A. (2023) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v7i4.5035 · Cited by 2
“One difficulty that may arise from appropriately measuring allocation harm using parity conditions is their incompatibility with each other, where all four parity conditions previously mentioned cannot be satisfied at the same time [15].”
BackgroundLocating disparities in machine learning
von Zahn, M., Hinz, O. F., & Feuerriegel, S. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386485 · Cited by 6
“Under certain mathematical conditions, it is even impossible for several of them to be simultaneously satisfied [26, 27, 28, 29].”
Background(Un)fair Exposure in Deep Face Rankings at a Distance
Atzori, A., Fenu, G., & Marras, M. (2023) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb57857.2023.10449007 · Cited by 2
“…research addressing this beyond accuracy objective is emerging, its scope has been restricted to pure biometric authentication and identification tasks [26, 4], leaving out potential adverse ramifications from a ranking standpoint and relying exclusively on close-range image data [24, 46, 30, 3].”
BackgroundÜber das Verhältnis von Ethik und Algorithmen
Reder, M., Müller, N. H., & Lehmann, R. (2024) · Edition Moderne, Postmoderne · doi:10.1515/9783839469057-002 · Cited by 4
HR in the Era of AI Paradoxes: Balancing Automation and Human Touch
Suneetha, S., Sudheer, A. K., Shankar, C., & Metta, S. (2024) · Advances in economics, business and management research/Advances in Economics, Business and Management Research · doi:10.2991/978-94-6463-433-4_40 · Cited by 8
A Canonical Data Transformation for Achieving Inter- and Within-Group Fairness
Lazri, Z. M., Brugere, I., Tian, X., Dachman-Soled, D., Polychroniadou, A., Dervovic, D., & Wu, M. H. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3416040 · Cited by 2
“As multiple works have shown [7, 20, 21], achieving a universal notion of fairness is impossible.”
BackgroundExploring latent discrimination through an Object-Relational Causal Inference method
Wang, Y., & Luo, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112148 · Cited by 2
Modeling Individual Fairness Beliefs and Its Applications
Zhang, C., Jacob, V. S., & Ryu, Y. U. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3682070 · Cited by 2
“Despite these attempts to improve fairness, there are reports on difficulties such as incompati-bilities among fairness algorithms [ 10 , 46 ] and the failure to deliver anticipated benefits [ 33 ].”
Bias Detection and Correction Methods for Machine Learning Algorithms
Dimitrakopoulou, G., Kapsalis, N., & Kokkinis, G. (2024) · 2024 5th International Conference in Electronic Engineering, Information Technology & Education (EEITE) · doi:10.1109/eeite61750.2024.10654404 · Cited by 2
“The problem of identifying a metric that simultaneously satisfies three major fairness measures, namely the calibration within groups, balance for the positive class, and balance for the negative class is feasible only in highly constrained special cases [21].”
Discrimination Bias Detection Through Categorical Association in Pre-Trained Language Models
Dusi, M., Arici, N., Gerevini, A., Putelli, L., & Serina, I. (2024) · IEEE Access · doi:10.1109/access.2024.3482010 · Cited by 5
“Some are also proven to be incompatible,, 2 meaning that not every fairness definition can be satisfied at the same time for the same system [24].”
BackgroundFair CoVariance Neural Networks
Cavallo, A., Navarro, M., Segarra, S., & Isufi, E. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888282 · Cited by 7
“Indeed, the downside of popular fairness definitions such as demographic parity (DP) [36] and equality of odds (EO) [37] is that they focus solely on outcomes without considering if the model exhibits preferential learning for certain groups [38], [39].”
Interdisciplinary Perspectives on the (Un)fairness of Artificial Intelligence
Starke, C., Blanke, T., Helberger, N., Smets, S., & de Vreese, C. H. (2025) · Minds and Machines · doi:10.1007/s11023-025-09722-3 · Cited by 2
“The wide range of fairness definitions reflects diverse perspectives on what fairness entails, yet these conceptions can also conflict with one another (Kleinberg et al., 2016).”
SoK: Fair Clustering: Critique, Caveats, and Future Directions
Dickerson, J. W. T., Esmaeili, S. A. H., Morgenstern, J., & Zhang, C. J. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00044 · Cited by 3
“In supervised learning, the work of [112] showed that two desired fairness notions (calibration and balance) cannot be satisfied simultaneously but the fair clustering literature has not considered the interaction of the different fairness notions with the exception of the recent work of [111] and…”
When Mitigating Bias is Unfair: Multiplicity and Arbitrariness in Algorithmic Group Fairness
Krčo, N., Laugel, T., Grari, V., Loubes, J., & Detyniecki, M. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00046 · Cited by 13
Role and Use of Race in Artificial Intelligence and Machine Learning Models Related to Health
Were, M. C., Li, A., Malin, B., Yin, Z., Coco, J., Collins, B. X., Clayton, E. W., Novak, L. L., Hendricks‐Sturrup, R., Oluyomi, A., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/73996 · Cited by 4
“Thus, the selection of fairness metrics should be based on the specific needs of each use case, recognizing that all metrics cannot be achieved at the same time [74].”
Kollaborative Diensteplattform. Digitalisierung als Mittel teamorientierter Selbstorganisation
Kubek, V., Velten, S., Uhde, A., Schlicker, N., & Blaudszun-Lahm, A. (2020) · Springer eBooks · doi:10.1007/978-3-662-60874-6_5 · Cited by 2
When users control the algorithms: Values expressed in practices on the twitter platform
Burrell, J., Kahn, Z., Jonas, A., & Griffin, D. (2019) · eScholarship (California Digital Library) · doi:10.1145/33592407 · Cited by 2
Achieving Fair Treatment in Algorithmic Classification
Morgan, A., & Pass, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03807-6_22 · Cited by 2
“, when ΨP = Ω C P = {0, 1}), fair treatment is equivalent to requiring “similar” false positive and false negative rates [8].”
BackgroundMethodInfluentialFairway: SE Principles for Building Fairer Software.
Chakraborty, J., Majumder, S., Wu, Z., & Menzies, T. (2020) · Cited by 2
Brayne, S. (2020) · Oxford University Press eBooks · doi:10.1093/oso/9780190684099.003.0003 · Cited by 2
Automated Risk Assessment in the Criminal Justice Process
Begby, E. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780198852834.003.0009 · Cited by 2
A Systematic Approach to Group Fairness in Automated Decision Making
Hertweck, C., & Heitz, C. (2021) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds51136.2021.00008 · Cited by 3
“Note that enforcing all fairness metrics at the same time is a mathematical impossibility in most cases (and thus essentially always in practice) [14], [24], [25].”
BackgroundA systematic approach to group fairness in automated decision making
Hertweck, C., & Heitz, C. (2021) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.21256/zhaw-23113 · Cited by 2
Machine Learning Equity and Accuracy in an Applied Justice Setting
Russell, J. R. (2021) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp52413.2021.00050 · Cited by 2
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0003 · Cited by 2
AI and the UN Sustainable Development Goals
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0012 · Cited by 2
Yet Another Impossibility Theorem in Algorithmic Fairness
Beigang, F. (2023) · Minds and Machines · doi:10.1007/s11023-023-09645-x · Cited by 2
“Most famously, Chouldechova (2017) and Kleinberg et al. (2016) have shown that under realistic conditions, equalized odds and predictive parity are mutually inconsistent.”
FairGridSearch: A Framework to Compare Fairness-Enhancing Models
Ma, S.-C., Ermakova, T., & Fabian, B. (2023) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat59888.2023.00064 · Cited by 2
“Several studies have shown that it is difficult to satisfy some of the group fairness constraints at once except in highly constrained special cases [25], or even impossible [26], [27], [28].”
BackgroundEvaluating and Improving the Performance and Racial Fairness of Algorithms for GFR Estimation
Zhang, L., Richter, L. R., Kim, T., & Hripcsak, G. M. (2024) · IEEE International Conference on Artificial Intelligence for Medicine, Health and Care (AIMHC) · doi:10.1109/aimhc59811.2024.00051 · Cited by 3
“However, previous studies have shown that an algorithm can be fair by some error metrics but not others [15], [22].”
BackgroundThe invisible power of fairness. How machine learning shapes democracyPreprint
Beretta, E., Santangelo, A. D. M., Lepri, B., Vetrò, A., & Martin, J. C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.09493 · Cited by 2
“Considering that more than one definition of fairness cannot be achieved simultaneously [18], design choices have a relevant impact on the effect that the algorithm’s outcomes will have on society.”
BackgroundFair quantile regressionPreprint
Yang, D., Lafferty, J., & Pollard, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.08646 · Cited by 6
Probabilistic Verification of Fairness Properties via ConcentrationPreprint
Bastani, O., Zhang, X., & Solar-Lezama, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02573 · Cited by 2
“For example, we can straightforwardly support other kinds of fairness for supervised learning [Galhotra et al. 2017; Kleinberg et al. 2017; Zafar et al. 2017].”
MethodSociotechnical Aspects of Automated Recommendations: Algorithms, Ethics, and Evaluation
Paraschakis, D. (2020) · doi:10.24834/isbn.9789178770755 · Cited by 2
Malinowska, J. K., & Żuradzki, T. (2020) · AJOB Neuroscience · doi:10.1080/21507740.2020.1778126 · Cited by 2
Online Multivalid Learning: Means, Moments, and Prediction IntervalsPreprint
Gupta, V., Jung, C., Noarov, G., Pai, M. M., & Roth, A. (2021) · RePEc: Research Papers in Economics · doi:10.4230/lipics.itcs.2022.82 · Cited by 19
Fairness-Aware Learning from Corrupted Data.Preprint
Konstantinov, N., & Lampert, C. H. (2021) · arXiv (Cornell University) · Cited by 2
Perrier, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04234 · Cited by 3
“Related work (Kleinberg, Mullainathan, and Raghavan 2016) on impossibility results that even probabilistic classification of being fair to different groups can be incompatible or potentially uncomputable provide another example of how complexity considerations can assist FML analysis.”
BackgroundFair Decision Rules for Binary ClassificationPreprint
Lawless, C., & Günlük, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.01325 · Cited by 8
“In fact, in most non-trivial applications strong adherence to fairness criteria come at a large cost to accuracy [26] and therefore one needs to consider how much these conditions are violated as a measure of fairness.”
BackgroundLocal Justice and the Algorithmic Allocation of Societal ResourcesPreprint
Das, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01236 · Cited by 2
“Notably there have been some impossibility results, showing that several different fairness criteria that all seem intuitively reasonable cannot be satisfied simultaneously (Kleinberg, Mullainathan, and Raghavan 2017; Pleiss et al. 2017; Feller et al. 2016).”
Affirmative Action with Multidimensional IdentitiesPreprint
Carvalho, J., Pradelski, B., & Williams, C. R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4070930 · Cited by 2
(2021) · doi:10.1287/orms.2021.03.20 · Cited by 2
Abstraction, mimesis and the evolution of deep learning
Eklöf, J., Hamelryck, T., Last, C., Grima, A., & Snis, U. L. (2023) · AI & Society · doi:10.1007/s00146-023-01688-z · Cited by 2
“However, there might be multiple conditions of fairness that (with mathematical certainty) cannot be satisfied at the same time (Kleinberg et al. 2017).”
A Fair price to pay: exploiting causal graphs for fairness in insurancePreprint
Côté, O., Côté, M., & Arthur Charpentier, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4709243 · Cited by 2
Building Non-Discriminatory Algorithms in Selected Data
Arnold, D. S., Dobbie, W., & Hull, P. (2024) · National Bureau of Economic Research · doi:10.3386/w32403 · Cited by 2
Selection Bias in Insurance: Why Portfolio-Specific Fairness Fails to Extend Market-WidePreprint
Côté, M., eacute, O. C. O. T., & Arthur Charpentier, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5018749 · Cited by 2
COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation
Wang, N., Wang, Q., Wang, Y.-C., Sanjabi, M., Liu, J., Firooz, H., Wang, H., & Nie, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.819 · Cited by 2
Ensuring Ethical, Transparent, and Auditable Use of Education Data and Algorithms on AutoML
Griep, K., Stone, L., Edwards, S. T., Hill, M., Cui, J., Liu, C., Jia, Q., & Song, Y. (2023) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1145/3639479.3639492 · Cited by 1
“…have been mentioned by researchers - 1) there is no holistic definition of fairness, instead, it can be defined in many ways [14]; 2) some fairness evaluation metrics are contradict each other, making it is almost impossible to achieve high fairness across many metrics at the same time [15].”
BackgroundVallor, S. (2024) · Oxford University Press eBooks · doi:10.1093/oso/9780197759066.003.0001 · Cited by 1
Explainable Disparity Compensation for Efficient Fair Ranking
Gale, A., & Marian, A. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00174 · Cited by 1
“In [28] the authors prove that different fairness metrics are incompatible, even in the approximate case.”
BackgroundGroup Fairness Refocused: Assessing the Social Impact of ML Systems
Hertweck, C., Loi, M., & Heitz, C. (2024) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds60720.2024.00034 · Cited by 2
“Other important works in this area include [5] and [21].”
BackgroundHuman–Robot Interactions and Procedural Law
Gless, S., & Whalen-Bridge, H. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009431453.008 · Cited by 1
Spanning and Spacing: Commentary on ‘New Possibilities for Fair Algorithms’
Eva, B. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00823-y · Cited by 1
“…is clearly entailed by the requirement that the average of risk score assigned to the group’s members be equal to the group’s base rate, which in turn is entailed by the well known ‘calibration within groups’ criterion (see e.g. Chouldechova (2017), Kleinberg et al. (2017), Pleiss et al. (2017)).”
CRAFT: Identifying Key Nodes in the Science Communication Community via Counterfactual Fairness
Jiang, W., Tang, Q., Lin, L., Han, Y., Wang, R., & He, H. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3501740 · Cited by 1
“This structure serves as a reference point for evaluating events and entities, constructed based on previous experiences and various cognitive and social information processes [9].”
Algorithmic Fairness as an Inconsistent Concept
Hummel, P. (2025) · American Philosophical Quarterly · doi:10.5406/21521123.62.1.04 · Cited by 1
“…in classification and prediction in the 1970s (Hutchinson and Mitchell 2019, 50–51) and proven more recently by several authors (Chouldechova 2017; Kleinberg, Mullainathan, and Raghavan 2017; Berk et al. 2018), setting aside highly contrived scenarios, it is practically impossible to satisfy…”
InfluentialSmuha, N. A. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009367783.002 · Cited by 1
The Cost of Representation by Subset Repairs
Liu, Y.-X., Shen, F., Ghosh, K., Gilad, A., Kimelfeld, B., & Roy, S. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3705829.3705860 · Cited by 2
Perceptions of the Fairness Impacts of Multiplicity in Machine Learning
Meyer, A. P., Kim, Y.-S., D'Antoni, L., & Albarghouthi, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713524 · Cited by 1
“We do not include optimizing for fairness as one of the possible multiplicity resolution techniques because the task of choosing the fairest model is typically not well-defined [17, 51]. decision making.”
Metadata
- Authors
- Kleinberg, J., Mullainathan, S., & Raghavan, M.
- Affiliations
- Cornell University (US)
- Harvard University (US)
- University of California, Berkeley (US)
- Venue
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Publisher
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W2962922665
- Cited by
- 943
- References
- 35
- Citations by year
- 2026: 49 · 2025: 74 · 2024: 112 · 2023: 139 · 2022: 102 · 2021: 153 · 2020: 129 · 2019: 107 · 2018: 54 · 2017: 21
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Leibniz international proceedings in informatics (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, risk score, probabilistic classification, fairness conditions, impossibility results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
