The DOI matches its DataCite record; 21 of the 35 references it cites were found in a registry or an index.
Inherent Trade-Offs in the Fair Determination of Risk Scores
Open accessAbstract
Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“each technical definition carry strong values assumptions which can often be in tension with each other (Friedler et al., 2016; Kleinberg et al., 2017).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Kleinberg et al., 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature often treats fairness and performance as a trade-off problem”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Kleinberg et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Many fairness metrics have been proposed, and the most widely used criteria were shown to be incompatible ([20], [41]), and to present risks for the populations they were meant to protect ([24]).”
From Fairness and explainability in automatic decision-making systems. A challenge for computer science and law (2023) · cites this paper as [41]
The cited source says what the claim attributes to it.
In the citing paper
“commonly used statistical fairness criteria can be mutually incompatible except under restrictive conditions”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Kleinberg et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“Kleinberg, Mullainathan and Raghavan (2017) shows theoretically that these notions of fairness are usually incompatible.”
From An algorithm for removing sensitive information: Application to race-independent recidivism prediction (2019) · cites this paper as Kleinberg, Mullainathan and Raghavan (2017)
The cited source says what the claim attributes to it.
In the citing paper
“These two notions of fairness are indeed distinct.”
From An algorithm for removing sensitive information: Application to race-independent recidivism prediction (2019) · cites this paper as Kleinberg, Mullainathan and Raghavan (2017)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“propose a framework based on a task-specific externally defined similarity metric between individuals, seeking to achieve fairness through the goal that “similar people [be] treated similarly” [11].”
BackgroundDiscrimination and racial disparities in health: evidence and needed research
Williams, D. R., & Mohammed, S. A. (2009) · Journal of Behavioral Medicine · doi:10.1007/s10865-008-9185-0 · Cited by 3,371 · cited 1×
“Here too we can ask whether such decision-making is being applied uniformly across different groups of patients [16, 27], and in particular how medical tests may play a differential role for conditions that vary widely in frequency between these groups.”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“Barocas and Selbst survey a range of ways in which data-analysis algorithms can lead to discriminatory outcomes [3], and review articles by Romei and Ruggieri [25] and Zliobaite [30] survey data-analytic and algorithmic methods for measuring discrimination.”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“consider the natural analogues of our conditions (B) and (C), balance for the negative and positive classes, in the case of classifiers that output binary “yes/no” predictions rather than real-valued scores as in our case [19].”
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 3×
“Also concurrent with our work and that of Hardt et al., Chouldechova [5] and Corbett-Davies et al. [6] (and see also [7]) consider binary prediction subject to these same analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction (requiri…”
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1×
“This latter global requirement is a version of statistical parity [12, 4, 21, 22].”
BackgroundAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. Z. (2017) · Knowledge Discovery and Data Mining · doi:10.1145/3097983.309809 · Cited by 1,464 · cited 1×
“[7] (and see also [6]) consider binary prediction subject to analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction.”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”
BackgroundThree naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · cited 3×
“Work in these directions has developed learning algorithms that penalize violations of statistical parity [4, 22].”
BackgroundAutomated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676 · cited 1×
“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”
Background
Flagged first, then by citations
14 of the 35 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [9][9] Amit Datta, Michael Carl Tschantz, and Anupam Datta. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies , 2015(1):92-112, 2015. -
- [14][14] Dean P. Foster and Rakesh V. Vohra. Asymptotic calibration. Biometrika , 85(2):379-390, 1998. -
- [29][29] Richard S Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning Fair Representations. Proceedings of the 30th International Conference on Machine Learning , 28:325-333, 2013. -
Cited by
Sorted by citations
Items 301–400 of 945
Pfohl, S. R., Zhang, H., Xu, Y., Foryciarz, A., Ghassemi, M., & Shah, N. H. (2022) · Scientific Reports · doi:10.1038/s41598-022-07167-7 · Cited by 25
“…can reduce the fit of the model or induce miscalibration for all groups, including the ones for whom an unconstrained model performed poorly, due to differences in the distribution of the data collected for those subpopulations that limit the best-achievable values for the metric of interest [25–30].”
BackgroundCensored Fairness through Awareness
Zhang, W., Hernandez‐Boussard, T., & Weiss, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26708 · Cited by 25
“…or race aka sensitive attributes, achieve approximate parity of some statistic over class labels, such as statistical parity (Zhang and Ntoutsi 2019), disparate impact (Zafar et al. 2017), equality of opportunity (Hardt et al. 2016) and calibration (Kleinberg, Mullainathan, and Raghavan 2016).”
BackgroundArbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification
Cooper, A. F., Lee, K., Choksi, M. Z., Barocas, S., De, C., Grimmelmann, J., Kleinberg, J. M., Sen, S., & Zhang, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30203 · Cited by 22
“While the field has put forth numerous theoretical results about (un)fairness regarding single models — impossibility of satisfying multiple metrics (Kleinberg et al. 2017), post-processing individual models to achieve a particular metric (Hardt et al. 2016) — these results seem to miss the point.”
BackgroundMethodSelf-citationFairness in online social network timelines: Measurements, models and mechanism design
Hargreaves, E. M., Agosti, C., Menasché, D. S., Neglia, G., Reiffers-Masson, A., & Altman, E. (2018) · Performance Evaluation · doi:10.1016/j.peva.2018.09.009 · Cited by 21
Achieving Outcome Fairness in Machine Learning Models for Social Decision Problems
Fang, B., Jiang, M., Cheng, P.-Y., Shen, J., & Fang, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/62 · Cited by 25
MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset
Meng, C., Trinh, L., Xu, N., & Liu, Y. (2021) · doi:10.21203/rs.3.rs-402058/v1 · Cited by 21
“Interestingly, it has been proven that some of the competing definitions and statistics previously proposed are mutually exclusive [74].”
BackgroundAlgorithmic bias: Social science research integration through the 3-D Dependable AI Framework
Ukanwa, K. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101836 · Cited by 24
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B. W., & et al. Vogeli, C. (2020) · Yearbook of pediatric endocrinology · doi:10.1530/ey.17.12.7 · Cited by 26
The Myth of Complete AI-Fairness
Dignum, V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77211-6_1 · Cited by 21
“This means that if we calibrate data, we need to be prepared to accept higher levels of false positives and false negatives for some groups, and to deal with their human and societal impact [6].”
BackgroundAdaptive Sampling Strategies to Construct Equitable Training Datasets
Cai, W., Encarnación, R., Chern, B., Corbett‐Davies, S., Bogen, M., Bergman, S., & Goel, S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533203 · Cited by 21
“Machine learning models like this—as well as related models in natural language processing, healthcare, criminal justice, and beyond—have been shown to exhibit sharp disparities, often performing worse on subgroups of the population defined by race, ethnicity, gender, language, and nationality [9, 11, 15, 18, 20, 22,…”
BackgroundUsing artificial intelligence to identify administrative errors in unemployment insurance
Young, M., Himmelreich, J., Honcharov, D., & Soundarajan, S. (2022) · Government Information Quarterly · doi:10.1016/j.giq.2022.101758 · Cited by 28
Cox, M., Panagides, J. C., Tabari, A., Kalva, S. P., Kalpathy–Cramer, J., & Daye, D. (2022) · PLoS ONE · doi:10.1371/journal.pone.0277507 · Cited by 22
“Fairness is a complex social and mathematical concept with multiple conflicting definitions [46], and the problem of ensuring fairness in machine learning may not be solvable by computation alone [47].”
BackgroundImprove individual fairness in federated learning via adversarial training
Li, J., Zhu, T., Ren, W., & Choo, K.-K. R. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103336 · Cited by 21
Machine learning and the optimization of prediction-based policies
Battiston, P., Gamba, S., & Santoro, A. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.123080 · Cited by 26
Avoiding Disparity Amplification under Different Worldviews
Yeom, S., & Tschantz, M. C. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445892 · Cited by 36
Give us a little social credit: to design or to discover personal ratings in the era of Big Data
Devereaux, A. N., & Peng, L. (2020) · Journal of Institutional Economics · doi:10.1017/s1744137419000754 · Cited by 21
“…credit score, the coefficients as scores for each bin, and the function PRS as a mapping from individual i with a vector of coefficients α i to a score proxying a single aspect (like trustworthiness), our formulation corresponds with generalized models of risk assessment (Kleinberg et al ., 2016).”
The Intuitive-Override Model: Nudging Judges Toward Pretrial Risk Assessment InstrumentsPreprint
DeMichele, M. T., Comfort, M. L., Misra, S., Barrick, K., & Baumgartner, P. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3168500 · Cited by 24
Achieving Equalized Odds by Resampling Sensitive AttributesPreprint
Romano, Y., Bates, S., & Candès, E. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04292 · Cited by 28
“As a final remark, note that a naive resampling scheme where we instead randomly resample A unconditionally would result in a test of the hypothesis Ŷ ⊥ A, a property called demographic parity [2, 3].”
BackgroundInfluentialExplainability for fair machine learningPreprint
Begley, T., Schwedes, T., Frye, C., & Feige, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07389 · Cited by 54
“l reasoning, some imposing constraints on group outcomes, others at the individual level — and each notion is often incompatible with its alternatives (Berk et al., 2018; Corbett-Davies et al., 2017; Kleinberg et al., 2017; Lipton et al., 2018; Pleiss et al., 2017). Deciding which measure of fairness to impose on fthu…”
BackgroundMachine Learning and Mobile Phone Data Can Improve the Targeting of Humanitarian Assistance
Aiken, E., Bellue, S., Karlan, D., Udry, C., & Blumenstock, J. E. (2021) · National Bureau of Economic Research · doi:10.3386/w29070 · Cited by 23
Fairness Regulation of Prices in Competitive Markets
Yang, Z., Fu, X., Gao, P., & Chen, Y. (2022) · Manufacturing & Service Operations Management · doi:10.1287/msom.2022.0552 · Cited by 22
Towards Fairness-Aware Disaster Informatics: an Interdisciplinary Perspective
Yang, Y., Zhang, C., Fan, C. H., Mostafavi, A., & Hu, X. (2020) · IEEE Access · doi:10.1109/access.2020.3035714 · Cited by 19
Comparing minds and machines: implications for financial stability
Buckmann, M., Haldane, A., & Hüser, A. (2021) · Oxford Review of Economic Policy · doi:10.1093/oxrep/grab017 · Cited by 24
Calibrate: Interactive Analysis of Probabilistic Model Output
Xenopoulos, P., Rulff, J., Nonato, L. G., Barr, B., & Silva, C. T. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209489 · Cited by 21
“show that, if groups have different base rates for their labels, it is statistically impossible to ensure fairness across the balance of the positive and negative classes, as well as the calibration of the model [26].”
BackgroundSpecification overfitting in artificial intelligence
Roth, B., de Araujo, P. H. L., Xia, Y., Kaltenbrunner, S., & Korab, C. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11040-6 · Cited by 21
“It has been shown that no method can satisfy these fairness conditions simultaneously (Kleinberg et al, 2017).”
Christos Dimitrakakis, A., Liu, Y., Parkes, D. C., & Radanović, G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301509 · Cited by 24
“For instance, [Dwork et al., 2012; Chouldechova, 2016; Corbett-Davies et al., 2017; Kleinberg, Mullainathan, and Raghavan, 2016; Kilbertus et al., 2017] studied fairness under a setting where the model is known.”
BackgroundInfluentialSun, Y., Haghighat, F., & Fung, B. C. M. (2021) · Energy · doi:10.1016/j.energy.2021.122273 · Cited by 21
“Since not all fairness measures can be met simultaneously in most cases [46], researchers should select the ones relevant to research objective.”
BackgroundSelective Classification Can Magnify Disparities Across GroupsPreprint
Jones, E., Sagawa, S., Koh, P. W., Kumar, A., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.14134 · Cited by 34
“These disparities have implications for model robustness and equity, and mitigating them is an important open challenge (Dwork et al., 2012; Hardt et al., 2016; Kleinberg et al., 2017; Duchi et al., 2019; Sagawa et al., 2020).”
BackgroundThe Possibility of Fairness: Revisiting the Impossibility Theorem in Practice
Bell, A., Bynum, L., Drushchak, N., Zakharchenko, T., Rosenblatt, L., & Stoyanovich, J. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594007 · Cited by 21
“1 (Impossibility Result Variation [15] [37]).”
BackgroundInfluentialDe-centering the (Traditional) user: Multistakeholder evaluation of recommender systems
Burke, R., Adomavičius, G., Bogers, T., Noia, T. D., Kowald, D., Neidhardt, J., Özgöbek, Ö., Pera, M. S., Tintarev, N., & Ziegler, J. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103560 · Cited by 25
The Dangers of Risk Prediction in the Criminal Justice System
Dressel, J., & Farid, H. (2021) · doi:10.21428/2c646de5.f5896f9f · Cited by 22
A Capability Approach to AI Ethics
Ratti, E., & Graves, M. (2025) · American Philosophical Quarterly · doi:10.5406/21521123.62.1.01 · Cited by 21
Fair Allocation through Selective Information Acquisition
Cai, W., Gaebler, J., Garg, N., & Goel, S. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375823 · Cited by 27
“In their setting, the decision maker must acquire additional features for each individual to ensure classification decisions have similar errors rates across groups— including parity in false negative and false positive rates—a common measure of fairness in the machine learning community [8, 16, 18].”
BackgroundFairness-Aware Range Queries for Selecting Unbiased Data
Shetiya, S., Swift, I. P., Asudeh, A., & Das, G. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00111 · Cited by 20
“Using biased data for algorithmic decisions create fairness dilemmas such as impossibility and inherent trade-offs of fairness [5], [6], [7].”
BackgroundOptimized Score Transformation for Fair ClassificationPreprint
Wei, D., Ramamurthy, K. N., & Calmon, F. P. (2019) · arXiv (Cornell University) · Cited by 22
Predicting the Replicability of Social Science Lab ExperimentsPreprint
Altmejd, A., Dreber, A., Forsell, E., Ho, T., Huber, J., Imai, T., Johannesson, M., Kirchler, M., Nave, G., & Camerer, C. F. (2019) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/zamry · Cited by 23
BeFair: Addressing Fairness in the Banking Sector
Castelnovo, A., Crupi, R., Gamba, G. D., Greco, G., Naseer, A., Regoli, D., & González, B. S. M. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9377894 · Cited by 21
“Moreover, some of the proposed definitions are mutually incompatible and exclusive under some conditions, while some of them are related in non-trivial ways [12], [13].”
BackgroundTo Split or not to Split: The Impact of Disparate Treatment in Classification
Wang, H., Hsu, H., Díaz, M., & Calmon, F. P. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3075415 · Cited by 19
Non-Discriminatory Machine Learning Through Convex Fairness Criteria
Goel, N., Yaghini, M., & Faltings, B. V. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11662 · Cited by 18
The Complex Economics of Artificial IntelligencePreprint
Mateos-Garcia, J. C. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3294552 · Cited by 22
A Distributionally Robust Approach to Fair ClassificationPreprint
Taşkesen, B., Nguyen, V. A., Kühn, D., & Blanchet, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09530 · Cited by 65
“As it is not possible to satisfy multiple notions of fairness simultaneously [4, 40], we focus on unfairness measures related to equalized opportunities.”
BackgroundEthical Quantum Computing: A RoadmapPreprint
Perrier, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.00759 · Cited by 52
“Work in this area involves considering how the networked distribution, say of individuals, affects welfare aggregation functions, including cluster analysis (Kleinberg, Mullainathan, and Raghavan 2016; Pierre Barthelemy and Monjardet 1981) along with graph-theoretic methods (Hammond 1988).”
BackgroundBeyond "Fairness:" Structural (In)justice Lenses on AI for EducationPreprint
Madaio, M., Blodgett, S. L., Mayfield, E., & Dixon‐Román, E. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08847 · Cited by 39
“Current approaches to assessing biases in such algorithms have developed mathematical formulations of group fairness [Kleinberg et al., 2016, Corbett-Davies and Goel, 2018] and individual fairness [Dwork et al., 2012], largely for algorithms that perform classification tasks (i.e., predicting one…”
BackgroundAdvancing accountability in AI
OECD (2023) · OECD digital economy papers · doi:10.1787/2448f04b-en · Cited by 42
Measuring Fairness in an Unfair World
Herington, J. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375854 · Cited by 22
“Examples of such measures include “balance” [19], and “predictive equality”[11].”
BackgroundDebiasing Vandalism Detection Models at Wikidata
Heindorf, S., Scholten, Y., Engels, G., & Potthast, M. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313507 · Cited by 17
Towards Fair Unsupervised Learning
Buet-Golfouse, F., & Utyagulov, I. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533197 · Cited by 20
“On the other hand, in supervised learning, multiple technical definitions of fairness co-exist and have been reviewed in-depth [4, 23, 32, 40], bringing to the fore impossibility theorems [10, 24] that proved that these different acceptations of fairness cannot be satisfied at once.”
BackgroundMethodInfluentialLearning Fair Representations via Distance Correlation Minimization
Guo, D., Wang, C., Wang, B., & Hongyuan Zha, B. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3187165 · Cited by 21
“A lot of work has been performed on mathematical definitions of fairness [21], [22].”
BackgroundDemographic Bias in Low-Resolution Deep Face Recognition in the Wild
Atzori, A., Fenu, G., & Marras, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3249485 · Cited by 17
“analyzing this beyond-accuracy perspective is emerging, but unfortunately focused on sensitive attributes separately [7] or on images acquired in high-resolution (HR) settings [8], [9], [10],”
BackgroundOn (assessing) the fairness of risk score models
Petersen, E., Ganz, M., Holm, S., & Feragen, A. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594045 · Cited by 25
“Separation would require the conditional risk distributions p(R ∣ G, Y ) to be identical across groups, which is impossible to achieve if R is calibrated by groups and there are di erences in the base rates p(Y = 1 ∣ G) [46].”
BackgroundThe Privacy-Bias Tradeoff: Data Minimization and Racial Disparity Assessments in U.S. Government
King, J., Ho, D. E., Gupta, A., Wu, V. Y., & Webley-Brown, H. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594015 · Cited by 17
“While early work in algorithmic fairness has grappled with the fairness-accuracy tradeoff [32, 52, 70, 73, 120, 148], one of the emerging tradeoffs is between fairness and individual privacy [25, 67].”
BackgroundA Novel Approach for Fair Principal Component Analysis Based on Eigendecomposition
Pelegrina, G. D., & Duarte, L. T. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3298291 · Cited by 18
“Atopic of interest when dealing with disparities is the tradeoff between model accuracy and fairness [17], [18], [19], [20], [21], [22].”
BackgroundThe Thoughts the Civilized Keep
Vallor, S. (2024) · Oxford University Press eBooks · doi:10.1093/oso/9780197759066.003.0005 · Cited by 16
Benchmark suites instead of leaderboards for evaluating AI fairness
Wang, A., Hertzmann, A., & Russakovsky, O. (2024) · Patterns · doi:10.1016/j.patter.2024.101080 · Cited by 19
“32,33 Even when considering one metric at a time, pursuing metrics can create serious problems.”
Algorithmic bias and racial inequality: a critical review
Kasy, M. (2024) · Oxford Review of Economic Policy · doi:10.1093/oxrep/grae031 · Cited by 22
Disparate Vulnerability to Membership Inference Attacks
Kulynych, B., Yaghini, M., Cherubin, G., Veale, M., & Troncoso, C. (2021) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2022-0023 · Cited by 73
Values Embedded in Legal Artificial IntelligencePreprint
Surden, H. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2932333 · Cited by 17
Fairness and bias in AI: a sociotechnical perspective
Mimouni, S. E., & Bouhdadi, M. (2025) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-12-2024-0182 · Cited by 18
Search results diversification for effective fair ranking in academic search
McDonald, G., Macdonald, C., & Ounis, I. (2021) · Information Retrieval · doi:10.1007/s10791-021-09399-z · Cited by 18
“Such classifiers might be deployed in decision-making tasks, such as loan or parole applications (Choul-dechova 2017; Hardt et al. 2016; Kleinberg et al. 2016; Woodworth et al. 2017; Zafar et al. 2017), where discrimination amongst individuals or sections of society can have serious implications…”
BackgroundSmith, L., & Lamprecht, C. (2024) · Journal of financial reporting & accounting · doi:10.1108/jfra-05-2023-0280 · Cited by 18
Fairness and Bias in Robot Learning
Londoño, L., Hurtado, J. V., Hertz, N., Kellmeyer, P., Vöeneky, S., & Valada, A. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3403898 · Cited by 43
“Besides the social-related difficulty, it has been shown that it is not feasible, from a technical point of view, to satisfy all fairness definitions simultaneously [100].”
BackgroundA Framework for Assurance Audits of Algorithmic Systems
Lange, B., Lam, K., Blili-Hamelin, B., Davidovic, J., Brown, S., & Hasan, A. (2024) · PhilPapers (PhilPapers Foundation) · Cited by 17
Improving medical machine learning models with generative balancing for equity and excellence
Theodorou, B., Danek, B. P., Tummala, V., Kumar, S. D. M., Malin, B., & Sun, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01438-z · Cited by 18
A Framework for Benchmarking Discrimination-Aware Models in Machine Learning
Cardoso, R. L., Meira, W., Almeida, V. A. F., & Zaki, M. J. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314262 · Cited by 16
“The recent study by [11] discusses the impossibility of a classifier to satisfy multiple notions of fairness.”
BackgroundEpistemic Therapy for Bias in Automated Decision-Making
Gilbert, T. K., & Mintz, Y. D. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314294 · Cited by 16
“There have been some technical critiques leveled against these notions of optimal fairness [11, 25], as well as several impossibility results that show it is difficult to satisfy all of these notions of fairness simultaneously [14, 21].”
BackgroundTuning Fairness by Balancing Target Labels
Kehrenberg, T., Chen, Z., & Quadrianto, N. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00033 · Cited by 20
Mitigating Bias in AI: A Framework for Ethical and Fair Machine Learning ModelsPreprint
Kamatala, S., Naayini, P., & Myakala, P. K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5138366 · Cited by 16
“• Algorithmic Bias: Arises from algorithmic design choices that unintentionally favor certain groups over others [3].”
InfluentialGaebler, J. D., Goel, S., Huq, A., & Tambe, P. (2024) · Behavioral Science & Policy · doi:10.1177/23794607251320229 · Cited by 15
A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness
Yaghini, M., Krause, A., & Heidari, H. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462583 · Cited by 22
“, [10, 13, 30]), existing formulations fail to take the decision-making context into account and consider the social backdrop and ramifications”
BackgroundInfluentialFirst- and Second-Level Bias in Automated Decision-making
Franke, U. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00500-y · Cited by 16
“Page 11 of 20 21 First and Second Level Bias in Automated Decision making - - - in trivial settings (Chouldechova, 2017; Kleinberg et al., 2017).”
BackgroundLeuthe, D., Meyer-Hollatz, T., Plank, T., & Senkmüller, A. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10526-6 · Cited by 25
“As highlighted in the introduction to this subsection, a holistic definition of fairness that covers each element is not possible (Kleinberg et al., 2017).”
FairXAI - A Taxonomy and Framework for Fairness and Explainability Synergy in Machine Learning
Ramachandranpillai, R., Baeza-Yates, R. A., & Heintz, F. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3528321 · Cited by 18
“According to [22], it is impossible to satisfy all fairness constraints at once.”
Paradoxes in Fair Machine Learning
Gölz, P., Kahng, A., & Procaccia, A. D. (2019) · neural information processing systems · Cited by 13
Two Kinds of Discrimination in AI-Based Penal Decision-Making
Hübner, D. (2021) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3468507.3468510 · Cited by 19
“There are several “impossibility theorems” demonstrating this unfavorable constellation [16, 25].”
BackgroundAlgorithmic Design: Fairness Versus Accuracy
Liang, A., Lu, J., & Mu, X. (2021) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3490486.3538237 · Cited by 19
“Kleinberg et al. (2017) and Chouldechova (2017) demonstrated that certain notions of fairness (equal false positive rates, equal false negative rates, calibration) cannot be simultaneously satisfied.”
BackgroundLearning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness ConstraintsPreprint
Vogel, R., Bellet, A., & Clémençon, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08159 · Cited by 15
FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., & Chau, D. H. (2019) · 2019 IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast47406.2019.8986948 · Cited by 14
“Two papers [14, 24] simultaneously proved that if groups have different base rates in their labels, it is statistically impossible to ensure fairness across three base fairness metrics — balance for the positive class, balance for the negative class, and calibration of the model.”
BackgroundDisparities in seizure outcomes revealed by large language models
(2024) · doi:10.1093/jamia/ocae047 · Cited by 17
Explaining Bias in Deep Face Recognition via Image Characteristics
Atzori, A., Fenu, G., & Marras, M. (2022) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb54206.2022.10007937 · Cited by 14
“Focusing on groups characterized by a sensitive attribute does not necessarily provide guarantees and generalization to groups based on another sensitive attribute [14].”
BackgroundHerzog, C., Blank, S., & Stahl, B. C. (2024) · AI & Society · doi:10.1007/s00146-024-02082-z · Cited by 18
“For instance, the very goal of devising a bias-free AI solution may be unviable, as different notions of fairness may not be simultaneously enforceable incurring inherent trade-offs (Kleinberg et al. 2017).”
A LEGAL FRAMEWORK FOR ARTIFICIAL INTELLIGENCE FAIRNESS REPORTING
Yap, J. Q., & Lim, E. W. K. (2022) · The Cambridge Law Journal · doi:10.1017/s0008197322000460 · Cited by 20
Almost politically acceptable criminal justice risk assessment
Berk, R. A., & Elzarka, A. A. (2020) · Criminology & Public Policy · doi:10.1111/1745-9133.12500 · Cited by 16
Auditing Black-Box Models Using Transparent Model Distillation With Side InformationPreprint
Tan, S., Caruana, R., Hooker, G., & Lou, Y. (2017) · arXiv (Cornell University) · Cited by 14
Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized OddsPreprint
Mishler, A., Kennedy, E. H., & Chouldechova, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.02841 · Cited by 14
“risk of neglect or abuse would be if they had been of a different race”
BackgroundPublic perception of accuracy-fairness trade-offs in algorithmic decisions in the United States
Mourali, M., Novakowski, D., Pogacar, R., & Brigden, N. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319861 · Cited by 16
Learning from Outcomes: Evidence-Based Rankings
Dwork, C., Kim, M. P., Reingold, O., Rothblum, G. N., & Yona, G. (2019) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2019.00016 · Cited by 15
“To this end, we turn to the concept of calibration studied in scoring and prediction, and recently in the context of fair prediction [14], [37].”
MethodGittens, A., Yener, B., & Yung, M. M. (2022) · IEEE Access · doi:10.1109/access.2022.3218715 · Cited by 22
“Unfortunately, a series of works have established that, in general, multiple forms of group fairness cannot simultaneously hold if the classifier is nontrivial [45], [46], [47].”
BackgroundAlgorithm-Based Sentencing and Discrimination
Lippert‐Rasmussen, K. (2022) · doi:10.1093/oso/9780197539538.003.0005 · Cited by 13
PreFair: Privately Generating Justifiably Fair Synthetic Data
Pujol, D., Gilad, A., & Machanavajjhala, A. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3583140.3583168 · Cited by 18
Monitoring Algorithmic Fairness
Henzinger, T. A., Karimi, M., Kueffner, K., & Mallik, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37703-7_17 · Cited by 22
Bird, K. (2023) · The AIR professional file · doi:10.34315/apf1612023 · Cited by 13
The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Goyal, N., Baumler, C., Nguyen, T. T., & Daumé, H. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645210 · Cited by 20
“There are other measures of fairness [46], however, not all definitions can be simultaneously satisfied [16, 36].”
BackgroundModel Projection: Theory and Applications to Fair Machine Learning
Alghamdi, W., Asoodeh, S., Wang, H., Calmon, F. P., Wei, D., & Ramamurthy, K. N. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9173988 · Cited by 15
Sarram, G., & Ivey, S. S. (2022) · Journal of Urban Management · doi:10.1016/j.jum.2022.01.001 · Cited by 13
Liang, A., & Madsen, E. (2020) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3391403.3399474 · Cited by 13
Sequential Fair Allocation: Achieving the Optimal Envy-Efficiency Trade-off Curve
Sinclair, S. R., Jain, G., Banerjee, S., & Yu, C. L. (2022) · Operations Research · doi:10.1287/opre.2022.2397 · Cited by 13
Why Fair Labels Can Yield Unfair Predictions: Graphical Conditions for Introduced Unfairness
Ashurst, C., Carey, R., Chiappa, S., & Everitt, T. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21182 · Cited by 13
“3) Group fairness definitions may overlook (un)fairness to individuals (Dwork et al. 2012; Kleinberg, Mullainathan, and Raghavan 2017).”
BackgroundAlgorithmic Fairness in Mortgage Lending: From Absolute Conditions to Relational Trade-OffsPreprint
Lee, M. S. A., & Floridi, L. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3559407 · Cited by 13
Fairness of Exposure in Stochastic BanditsPreprint
Wang, L., Bai, Y., Sun, W., & Joachims, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02735 · Cited by 21
“Algorithmic fairness has been extensively studied in binary classification (Hardt et al., 2016; Chouldechova, 2017; Kleinberg et al., 2017; Agarwal et al., 2018).”
BackgroundApproximation Algorithms for Socially Fair ClusteringPreprint
Makarychev, Y., & Vakilian, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02512 · Cited by 16
“Due to increasing use of machine learning in decision making, there has been an extensive line of research on the societal aspects of algorithms (Galindo and Tamayo, 2000; Kleinberg et al., 2017; Chouldechova, 2017; Dressel and Farid, 2018).”
Metadata
- Authors
- Kleinberg, J., Mullainathan, S., & Raghavan, M.
- Affiliations
- Cornell University (US)
- Harvard University (US)
- University of California, Berkeley (US)
- Venue
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Publisher
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W2962922665
- Cited by
- 943
- References
- 35
- Citations by year
- 2026: 49 · 2025: 74 · 2024: 112 · 2023: 139 · 2022: 102 · 2021: 153 · 2020: 129 · 2019: 107 · 2018: 54 · 2017: 21
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Leibniz international proceedings in informatics (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, risk score, probabilistic classification, fairness conditions, impossibility results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
