Inherent Trade-Offs in the Fair Determination of Risk Scores

“propose a framework based on a task-specific externally defined similarity metric between individuals, seeking to achieve fairness through the goal that “similar people [be] treated similarly” [11].”

“Here too we can ask whether such decision-making is being applied uniformly across different groups of patients [16, 27], and in particular how medical tests may play a differential role for conditions that vary widely in frequency between these groups.”

“Barocas and Selbst survey a range of ways in which data-analysis algorithms can lead to discriminatory outcomes [3], and review articles by Romei and Ruggieri [25] and Zliobaite [30] survey data-analytic and algorithmic methods for measuring discrimination.”

“consider the natural analogues of our conditions (B) and (C), balance for the negative and positive classes, in the case of classifiers that output binary “yes/no” predictions rather than real-valued scores as in our case [19].”

“Also concurrent with our work and that of Hardt et al., Chouldechova [5] and Corbett-Davies et al. [6] (and see also [7]) consider binary prediction subject to these same analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction (requiri…”

“This latter global requirement is a version of statistical parity [12, 4, 21, 22].”

“[7] (and see also [6]) consider binary prediction subject to analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction.”

“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”

“Work in these directions has developed learning algorithms that penalize violations of statistical parity [4, 22].”

“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”

“For example, the sufficiency and separation principles can be incompatible with each other (Kleinberg et al., 2016; Friedler et al., 2021), and classification parity / calibration can harm the very groups that the algorithms are designed to protect (Corbett-Davies and Goel, 2018).”

“Buolam-wini and Gebru, [9] demonstrated that subgroup harms can be substantially larger at the intersection of multiple axes than at any single axis, but the methodological apparatus for measuring and remediating intersectional bias remains under-developed.”

“Prior works have analyzed the tradeoffs between multiple fairness definitions [Kleinberg et al. , 2017; Friedler et al. , 2021; Her-tweck and R¨az] and with performance and fairness measures [Brandao, 2021; Chen and Hooker, 2020; Liu et al. , 2022; Radovanovic et al. , 2020; Andrews et al. , 2022;…”

“…casts training fair models as a constrained optimization problem: the goal is to maximize accuracy , usually operationalized as predictive error, subject to a fairness constraint, usually operationalized as differences in model error rates between groups [9, 18, 21, 32, 40, 54–56, 65, 82, 83].”

“Leininger et al. (2025) show that several common fairness notions – on the group and on the individual level – are simultaneously fulfilled in the FiND world, overcoming the fairness-accuracy trade-off and the impossibility theorem (Choulde-chova, 2017; Kleinberg et al., 2017).”

“11 Chouldechova (2017); Kleinberg et al. (2017).”

“Algorithmic fairness has received extensive theoretical treatment for unconstrained classifiers[4][5], with well-characterized trade-offs among fairness criteria [6][7] and practical frameworks[8][9].”

“Surveys by Verma and Rubin [ 12], Mehrabi et al. [26 ], and Caton and Haas [27 ] document this landscape, while impossibility theorems demonstrate that these criteria cannot be jointly satisfied [ 8, 9 ].”

“When group base rates differ, calibration and balanced error rates cannot hold simultaneously [3].”

“As discussed in the literature, they cannot be satisfied simultaneously, when the base rates differ or when the models are well calibrated [ 7, 15, 20].”

“Where base rates differ, calibration within groups and equal error rates can - not in general hold together [ 86 , 87 ], so no single fairness metric is prescribed; the evidence for Q3 and Q5 must name the metric and justify it by the clinical harm of each type of error, for instance parity of…”

“There has been much recent interest in designing algorithms that make fair predictions [4, 6, 9, 10, 12, 14–17, 19, 21, 30–33].”

“This is perhaps consistent with the emerging theoretical literature demonstrating the lack of a single “right” definition of fairness [7, 12, 22], and our work adds evidence to the view that fairness is a topic demanding careful domain-specific considerations.”

“tions about what the purpose of these tools is and should be in the first place [26, 29, 48].”

“Recent work on fairness in machine learning proposes and analyzes competing criteria for assessing the fairness of machine learning algorithms, where some adjustments attempt to equalize accuracy metrics across groups (Corbett-Davies et al., 2017; Kleinberg et al., 2017; Hardt et al., 2016).”

“s of direct and indirect discrimination and provide mechanisms [17, 21, 32, 40, 47, 60–62] for non-discriminatory learning, as well as examining the feasibility of making non-discriminatory decisions [12, 14, 24, 35]. Existing studies of algorithmic fairness are largely normative (prescriptive) in nature, i.e., they b…”

“…across groups, as is observed to be the case in ProPublica’s Broward County data, a tool cannot simultaneously satisfy both predictive parity (calibration) and error rate balance (resp. balance for the positive and negative class) (Kleinberg et al., 2016; Chouldechova, 2017; Berk et al., 2017).”