Inherent Trade-Offs in the Fair Determination of Risk Scores

“propose a framework based on a task-specific externally defined similarity metric between individuals, seeking to achieve fairness through the goal that “similar people [be] treated similarly” [11].”

“Here too we can ask whether such decision-making is being applied uniformly across different groups of patients [16, 27], and in particular how medical tests may play a differential role for conditions that vary widely in frequency between these groups.”

“Barocas and Selbst survey a range of ways in which data-analysis algorithms can lead to discriminatory outcomes [3], and review articles by Romei and Ruggieri [25] and Zliobaite [30] survey data-analytic and algorithmic methods for measuring discrimination.”

“consider the natural analogues of our conditions (B) and (C), balance for the negative and positive classes, in the case of classifiers that output binary “yes/no” predictions rather than real-valued scores as in our case [19].”

“Also concurrent with our work and that of Hardt et al., Chouldechova [5] and Corbett-Davies et al. [6] (and see also [7]) consider binary prediction subject to these same analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction (requiri…”

“This latter global requirement is a version of statistical parity [12, 4, 21, 22].”

“[7] (and see also [6]) consider binary prediction subject to analogues of the balance conditions for the negative and positive classes, together with a form of calibration adapted to binary prediction.”

“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”

“Work in these directions has developed learning algorithms that penalize violations of statistical parity [4, 22].”

“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”

“…the performance-fairness trade-off —the inevitable situation in which the lesser the fairness of an algorithm, the greater its predictive capabilities and vice-versa (Kleinberg et al., 2016)—should be tunable to satisfy the ethical, legal, and societal needs of the end user (i.e., domain expert).”

“…in very specific settings, which have produced divergent findings that additionally vary with competing definitions of algorithmic fairness (Kleinberg, Mullainathan, and Raghavan 2016; Angwin et al. 2016; Hamilton 2019; Hellman 2020; Wang et al. 2023; Kwegyir-Aggrey et al. 2023; Imai et…”

“This result is analogous to the results for Calibration obtained by Kleinberg et al. (2017).”

“Recently, algorithmic fairness has been receiving significant attention from a wide spectrum of research communities like social science, computer science and statistics [3, 4, 6, 8, 11, 15, 18, 22, 23, 27, 30, 32].”

“As a result there has been a significant interest in recent years in the area of algorithmic fairness [7, 10, 25].”

“Other commonly used definitions include predictive parity [13, 38], predictive equality [13, 16], equal opportunity [28] equalized odds [13, 28, 38, 62] and conditional use accuracy equality [6].”

“Larson et al. (2015) show that a test preparation provider charged Asian-Americans higher prices, even when controlling for income.”

“In particular, we consider four different statistical notions of fairness, namely statistical parity [22, 13, 33, 23, 51], predictive equality [15, 16], equal opportunity [30] and equalized odds [15, 34, 30, 50].”

“While the initial discussion has focused on the algorithmic assistance of judges in sentencing, pre-trial, or bail decisions (Kleinberg et al., 2017), similar algorithmic decision aids are rapidly expanding to other areas of public decision-making (Huq, 2020b).”

“What sparked a substantive debate, however, were formal results by Choul-dechova (2017) and Kleinberg, Mullainathan, and Raghavan (2016), which showcase how different sensible fairness measurements cannot be jointly satisfied under fairly common conditions (when the decision-problem has no perfect…”

“As some common definitions are fairness are mathemati - cally incompatible, except in some highly constrained cases [136], the ethical determination of which fairness measure to use depends upon external criteria, of which the harm and suffering caused are highly relevant.”

“[6] The inherent tradeoffs between accuracy and fairness in risk scoring systems are explored in this research, along with the difficulties in accomplishing both at once.[7] The authors look at how particular geographic areas are underrepresented in open datasets that are used to train”

“Positive balance [13] is similar to equal opportunity, but instead of taking the difference of probability of binary prediction ˆ Y , it takes the difference of the expected value of the score ˆ V , which may be non-binary, such as the score of COMPAS.”

“Amongst researchers, it has led to sophisticated debates about such issues as algorithmic fairness [2, 12, 20, 34, 35, 49, 50], value alignment [3, 10, 27], and how new technologies shape the perception of fundamental human values [15, 16, 41, 64].”

“This realization has recently spawned a new active area of research into quantifying and guaranteeing fairness for machine learning [4, 12, 17].”

“In particular, these notions can model discrimination either against groups [49] or individuals [35] and it is well-established that trade-offs exist between the two types [11,37].”

“From Natural Language Processing (NLP) community, an increasing number of papers have been introduced, examining five different ethical categories: (i) Fairness [27], (ii) Safety [37], (iii) Prosocial [36, 38], (iv) Utility [10, 29], and (v) Com-monsense Morality [18].”

“We successfully accomplished our purpose mainly because sometimes there is a trade-off in sacrificing the accuracy of the major group to improve the accuracy of the minor group [34].”

“) Members of the data management community who are interested in this topic may consider a growing body of work on impossibility results, which show that different notions of fairness cannot be enforced simultaneously, and so require explicit trade-offs (Chouldechova 2017; Friedler et al. 2016; Kleinberg et al. 2017).”

“This situation is not unlike that in learning, where fairness has emerged as an important property in classification tasks (Zafar et al. 2017; Kleinberg, Mullainathan, and Raghavan 2017), with multiple notions of fairness emerging in this context.”

“Fair calibration demands this property holds across groups: [54], [55].”

“We should caution that the model approval decisions should not be made solely because one or two fairness metrics are out of range (e.g., Meng 2020) due to the fact that it is impossible to simultaneously meet all fairness requirements (e.g., Kleinberg et al. 2017 and Barocas et al. 2023, Chap.”

“[146-148].”

“However, these metrics are often mutually exclusive [10, 15, 32].”

“In fact, they might appear innocuous at first glance [26].”

“…of findings that elucidate risk assessment quandaries forensic scientists have been grappling with for some time (for e.g. that fairness and accuracy are incompatible when demographic groups have varying offending base rates, Chouldechova, 2016; Corbett-Davies et al., 2017; Kleinberg et al., 2016).”

“Unfortunately, these criteria cannot all be achieved in a single model without either having a perfect or trivial classifier [3], [12].”

“A predictive algorithm is relatively calibrated (Chouldechova, 2017; Corbett-Davies and Goel, 2018) if If the risk score further satisfies P ( Y = 1 | S,G ) = S , we say that it is absolutely calibrated (Kleinberg et al., 2017).”

“A truly representative pattern implies statistical parity with regard to the control variable, although it is worth noting that both Dwork and Mullainathan explicitly mention that statistical parity should not be equated with fairness, as it can potentially be “blatantly unfair” on an individual level [16], [26].”

“[21], Figure 2c also suggests that one cannot have both individual and group fairness in a scenario with high mean error.”

“Kleinberg et al. (2016) showed that 1 Most of the basic fairness objectives can also be analogously extended to the case of numeric prediction.”

“Despite some impossibility results in certain settings [20, 31], it has been used successfully combined for classification [28, 40, 48], ranking [15, 16], regression [3], graph embeddings [14] and indeed clustering.”

“Faced with the impossibility results (Kleinberg, Mullainathan, and Raghavan 2016; Chouldechova 2017) for different statistical measures of fairness, we can make a principled choice, on the basis of well-articulated moral reasons, to drop the pursuit of a redundant fairness notion or to trade off…”

“mon fairness metrics and it is often impossible to optimize them simultaneously [22, 40].”

“” One such fairness criterion is that of group calibration, also known as sufficiency [34, 35, 36]: Definition 5.”

“Different applications may prescribe different notions of fairness, and these notions may even conflict with each other (Kleinberg, Mullainathan, and Raghavan 2017).”

“In summary, algorithms are not objective and fair, but rather inherently biased [38].”

“Unfortunately, several impossibility theorems have recently demonstrated that no single AI system can jointly satisfy all of these criteria (Chouldechova, 2017; Corbett-Davies et al., 2017; Kleinberg et al., 2016; Miconi, 2017).”

“Suf6iciency demands that the actual outcome (whether the defendant does in fact reoffend) should be independent of the protected attribute (the ethnicity of the offender) given the outcome predicted by the system (the likelihood of reoffending).”

“An equal base rate is defined as having an equal number of objects for different subgroups in the data set [23].”

“Much of this work has focused on the immediate predictive disparities that arise when supervised learning techniques are applied to batches of training data sampled from a fixed un- derlying population [Dwork et al., 2012; Zafar et al., 2017; Hardt et al., 2016; Kleinberg et al., 2017].”

“Indeed, it was shown that an algorithm cannot generally satisfy these two metrics at the same time (Kleinberg et al., 2016; Loftus et al., 2018).”

“Additionally, different definitions of group fairness are innately incompatible with each other [10], [11], [12].”

“Following the definitions of T , there is not an 1We note that this has been disputed [57, 29], and that it has been suggested that such forecasts [71] are difficult to make, in general, due to the cohort differences in group-based arrest trajectories.”

“So do (5) positive and (6) negative class balance (Kleinberg et al., 2017), two notions applicable when a binary classification is accomplished through some intermediate scoring mechanism.”

“However, in many real-world contexts, there is no agreed-upon definition of fairness when training AI (Kleinberg, Mullainathan, and Raghavan 2016; Barocas, Hardt, and Narayanan 2019).”

“Model fairness can be defined in many ways and the method of evaluation can reveal different patterns of disparities [13].”

“This notion is called balance for the positive (or negative) class in Kleinberg et al. (2017).”

“(b) Even if we could somehow “operationalize” the fairness program, these measures are usually incompatible: it is provably impossible to design an algorithm that obeys all notions of fairness simultaneously [15, 16].”

“iversity (3 weeks). A taxonomy of fairness definitions; individual and group fairness [34, 37, 75]. The importance of a socio-technical perspective: stakeholders and trade-offs. Impossibility results [20, 59]; causal definitions [90]; fairness beyond classification and risk assessment [112]. Diversity Teaching Respons…”

“However, given the higher proportion of pain instances in the female test data compared to male pain images, these metrics (8) (9) do not provide conclusive evidence of algorithmic bias in the models [48].”

“Importantly, different fairness criteria cannot be simultaneously satisfied—they are “mathemati-cally incompatible” (p. 5) (see Chouldechova, 2017; Kleinberg et al., 2017, for more details).”

“[6].”