Fair Regression with Wasserstein Barycenters

“The next result establishes that as long as one of the measures in the definition of the Wasserstein-2 distance admits a density, then the optimal coupling in the infimum in Definition 2.1 is deterministic (see e.g., [41, Theorem 2.18] or [38, Theorems 2.5 and 2.9]).”

“This step is also known as jittering , often used for data visualization [11] for tie-breaking.”

“…field to designing methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those complying with the…”

“The next result is standard and can be found for instance in [2, Section 6.1] or [38, Section 5.5.5].”

“(8) is built upon the well-known Dvoretzky–Kiefer–Wolfowitz inequality [33, Corollary 1].”

“The next result is standard and can be found for instance in [2, Section 6.1] or [38, Section 5.5.5].”

“…methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those complying with the Demographic…”

“We refer to [3, 15, 29, 30, 39] for various examples of estimators and additional assumptions such that the bound in Assumption 4.3 is satisfied.”

“Therefore, a substantial effort is being devoted in the field to designing methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47, 49] and references therein.”

“…in the field to designing methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those…”

“We refer to [3, 15, 29, 30, 39] for various examples of estimators and additional assumptions such that the bound in Assumption 4.”

“We refer to [3, 15, 29, 30, 39] for various examples of estimators and additional assumptions such that the bound in Assumption 4.3 is satisfied.”

“We employ the 2-steps 10-fold CV procedure considered by [16] to select the best hyperparameters with the training set.”

“Similar proof techniques are also used in randomization inference via permutations [18, 21], conformal prediction [30, 42], knockoff estimation [4] to name a few.”

“…methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those complying with the Demographic…”

“Demographic Parity requires the probability distribution of the predicted output to be independent of the sensitive attribute and has been used extensively in the literature, both in the context of classification and regression [1, 12, 19, 23, 34].”

“Similar proof techniques are also used in randomization inference via permutations [18, 21], conformal prediction [30, 42], knockoff estimation [4] to name a few.”

“Our distribution-free fairness guarantees share similarities with contributions on prediction sets [30, 31] and conformal prediction literature [42, 48] as they also rely on results on rank statistics.”

“In this work we use the following definition of (strong) Demographic Parity, which was previously used in the context of regression by [1, 12, 24].”

“Demographic Parity requires the probability distribution of the predicted output to be independent of the sensitive attribute and has been used extensively in the literature, both in the context of classification and regression [1, 12, 19, 23, 34].”

“Similar proof techniques are also used in randomization inference via permutations [18, 21], conformal prediction [30, 42], knockoff estimation [4] to name a few.”

“…and test it against the state-of-the-art alternatives, we consider five benchmark datasets, CRIME, LAW, NLSY, STUD, and UNIV, which are briefly described below: Communities&Crime (CRIME) contains socio-economic, law enforcement, and crime data about communities in the US [37] with 1994 examples.”

“…designing methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those complying with the…”

“In the case of linear models we consider the following methods: Linear RLS plus [6] (RLS+Berk), Linear RLS plus [34] (RLS+Oneto), and Linear RLS plus Our Method (RLS+Ours), where RLS is the abbreviation of Regularized Least Squares.”

“Therefore, a substantial effort is being devoted in the field to designing methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47, 49] and references therein.”

“Following [9], we made a binary sensitive attribute s as to the percentage of black population, which yielded 970 instances of s =1 with a mean crime rate 0 .”

“We refer to [3, 15, 29, 30, 39] for various examples of estimators and additional assumptions such that the bound in Assumption 4.3 is satisfied.”

“We refer to [3, 15, 29, 30, 39] for various examples of estimators and additional assumptions such that the bound in Assumption 4.”

“…methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those complying with the Demographic Parity…”

“In the case of non-linear models we compare to the following methods. i) For Kernel RLS (KRLS): KRLS plus [34] (KRLS+Oneto), KRLS plus [35] (KRLS+Perez), KRLS plus Our Method (KRLS+Ours); ii) For Random Forests (RF): RF plus [36] (RF+Raff), RF plus [1] 5 (RF+Agar), and RF plus Our Method (RF+Ours).”

“Connections between algorithmic fairness and Optimal Transport, and in particular the problem of Wasserstein barycenters, has been studied in [12, 19, 23, 43] but mainly in the context of classification.”

“In the case of non-linear models we compare to the following methods. i) For Kernel RLS (KRLS): KRLS plus [34] (KRLS+Oneto), KRLS plus [35] (KRLS+Perez), KRLS plus Our Method (KRLS+Ours); ii) For Random Forests (RF): RF plus [36] (RF+Raff), RF plus [1] 5 (RF+Agar), and RF plus Our Method (RF+Ours).”

“Demographic Parity requires the probability distribution of the predicted output to be independent of the sensitive attribute and has been used extensively in the literature, both in the context of classification and regression [1, 12, 19, 23, 34].”

“…to designing methods that satisfy “fairness” requirements, while still optimizing prediction performance, see for example [5, 10, 13, 16, 17, 20, 22, 24, 25, 27, 32, 45–47 In this paper we study the problem of learning a real-valued regression function which among those complying with the…”

“Analogously to [26] we model a virtual company’s hiring decision assuming that the company does not have access to the applicants’ academic scores.”

“…2019 JSAI Kim et al. [180] 2020 ICML Jiang et al. [181] 2020 UAI Savani et al. [355] 2020 NeurIPS Chzhen et al. [356] 2020 NeurIPS Chzhen et al. [266] 2020 NeurIPS Awasthi et al. [357] 2020 PMLR Chzhen and Schreuder [271] 2020 arxiv Schreuder and Chzhen [358] 2021 UAI Kanamori and Arimura [359]…”

“More recently, statistical and learning guarantees for fair regression were derived (Agarwal et al., 2019, Chiappa et al., 2020, Chzhen et al., 2020a,b, Fitzsimons et al., 2019, Le Gouic et al., 2020, Plečko and Meinshausen, 2019).”

“DP can also be applied to the regression model rather than to the classification model [37]: • General Demographic Parity (GDP) [75] extends the demographic parity on the continuous sensitive attribute: where E [ˆ y | s ] is the local average prediction of the model conditioned on the sensitive…”

“DP can also be applied to the regression model rather than to the classification model [37] sup | 2) General demographic parity (GDP) [75] extends the DP on the continuous sensitive attribute where E [ˆ y | s ] is the local average prediction of the model conditioned on the sensitive attribute and E…”

“See [16, 17] on a similar, more formal critique on individual fairness [20] notions.”

“Prior research conducted by (Chzhen et al. 2020b; Gouic, Loubes, and Rigollet 2020) demonstrates that, Therefore, f B represents the optimal fair predictor in terms of minimizing unfairness-risk.”

“To this end various algorithms have been developed to find fair machine models (FML) that satisfy different fairness measures [4], [5], [6], [7].”

“[10] focused on demographic parity that requires the distribution of the predicted output independent of the sensitive attribute.”

“WB has been applied in diverse areas: image morphing [6], atmospheric data averaging [7], graph learning [8], fairness [9], clustering [10], Bayesian learning [11], stain normalization [12], forecast aggregation [13], multi-media fusion [14], multilingual alignment [15], COVID-19 pattern clustering…”

“…of demographic parity as well as other criteria (Black, S., and Fredrikson 2019; Del Barrio et al. 2019; Johndrow and Lum 2019; Risser et al. 2019; Wang, Ustun, and Calmon 2019; Chzhen et al. 2020; Del Barrio, Gordaliza, and Loubes 2020; Le Gouic and Loubes 2020; Yurochkin, Bower, and Sun 2020).”

“This approach has been followed in articles such as [39], [40], [41], [42], and [43].”

“Research in that field has led to several approaches, such as fair representation learning [4, 50, 63, 67], constrained optimization [1, 42], adversarial training [13], and post-processing methods [10, 17].”

“Chzhen et al. (2020) considers the following situation.”

“…demographic parity (Dwork et al., 2012) and equalized odds (Hardt et al., 2016)) in the settings of classification (Hardt et al., 2016; Zafar et al., 2017; Zhao and Gordon, 2019; Jiang et al., 2020) and regression (Komiyama et al., 2018; Agarwal et al., 2019; Chzhen et al., 2020; Chi et al., 2021).”

“Under-Performance of Adversarial Debiasing under Covariate Shift : We study the variation of the performance of a State of The Art Method, namely Adversarial Debias-ing (Zhang, Lemoine, and Mitchell 2018) against the magnitude of shift (cid:13) on the Adult dataset.”

“For a formal discussion on effort and its role on individual fairness (Dwork et al., 2012), see Chzhen and Schreuder (2022) and Chzhen et al. (2020).”

“It follows from [12, 18] that the optimal fair regression outcome can be characterized by the Wasserstein-2 barycenter.”

“To this end various algorithms have been developed to find fair machine models (FML) that satisfy different fairness measures [2, 3, 4, 5].”

“Finally, the Wasserstein distance has been used to express fairness constraints in several in-processing methods (Chzhen et al. 2020; Chzhen and Schreuder 2022).”

“Email addresses: ferran.arque@upc.edu (Ferran Arqué), cauribe@rice.edu (César A. Uribe), carlos.ocampo@upc.edu (Carlos Ocampo-Martinez). ization (Cuturi et al. 2020) or fair regression (Chzhen et al. 2020).”

“For example, [36] proposes a fair classifier that maximizes fairness with accuracy constraints and [6] develops a fair regression model evaluated with Demographic Parity and formulas the objective as solving for the Wasserstein barycenter.”

“Another set of approaches [26, 43] involves applying projections based on the Wasserstein distance in order to explicitly solve an empirical risk minimization problem with a fairness constraint.”

“(i) Pooled: bias(s) = E[YS ]− E[Ys] + β>Pool(µs − µ̄); (ii) PTF in (18): bias(s) = E[YS ]− E[Ys]; (iii) Conditional expectation: bias(s) = E[ŶC(Xs)]− E[Ys]; (iv) FEO: bias(s) = E[YS ]− E[Ys] + β>F (µs − µ̄); (v) SEO: bias(s) = E[YS ]− E[Ys].”

“…is encapsulated in the more strict definition of threshold-invariant fairness defined by Chen et al. [8] (also coined strong demographic parity by Jiang et al. [29]), and has been employed in a variety of works to enforce inter-group-fair classification [8,26,29,38,39] and regression [38,40].”

“An optimal transport (OT)-based repair provides a compelling setting for this problem, and there are many works which consider OT for data repair [4], [20], [21].”

“Chzhen et al. (2020); Le Gouic et al. (2020) showed that under mild additional assumptions1 on the distribution P, the optimal fair prediction g∗ can be obtained for all (x, s) ∈ Rp × {1, 2} as g∗(x, s) = ( p1G −1 1 + p2G −1 2 ) ◦Gs ( E[Y”

“In many decision-making scenarios, such as hiring, recommendation systems, and criminal justice, concerns have been raised regarding the fairness of decisions from the learning process (Chzhen et al., 2020).”

“Agarwal (2019), Fitzsimons (2019) and Chzhen et al. (2020) studied 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Sydney, Australia. demographic parity.”

“Recently, Chzhen et al. (2020a) and Le Gouic et al. (2020) concurrently with each other derived an analytic bound to characterize the price of statistical parity in regression using Wasserstein barycentres when the learner can take the sensitive attribute explicitly as an input.”

“For example, Zafar et al. [42] proposes a fair classifier that maximizes fairness with accuracy constraints and Chzhen et al. [43] develops a fair regression model evaluated with Demographic Parity and formulas the objective as solving for the Wasserstein barycenter.”

“(2020), Chzhen et al .”

“…identify the most informative features in free movements and social interactions associated with MoCA and Mini-BESTest scores, as well as the demographic biases (sex and race) of the model and bias mitigation analysis for developing a demographically fair model within a fair ML framework [28,51].”