Marrying Fairness and Explainability in Supervised Learning
Open accessAbstract
Machine learning algorithms that aid human decision-making may inadvertently discriminate against certain protected groups. Therefore, we formalize direct discrimination as a direct causal effect of the protected attributes on the decisions, while induced discrimination as a change in the causal influence of non-protected features associated with the protected attributes. The measurements of marginal direct effect (MDE) and SHapley Additive exPlanations (SHAP) reveal that state-of-the-art fair learning methods can induce discrimination via association or reverse discrimination in synthetic and real-world datasets. To inhibit discrimination in algorithmic systems, we propose to nullify the influence of the protected attribute on the output of the system, while preserving the influence of remaining features. We introduce and study post-processing methods achieving such objectives, finding that they yield relatively high model accuracy, prevent direct discrimination, and diminishes various disparity measures, e.g., demographic disparity.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper formalizes direct discrimination as a direct causal effect of protected attributes on decisions and induced discrimination as a change in the causal influence of non-protected features associated with protected attributes. It proposes marginal interventional mixture (MIM) and optimization-based post-processing methods that remove protected-attribute influence while preserving non-protected feature influence, reporting relatively high accuracy and reduced disparity measures on synthetic and real-world datasets.
Key findings
- Using MDE and SHAP, the authors find state-of-the-art fair learning methods can induce discrimination via association or reverse discrimination; e.g., in synthetic Scenario A, traditional learning that drops Z induces indirect discrimination via X1 as r(X1,Z) increases, and in Scenario B Exp Grad methods become increasingly influenced by X1 even though X1 has no impact on Y.
- The proposed MIM minimizes L_MDE(X) and L_SHAP(X); OPT-MDE preserves MDE of individual Xi, OPT-SHAP preserves SHAP of individual Xi, and MIM preserves pooled X influence. MIM is slightly more accurate than OPT-SHAP at the cost of minimally higher demographic disparity and equal opportunity difference.
- On COMPAS, German Credit, and Adult Census Income, MIM is not influenced by the protected attribute and preserves influence for the two attributes most correlated with it; most evaluated models outperform MIM on demographic disparity but either use the protected attribute or fail to preserve at least one correlated attribute and have significantly lower accuracy (e.g., Exp Grad for COMPAS).
- Theoretical comparison: MCM is more accurate than PSCF, with E(Y − Y_PSCF)^2 = E(Y − Y_c)^2 + Δ^2, and MIM has larger error than MCM, E(Y − Y_MIM)^2 = E(Y − Y_c)^2 + (θ_ym ε_m + θ_yl θ_lm ε_m)^2.
Methods
- Design: formalizes direct/indirect/induced discrimination via controlled direct effect (CDE), marginal direct effect (MDE), and SHAP; constructs losses L_MDE(X), L_SHAP(X), and feature-specific L_IND_MDE, L_IND_SHAP; proposes MIM closed-form, OPT-MDE/OPT-SHAP via transfer learning/gradient descent, and marginal counterfactual mixture (MCM) for nested indirect discrimination.
- Data: synthetic datasets drawn from multivariate normal with standard normal marginals and given correlations; Scenario A Y=σ(X1+X2+Z+1), Scenario B Y=σ(0*X1+X2+0*Z+1) with corr(X2,Z)=0.5; real-world COMPAS, German Credit, Adult Census Income; 20:80 test/train split; 30 trials for COMPAS and German Credit, 10 for Adult Census Income; COMPAS racial bias exacerbated by removing 500 positive outcomes for African-Americans.
- Analysis: compares against Reweighing (pre-processing), reductions with DP/EO/TPR/ER constraints (in-processing), calibrated equalized odds (post-processing), traditional logistic regression, and drop-Z; measures SHAP, MDE, accuracy, demographic disparity, equalized odds/equal opportunity; underlying classifier logistic regression; AIF360 implementations.
Limitations
- Stated: focuses on MDE and SHAP; SHAP can be fooled into stating a protected attribute has no influence, so adversarial scenarios may trick the approach; theoretical guarantees cover wide model classes but experiments use simple logistic models; any fairness objective can be misused, and omitting the first debiasing step for an unfairly influenced Xi would result in indirect discrimination.
- Stated/evident: methods assume no unmeasured confounders and direct causal links from X and Z to Y; if this does not hold (e.g., Y→X←Z), supervised learning may suffer collider bias (Berkson’s paradox).
- Evident: the paper does not test complex non-linear models or many other influence measures/loss functions; it leaves exploration of other losses and measures for future work.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 47 cited sources checked · 0 supported · 0 flagged
- Findingown data§ ABSTRACT · p. 1
Measurements of marginal direct effect (MDE) and SHapley Additive exPlanations (SHAP) reveal that state-of-the-art fair learning methods can induce discrimination via association or reverse discrimination in synthetic and real-world datasets.
“The measurements of marginal direct effect (MDE) and SHapley Additive exPlanations (SHAP) reveal that state-of-the-art fair learning methods can induce discrimination via association or reverse discrimination in synthetic and real-world datasets.”
Quote in the paperSubstantial claim - Findingown data§ ABSTRACT · p. 1
The proposed post-processing methods diminish various disparity measures, e.g., demographic disparity.
“finding that they yield relatively high model accuracy, prevent direct discrimination, and diminishes various disparity measures, e.g., demographic disparity.”
Quote in the paperSubstantial claim - Findingnone given§ 1 INTRODUCTION · p. 2
Dropping the protected attribute before training in standard supervised learning results in increased influence of features associated with the protected attribute.
“Third, we show that dropping the protected attribute before training in standard supervised learning would result in increased influence of features associated with the protected attribute.”
Quote in the paperSubstantial claim- The authors state this as an assumption: There are direct causal links from X and Z to Y.
- The authors state this as an assumption: There are no unmeasured confounders.
- Findingnone given§ 1 INTRODUCTION · p. 2
Marginal interventional mixture models keep the influence of X and Z close to the target values.
“We show that this method keeps influence of 𝑿 and 𝒁 close to the target values”
Quote in the paperSubstantial claim- The authors state this as an assumption: There are direct causal links from X and Z to Y.
- The authors state this as an assumption: There are no unmeasured confounders.
- Findingown data§ ABSTRACT · p. 1
The proposed post-processing methods prevent direct discrimination.
“finding that they yield relatively high model accuracy, prevent direct discrimination, and diminishes various disparity measures, e.g., demographic disparity.”
Quote in the paperSubstantial claim
- Findingown data§ ABSTRACT · p. 1
The proposed post-processing methods yield relatively high model accuracy.
“finding that they yield relatively high model accuracy, prevent direct discrimination, and diminishes various disparity measures, e.g., demographic disparity.”
Quote in the paperSubstantial claim - Findingreasoning§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7
The mean squared error w.r.t. Y is larger for PSCF than for MCM by the square of the difference, i.e., E(Y - ^Y_PSCF)^2 = E(Y - ^Y_c)^2 + Δ^2.
“In fact, the mean squared error w.r.t. 𝑌 is larger for PSCF than for MCM by the square of the difference, i.e., E ( 𝑌 -^ 𝑌 PSCF ) 2 = E ( 𝑌 -^ 𝑌 c ) 2 + Δ 2 .”
Quote in the paperSubstantial claim- The authors state this as an assumption: The causal influence of the protected attribute Z on decisions Y and the mediator M is assumed unfair and all other influences are fair.
- The authors state this as an assumption: We have enough samples to have perfect estimates of a well-specified model's parameters.
- Findingreasoning§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7
The MIM has a larger error than the MCM, i.e., E(Y - ^Y_MIM)^2 = E(Y - ^Y_c)^2 + (θ_ym ε_m + θ_yl θ_lm ε_m)^2.
“Thus resulting in a larger error for the MIM than the MCM, i.e., E ( 𝑌 -^ 𝑌 MIM ) 2 = E ( 𝑌 -^ 𝑌 c ) 2 + ( 𝜃 𝑦 𝑚 𝜖 𝑚 + 𝜃 𝑦 𝑙 𝜃 𝑙 𝑚 𝜖 𝑚 ) 2”
Quote in the paperSubstantial claim- Rests on an assumption: Each variable can contain exogenous noise, i.e., unobserved intrinsic noise not associated with any other variable.
- Findingreasoning§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7
The result that MCM is at least as accurate as PSCF is true for any linear model and any choice of the reference z'.
“The above result that MCM is at least as accurate as PSCF is true for any linear model and any choice of the reference 𝑧 ′ .”
Quote in the paperSubstantial claim- The authors state this as an assumption: We have enough samples to have perfect estimates of a well-specified model's parameters.
- Findingreasoning§ 4.1 Minimizing 𝐿 MDE ( 𝑿 ) or 𝐿 SHAP ( 𝑿 ) · p. 6
For a real-valued and analytic y(x,z), the MIM is an interventional mixture that minimizes the objective L_SHAP(X).
“Proposition 2. For a real-valued and analytic 𝑦 ( 𝑥, 𝑧 ) , the MIM is an interventional mixture that minimizes the objective 𝐿 SHAP ( 𝑿 ) .”
Quote in the paperSubstantial claim - Findingreasoning§ 5.2 Synthetic datasets · p. 8
The MIM preserves the influence of pooled X.
“the MIM preserves the influence of pooled 𝑿 (Figure 3D),”
Quote in the paperSubstantial claim - Findingnone given§ 4.1 Minimizing 𝐿 MDE ( 𝑿 ) or 𝐿 SHAP ( 𝑿 ) · p. 6
For variable Y, the objective L_MDE(X) is minimized by the MIM.
“Proposition 1. For variable 𝑌 , the objective 𝐿 MDE ( 𝑿 ) is minimized by the MIM.”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Synthetic datasets · p. 9
Other reductions approaches, constrained on equal opportunity and error ratio, yield similar outcomes.
“Other reductions approaches, constrained on equal opportunity and error ratio, yield similar outcomes (Appendix B).”
Quote in the paperNot a substantial claim - Findingreasoning§ 3.2 Problem formulation based on causal influence measures · p. 5
The marginal direct effect (MDE) yields the same value for all graphs in Figure 2.
“This measure yields the same value for all graphs in Figure 2.”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Synthetic datasets · p. 8
All introduced methods remove the influence of Z.
“All introduced methods achieve their objectives (compare them against the blue lines in Figure 3), i.e., they all remove the influence of 𝒁 (Figures 3C, 3H),”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Synthetic datasets · p. 9
In Scenario A, the influence of X1 increases with correlation for the traditional method that simply drops Z, i.e., it induces indirect discrimination via X1.
“As expected in Scenario A, the influence of 𝑋 1 increases with correlation for the traditional method that simply drops 𝑍 , i.e., it induces indirect discrimination via 𝑋 1 (dotted blue line in Figure 4A).”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Synthetic datasets · p. 9
The MIM preserves X1's influence with respect to the standard full model as r(X1,Z) increases.
“The MIM preserve 𝑋 1 's influence with respect to the standard full model as 𝑟 ( 𝑋 1 , 𝑍 ) increases (red and solid blue lines in Figures 4A, 4B, 4F, 4G).”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Synthetic datasets · p. 8
The OPT-SHAP preserves the SHAP of individual X_i.
“and the OPT-SHAP preserves the SHAP of individual 𝑿 𝑖 (Figures 3F, 3G).”
Quote in the paperSubstantial claim - Findingreasoning§ 5.2 Synthetic datasets · p. 8
The MIM and the two OPT methods perform comparably.
“As the MIM and the two OPT methods minimize loss functions based on the preservation of the influence of non-protected attributes, the resulting models perform comparably (red and two orange lines in Figure 3).”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Synthetic datasets · p. 8
The OPT-MDE preserves the MDE of individual X_i.
“the OPT-MDE preserves the MDE of individual 𝑿 𝑖 (Figures 3A, 3B),”
Quote in the paperSubstantial claim - Findingreasoning§ 5.2 Synthetic datasets · p. 8
The MIM performs nearly the same as the OPT-SHAP across all measures, despite not being designed to achieve the feature-specific loss of OPT-SHAP.
“Interestingly, the MIM performs nearly the same as the OPT-SHAP across all measures, despite not being designed to achieve the feature-specific loss of OPT-SHAP (Equation 11).”
Quote in the paperSubstantial claim - Findingreasoning§ APPENDIX A: PROOF OF PROPOSITION 2 · p. 13
The MIM is an interventional mixture that minimizes L_SHAP(X) for any n and any function y(x,z) that has a Taylor expansion.
“Thus, by induction, the MIM is an interventional mixture that minimizes 𝐿 SHAP ( 𝑿 ) for any 𝑛 and any function 𝑦 ( 𝑥, 𝑧 ) that has a Taylor expansion.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The function y(x,z) is analytic (can be expanded into a Taylor series).
- Findingreasoning§ APPENDIX A: PROOF OF PROPOSITION 2 · p. 13
For y1(x,z)=α1 x^k z^l, the optimal mixing distribution π*(Z) is the marginal distribution.
“First, we show that for 𝑦 1 ( 𝑥, 𝑧 ) = 𝛼 1 𝑥 𝑘 𝑧 𝑙 , where 𝑘 = 𝑖 ( 1 ) and 𝑙 = 𝑗 ( 1 ) , the optimal mixing distribution 𝜋 ∗ ( ˜ 𝑍 ) is the marginal distribution, i.e., 𝜋 ∗ ( ˜ 𝑍 = 𝑧 ) = 𝑃 ( 𝑍 = 𝑧 ) .”
Quote in the paperSubstantial claim- Rests on an assumption: X', Z', and Z'' have the same means as X and Z, respectively.
- Findingreasoning§ APPENDIX A: PROOF OF PROPOSITION 2 · p. 13
The necessary conditions for extremum are met if the distribution of Z̃ is the same as the marginal distribution of Z.
“Hence, the necessary conditions for extremum are met if the distribution of ˜ 𝑍 is the same as the marginal distribution of 𝑍 .”
Quote in the paperSubstantial claim- Rests on an assumption: X', Z', and Z'' have the same means as X and Z, respectively.
- Findingreasoning§ 6 LIMITATIONS AND FUTURE WORK · p. 9
Our theoretical guarantees for the preservation of MDE or SHAP hold for wide classes of models.
“While our theoretical guarantees for the preservation of MDE or SHAP hold for wide classes of models, our experiments compare”
Quote in the paperSubstantial claim - Findingnone given§ 7 CONCLUSIONS · p. 11
State-of-the-art methods addressing discrimination often return biased models when they are trained on datasets that are or are not affected by discrimination.
“Second, we show that state-of-the-art methods addressing discrimination often return biased models when they are trained on datasets that are or are not affected by discrimination.”
Quote in the paperSubstantial claim - Premisecitation§ 2 RELATED WORKS · p. 2
Notions of discrimination based on statistical independence between Y and Z are misaligned with their legal counterparts, resulting in shortcomings.
“Hence, the notions of discrimination based on the statistical independence between 𝑌 and 𝑍 are misaligned with their legal counterparts [33], which results in shortcomings.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [33] [33] Zachary C. Lipton and Jacob Steinhardt. 2019. Troubling trends in machinelearning scholarship. Queue 17, 1 (2019), 1-15. https://doi.org/10.1145/3317287. 3328534 arXiv:1807.03341 -
Not checked in the free run.
- Premisereasoning§ 2 RELATED WORKS · p. 2
The corresponding indirect discrimination in causal-influence definitions is not consistent with legal systems, since the business necessity clause allows use of an attribute that depends on the protected feature if the attribute is judged relevant to the decisions made.
“While this notion of direct discrimination is consistent with the concept of disparate treatment in legal systems, the corresponding indirect discrimination is not consistent with them, since the business necessity clause allows the use of an attribute that depends on the protected feature (causally or otherwise), if the attribute is judged relevant to the decisions made.”
Quote in the paperSubstantial claim - Premisecitation§ 1 INTRODUCTION · p. 2
Data-rich machine learning systems often can find surprisingly accurate surrogates for protected attributes when a large enough set of legitimate-looking variables is available, resulting in discrimination via association.
“This is a particularly acute problem for data-rich machine learning systems, since they often can find surprisingly accurate surrogates for protected attributes when a large enough set of legitimate-looking variables is available, resulting in discrimination via association [57].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [57] [57] Sandra Wachter. 2019. Affinity Profiling and Discrimination by Association in Online Behavioural Advertising. SSRN Electronic Journal (2019), 1-74. https: //doi.org/10.2139/ssrn.3388639 -
Not checked in the free run.
- Premisecitation§ 2 RELATED WORKS · p. 2
Algorithms that constrain disparities in treatment and impact could negatively affect females with short hair and/or programming skills because of those features' association with males.
“For instance, the algorithms that put constraints on the aforementioned disparities in treatment and impact [17, 41, 61] could negatively affect females with short hair and/or programming skills, because of those features' (fair or unfair) association with males [32].”
Quote in the paperNot a substantial claim0 of 4 sources checked- cites [17] [17] Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2014. Certifying and removing disparate impact. (2014), 259-268. https://doi.org/10.1145/2783258.2783311 arXiv:1412.3756 - source
Not checked in the free run.
- cites [41] [41] Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-aware data mining. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08 . ACMPress, New York, New York, USA, 560. https://doi.org/10.1145/1401890.1401959 - source
Not checked in the free run.
- cites [61] [61] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2015. Fairness Constraints: Mechanisms for Fair Classification. Fairness, Accountability, and Transparency in Machine Learning (jul 2015). arXiv:1507.05259 http://arxiv.org/abs/1507.05259 - source
Not checked in the free run.
- cites [32] [32] Zachary C. Lipton, Alexandra Chouldechova, and Julian McAuley. 2018. Does mitigating ML's impact disparity require treatment disparity? Advances in Neural Information Processing Systems 2018-Decem, ML (2018), 8125-8135. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation§ 3.2 Problem formulation based on causal influence measures · p. 4
If there are not direct causal links from X and Z to Y, e.g., because supervised learning is used for nowcasting instead of forecasting, then the methodology may suffer collider bias.
“If this assumption does not hold, e.g., because supervised learning is used for nowcasting instead of forecasting, then the following methodology may suffer collider bias (Berkson's paradox) [11, 51].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [11] [11] Anthony C. Constantinou, Yang Liu, Kiattikun Chobtham, Zhigao Guo, and Neville K. Kitson. 2020. Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data. arXiv:2005.09020 [cs.LG] -
Not checked in the free run.
- cites [51] [51] Peter Spirtes and Kun Zhang. 2016. Causal discovery and inference: concepts and recent methodological advances. Applied Informatics 3, 1 (2016). https: //doi.org/10.1186/s40535-016-0018-x -
Not checked in the free run.
- The authors state this as an assumption: There are direct causal links from X and Z to Y.
- Premisecitation§ 3.1 Legal notions and responsibility for decision-making models · p. 3
Features perceived as fair tend to be volitional.
“Prior research shows that features perceived as fair tend to be volitional [20], as in the above example.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [20] [20] Nina Grgic-Hlaca, Elissa M. Redmiles, Krishna P. Gummadi, and Adrian Weller. 2018. Human Perceptions of Fairness in Algorithmic Decision Making. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 . ACM Press, New York, New York, USA, 903-912. https://doi.org/10.1145/ 3178876.3186138 arXiv:1802.09548 - source
Not checked in the free run.
- Premisenone given§ 2 RELATED WORKS · p. 2
If there is no limit on the influence that can pass through a fair causal path, then the path can be used for discrimination, as in redlining.
“However, if there is no limit on the influence that can pass through such a path, then the path can be used for discrimination, as in the aforementioned case of redlining .”
Quote in the paperNot a substantial claim - Interpretationreasoning§ 5.2 Synthetic datasets · p. 8
The MIM can achieve both the pooled and individual objectives, as in this case.
“hence the MIM can achieve both the pooled and individual objectives, as in this case.”
Quote in the paperSubstantial claim - Premisecitation§ 3.2 Problem formulation based on causal influence measures · p. 4
If the underlying causal graph is Y → X ← Z, then conditioning on X makes Y and Z depend on each other, despite Z not causally influencing Y, so supervised learning based on samples (x,z,y) would yield a model in which Z unfaithfully influences the model of Y.
“For instance, if the underlying causal graph is 𝑌 → 𝑋 ← 𝑍 , then conditioning on 𝑋 makes 𝑌 and 𝑍 depend on each other, despite the fact that 𝑍 does not causally influence 𝑌 , so supervised learning based on samples ( 𝑥, 𝑧, 𝑦 ) would yield a model in which 𝑍 unfaithfully (w.r.t. the causal graph) influences the model of 𝑌 .”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [11] [11] Anthony C. Constantinou, Yang Liu, Kiattikun Chobtham, Zhigao Guo, and Neville K. Kitson. 2020. Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data. arXiv:2005.09020 [cs.LG] -
Not checked in the free run.
- cites [51] [51] Peter Spirtes and Kun Zhang. 2016. Causal discovery and inference: concepts and recent methodological advances. Applied Informatics 3, 1 (2016). https: //doi.org/10.1186/s40535-016-0018-x -
Not checked in the free run.
- Interpretationreasoning§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7
The multi-stage MCM corrects the decision through a correction on all the variables that are influenced by the protected attribute along unfair pathways.
“Similar to PCSF, the multi-stage MCM corrects the decision through a correction on all the variables that are influenced by the protected attribute along unfair pathways.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The causal influence of the protected attribute Z on decisions Y and the mediator M is assumed unfair and all other influences are fair.
- Premisecitation§ 5.2 Synthetic datasets · p. 8
SHAP meets the completeness axiom, which says that the sum of influence of individual features equals the influence of all features pooled together.
“and SHAP meets the completeness axiom (a.k.a. additivity axiom) [13, 23], which says that the sum of influence of individual features equals to the influence of all features pooled together,”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [13] [13] Anupam Datta, Shayak Sen, and Yair Zick. 2016. Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems. Proceedings - 2016 IEEE Symposium on Security and Privacy, SP 2016 (2016), 598617. https://doi.org/10.1109/SP.2016.42 - source
Not checked in the free run.
- cites [23] [23] Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. 2019. Feature relevance quantification in explainable AI: A causal problem. 2015 (oct 2019). arXiv:1910.13413 http://arxiv.org/abs/1910.13413 - source
Not checked in the free run.
- Premisecitation§ 5.1 Evaluated learning methods · p. 8
Recent works expose the impossibility of simultaneously satisfying multiple non-discriminatory objectives, such as equalized opportunity and parity mistreatment.
“Recent works expose the impossibility of simultaneously satisfying multiple non-discriminatory objectives, such as equalized opportunity and parity mistreatment [10, 19, 27].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [10] [10] Alexandra Chouldechova. 2017. Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data 5, 2 (jun 2017), 153-163. https://doi.org/10.1089/big.2016.0047 arXiv:1703.00056 - source
Not checked in the free run.
- cites [19] [19] Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im)possibility of fairness. (2016). arXiv:1609.07236 http://arxiv.org/abs/ 1609.07236 - source
Not checked in the free run.
- cites [27] [27] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores. In Proceedings of Innovations in Theoretical Computer Science (ITCS) . https://doi.org/10.1111/j.1740-9713.2017. 01012.x arXiv:1609.05807 -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationnone given§ 3.2 Problem formulation based on causal influence measures · p. 5
The choice to use X_z as a reference value in the definitions of natural direct effects is arbitrary.
“Furthermore, the choice to use 𝑿 𝒛 as a reference value in the definitions of natural direct effects is arbitrary, e.g., one could use 𝑿 𝒛 ′ instead.”
Quote in the paperSubstantial claim - Premisecitation§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 6
Path-specific counterfactual fairness (PSCF) requires the knowledge of a full causal graph.
“In contrast to our proposed methods, path-specific counterfactual fairness (PSCF) requires the knowledge of a full causal graph.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [8] [8] Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. Proceedings of the AAAI Conference on Artificial Intelligence 33 (jul 2019), 7801-7808. https: //doi.org/10.1609/aaai.v33i01.33017801 - source
Not checked in the free run.
- Premisecitation§ 5.1 Evaluated learning methods · p. 7
Such methods can induce a discriminatory bias in model parameters.
“Such methods, however, can induce a discriminatory bias in model parameters [32].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [32] [32] Zachary C. Lipton, Alexandra Chouldechova, and Julian McAuley. 2018. Does mitigating ML's impact disparity require treatment disparity? Advances in Neural Information Processing Systems 2018-Decem, ML (2018), 8125-8135. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationnone given§ 4 LEARNING FAIR AND EXPLAINABLE MODELS · p. 6
Either of these approaches can be used to remove direct or indirect discrimination.
“Either of these approaches can be used to remove direct or indirect discrimination (see example in Subsection 4.3).”
Quote in the paperNot a substantial claim - Interpretationnone given§ 3.2 Problem formulation based on causal influence measures · p. 4
Removing direct discrimination by constructing a model that does not use Z may induce discrimination via attributes Xi associated with Z, even if there is no causal link from Z to Xi.
“To remove direct discrimination, one can construct a model ^ 𝑌 that does not use 𝒁 . However, the removal of direct discrimination may induce discrimination via the attributes 𝑋 𝑖 associated with the protected attributes 𝒁 , even if there is no causal link from 𝒁 to 𝑋 𝑖 .”
Quote in the paperNot a substantial claim - Premisecitation§ 6 LIMITATIONS AND FUTURE WORK · p. 9
Prior studies show that input influence measures like SHAP can be fooled into stating that a protected attribute has no influence on a model.
“Prior studies show that input influence measures like SHAP can be fooled into stating that a protected attribute has no influence on a model [49].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [49] [49] Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. 2020. Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods. In AAAI/ACM Conference on AI, Ethics, and Society (AIES) . - source
Not checked in the free run.
- Interpretationcitation§ 6 LIMITATIONS AND FUTURE WORK · p. 9
Someone may be able to trick our approach into believing a model was fair by our definition, even though in reality it was not.
“With this, someone may be able to trick our approach into believing a model was fair by our definition, even though in reality it was not.”
Quote in the paperNot a substantial claim - Interpretationnone given§ 7 CONCLUSIONS · p. 11
The marginal interventional mixture of full models prevents the induction of discrimination via association.
“Third, for discrimination prevention we propose to use a marginal interventional mixture of full models, which prevents the induction of discrimination via association.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The function y(x,z) is analytic (can be expanded into a Taylor series).
- May combine two claims. 30
- The quote only partly states this claim. 1
- Classified as probably not a substantial claim. 27
May combine two claims. The quote only partly states this claim.
Findingnone given§ 1 INTRODUCTION · p. 2Marginal interventional mixture models decrease popular disparity measures while keeping high model accuracy.
“in addition, decreases popular disparity measures, while keeping high model accuracy.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: There are direct causal links from X and Z to Y.
- The authors state this as an assumption: There are no unmeasured confounders.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone given§ 5.3 Real-world datasets · p. 9The two attributes most correlated with a person's sex in the Adult Census Income dataset are if they are a husband and if they have a spouse in the armed forces.
“The two attributes most correlated with a person's sex are if they are a husband and if they have a spouse in the armed forces.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ 5.2 Synthetic datasets · p. 9In Scenario A, the method optimizing for parity of impact ('Exp Grad DP') unnecessarily decreases the influence of X1, which leads to an accuracy loss, because its goal is to remove the correlation between ^Y and Z.
“On the one hand, the method optimizing for parity of impact ('Exp Grad DP') in Scenario A unnecessarily decreases the influence of 𝑿 1 (brown line in Figure 4A), which leads to an accuracy loss (Figure 4D), because its goal is to remove the correlation between ^ 𝑌 and 𝑍 .”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingnone given§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 6The interventional mixture calculates the value of ^X_i had the causal influence of Z been removed from it given the values of all observed variables, but not the values of exogenous noise.
“Note that the interventional mixture calculates the value of ^ 𝑋 𝑖 had the casual influence of 𝒁 been removed from it given the values of all observed variables, but not the values of exogenous noise.”
Quote in the paperSubstantial claim- Rests on an assumption: Each variable can contain exogenous noise, i.e., unobserved intrinsic noise not associated with any other variable.
May combine two claims.
Findingown data§ 5.2 Synthetic datasets · p. 9In Scenario B, the affected methods ('Exp Grad') are increasingly influenced by X1 as it gets more associated with the protected attribute, despite X1 not having impact on Y, because this enables them to increasingly utilize X2 in their model of Y and improve accuracy under a respective fairness constraint.
“In Scenario B, the changes in the influence of 𝑋 1 with the growing correlation are especially noteworthy. The affected methods ('Exp Grad') are increasingly influenced by 𝑋 1 as it gets more associated with the protected attribute (Figure 4F), despite 𝑋 1 not having impact on 𝑌 , because this enables them to increasingly utilize 𝑋 2 in their model of 𝑌 (Figure 4G) and improve accuracy (Figure 4I)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 5.2 Synthetic datasets · p. 9Methods allowing the influence of Z perform relatively well in Scenario A, because they counteract discrimination by using Z directly to maintain stable influence of X1 and X2 on ^Y and a high model accuracy, independently of r(X1,Z).
“On the other hand, the methods allowing the influence of 𝑍 perform relative well in Scenario A, because they counteract discrimination by using 𝑍 directly (violet and teal lines in Figures 4C, 4H) to maintain stable influence of 𝑋 1 and 𝑋 2 on ^ 𝑌 (Figures 4A, 4B, 4F, 4G) and a high model accuracy (Figures 4D, 4I), independently of 𝑟 ( 𝑋 1 , 𝑍 ) .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone given§ 5.3 Real-world datasets · p. 9The two attributes most correlated with an applicant's sex in the German Credit dataset are their age and number of dependents.
“The two attributes most correlated with a applicant's sex are their age and number of dependents.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.3 Real-world datasets · p. 9In line with the synthetic results, the MIM (and OPT methods) is not influenced by the protected attribute and, with respect to the traditional model, preserves the influence for the two attributes most correlated with the protected attribute in the real-world scenarios.
“In line with the synthetic results, the MIM (and OPT methods) is not influenced by the protected attribute (leftmost column in Figure 5) and, with respect to the traditional model, preserves the influence for the two attributes most correlated with the protected attribute in these real-world scenarios (blue and red bars in the two middle columns of Figure 5).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 5.2 Synthetic datasets · p. 9The results for MDE are qualitatively the same as the results for SHAP.
“In the remainder of the paper we report results for the SHAP influence, since the results for MDE are qualitatively the same (Appendix C).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Synthetic datasets · p. 8The MIM is slightly more accurate than the OPT-SHAP at the cost of minimally higher demographic disparity and equal opportunity difference (accuracy disparity).
“Note, however, that the MIM is slightly more accurate than the OPT-SHAP (Figure 3E) at the cost of minimally higher demographic disparity (Figure 3I) and equal opportunity difference, i.e., accuracy disparity (Figure 3J).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone given§ 4.2 Minimizing 𝐿 IND MDE ( 𝑿 ) and 𝐿 IND SHAP ( 𝑿 ) via transfer learning · p. 6The minimization of feature-specific losses L_IND_MDE(X) and L_IND_SHAP(X) does not result in closed-form solutions.
“Theminimization of the feature-specific losses, 𝐿 IND MDE ( 𝑿 ) and 𝐿 IND SHAP ( 𝑿 ) , does not result in closed-form solutions, so we apply a respective gradient descent.”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning§ 4.1 Minimizing 𝐿 MDE ( 𝑿 ) or 𝐿 SHAP ( 𝑿 ) · p. 6Minimizing L_SHAP gives a potentially infinite set of conditions E[Z~^l]=E[Z^l] for the respective moments of Z~; these conditions are met if P(Z~=z)=P(Z=z).
“Minimizing this objective gives a potentially infinite set of conditions E [ ˜ 𝑍 𝑙 ] = E [ 𝑍 𝑙 ] for the respective moments of ˜ 𝑍 . Since 𝑙 can be any positive integer, these conditions are met if 𝑃 ( ˜ 𝑍 = 𝑧 ) = 𝑃 ( 𝑍 = 𝑧 ) .”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 5.2 Synthetic datasets · p. 9Even though the MIM does not optimize for any fairness measure, it performs better in demographic disparity and all other fairness measures than the traditional method dropping Z.
“Importantly, even though the MIM does not optimize for any fairness measure, it performs better in demographic disparity (Figure 4E) and all other fairness measures (Appendix B) than the traditional method dropping 𝑍 .”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 5.2 Synthetic datasets · p. 9In Scenario B, where there is no discrimination, methods allowing the influence of Z introduce reverse discrimination to counteract the correlation between X2 and Z, without considering the possibility that this correlation is a fair relationship, and achieve lower accuracy than the MIM.
“However, in Scenario B, where there is no discrimination, these methods introduce reverse discrimination to counteract the correlation between 𝑋 2 and 𝑍 , without considering the possibility that this correlation is a fair relationship , and achieve lower accuracy than the MIM (Figure 4I).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 5.3 Real-world datasets · p. 9The two attributes most correlated with race in the COMPAS dataset are age and number of prior counts.
“The two attributes most correlated with race are age and number of prior counts.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7Applying only the MIM would not take advantage of estimating the noise terms and yields less-accurate estimators than MCM.
“If we applied only the MIM, we would not take advantage of estimating the noise terms and yield the following less-accurate estimators,”
Quote in the paperSubstantial claim- Rests on an assumption: Each variable can contain exogenous noise, i.e., unobserved intrinsic noise not associated with any other variable.
May combine two claims.
Findingnone given§ 7 CONCLUSIONS · p. 11In scenarios where discrimination does not affect the training data, the proposed learning algorithm falls back to traditional learning, ensuring that the method does not bias the model needlessly.
“In the scenarios where discrimination does not affect the training data, the proposed learning algorithm falls back to a traditional learning, which ensures that the method does not bias the model needlessly.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone given§ APPENDIX C: MARGINAL DIRECT EFFECT (MDE) · p. 13All results for MDE are qualitatively equivalent to that of SHAP for each dataset.
“All results for MDE are qualitative equivalent to that of SHAP for each dataset.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ APPENDIX A: PROOF OF PROPOSITION 2 · p. 13For the special case of k=0 or l=0, the MIM minimizes the objective globally, achieving L_SHAP(X)=0.
“For the special case of 𝑘 = 0 or 𝑙 = 0 the MIM minimizes the objective globally, by achieving 𝐿 SHAP ( 𝑿 ) = 0 .”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.3 Real-world datasets · p. 9Most of the evaluated models outperform the MIM in terms of demographic disparity, but they are either influenced by the protected attribute or do not preserve the influence of at least one of the most correlated attributes and have significantly lower accuracy.
“While most of the evaluated models outperform the MIM in terms of demographic disparity (the rightmost column in Figure 5), they are either influenced by the protected attribute (the leftmost column in Figure 5) or do not preserve the influence of at least one of the most correlated attributes (the two middle columns in Figure 5) and have significantly lower accuracy (Figure 6), e.g., 'Exp Grad'”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone given§ APPENDIX A: PROOF OF PROPOSITION 2 · p. 13The corresponding Hessian matrix is positive semi-definite, so the extremum is a minimum.
“The corresponding Hessian matrix is positive semi-definite, so the extremum is a minimum.”
Quote in the paperNot a substantial claim- Rests on an assumption: X', Z', and Z'' have the same means as X and Z, respectively.
May combine two claims.
Premisecitation§ 3.1 Legal notions and responsibility for decision-making models · p. 3In graduate admissions to University of California Berkeley, females were less often admitted than males, but females applied to departments with lower admission rates and the overall admission process was judged legal.
“For instance, in graduate admissions to University of California Berkeley it was found that females were less often admitted than males [7]. However, females applied to departments with lower admission rates than males and the overall admission process was judged legal.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [7] [7] P. J. Bickel, E. A. Hammel, and J. W. O'Connell. 1975. Sex Bias in Graduate Admissions: Data from Berkeley. Science 187, 4175 (feb 1975), 398-404. https: //doi.org/10.1126/science.187.4175.398 - source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ 2 RELATED WORKS · p. 2Path-specific notions of causal fairness introduce fair causal paths, enabling business necessity to a limited extent.
“Path-specific notions of causal fairness address this issue to a limited extent [8, 37, 59]. These methods introduce fair causal paths , i.e., the paths through which the impact of the protected attribute is permitted, hence enabling business necessity.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [8] [8] Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. Proceedings of the AAAI Conference on Artificial Intelligence 33 (jul 2019), 7801-7808. https: //doi.org/10.1609/aaai.v33i01.33017801 -
Not checked in the free run.
- cites [37] [37] Razieh Nabi, Daniel Malinsky, and Ilya Shpitser. 2019. Learning Optimal Fair Policies. In Proceedings of the 36th International Conference on Machine Learning . PMLR 97:4674-4682. arXiv:1809.02244 http://arxiv.org/abs/1809.02244 -
Not checked in the free run.
- cites [59] [59] Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. 2019. PC-Fairness: A unified framework for measuring causality-based fairness. Advances in Neural Information Processing Systems 32, NeurIPS (2019). arXiv:1910.12586 -
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 3.1 Legal notions and responsibility for decision-making models · p. 3The U.S. Civil Rights Act of 1968 determines that decisions about sale, rental, and financing of housing shall not be influenced by race, creed, and national origin.
“the U.S. Civil Rights Act of 1968 (Titles VIII and IX, known as Fair Housing Act) [53] determines that decisions about sale, rental, and financing of housing shall not be influenced by race, creed, and national origin”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [53] [53] The Fair Housing Act. 1968. 42 U.S.C.A., 3601-3631. -
Not checked in the free run.
May combine two claims.
Premisecitation§ 3.1 Legal notions and responsibility for decision-making models · p. 4A model is causal with respect to its own predictions, and the parents of these predictions are X and possibly Z.
“Either way, whether the model at hand does or does not represent causal relations between variables in the real world, the model is causal w.r.t. its own predictions and the parents of these predictions are 𝑿 and, possibly, 𝒁 , as detailed in the causal explainability literature [23].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23] [23] Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. 2019. Feature relevance quantification in explainable AI: A causal problem. 2015 (oct 2019). arXiv:1910.13413 http://arxiv.org/abs/1910.13413 - source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone given§ 1 INTRODUCTION · p. 2In supervised learning, potentially any feature that improves model predictiveness on deployment could be claimed to fulfil a business necessity.
“In supervised learning, potentially any feature that improves model predictiveness on deployment could be claimed to fulfil a business necessity.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisecitation§ 3.1 Legal notions and responsibility for decision-making models · p. 4Machine learning models are developed in best faith to maximize accuracy, but often are not causal and not robust to covariate shifts.
“Machine learning models are developed in best faith to maximize accuracy, but often are not causal and not robust to covariate shifts [28, 45], i.e., they constitute an inaccurate epistemic state.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [28] [28] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2020. WILDS: A Benchmark of in-theWild Distribution Shifts. (2020), 1-87. arXiv:2012.07421 http://arxiv.org/abs/ 2012.07421 -
Not checked in the free run.
- cites [45] [45] Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. 2020. An Investigation of Why Overparameterization Exacerbates Spurious Correlations. In ICML'20 . arXiv:2005.04345 http://arxiv.org/abs/2005.04345 -
Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ 3.1 Legal notions and responsibility for decision-making models · p. 4A model can suffer the effects of training on discriminatory data.
“That model can suffer the effects of training on discriminatory data.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).
Premisecitation§ 2 RELATED WORKS · p. 2Direct and indirect discrimination can be defined as direct and indirect causal influence of Z on Y, respectively.
“A relevant line of research proposes to define direct and indirect discrimination as direct and indirect causal influence of 𝒁 on 𝑌 , respectively [65, 66].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [65] [65] Junzhe Zhang and Elias Bareinboim. 2018. Fairness in Decision-Making - The Causal Explanation Formula. AAAI (2018), 2037-2045. https://www.aaai.org/ ocs/index.php/AAAI/AAAI18/paper/view/16949 -
Not checked in the free run.
- cites [66] [66] Lu Zhang, Yongkai Wu, and Xintao Wu. 2017. A causal framework for discovering and removing direct and indirect discrimination. IJCAI International Joint Conference on Artificial Intelligence 0 (2017), 3929-3935. https: //doi.org/10.24963/ijcai.2017/549 arXiv:1611.07509
Not checked in the free run.
May combine two claims.
Premisereasoning§ 2 RELATED WORKS · p. 2The limit on influence through fair causal paths is not a focus of prior works but is crucial to prevent induced discrimination in machine learning.
“This limit is not a focus of prior works [8, 25, 29, 37, 46, 59, 65], but it is crucial to prevent induced discrimination in machine learning.”
Quote in the paperSubstantial claim0 of 7 sources checked- cites [8] [8] Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. Proceedings of the AAAI Conference on Artificial Intelligence 33 (jul 2019), 7801-7808. https: //doi.org/10.1609/aaai.v33i01.33017801 -
Not checked in the free run.
- cites [25] [25] Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding Discrimination through Causal Reasoning. In Advances in Neural Information Processing Systems 30 . Curran Associates, Inc., 656-666. arXiv:1706.02744 http://arxiv.org/abs/1706.02744http://papers.nips.cc/paper/6668-avoidingdiscrimination-through-causal-reasoning.pdf -
Not checked in the free run.
- cites [29] [29] Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In Advances in Neural Information Processing Systems 30 , I Guyon, U V Luxburg, S Bengio, H Wallach, R Fergus, S Vishwanathan, and R Garnett (Eds.). Curran Associates, Inc., 4066-4076. arXiv:1703.06856 http://arxiv.org/abs/ 1703.06856http://papers.nips.cc/paper/6995-counterfactual-fairness.pdf -
Not checked in the free run.
- cites [37] [37] Razieh Nabi, Daniel Malinsky, and Ilya Shpitser. 2019. Learning Optimal Fair Policies. In Proceedings of the 36th International Conference on Machine Learning . PMLR 97:4674-4682. arXiv:1809.02244 http://arxiv.org/abs/1809.02244 -
Not checked in the free run.
- cites [46] [46] Babak Salimi, Luke Rodriguez, Bill Howe, and Dan Suciu. 2019. Capuchin: Causal Database Repair for Algorithmic Fairness. (feb 2019). arXiv:1902.08283 http: //arxiv.org/abs/1902.08283 -
Not checked in the free run.
- cites [59] [59] Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. 2019. PC-Fairness: A unified framework for measuring causality-based fairness. Advances in Neural Information Processing Systems 32, NeurIPS (2019). arXiv:1910.12586 -
Not checked in the free run.
- cites [65] [65] Junzhe Zhang and Elias Bareinboim. 2018. Fairness in Decision-Making - The Causal Explanation Formula. AAAI (2018), 2037-2045. https://www.aaai.org/ ocs/index.php/AAAI/AAAI18/paper/view/16949 -
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone given§ 3.1 Legal notions and responsibility for decision-making models · p. 4Legal systems may acquit model developers if the intentions and reasoning behind the development process of models of Y and ^Y were legally admissible, despite an incorrect epistemic state.
“In such situations, legal systems may acquit model developers if the intentions and reasoning behind the development process of models of 𝑌 and ^ 𝑌 were legally admissible, despite the incorrect epistemic state.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 3.1 Legal notions and responsibility for decision-making models · p. 3The U.S. Civil Rights Act of 1964 (Title VII) determines that hiring decisions shall not be influenced by race, color, religion, sex, and national origin.
“the U.S. Civil Rights Act of 1964 (Title VII) [54] determines that hiring decisions shall not be influenced by race, color, religion, sex, and national origin.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [54] [54] Title VII of the Civil Rights Act. 1964. 7, 42 U.S.C., 2000e et seq. -
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Premisenone given§ 3.1 Legal notions and responsibility for decision-making models · p. 3Legal texts typically clearly define unfair influence, but fair relationships are determined on a case-by-case basis.
“Legal texts typically clearly define unfair influence, but fair relationships are determined on case-by-case basis.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone given§ 3.1 Legal notions and responsibility for decision-making models · p. 4In practice it may be impossible to test the causal validity of a model y(x,z) because of limited and unobserved data, privacy concerns, and the infeasibility or prohibitive costs of causal experimentation.
“Unfortunately, in practice it may be impossible to test causal validity of model 𝑦 ( 𝒙 , 𝒛 ) , because of limited and unobserved data, privacy concerns, and the infeasibility or prohibitive costs of causal experimentation.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisenone given§ 3.2 Problem formulation based on causal influence measures · p. 4The methods proposed in the paper rely only on the notion of intervention, which tends to have a consistent meaning across causal frameworks.
“The methods proposed here rely only on the notion of intervention, which tends to have a consistent meaning across causal frameworks.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Premisenone given§ 3 PROBLEM FORMULATION OF FAIR AND EXPLAINABLE LEARNING · p. 3The process generating decisions Y corresponds either to a real-world causal mechanism or its model, and the inducement of indirect discrimination shall be prevented on legal grounds in either case.
“The process generating decisions 𝑌 corresponds either to a real-world causal mechanism or its model, while the inducement of indirect discrimination shall be prevented on legal grounds in either case (see Subsection 3.1.2).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisecitation§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7PSCF is based on NDE, was introduced for binary Z, and relies on a choice of reference value z' (baseline).
“PSCF is based on NDE (Equation 2), it was introduced for binary 𝑍 , and relies on a choice of reference value, 𝑧 ′ , also known as baseline, which is assumed 𝑧 ′ = 0 in the above example.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [8] [8] Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. Proceedings of the AAAI Conference on Artificial Intelligence 33 (jul 2019), 7801-7808. https: //doi.org/10.1609/aaai.v33i01.33017801 -
Not checked in the free run.
- The authors state this as an assumption: The causal influence of the protected attribute Z on decisions Y and the mediator M is assumed unfair and all other influences are fair.
Classified as probably not a substantial claim.
Premisecitation§ 3.3 Problem formulation based on input influence measures · p. 5A popular measure of the influence of input w_i is based on the Shapley value (SHAP), which averages the marginal influence over all possible subsets T of all features except for i.
“A popular measure of the influence of input 𝑤 𝑖 is based on the Shapley value (SHAP), which averages the marginal influence over all possible subsets 𝑇 of all features except for 𝑖 [13, 34],”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [13] [13] Anupam Datta, Shayak Sen, and Yair Zick. 2016. Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems. Proceedings - 2016 IEEE Symposium on Security and Privacy, SP 2016 (2016), 598617. https://doi.org/10.1109/SP.2016.42 -
Not checked in the free run.
- cites [34] [34] Scott M. Lundberg and Su In Lee. 2017. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 2017-Decem, Section 2 (2017), 4766-4775. arXiv:1705.07874 -
Not checked in the free run.
May combine two claims.
Premisenone given§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 6A feature X_i can be unfairly and illegally influenced by Z; if decisions Y used such X_i, then Y would be indirectly discriminatory.
“However, a feature 𝑋 𝑖 can be unfairly and illegally influenced by 𝒁 . If decisions 𝑌 used such 𝑋 𝑖 , then 𝑌 would be indirectly discriminatory.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 5.1 Evaluated learning methods · p. 7Several methods have been proposed to train machine learning models that prevent a combination of disparate treatment and impact.
“Several methods have been proposed to train machine learning models that prevent a combination of disparate treatment and impact [17, 41, 61].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [17] [17] Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2014. Certifying and removing disparate impact. (2014), 259-268. https://doi.org/10.1145/2783258.2783311 arXiv:1412.3756 - source
Not checked in the free run.
- cites [41] [41] Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-aware data mining. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08 . ACMPress, New York, New York, USA, 560. https://doi.org/10.1145/1401890.1401959 - source
Not checked in the free run.
- cites [61] [61] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2015. Fairness Constraints: Mechanisms for Fair Classification. Fairness, Accountability, and Transparency in Machine Learning (jul 2015). arXiv:1507.05259 http://arxiv.org/abs/1507.05259 - source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Premisenone given§ 1 INTRODUCTION · p. 2The big challenge in introducing non-discriminatory supervised learning algorithms is preventing direct discrimination without inducing indirect discrimination while enabling the necessity of businesses to maximize model accuracy.
“The big challenge in introducing non-discriminatory supervised learning algorithms is preventing direct discrimination without inducing indirect discrimination while enabling the necessity of businesses to maximizing model accuracy.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone given§ 4 LEARNING FAIR AND EXPLAINABLE MODELS · p. 6L_MDE(X) or L_SHAP(X) lead to a simple closed-form solution.
“We seek models ^ 𝑌 that remove the influence of the protected attributes 𝒁 , while preserving the influence of non-protected attributes 𝑿 by minimizing 𝐿 MDE ( 𝑿 ) or 𝐿 SHAP ( 𝑿 ) , which lead to a simple closed-form solution, or their feature-specific versions, i.e., 𝐿 IND MDE ( 𝑿 ) or 𝐿 IND SHAP ( 𝑿 ) , which we solve via transfer learning.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 7The MCM can be applied to continuous Z and it results in more accurate models.
“Thanks to this, the MCM can be applied to continuous 𝑍 and it results in more accurate models.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The causal influence of the protected attribute Z on decisions Y and the mediator M is assumed unfair and all other influences are fair.
May combine two claims.
Interpretationreasoning§ 5.1 Evaluated learning methods · p. 8There exist multiple supervised learning methods for addressing discrimination, but they are often mutually exclusive.
“Thus, there exist multiple supervised learning methods for addressing discrimination, but they are often mutually exclusive.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisecitation§ 3.3 Problem formulation based on input influence measures · p. 5Prior works suggest making a probabilistic intervention on a feature by replacing it with an independent random variable.
“To measure the influence of a certain feature 𝑊 𝑖 prior works suggest to make a probabilistic intervention on that variable by replacing it with an independent random variable [13, 23, 34].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [13] [13] Anupam Datta, Shayak Sen, and Yair Zick. 2016. Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems. Proceedings - 2016 IEEE Symposium on Security and Privacy, SP 2016 (2016), 598617. https://doi.org/10.1109/SP.2016.42 - source
Not checked in the free run.
- cites [23] [23] Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. 2019. Feature relevance quantification in explainable AI: A causal problem. 2015 (oct 2019). arXiv:1910.13413 http://arxiv.org/abs/1910.13413 - source
Not checked in the free run.
- cites [34] [34] Scott M. Lundberg and Su In Lee. 2017. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 2017-Decem, Section 2 (2017), 4766-4775. arXiv:1705.07874 - source
Not checked in the free run.
May combine two claims.
Interpretationnone given§ 3 PROBLEM FORMULATION OF FAIR AND EXPLAINABLE LEARNING · p. 3The proposed method allows addition of features to the model that increase model predictiveness, while preventing them from being used inadvertently as proxies for the protected features.
“This method allows addition of features to the model that increase model predictiveness, while preventing them from being used inadvertently as proxies for the protected features.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisecitation§ 4.3 Removal of indirect discrimination via nested use of proposed methods · p. 6Causality literature posits a causal hierarchy and distinguishes between interventional and counterfactual estimates.
“Causality literature posits a causal hierarchy and distinguishes between interventional and counterfactual estimates [40].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [40] [40] Judea Pearl, Madelyn Glymour, and Nicolas P. Jewell. 2016. Causal Inference in Statistics: A Primer . -
Not checked in the free run.
May combine two claims.
Interpretationreasoning§ 5.3 Real-world datasets · p. 9The changes in influence for the features most correlated with the protected attribute indicate that these methods induce indirect discrimination during training, despite having better performance for certain fairness measures.
“As with the synthetic results, the changes in influence for the features most correlated with the protected attribute indicate that these methods induce indirect discrimination during training, despite having better performance for certain fairness measures.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ 6 LIMITATIONS AND FUTURE WORK · p. 9In adversarial scenarios, our approach may experience the limitations of other discrimination preventing methods where satisfying a specified fairness objective still leads to discrimination.
“In such adversarial scenarios, our approach may experience the limitations of other discrimination preventing methods where satisfying a specified fairness objective still leads to discrimination.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone given§ 6 LIMITATIONS AND FUTURE WORK · p. 10If a company develops a model of Y using X and some Xi is unfairly influenced, omitting the step of first applying the method to a model of Xi would result in indirect discrimination.
“An omission of the first step, whether intentional or unintentional, would result in indirect discrimination.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
The purpose of business is to develop a model that on deployment is the most predictive possible.
“It is reasonable to assume that the purpose of business is to develop a model that on deployment is the most predictive possible.”
The authors state this as an assumption.
There are direct causal links from X and Z to Y.
“We assume that there are direct causal links from 𝑿 and 𝒁 to 𝑌 .”
The authors state this as an assumption.
There are no unmeasured confounders.
“Finally, it is assumed that there are no unmeasured confounders.”
The authors state this as an assumption.
The causal influence of the protected attribute Z on decisions Y and the mediator M is assumed unfair and all other influences are fair.
“The causal influence of the protected attribute 𝑍 on decisions 𝑌 and the mediator 𝑀 is assumed unfair and all other influences are fair.”
The authors state this as an assumption.
We have enough samples to have perfect estimates of a well-specified model's parameters.
“Without loss of generality, let us consider a scenario where we have enough samples to have perfect estimates of a well-specified model's parameters, so that the estimated model is ^ 𝑚 = 𝜃 𝑚 + 𝜃 𝑚 𝑧 𝑧 + 𝜃 𝑐 𝑚 𝑐 .”
The paper relies on this without support.
Each variable can contain exogenous noise, i.e., unobserved intrinsic noise not associated with any other variable.
“However, each variable can contain exogenous noise, i.e., unobserved intrinsic noise not associated with any other variable.”
The authors state this as an assumption.
The function y(x,z) is analytic (can be expanded into a Taylor series).
“Assuming that the function 𝑦 ( 𝑥, 𝑧 ) is analytic, we can expand 𝑦 ( 𝑥, 𝑧 ) into a Taylor series around the point ( 𝑥 = 0 , 𝑧 = 0 )”
The paper relies on this without support.
X', Z', and Z'' have the same means as X and Z, respectively.
“Since 𝑋 ′ , 𝑍 ′ , and 𝑍 ′′ have the same means as 𝑋 and 𝑍 , respectively, so the first derivative can be simplified as follows,”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The introduction of SHAP values has seen an increase in attention from fairness researchers.
“This is a relatively under-addressed area in the fairness literature, perhaps due to the introduction of SHAP valuesfoot_3 [203], which has seen an increase in attention from fairness researchers (e.g.[126]).”
From Fairness in machine learning: A survey (2024) · cites this paper as [126]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Taylor-series proof sketch omits zero-degree terms
MinorThe proof sketch says both exponents start at 1, whereas the appendix states that they start at 0. Starting at 1 omits constant terms and terms involving only one variable.
“The series is a sum of components 𝐶𝑥 𝑘 𝑧 𝑙 , where 𝐶 is a constant and 𝑘 and 𝑙 are integers from 1 to ∞ .”
“which is a series of components 𝐶𝑥 𝑘 𝑧 𝑙 , where 𝐶 is a real-valued constant and 𝑘 and 𝑙 are integers from 0 to ∞ .”
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Permutation importance: a corrected feature importance measure
Altmann, A., Toloşi, L., Sander, O., & Lengauer, T. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq134 · Cited by 2,903 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581 · see citing works
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305 · see citing works
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · see citing works
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375830 · Cited by 900 · see citing works
2020 International Joint Conference on Neural Networks (IJCNN)
(2020) · doi:10.1109/ijcnn48605.2020 · Cited by 786 · see citing works
Datta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728 · see citing works
Discrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · see citing works
How to Tell When Simpler, More Unified, or LessAd HocTheories will Provide More Accurate Predictions
FORSTER, M., & Sober, E. R. (1994) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/45.1.1 · Cited by 661 · see citing works
Bickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975) · Science · doi:10.1126/science.187.4175.398 · Cited by 659 · see citing works
Fairness Constraints: Mechanisms for Fair Classification
Zafar, M. B., Valera, I., Rodriguez, M. G., & Gummadi, K. P. (2015) · arXiv · doi:10.48550/arxiv.1507.05259 · Cited by 645 · see citing works
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · cited 1× · see citing works
“Machine learning models are developed in best faith to maximize accuracy, but often are not causal and not robust to covariate shifts [43, 44], i.”
BackgroundOn Fairness and CalibrationPreprint
Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J. M., & Weinberger, K. Q. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.02012 · Cited by 509 · see citing works
A Reductions Approach to Fair ClassificationPreprint
Agarwal, A., Beygelzimer, A., Dudı́k, M., Langford, J. C., & Wallach, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.02453 · Cited by 364 · see citing works
Avoiding Discrimination through Causal Reasoning
Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., & Schölkopf, B. (2017) · arXiv · doi:10.48550/arxiv.1706.02744 · Cited by 358 · see citing works
Responsibility and Blame: A Structural-Model Approach
Chockler, H., & Halpern, J. Y. (2004) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1391 · Cited by 325 · see citing works
IJCAI-83: International joint conference on artificial intelligence
(1982) · Artificial Intelligence · doi:10.1016/0004-3702(82)90046-7 · Cited by 317 · see citing works
Causal discovery and inference: concepts and recent methodological advances
Spirtes, P., & Zhang, K. (2016) · Applied Informatics · doi:10.1186/s40535-016-0018-x · Cited by 304 · see citing works
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · see citing works
Understanding Global Feature Contributions With Additive Importance MeasuresPreprint
Covert, I., Lundberg, S., & Lee, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00668 · Cited by 237 · see citing works
Lipton, Z. C., & Steinhardt, J. (2019) · Queue · doi:10.1145/3317287.3328534 · Cited by 231 · see citing works
Consumer-Lending Discrimination in the FinTech Era
Bartlett, R. P., Morse, A., Stanton, R., & Wallace, N. E. (2019) · National Bureau of Economic Research · doi:10.3386/w25943 · Cited by 213 · see citing works
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207 · see citing works
Feature relevance quantification in explainable AI: A causal problem
Janzing, D., Minorics, L., & Blöbaum, P. (2019) · arXiv · doi:10.48550/arxiv.1910.13413 · Cited by 207 · see citing works
Fairness in Decision-Making — The Causal Explanation Formula
Zhang, J., & Bareinboim, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11564 · Cited by 202 · see citing works
Learning Non-Discriminatory Predictors
Woodworth, B., Gunasekar, S., Ohannessian, M. I., & Srebro, N. (2017) · arXiv · doi:10.48550/arxiv.1702.06081 · Cited by 182 · see citing works
Redlining Revisited: Mortgage Lending Patterns in Sacramento 1930–2004
Hernández, J. F. (2009) · International Journal of Urban and Regional Research · doi:10.1111/j.1468-2427.2009.00873.x · Cited by 178 · see citing works
Racial Discrimination and Redlining in Cities
Zénou, Y., & Boccard, N. (2000) · Journal of Urban Economics · doi:10.1006/juec.1999.2166 · Cited by 151 · see citing works
A Causal Framework for Discovering and Removing Direct and Indirect Discrimination
Zhang, L., Wu, Y., & Wu, X. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/549 · Cited by 144 · see citing works
An Investigation of Why Overparameterization Exacerbates Spurious CorrelationsPreprint
Sagawa, S., Raghunathan, A., Koh, P. W., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04345 · Cited by 132 · see citing works
Lippert-Rasmussen, K. (2006) · Ethical Theory and Moral Practice · doi:10.1007/s10677-006-9014-x · Cited by 126 · see citing works
From Parity to Preference-based Notions of Fairness in ClassificationPreprint
Zafar, M. B., Valera, I., Gomez-Rodriguez, M., Gummadi, K. P., & Weller, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00010 · Cited by 121 · see citing works
Affinity Profiling and Discrimination by Association in Online Behavioural AdvertisingPreprint
Wachter, S. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3388639 · Cited by 85 · see citing works
Grgić-Hlača, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · arXiv · doi:10.48550/arxiv.1802.09548 · Cited by 75 · see citing works
PC-Fairness: A Unified Framework for Measuring Causality-based FairnessPreprint
Wu, Y., Zhang, L., Wu, X., & Tong, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12586 · Cited by 51 · see citing works
General Fair Empirical Risk Minimization
Oneto, L., Donini, M., & Pontil, M. (2019) · IEEE International Joint Conference on Neural Network · doi:10.1109/ijcnn48605.2020.9206819 · Cited by 50 · see citing works
Learning Optimal Fair Policies
Nabi, R., Malinsky, D., & Shpitser, I. (2018) · PubMed · doi:10.48550/arxiv.1809.02244 · Cited by 43 · see citing works
Fairness and Transparency in the Age of the Algorithm
Olhede, S. C., & Rodrigues, R. (2017) · Significance · doi:10.1111/j.1740-9713.2017.01012.x · Cited by 24 · see citing works
Path-Specific Counterfactual Fairness
Chiappa, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017801 · Cited by 16 · see citing works
Disentangling Influence: Using Disentangled Representations to Audit Model PredictionsPreprint
Marx, C. T., Phillips, R. L., Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08652 · Cited by 16 · see citing works
Shafer, G. R. (2002) · Studies in fuzziness and soft computing · doi:10.1007/978-3-7908-1792-8_23 · Cited by 13 · see citing works
Capuchin: Causal Database Repair for Algorithmic FairnessPreprint
Salimi, B., Rodriguez, L., Howe, B., & Suciu, D. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08283 · Cited by 12 · see citing works
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for SupportPreprint
Madaio, M., Egede, L., Subramonyam, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.05675 · Cited by 12 · see citing works
Optimization Hierarchy for Fair Statistical Decision ProblemsPreprint
Aswani, A. J., & Olfat, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.08520
Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data
Constantinou, A. C., Liu, Y., Chobtham, K., Guo, Z., & Kitson, N. K. (2020) · arXiv · doi:10.48550/arxiv.2005.09020
Flagged first, then by citations
Research works printed but not linked: 13
- Andrew Altman. 2016. Discrimination. In The Stanford Encyclopedia of Philosophy(2016 ed.), Edward N Zalta (Ed.). Metaphysics Research Lab, Stanford University.
- 10.1093/bioinformatics · DOI
- A Philip Dawid. 2008. Beware of the DAG!JMLR: Workshop and Conference Proceedings 6 (2008), 59–86.
- Michele Donini Luca Oneto Shai Ben-David John Shawe-Taylor and Massimiliano Pontil. 2018. Empirical risk minimization under fairness constraints. Advances in Neural Information Processing Systems 2018-Decem NeurIPS(2018) 2791–2801.
- 10.1093/bjps · DOI
- Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. Pro Publica (2016). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
- 10.1007/sl0677-006-9014-x · DOI
- Zachary C. Lipton Alexandra Chouldechova and Julian McAuley. 2018. Does mitigating ML’s impact disparity require treatment disparity?Advances in Neural Information Processing Systems 2018-Decem ML(2018) 8125–8135.
- 10.5555/1642718 · DOI
- Ilya Shpitser. 2015. Segregated graphs and marginals of chain graph models. Advances in Neural Information Processing Systems 2015-Janua (2015), 1720–1728.
- Kwame Ture, Charles V Hamilton, and Stokely Carmichael. 1968. Black power: The politics of liberation in America: With new afterwords by the authors. Vintage Books.
- Margery Austin Turner and Felicity Skidmore. 1999. Mortgage Lending Discrimination : A Review of Existing Evidence Lending Discrimination : A Review of existing Evidence. In The Urban Institute. 1–176.
- Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2017. Fairness Constraints: Mechanisms for Fair Classification. Artificial Intelligence and Statistics 54 (2017). arxiv:1507.05259https://arxiv.org/abs/1507.05259
Cited by
Sorted by citations
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Fairness and Diversity in Recommender Systems: A Survey
Zhao, Y., Wang, Y., Liu, Y., Cheng, X., Aggarwal, C. C., & Derr, T. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3664928 · Cited by 121 · see citing works
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 128 · see citing works
WEIRD FAccTs: How Western, Educated, Industrialized, Rich, and Democratic is FAccT?
Septiandri, A. A., Constantinides, M., Tahaei, M., & Quercia, D. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593985 · Cited by 58 · see citing works
Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models
Balkır, E., Kiritchenko, S., Nejadgholi, I., & Fraser, K. (2022) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2022.trustnlp-1.8 · Cited by 43 · see citing works
A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Wang, Z., Huang, C., & Yao, X. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3702004 · Cited by 32 · see citing works
A review of causality-based fairness machine learning
Su, C., Yu, G., Wang, J., Yan, Z., & Cui, L. (2022) · Intelligence & Robotics · doi:10.20517/ir.2022.17 · Cited by 24 · see citing works
Black, E., Naidu, R. C. A., Ghani, R., Rodolfa, K. T., Ho, D. E., & Heidari, H. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623259 · Cited by 19 · see citing works
Fix Fairness, Don’t Ruin Accuracy: Performance Aware Fairness Repair using AutoML
Nguyen, G., Biswas, S., & Rajan, H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616257 · Cited by 15 · see citing works
FairXAI - A Taxonomy and Framework for Fairness and Explainability Synergy in Machine Learning
Ramachandranpillai, R., Baeza-Yates, R. A., & Heintz, F. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3528321 · Cited by 15 · see citing works
Discrimination-Aware Process Mining: A Discussion
Pohl, T., Qafari, M. S., & van der Aalst, W. M. P. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-27815-0_8 · Cited by 9 · see citing works
Ruggieri, S., Álvarez, J. M., Pugnana, A., State, L., & Turini, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26798 · Cited by 9 · see citing works
How fair is machine learning in credit lending?
Babaei, G., & Giudici, P. (2024) · Quality and Reliability Engineering International · doi:10.1002/qre.3579 · Cited by 8 · see citing works
Fairness in Algorithmic Profiling: The AMAS Case
Achterhold, E., Mühlböck, M., Steiber, N., & Kern, C. A. (2025) · Minds and Machines · doi:10.1007/s11023-024-09706-9 · Cited by 4 · see citing works
Legal NLP in India: a comprehensive survey of tasks, challenges, and future directions
Narzary, M., Singh, P. K., & Brahma, M. (2025) · AI & Society · doi:10.1007/s00146-025-02373-z · Cited by 4 · see citing works
Detection of Credit Card Fraud with Machine Learning Methods and Resampling Techniques
Rahmatullah, M. B. S., Hanani, A. L. S., Naim, A. M., Sari, Z., & Azhar, Y. (2022) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v6i6.4213 · Cited by 4 · see citing works
Explainable Global Fairness Verification of Tree-Based Classifiers
Calzavara, S., Cazzaro, L., Lucchese, C., & Marcuzzi, F. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00011 · Cited by 3 · see citing works
Bias Mitigation Methods: Applicability, Legality, and Recommendations for Development
Waller, M., Rodrigues, O., Lee, M. S. A., & Cocarascu, O. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.16759 · Cited by 3 · see citing works
Discrimination Induced by Algorithmic Recourse Objectives
Perello, N., Cousins, C., Zick, Y., & Grabowicz, P. A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732110 · Cited by 2 · see citing works
Self-citationFairSense: Long-Term Fairness Analysis of ML-Enabled Systems
She, Y., Biswas, S., Kästner, C., & Kang, E. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00159 · Cited by 2 · see citing works
Explainability, fairness and the Simpson’s paradox in credit lending
Babaei, G., Giudici, P., & Neelakantan, P. (2025) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2025.131030 · Cited by 2 · see citing works
Explainable Fairness and Propensity Score Matching
Giudici, P., & Babaei, G. (2025) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.70047 · Cited by 2 · see citing works
Pelegrina, G. D., Couceiro, M., & Duarte, L. T. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658905 · Cited by 2 · see citing works
FairCR – An Evaluation and Recommendation System for Fair Classification Algorithms
Lässig, N., Herschel, M., & Nies, O. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00431 · Cited by 2 · see citing works
Pelegrina, G. D., Siraj, S., Duarte, L. T., & Michel Grabisch, B. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109427 · Cited by 2 · see citing works
Learning from Discriminatory Training Data
Grabowicz, P. A., Perello, N., & Takatsu, K. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604710 · Cited by 2 · see citing works
Self-citationConditional Fairness, with Application to Credit LendingPreprint
Babaei, G., & Neelakantan, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4814600 · Cited by 1 · see citing works
Explainable Fairness, with Application to Credit LendingPreprint
Stefano, P. G., Babaei, G., & Neelakantan, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5082737 · Cited by 1 · see citing works
Leveraging Group Contrastive Explanations for Handling Fairness
Castelnovo, A., Inverardi, N., Malandri, L., Mercorio, F., Mezzanzanica, M., & Seveso, A. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44070-0_17
Lässig, N., & Herschel, M. (2025) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65448.2025.00096
A structural model to explain unfairness
Giudici, P., Neelakantan, P., Pavarana, S., & Schmidt, T. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228180
The Future Perspective of Human–AI Communication
Geroimenko, V. (2026) · Human-computer interaction series · doi:10.1007/978-3-032-21689-2_8
GESD: Beyond Outcome-Oriented Fairness
Popoola, G., & Sheppard, J. (2026) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai68641.2026.11536479
On the Interplay of Explainability and Fairness in AI: A Survey
Fragkathoulas, C., Papanikou, V., Karidi, D. P., & Pitoura, E. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3706635
Are bank decisions fair? a statistical analysis of fairness in credit lending
Giudici, P., Neelakantan, P., Pavarana, S., & Schmidt, T. (2026) · Statistics · doi:10.1080/02331888.2026.2700452
Racismo Algorítmico em Sistemas de Inteligência Artificial: Uma Revisão Sistemática da Literatura
Campos, A. M. D. S., dos Santos, N. D. S., da Silva, D. T. A., & Cunha, L. D. S. B. (2026) · Anais do Workshop sobre as Implicações da Computação na Sociedade (WICS) · doi:10.5753/wics.2026.23828
- Authors
- Grabowicz, P. A., Perello, N., & Mishra, A.
- Affiliations
- University of Massachusetts Amherst (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 1905-1916
- OpenAlex
- W4226346920
- Cited by
- 37
- References
- 31
- Citations by year
- 2026: 5 · 2025: 10 · 2024: 9 · 2023: 9 · 2022: 4
- FWCI
- 7.95
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- DARPA (W911NF-20-2-0005)
- Volkswagen Foundation (92136)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, direct discrimination, protected characteristics, causal inference, SHAP, explainability, post-processing, reverse discrimination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
