CoChat
Sign Up
Cited by 146Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Privacy-Preserving Technologies in Data

Interventional Fairness: Causal Database Repair for Algorithmic Fairness

Open access
Salimi, Babak, Rodriguez, Luke, Howe, Bill, Suciu, Dan·2019·Proceedings of the 2019 International Conference on Management of Data
DOI 10.1145/3299869.3319901
Open this paper in CoChat — free accountOpen PDF

Abstract

Fairness is increasingly recognized as a critical component of machine learning systems. However, it is the underlying data on which these systems are trained that often reflect discrimination, suggesting a database repair problem. Existing treatments of fairness rely on statistical correlations that can be fooled by statistical anomalies, such as Simpson's paradox. Proposals for causality-based definitions of fairness can correctly model some of these situations, but they require specification of the underlying causal models. In this paper, we formalize the situation as a database repair problem, proving sufficient conditions for fair classifiers in terms of admissible variables as opposed to a complete causal model. We show that these conditions correctly capture subtle fairness violations. We then use these conditions as the basis for database repair algorithms that provide provable fairness guarantees about classifiers trained on their training labels. We evaluate our algorithms on real data, demonstrating improvement over the state of the art on multiple fairness metrics proposed in the literature while retaining high utility.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper formalizes algorithmic fairness as a causal database repair problem, introducing “justifiable fairness” based on admissible variables rather than a full causal model. It proves sufficient conditions for fair classifiers and develops repair algorithms that improve fairness metrics on real data while retaining high utility.

Key findings

  • The paper defines justifiable fairness and proves that if all directed paths from protected attribute \(S\) to outcome \(O\) go through admissible attributes \(A\), the algorithm is justifiably fair; under faithfulness, the converse also holds.
  • It gives a data-testable sufficient condition: any reasonable classifier trained on \(X \subseteq V\) is justifiably fair if either \(Pr\) satisfies \((Y \perp X \cap I \mid X \cap A)\), or \(X \supseteq A\) and \(Pr\) satisfies the saturated CI \((Y \perp I \mid A)\).
  • On Adult data, the MaxSAT soft-repair approach removed almost 50% of the bias while accuracy decreased by only 1%; the MaxSAT approach also removed up to 80% fewer tuples than a naive lineage-based repair.
  • All Capuchin repair methods reduced the Ratio of Observational Discrimination (ROD) on Adult and COMPAS; Capuchin outperformed Feldman et al. and Calmon et al. in utility, and all Capuchin methods outperformed Calmon’s algorithm significantly.

Methods

  • Design: a pre-processing/database-repair framework that takes a protected attribute, outcome, and admissible/inadmissible variable partition, then repairs training data to enforce a saturated conditional independence without requiring a complete causal model.
  • Data: Adult dataset (10 attributes, 48k rows) and COMPAS recidivism dataset (7 attributes, 7k rows), plus binned versions for comparison with prior methods.
  • Analysis: repairs are reduced to multivalued dependency repair and solved via weighted MaxSAT, matrix factorization, and Independent Coupling; evaluation uses 5-fold cross-validation with Linear Regression, Multi-layer Perceptron, and Random Forest, reporting accuracy and fairness metrics including ROD, DP, TPRB, TNRB, CDP, CTPB, and CTNB.

Limitations

  • The paper states that interventional fairness is too restrictive in practice, motivating justifiable fairness; it also states that repairing a database for a single CI is NP-hard, and scalability depends on MaxSAT solvers and matrix-factorization approximations.
  • The paper states that it focuses on a single CI and leaves extension to multiple CIs to future work.
  • The paper states that MaxSAT and matrix-factorization repairs satisfy the needed condition only approximately, giving slightly weaker fairness guarantees on unseen test data, while Independent Coupling satisfies it by construction; residual bias is expected because the classifier is only an approximation and test data are not repaired.
  • Evident from the text: experiments are limited to Adult and COMPAS datasets, and the comparison with prior methods uses binned data with restricted attributes, which the authors note makes bias and utility results mismatched with the main experiments.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3299869.3319901

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 38 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The paper's algorithms capture fairness situations other approaches cannot.

    “We find that our algorithms not only capture fairness situations other approaches cannot, but that they outperform the existing stateof-the-art pre-processing approaches even on other fairness metrics for which our algorithms were not necessarily designed.”
    Quote in the paperSubstantial claim
  • Findingown data

    The paper's algorithms outperform existing state-of-the-art pre-processing approaches even on other fairness metrics for which they were not necessarily designed.

    “We find that our algorithms not only capture fairness situations other approaches cannot, but that they outperform the existing stateof-the-art pre-processing approaches even on other fairness metrics for which our algorithms were not necessarily designed.”
    Quote in the paperSubstantial claim
  • Findingown data

    Dropping even one inadmissible variable can negatively affect the accuracy of the classifier.

    “as we will show in Sec.6, dropping even one inadmissible variable can negatively affect the accuracy of the classifier.”
    Quote in the paperSubstantial claim
  • Findingown data

    MaxSAT and MF approaches maintain a significantly better balance between accuracy and fairness.

    “Nevertheless, we empirically show in Sec 6 that MaxSAT and MF approaches maintain a significantly better balance between accuracy and fairness.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    The ROD in training labels became insignificant for COMPAS.

    “Because COMPAS data was excessively binned, the ROD in training labels became insignificant for COMPAS, and accuracy dropped by 2%.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 35
  • Classified as probably not a substantial claim. 29
  • The quote only partly states this claim. 2

  • May combine two claims.

    Findingown data

    The MaxSAT-based algorithm can remove almost 50% of the discrimination while decreasing accuracy by only 1% on adult data.

    “For example, our most flexible algorithm, which involves a reduction to MaxSAT, can remove almost 50% of the discrimination while decreasing accuracy by only 1% on adult data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The paper's algorithms improve over the state of the art on multiple fairness metrics proposed in the literature while retaining high utility.

    “We evaluate our algorithms on real data, demonstrating improvement over the state of the art on multiple fairness metrics proposed in the literature while retaining high utility.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The paper's repair algorithms can mitigate discrimination as well as prohibitively aggressive approaches such as dropping all inadmissible variables from the training set, while maintaining high accuracy.

    “Surprisingly, our results show that our repair algorithms can mitigate discrimination as well as prohibitively aggressive approaches, such as dropping all inadmissible variables from the training set, while maintaining high accuracy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The residual bias generally favored the protected group, as opposed to the bias in the original data.

    “We also note that the residual bias generally favored the protected group (as opposed to the bias in the original data).”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The potential influence of gender on income through some or all of the admissible variables is fair; the direct influence of gender on income, as well as its indirect influence on income through marital status, are discriminatory.

  • May combine two claims.

    Findingown data

    When partially repairing data using MS(Soft), almost 50% of the bias was removed while accuracy decreased by only 1%.

    “These repair methods delivered surprisingly good results: when partially repairing data using the MaxSAT approach, i.e, using MS(Soft), almost 50% of the bias was removed while accuracy decreased by only 1%.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The potential influence of gender on income through some or all of the admissible variables is fair; the direct influence of gender on income, as well as its indirect influence on income through marital status, are discriminatory.

  • May combine two claims.

    Findingown data

    The MaxSAT approach achieved better classification accuracy and balanced tuple insertion and deletion.

    “Further, it achieved better classification accuracy, and it balanced tuple insertion and deletion.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The potential influence of gender on income through some or all of the admissible variables is fair; the direct influence of gender on income, as well as its indirect influence on income through marital status, are discriminatory.

  • May combine two claims.

    Findingown data

    The repaired data improved associational fairness measures; the Capuchin repair methods had an effect similar to dropping the sensitive and inadmissible variables completely, but they delivered much higher accuracy.

    “As shown in Fig. 8 (see also Fig 15 in the appendix), the repaired data also improved associational fairness measures: the Capuchin repair methods had an effect similar to dropping the sensitive and inadmissible variables completely, but they delivered much higher accuracy (because the CI was enforced approximately).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The potential influence of gender on income through some or all of the admissible variables is fair; the direct influence of gender on income, as well as its indirect influence on income through marital status, are discriminatory.

  • May combine two claims.

    Findingown data

    Both reference algorithms enforced DP at the cost of increasing ROD.

    “We observe that both reference algorithms enforced DP at the cost of increasing ROD; however, in some cases the introduced bias was not statistically significant.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Partitioning had no effect on MF and IC on a single-core machine, but it sped up MaxSAT.

    “While partitioning had no effect on MF and IC on a single-core machine, as shown in Fig 12(b), it sped up MaxSAT .”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The potential influence of gender on income through some or all of the admissible variables is fair; the direct influence of gender on income, as well as its indirect influence on income through marital status, are discriminatory.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    ROD is sensitive to the choice of a context A_b = a_b by design.

    “ROD is sensitive to the choice of a context A b = a b by design.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Causality-based definitions of fairness can correctly model some situations but require specification of the underlying causal models.

    “Proposals for causality-based definitions of fairness can correctly model some of these situations, but they require specification of the underlying causal models.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given

    Existing treatments of fairness rely on statistical correlations that can be fooled by statistical anomalies such as Simpson's paradox.

    “Existing treatments of fairness rely on statistical correlations that can be fooled by statistical anomalies, such as Simpson's paradox.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    The paper's definition of fairness is based solely on the notion of intervention and can be guaranteed even in the absence of causal models.

    “Unlike previous measures of fairness based on causality [24,34,43], which require the presence of the underlying causal model, our definition is based solely on the notion of intervention [36] and can be guaranteed even in the absence of causal models.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [24]

      Not checked in the free run.

    • cites [34]

      Not checked in the free run.

    • cites [43]

      Not checked in the free run.

    • cites [36]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The protected attribute is frequently represented implicitly by some combination of proxy variables, so a classifier trained without it still learns discrimination reflected in training data.

    “However, since the protected attribute is frequently represented implicitly by some combination of proxy variables, the classifier still learns the discrimination reflected in training data.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [21]

      Not checked in the free run.

    • cites [46] source

      Not checked in the free run.

    • cites [4]

      Not checked in the free run.

    • cites [53]

      Not checked in the free run.

    • cites [12]

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The user need only distinguish admissible and inadmissible attributes, and this information is sufficient to mitigate discrimination.

    “The user need only distinguish admissible and inadmissible attributes; we prove that this information is sufficient to mitigate discrimination.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Only users can identify the classification of variables into admissible and inadmissible.

  • May combine two claims.

    Premisecitation

    A classifier trained without regard to the protected attribute fails to remove discrimination and can complicate detection and mitigation of discrimination downstream via in-processing or post-processing techniques.

    “As a result, a classifier trained without regard to the protected attribute not only fails to remove discrimination, but it can complicate the detection and mitigation of discrimination downstream via in-processing or post-processing techniques [9,10,18,24,25,34,43,51], such as those we describe next.”
    Quote in the paperSubstantial claim0 of 8 sources checked
    • cites [9] source

      Not checked in the free run.

    • cites [10] source

      Not checked in the free run.

    • cites [18] source

      Not checked in the free run.

    • cites [24]

      Not checked in the free run.

    • cites [25]

      Not checked in the free run.

    • cites [34]

      Not checked in the free run.

    • cites [43]

      Not checked in the free run.

    • cites [51] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In the 1973 UC Berkeley admissions case, the overall lower female admission rate was explained by females applying to departments with lower acceptance rates, and department-level analysis showed a slight bias toward female applicants.

    “It turned out that females tended to apply to departments with lower overall acceptance rates [44].When broken down by department, a slight bias toward female applicants was observed, a result that did not constitute evidence for gender-based discrimination.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Justifiable fairness subsumes and improves on several previous definitions.

    “Justifiable fairness subsumes and improves on several previous definitions and can correctly distinguish fairness violations and non-violations that would otherwise be hidden by statistical coincidences, such as Simpson's paradox.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The condition in Theorem 3.7 can be checked without knowing the causal DAG, but requires the computation of the Markov boundary; it is expressed in terms of the outcome O of the algorithm.

    “The condition in Theorem 3.7 can be checked without knowing the causal DAG, but requires the computation of the Markov boundary; moreover, it is expressed in terms of the outcome O of the algorithm.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    In terms of the causal DAG, their approach can be seen as modifying the underlying causal model to enforce the fairness constraint.

    “In terms of the causal DAG, this approach can be seen as modifying the underlying causal model to enforce the fairness constraint.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Justifiable fairness correctly identifies College I as discriminatory and College II as fair, unlike previous definitions of fairness.

    “To summarize, unlike previous definitions of fairness, justifiable fairness correctly identifies College I as discriminatory and College II as fair.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The algorithm in College II satisfies demographic parity and proxy fairness but fails to satisfy conditional statistical parity.

    “The algorithm satisfies demographic parity and proxy fairness but fails to satisfy conditional statistical parity since Pr(A = 1|G = 1, D = 'A') = 0.8 but Pr(A = 1|G = 0, D = 'A') = 0.2).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Interventional fairness is a more fine-grained notion than proxy fairness.

    “Therefore, interventional fairness is a more fine-grained notion than proxy fairness.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Theorem 3.7 is proved without assuming faithfulness.

    “but we prove it without assuming faithfulness in Appendix 8.2.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    In practice, interventional fairness is too restrictive.

    “In practice, interventional fairness is too restrictive, as we show below.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Condition (a) is the weaker assumption, while condition (b) has the advantage that the CI is saturated.

    “While condition (a) is the weaker assumption, condition (b) has the advantage that the CI is saturated.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Interventional fairness is the strongest notion of fairness that is testable from data, yet correctly captures group-level fairness.

    “In fact, we argue that interventional fairness is the strongest notion of fairness that is testable from data, yet correctly captures group-level fairness.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Any reasonable classifier trained on a set of variables X ⊆ V is justifiably fair w.r.t. a protected attribute S, if either (a) Pr satisfies the CI (Y ⊥ X ∩ I | X ∩ A), or (b) X ⊇ A and Pr satisfies the saturated CI (Y ⊥ I | A).

    “Corollary 3.8.Fix a training data D, Pr, where Y ∈ V is the training label, and A, I are admissible and inadmissible attributes.Then any reasonable classifier trained on a set of variables X ⊆ V is justifiably fair w.r.t. a protected attribute S, if either: (a) Pr satisfies the CI (Y ⊥ ⊥X ∩ I|X ∩ A), or (b) X ⊇ A and Pr satisfies the saturated CI (Y ⊥ ⊥I |A).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The algorithm is a good prediction of the response variable, i.e., Pr(Y = 1|X = x) ≈ Pr(O = 1|X = x) (a reasonable classifier).

  • May combine two claims.

    Premisecitation

    Identifying path-specific causality from data requires very strong assumptions and is often impractical.

    “however, identifying path-specific causality from data requires very strong assumptions and is often impractical [2].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The paper's definition correctly classifies the College II example as justifiably fair.

    “One can check that the algorithm is justifiably fair, and thus our definition also correctly classifies this example; for example, {D}fairness follows from Eq.( 2):”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    In Example 2.3, the admission process is proxy-fair but individually unfair and group-level unfair.

    “This admission process is proxy-fair 3 , because”
    • Also stated in: § 2.3 Causal Fairness · boosted
      “On the other hand, it is clearly individually-unfair, in fact it is grouplevel unfair (for all applicants to the same department).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    They consider the saturated CI (Y ⊥ I | A) as an integrity constraint that should always hold in training data D, Pr.

    “We consider the saturated CI (Y ⊥ ⊥I|A) as an integrity constraint that should always hold in training data D, Pr.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Interventional fairness correctly captures group-level fairness, unlike proxy fairness.

    “Unlike proxy fairness, this notion correctly captures group-level fairness, because it ensures that S does not affect O in any configuration of the system obtained by fixing other variables at some arbitrary values.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    Interventional fairness is based on the standard notion of intervention and hence is testable from the data, unlike counterfactual fairness.

    “Unlike counterfactual fairness, it does not attempt to capture fairness at the individual level, and therefore it uses the standard definition of intervention (the do-operator).”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    In the College I example, the admission process is discriminatory despite equal overall admission rates, and justifiable fairness correctly classifies it as unfair while proxy fairness does not.

    “Clearly, the admission process is discriminatory in this college because department A admits 80% of its male applicants and 20% of the female applicants, while department B admits 20% of male and 80% of female applicants.On the other hand, the admission rate for the entire college is the same 32% for both male and female applicants, falsely suggesting that the college is fair.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Adult data is inconsistent because its income attribute reports household income for married individuals, and there are more married males in the data.

    “However, it has been shown that Adult data is inconsistent [44] because its income attribute reports household income for married individuals, and there are more married males in data.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [44]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    They propose a new metric (ROD) for discovering evidence of potential discrimination from data that uses the causal framework but is applicable when only the Markov boundary admissibility is known.

    “Instead, we propose a new metric for discovering evidence of potential discrimination from data that uses the causal framework we described but is still applicable in situations where all we know is which attributes in the Markov boundary of O are admissible.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    They show how to modify (repair) the training data to enforce the CI and thus ensure that any reasonable classifier trained on it will be justifiably fair.

    “We show here how to modify (repair) the training data to enforce the CI and thus ensure that any reasonable classifier trained on it will be justifiably fair.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The algorithm is a good prediction of the response variable, i.e., Pr(Y = 1|X = x) ≈ Pr(O = 1|X = x) (a reasonable classifier).

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Focusing on a single CI suffices for many real world fairness applications.

    “In this paper we focused on a single CI, which suffices for many real world fairness applications.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    Their problem is to repair for a saturated CI, not an MVD, since that is what is required in Corollary 3.8.

    “However, our problem is to repair for a saturated CI, not an MVD, since that is what is required in Corollary 3.8.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    In many practical settings, we must make judgments about the fairness of an algorithm whose inputs are unknown, and we cannot assume access to an underlying causal DAG.

    “However, in many practical settings we must make judgments about the fairness of an algorithm whose inputs are unknown.We cannot assume access to an underlying causal DAG in these situations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    KL-divergence between Pr and Pr' is bounded by conditional mutual information I(X ⊥ Y | Z).

    “It is easy to see that KL-divergence between Pr and Pr ′ is bounded by conditional mutual information I (X⊥ ⊥Y|Z).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    Repairing to satisfy a CI is not a database constraint.

    “here we need to repair to satisfy a CI, which is not a database constraint.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The MaxSAT and MF approaches only approximately satisfy the condition, translating to slightly weaker fairness guarantees on unseen test data.

    “In contrast, the condition is only approximately satisfied by the MaxSAT and MF approaches, translating to slightly weaker fairness guarantees on unseen test data.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Several prior efforts in algorithmic fairness have reported gender discrimination in Adult data based on a strong statistical dependency between income and gender in favor of males.

    “Using this dataset, several prior efforts in algorithmic fairness have reported gender discrimination based on a strong statistical dependency between income and gender in favor of males [31,48,60].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [31] source

      Not checked in the free run.

    • cites [48]

      Not checked in the free run.

    • cites [60]

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    D does not satisfy φ if and only if D satisfies Q_φ, so the repair problem becomes modifying D to make Q_φ false.

    “It follows immediately that D ̸ |= φ iff D |= Q φ , and therefore the repair problem becomes: modify the database D to make Q φ false.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Justifiable fairness is the strongest notion of fairness that is testable from data.

    “We defined a new notion of fairness, called justifiable fairness, that addresses shortcomings of the previous definitions and argued that it is the strongest notion of fairness that is testable from data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Feldman's algorithm modifies each attribute so that the marginal distributions based on the subsets of the attribute with a given sensitive value are all equal.

    “Feldman's algorithm modifies each attribute so that the marginal distributions based on the subsets of the attribute with a given sensitive value are all equal.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    If Pr is a strictly positive distribution, then MB(V) is unique for all V ∈ V and can be learned from data in polynomial time.

    “It is known that if Pr is a strictly positive distribution (i.e., forall v ∈ Dom(V), P(v) > 0), then MB(V ) is unique for all V ∈ V and can be learned from data in polynomial time [32].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Under the faithfulness assumption, the Markov boundary of a node X in the causal graph consists of the parents of X, the children of X, and the parents of the children of X.

    “Note that under the faithfulness assumption, the Markov boundary of a node X in the causal graph consists of the parents of X , the children of X , and the parents of the children of X [35].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35]

      Not checked in the free run.

    • The authors state this as an assumption: The probability distribution of A is faithful to the causal DAG.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Calmon's algorithm randomly transforms all variables except for the sensitive attribute to reduce the dependence between training labels and the sensitive attribute subject to constraints that the joint distribution of the transformed data is close to the original distribution and individual attributes are not substantially distorted.

    “Calmon's algorithm randomly transforms all variables except for the sensitive attribute to reduce the dependence between training labels and the sensitive attribute subject to the following constraints: (1) the joint distribution of the transformed data is close to the original distribution, and (2) individual attributes are not substantially distorted.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7]

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Enforcing DP, while unnecessary, can severely affect the accuracy of a classifier and introduce bias in sub-populations.

    “This experiment shows that enforcing DP, while unnecessary, can severely affect the accuracy of a classifier and, even more importantly, introduce bias in sub-populations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The goal of Calmon's and Feldman's algorithm is essentially to reduce DP.

    “For binned Adult data, all methods significantly reduced ROD, even though the goal of Calmon's and Fledman's algorithm is essentially to reduce DP.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    Justifiable fairness addresses shortcomings of previous definitions.

    “We defined a new notion of fairness, called justifiable fairness, that addresses shortcomings of the previous definitions and argued that it is the strongest notion of fairness that is testable from data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The requirement can be satisfied in data by removing logical dependencies.

    “The requirement can be satisfied in data by removing logical dependencies [44].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The analysis in [16] was restricted to only a few attributes, and the data was excessively binned to few categories.

    “We note that the analysis in [16] was restricted to only a few attributes, and the data was excessively binned to few categories (to facilitate the definition of distortion function).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [16] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    An algorithm A is A-fair if MB(O) ⊆ A.

    “Proof of Theorem 3.7.We show that an algorithm A is A-fair if MB(O) ⊆ A.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    If δ(S;O|a_b) > 1, then the algorithm potentially discriminates against the protected group, and vice versa if δ(S;O|a_b) < 1.

    “If δ (S; O |a b ) > 1, then the algorithm potentially discriminates against the protected group, and vice versa if δ (S; O |a b ) < 1.”
    Quote in the paperSubstantial claim
58 of 58

  • The authors state this as an assumption.

    The causal graph is given.

    “In this section we assume that the causal graph is given.”

  • The authors state this as an assumption.

    The protected attribute S classifies the population into protected S = 1 and privileged S = 0.

    “For simplicity, we assume S classifies the population into protected S = 1 and privileged S = 0, for example, female and male.”

  • The paper relies on this without support.

    No existing proposals describe comprehensive systems for pre-processing data to mitigate causal discrimination.

    “Moreover, no existing proposals describe comprehensive systems for pre-processing data to mitigate causal discrimination.”

  • The paper relies on this without support.

    Only users can identify the classification of variables into admissible and inadmissible.

    “Only users can identify this classification, and therefore admissible variables are part of the problem definition:”

  • The authors state this as an assumption.

    G and Pr are Markov compatible.

    “We assume only that G and Pr are Markov compatible (Sec.2.2).”

  • The authors state this as an assumption.

    The algorithm is a good prediction of the response variable, i.e., Pr(Y = 1|X = x) ≈ Pr(O = 1|X = x) (a reasonable classifier).

    “As before, we assume a fairness application (A, S, A, I) is given and that the algorithm is a good prediction of the response variable, i.e.Pr(Y = 1|X = x) ≈ Pr(O = 1|X = x); we call the algorithm a reasonable classifier to indicate that it satisfies this condition.”

  • The authors state this as an assumption.

    The Markov boundary of a variable is unique.

    “It is usually assumed that the Markov boundary of a variable is unique (see Appendix 8.1).”

  • The paper relies on this without support.

    Releasing data that reflect discrimination can unintentionally reinforce and amplify discrimination in other contexts that the data is used.

    “Releasing data that reflect discrimination can unintentionally reinforce and amplify discrimination in other contexts that the data is used.”

  • The paper relies on this without support.

    Minimal repairs are crucial for preserving the utility of data.

    “Note that minimal repairs are crucial for preserving the utility of data.”

  • The authors state this as an assumption.

    All probabilities are rational numbers.

    “We assume that all probabilities are rational numbers.”

  • The authors state this as an assumption.

    The test and training data are drawn from the same distribution Pr.

    “It is generally assumed that the test and training data are drawn from the same distribution Pr.”

  • The paper relies on this without support.

    Data reflects historical income inequality that can be reinforced by ML algorithms.

    “Furthermore, data reflects historical income inequality that can be reinforced by ML algorithms.”

  • The authors state this as an assumption.

    The potential influence of gender on income through some or all of the admissible variables is fair; the direct influence of gender on income, as well as its indirect influence on income through marital status, are discriminatory.

    “As is common in the literature, we assumed that the potential influence of gender on income through some or all of the admissible variables was fair; However, the direct influence of gender on income, as well as its indirect influence on income through marital status, were assumed to be discriminatory.”

  • The authors state this as an assumption.

    It is fair to use the admissible attributes to predict recidivism even though they can potentially be influenced by race, and the only goal is to address the direct influence of race.

    “As is common in the literature, we assumed that it was fair to use the admissible attributes to predict recidivism even though they can potentially be influenced by race, and our only goal in this experiment was to address the direct influence of race.”

  • The authors state this as an assumption.

    For both datasets, all attributes were admissible.

    “For both datasets, we assumed all attributes were admissible.”

  • The authors state this as an assumption.

    The probability distribution of A is faithful to the causal DAG.

    “suppose the probability distribution of A is faithful to the causal DAG.”

  • The authors state this as an assumption.

    X_i is a non-descendant of X_{i+1}.

    “the assumption that X i is an non-descendant of X i+1”

  • The authors state this as an assumption.

    Pr(Y=1|X=x) ≈ Pr(O=1|X=x).

    “the assumptions Pr(Y = 1|X = x) ≈ Pr(O = 1|X = x)”

  • The paper relies on this without support.

    The classifier is a deterministic function trained on Pr_B'.

    “Because the classifier is a deterministic function trained on Pr B ′ , it follows that Pr T (o, s, i, a) = Pr B ′ (o, s, i, a) = Pr B ′ (o|a)Pr B ′ (i, s, a).”

  • The authors state this as an assumption.

    The probability distribution is strictly positive.

    “It is known that if Pr is a strictly positive distribution (i.e., forall v ∈ Dom(V), P(v) > 0), then MB(V ) is unique for all V ∈ V and can be learned from data in polynomial time [32].”
20 of 20

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Omission of sensitive variables from training data decreases model accuracy.

    “Omission has been shown to decrease model accuracy [61,136,253] and increase discrimination [54,89,165].”

    From Fairness in machine learning: A survey (2024) · cites this paper as [253]

  • The main challenge for causal models is the requirement for background information and context regarding the causal model which may not be accessible.

    “The main challenge for causal models is the requirement for background information and context regarding the causal model which may not be accessible [253].”

    From Fairness in machine learning: A survey (2024) · cites this paper as [253]

  • Causal models have been criticized for not well examining how they would be applied in practice.

    “They have also been criticized for not well examining how they would be applied in practice [253].”

    From Fairness in machine learning: A survey (2024) · cites this paper as [253]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • College II's conditional rates conflict with its stated department A rates

    Minor

    The paper says department A admits 80% of female applicants and 100% of male applicants, then gives 0.8 and 0.2 for the two genders in that department. Those conditional rates cannot both describe the stated College II admissions.

  • Normalized ROD has conflicting no-discrimination endpoints

    Minor

    The experimental setup says normalized ROD is 0 when there is no observational discrimination, while the comparison section says normalized ROD is 1 when there is no discrimination.

  • Example 2.3 gives the wrong intervention probability

    Minor

    With department and qualification independently uniform, setting either gender produces an admission only when the applicant is both qualified and in the department that admits that gender. The probability is 1/4, not 1/2; this is the value used to establish the example's proxy-fairness claim.

  • The same intervention is assigned two different probabilities

    Minor

    Both expressions specify do(G = 1) and do(U o = 1), but one gives probability 0 and the other gives probability 1. The second expression appears to intend do(G = 0).

4 of 4

Cites

Flagged first, then by citations

Research works printed but not linked: 8

  1. 10.5555/551350 · DOI
  2. 10.5555/2371212 · DOI
  3. Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . How we analyzed the compas recidivism algorithm. ProPublica (5 2016 ), 9, 2016. Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. How we analyzed the compas recidivism algorithm. ProPublica (5 2016), 9, 2016.
  4. Judea Pearl . Causality. Cambridge university press , 2009 . Judea Pearl. Causality. Cambridge university press, 2009.
  5. Judea Pearl and Azaria Paz . Graphoids: A graph-based logic for reasoning about relevance relations . University of California (Los Angeles). Computer Science Department , 1985 . Judea Pearl and Azaria Paz. Graphoids: A graph-based logic for reasoning about relevance relations. University of California (Los Angeles). Computer Science Department, 1985.
  6. Jennifer Valentino-Devries , Jeremy Singer-Vine , and Ashkan Soltani . Websites vary prices, deals based on users' information . Wall Street Journal , 10 : 60 -- 68 , 2012 . Jennifer Valentino-Devries, Jeremy Singer-Vine, and Ashkan Soltani. Websites vary prices, deals based on users' information. Wall Street Journal, 10:60--68, 2012.
  7. 10.5555/1958447.1958459 · DOI
  8. Lauren Weber and Elizabeth Dwoskin . Are workplace personality tests fair ? Wall Strreet Journal , 2014 . Lauren Weber and Elizabeth Dwoskin. Are workplace personality tests fair? Wall Strreet Journal, 2014.

Cited by

Sorted by citations

Related

Authors
Salimi, B., Rodriguez, L., Howe, B., & Suciu, D.
Affiliations
University of Washington (US)
ORCID
Venue
Proceedings of the 2019 International Conference on Management of Data
Publisher
ACM
Published
2019-06-25
Pages
793-810
ISSN
0730-8078 (ISSN-L)
DOI
10.1145/3299869.3319901
OpenAlex
W2948130259
Cited by
145
References
55
Citations by year
2026: 8 · 2025: 14 · 2024: 21 · 2023: 20 · 2022: 23 · 2021: 30 · 2020: 21 · 2019: 8
FWCI
21.43
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM SIGMOD International Conference on Management of Data (SIGMOD) (publishedVersion)
Funders
  • NSF (1740996)
  • NSF III (1703281)
  • NSF AITF (1535565)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
algorithmic fairness, database repair, Simpson's paradox, causal models, fairness guarantees, statistical anomalies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review