CoChat
Sign Up
Cited by 646Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Privacy-Preserving Technologies in Data

Fairness in machine learning: A survey

Open access
Caton, Simon, Haas, Christian·2024·ACM Computing Surveys
DOI 10.1145/3616865
Open this paper in CoChat — free accountOpen PDF

Abstract

When Machine Learning technologies are used in contexts that affect citizens, companies as well as researchers need to be confident that there will not be any unexpected social implications, such as bias towards gender, ethnicity, and/or people with disabilities. There is significant literature on approaches to mitigate bias and promote fairness, yet the area is complex and hard to penetrate for newcomers to the domain. This article seeks to provide an overview of the different schools of thought and approaches that aim to increase the fairness of Machine Learning. It organizes approaches into the widely accepted framework of pre-processing, in-processing, and post-processing methods, subcategorizing into a further 11 method areas. Although much of the literature emphasizes binary classification, a discussion of fairness in regression, recommender systems, and unsupervised learning is also provided along with a selection of currently available open source libraries. The article concludes by summarizing open challenges articulated as five dilemmas for fairness research.

AI snapshot

Written from the paper’s full text.

TL;DR

This survey provides an entry-level overview of fairness in machine learning, organizing technical interventions into pre-processing, in-processing, and post-processing approaches and subcategorizing them into 11 method areas. It also reviews fairness metrics, discusses settings beyond binary classification, and concludes with five open dilemmas for fairness research.

Key findings

  • The paper organizes fairness interventions into the widely accepted framework of pre-processing, in-processing, and post-processing methods, with a further 11 method areas; it notes that the literature is dominated by binary classification.
  • It summarizes fairness metrics into group-based categories (parity-based, confusion matrix-based, calibration-based, score-based) and individual/counterfactual metrics, and states that many fairness notions are difficult or impossible to combine.
  • It identifies five dilemmas for future research: fairness vs. model performance; disagreement/incompatibility of fairness notions; tensions with context and policy; democratization of ML vs. the fairness skills gap; and scientific advancement vs. the reality of data.
  • It notes that beyond binary classification, fairness has been studied in regression, recommender systems/ranking, and unsupervised methods such as fair clustering, but says these areas are less comprehensive in the survey.

Methods

  • Design: a survey/overview article that synthesizes existing fairness literature rather than presenting new empirical experiments.
  • Scope: focuses on technical approaches to fairness in classification and closely related domains such as regression and rating predictions; refers elsewhere for NLP and reinforcement learning.
  • Analysis: organizes approaches by intervention stage and method area, discusses metrics and trade-offs, and lists open source toolkits in an online appendix.

Limitations

  • The paper states that the fairness area is complex and hard to penetrate for newcomers, that there is no universal means to measure fairness, and that no clear guidelines exist on which measures are best.
  • The paper states that much of the literature emphasizes binary classification, that beyond-binary-classification coverage is less comprehensive, and that there is a need for more realistic datasets, better industry engagement, and more runtime benchmarking outside clustering.
  • Evident from the text: the survey is not a systematic meta-review; it explicitly notes the absence of a structured meta-review of fairness approaches and says offering advice on approach selection is unrealistic.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3616865

Cites

363 of 363 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 480 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Intrinsic biases in the sample will manifest themselves in any model built on the data.

    “There is a recognition in the literature that often data is the problem, i.e., intrinsic biases in the sample will manifest themselves in any model built on the data [33,42]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [33]

      Not checked in the free run.

    • cites [42] source

      Not checked in the free run.

  • Premisecitation

    There is a rich set of fairness-related work in a variety of disciplines, often with concepts that are similar or equal to current ML fairness research.

    “Speciically, there is a rich set of fairness-related work in a variety of disciplines, often with concepts that are similar or equal to current ML fairness research [147].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [147] source

      Not checked in the free run.

  • Premisecitation

    Inappropriate uses of data lead to (un)conscious bias(es).

    “inappropriate uses of data leading to (un)conscious bias(es) [41,42]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [41]

      Not checked in the free run.

    • cites [42] source

      Not checked in the free run.

  • Premisenone given

    Most approaches to mitigate unfairness, bias, or discrimination are based on the notion of protected or sensitive variables and on (un)privileged groups.

    “Most approaches to mitigate unfairness, bias, or discrimination are based on the notion of protected or sensitive variables (we will use the terms interchangeably) and on (un)privileged groups: groups (deined by one or more sensitive variables) that are disproportionately (less) more likely to be positively classiied.”
    Quote in the paperNot a substantial claim
  • Premisecitation

    Many variables are explicitly defined as 'sensitive' by specific legal frameworks.

    “We note that many variables are explicitly deined as łsensitivež by speciic legal frameworks, see [25,129,130,191,192,209,268,280,306] and the references therein.”
    Quote in the paperNot a substantial claim0 of 9 sources checked
    • cites [25] source

      Not checked in the free run.

    • cites [129] source

      Not checked in the free run.

    • cites [130] source

      Not checked in the free run.

    • cites [191]

      Not checked in the free run.

    • cites [192] source

      Not checked in the free run.

    • cites [209] source

      Not checked in the free run.

    • cites [268] source

      Not checked in the free run.

    • cites [280] source

      Not checked in the free run.

    • cites [306] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 142
  • May combine two claims. 65

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    We are yet to find a strategy using pre-, in-, and post-processing methods combined.

    “However, we are yet to ind a strategy using pre-, in-, and post-processing methods combined.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    A few notable projects have emerged that address fairness in machine learning from a more general perspective.

    “While many researchers publish their individual approaches on GitHub and similar platforms, a few notable projects have emerged that address fairness in ML from a more general perspective.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The fairness literature focuses on either the technical aspects of bias and fairness in ML, or theorises on the social, legal, and ethical aspects of ML discrimination.

    “The fairness literature focuses on either the technical aspects of bias and fairness in ML, or theorises on the social, legal, and ethical aspects of ML discrimination [124].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [124]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Data veracity and quality, data relativity and context shifts, and subjectivity filters are challenges in fairness.

    “data veracity and quality [320], data relativity and context shifts [42,122,249], and subjectivity ilters [41].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [320] source

      Not checked in the free run.

    • cites [42] source

      Not checked in the free run.

    • cites [122] source

      Not checked in the free run.

    • cites [249] source

      Not checked in the free run.

    • cites [41]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Much of the early literature into fairness considered regression and correlation-based scenarios, compared to the recent emphasis on classification.

    “It is noteworthy that much of the early literature into fairness considered regression and correlation-based scenarios (standardized test scores, see e.g., [81]), compared to the recent emphasis on classiication (binary decisions).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [81] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Discrimination in hiring decisions has been examined since the 1960s.

    “For example, discrimination in hiring decisions has been examined since the 1960s [133].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [133] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    No general consensus as to what constitutes fairness or if it can/should be quantified emerged based on the first wave of fairness research.

    “Despite the increased discussion of diferent aspects and viewpoints of fairness in the 1970s as well as the founding of many modern fairness concepts, no general consensus as to what constitutes fairness or if it can/should be quantiied emerged based on this irst wave of fairness research.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The area of fairness in ML is complex and hard to penetrate for newcomers to the domain.

  • May combine two claims.

    Premisecitation

    The general notions from early fairness literature transfer to (binary) classification settings and define essential concepts such as protected/demographic variables, notions of group vs individual fairness, impossibility conditions between fairness conditions, and metric-based fairness quantification.

    “However, the general notions transfer to (binary) classiication settings and thus deine essential concepts such as protected/demographic variables (e.g.[72,73]), notions of group vs individual fairness (e.g.[257,278]), impossibility conditions between fairness conditions ( [81]), and metric-based fairness quantiication (e.g., true positive rates, [74]).”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [72] source

      Not checked in the free run.

    • cites [73] source

      Not checked in the free run.

    • cites [257] source

      Not checked in the free run.

    • cites [278] source

      Not checked in the free run.

    • cites [81] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    The importance of fairness and related ethical principles is recognized as key to improving the trustworthiness of ML (and AI in general).

    “The importance of fairness and related ethical principles, however, is recognized as key to improving the trustworthiness of ML (and AI in general) [75,198].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [75]

      Not checked in the free run.

    • cites [198]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Two initial foci of fairness research were unfairness of standardized tests in higher education contexts and discrimination in employment contexts.

    “Two initial foci of fairness research were unfairness of standardized tests in higher education contexts [72,73] and discrimination in employment contexts [133].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [72] source

      Not checked in the free run.

    • cites [73] source

      Not checked in the free run.

    • cites [133] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Much of the literature assumes a priori knowledge of the set of quasi-identifiers, or seeks to discover them on a case-by-case basis.

    “Yet, for correlated or otherwise related variables (often referred to as proxies or quasiidentiiers), much of the literature assumes a priori knowledge of the set of quasi-identiiers [111], or seeks to discover them on a case-by-case basis (e.g.[150,217]) towards the idea of privacy preserving data mining [6].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [111] source

      Not checked in the free run.

    • cites [150] source

      Not checked in the free run.

    • cites [217]

      Not checked in the free run.

    • cites [6] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Key approaches seek to protect specific individuals and groups from being identifiable, i.e., minimize disclosure risk.

    “Key approaches (e.g.[194,205,276]) seek to protect speciic individuals and groups from being identiiable, i.e., minimize disclosure risk.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [194] source

      Not checked in the free run.

    • cites [205] source

      Not checked in the free run.

    • cites [276] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Sensitive data disclosure is a long-standing challenge in protecting anonymity when data is published and/or analysed.

    “In particular, sensitive data disclosure is a long-standing challenge in protecting anonymity when data is published and/or analysed [6,111,195].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [6] source

      Not checked in the free run.

    • cites [111] source

      Not checked in the free run.

    • cites [195] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Pre-processing is argued as the most flexible part of the data science pipeline, as it makes no assumptions with respect to the choice of subsequently applied modeling technique.

    “Pre-processing is argued as the most lexible part of the data science pipeline, as it makes no assumptions with respect to the choice of subsequently applied modeling technique [88].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [88] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    All technical fairness interventions operate at specific locations in the ML pipeline, i.e., before, during, or after model training.

    “All technical fairness interventions operate at speciic locations in the ML pipeline, i.e., before, during, or after model training.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Pre-processing approaches tend to alter the sample distributions of protected variables, or more generally perform specific transformations on the data with the aim to remove discrimination from the training data.

    “Thus, pre-processing approaches tend to alter the sample distributions of protected variables, or more generally perform speciic transformations on the data with the aim to remove discrimination from the training data [165].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [165] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Modification of the data and/or model output may have legal implications and can mean models are less interpretable, which may be at odds with current data protection legislation with respect to explainability.

    “Yet, modiication of the data and/or model output may have legal implications [17] and can mean models are less interpretable [192,202], which may be at odds with current data protection legislation with respect to explainability.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [17]

      Not checked in the free run.

    • cites [192] source

      Not checked in the free run.

    • cites [202]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    In-processing approaches often incorporate one or more fairness metrics into the model optimization functions in a bid to converge towards a model parameterization that maximizes performance and fairness.

    “In-processing approaches tackle this by often incorporating one or more fairness metrics into the model optimization functions in a bid to converge towards a model parameterization that maximizes performance and fairness.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    There is no universal means to measure fairness, and no clear guideline(s) on which measures are 'best'.

    “Many are diicult / impossible to combine [69,177], but ultimately, we must keep in mind (as noted in [68]) there is no universal means to measure fairness, and no clear guideline(s) on which measures are łbestž.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [68]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Increasing fairness often results in lower overall accuracy or related metrics, leading to the necessity of analyzing potentially achievable trade-offs in a given scenario.

    “However, increasing fairness often results in lower overall accuracy or related metrics, leading to the necessity of analyzing potentially achievable trade-ofs in a given scenario [135].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [135]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Post-processing approaches tend to apply transformations to model output to improve prediction fairness.

    “Thus, post-processing approaches tend to apply transformations to model output to improve prediction fairness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Equal opportunity promotes that the TPR is the same across different groups.

    “Such that equal opportunity promotes that the TPR is the same across diferent groups.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [139]

      Not checked in the free run.

    • cites [239]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Equalized odds simultaneously considers TPR and FPR.

    “Similarly to equal opportunity, in addition to TPR equalized odds simultaneously considers FPR as well, i.e., the percentage of actual negatives that are predicted as positive.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    General fairness desiderata are categorized into three 'non-discrimination' criteria: Independence, Separation, and Sufficiency.

    “Using these properties, general fairness desiderata are categorized into three łnon-discriminationž criteria [16]: Independence aims for classiiers to make their scoring independent of the group membership:”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [16]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Group-based fairness metrics essentially compare the outcome of the classification algorithm for two or more groups.

    “Group-based fairness metrics essentially compare the outcome of the classiication algorithm for two or more groups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Most group fairness metrics use metrics based on the binary classification confusion matrix to define fairness.

    “Over time, many diferent approaches have been suggested, most of which use metrics based on the binary classiication confusion matrix to deine fairness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Confusion matrix-based metrics consider additional aspects such as True Positive Rate (TPR), True Negative Rate (TNR), False Positive Rate (FPR), and False Negative Rate (FNR).

    “While parity-based metrics typically consider variants of the predicted positive rate ( = 1), confusion matrixbased metrics consider additional aspects such as True Positive Rate (TPR), True Negative Rate (TNR), False Positive Rate (FPR), and False Negative Rate (FNR).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Statistical/Demographic Parity defines fairness as an equal probability of being classified with the positive label.

    “Statistical/Demographic Parity: One of the earliest deinitions of fairness, this metric deines fairness as an equal probability of being classiied with the positive label [77,98,168,313].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [77]

      Not checked in the free run.

    • cites [98] source

      Not checked in the free run.

    • cites [168] source

      Not checked in the free run.

    • cites [313]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Most quantitative definitions and measures of fairness are centered around three fundamental aspects of a (binary) classifier: the sensitive variable, the target variable, and the classification score.

    “Most quantitative deinitions and measures of fairness are centered around three fundamental aspects of a (binary) classiier: 5 First, the sensitive variable that deines the groups for which we want to measure fairness.Second, the target variable .In binary classiication, this represents the two classes that we can predict: = 0 or = 1.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Disparate impact considers the ratio between unprivileged and privileged groups.

    “In contrast to parity, it considers the ratio between unprivileged and privileged groups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Overall accuracy equality looks at relative accuracy rates across different groups.

    “[27] adjusts this concept by looking at relative accuracy rates across diferent groups.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    A disadvantage of statistical parity is that potential differences between groups are not being taken into account.

    “A disadvantage of this notion, however, is that potential diferences between groups are not being taken into account.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Legal fairness considerations for selection procedures sometimes use an 80% rule to define if a process has disparate impact (ratio smaller than 0.8) or not.

    “Its origins are in legal fairness considerations for selection procedures which sometimes use an 80% rule to deine if a process has disparate impact (ratio smaller than 0.8) or not [98].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [98] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Test fairness or calibration wants to guarantee that the probability of Y=1 is the same given a particular score.

    “Test fairness/ calibration / matching conditional frequencies ( [69,139]): Essentially, test fairness or calibration wants to guarantee that the probability of = 1 is the same given a particular score.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [69] source

      Not checked in the free run.

    • cites [139]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given

    Many of the most contentious application areas that motivated the domain are binary decisions.

    “1) many of the most contentious application areas that motivated the domain are binary decisions (hiring vs. not hiring; ofering a loan vs. not ofering a loan etc.);”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Treatment equality considers the ratio of False Negative Predictions (FNR) to False Positive Predictions.

    “Treatment equality [27]: Treatment equality considers the ratio of False Negative Predictions (FNR) to False Positive Predictions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Calibration-based metrics take the predicted probability, or score, into account, differentiating them from metrics that use predicted and actual values.

    “Related to the Suiciency criterion, calibration-based metrics take the predicted probability, or score, into account, diferentiating them from metrics above that use predicted and actual values.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Counterfactual fairness builds on causal fairness models and is related to both individual and group fairness concepts.

    “[182] propose the concept of counterfactual fairness which builds on causal fairness models and is related to both individual and group fairness concepts.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [182]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    There is an abundance of pre-and in-processing vs. post-processing methods and many researchers increasingly develop intervention strategies that belong to multiple stages.

    “We note an abundance of pre-and in-processing vs. post-processing methods and that many researchers increasingly develop intervention strategies that belong to multiple stages.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Quantifying fairness on a binary dependent variable is mathematically more convenient; addressing multi-class problems would add terms to the fairness quantity.

    “2) quantifying fairness on a binary dependent variable is mathematically more convenient; addressing multi-class problems would add terms to the fairness quantity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Individual and counterfactual fairness metrics do not focus on comparing two or more groups as defined by a sensitive variable, but consider the outcome for each participating individual.

    “As compared to group-based metrics which compare scores across diferent groups, individual and counterfactual fairness metrics do not focus on comparing two or more groups as deined by a sensitive variable, but consider the outcome for each participating individual.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Definitions of fairness and bias expressed mathematically often do not map to normative social, economic, or legal understandings of fairness.

    “With a reliance on expressing fairness and bias mathematically, [76,128] argue that these deinitions often do not map to normative social, economic, or legal understandings of fairness.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [76]

      Not checked in the free run.

    • cites [128]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Many approaches to group fairness tackle only between-group issues, as a consequence worsening within-group fairness.

    “[270] note that many approaches to group fairness tackle only between-group issues, as a consequence worsening within-group fairness.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [270] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    In the absence of such a review, offering advice on approach selection is unrealistic.

    “In the absence of such a review, ofering advice on approach selection is unrealistic.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The risk of omitted variable bias should not be downplayed.

    “Yet, determining which combination(s) of variables to blind is non-trivial and the risk of omitted variable bias should not be downplayed [71,160].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [71] source

      Not checked in the free run.

    • cites [160]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Both omission and immunity overlook relationships with proxy variables.

    “Both omission and immunity overlook relationships with proxy variables (as discussed in subsection 2.1, we note [291] as an exception here who omit proxies), which can result in increasing bias and discrimination [177], or indirectly concealing discrimination [88].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [177] source

      Not checked in the free run.

    • cites [88] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Causal models have been criticized for not well examining how they would be applied in practice.

    “They have also been criticized for not well examining how they would be applied in practice [253].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [253] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Directed acyclic graphs (DAGs) are a common means to represent conditional independence assumptions between variables.

    “Directed acyclic graphs (DAGs) are a common means to represent conditional independence assumptions between variables [155].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [155]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    DAGs can have high utility in studies of fairness in industry settings.

    “This means they can have high utility in studies of fairness in industry settings.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Causal methods are specifically well suited to identifying which subgroups are most (un)fairly treated and differentiating the types of bias exhibited.

    “Thus, causal methods are speciically well suited to identifying 1) proxies of sensitive variables as discussed in subsection 2.1; 2) which subgroups are most (un)fairly treated and diferentiate the types of bias exhibited [118]; and 3) provide transparency with respect to how (classiication) decisions were made [143].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [118] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Initial results with data repair methods have shown to result in debiased classifiers which are robust to unseen test data.

    “Initial results with data repair methods have shown to result in łdebiasedž classiiers which are robust to unseen test data, yet this process requires signiicant computational resources.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [251ś253]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [134] also learn proxy groupings.

    “[134] also learn proxy groupings.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [134]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [290] propose an iterative human-in-the-loop resampling approach where users can evaluate potential fairness issues of a model through interactive visualizations.

    “[290] propose an iterative human-in-the-loop resampling approach where users can evaluate potential fairness issues of a model through interactive visualizations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [290] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Probabilistic verification of fairness measures via the sampling of sensitive variables has been proposed to evaluate a trained model within some (small) confidence bound.

    “Probabilistic veriication (e.g.[18]) of fairness measures via the sampling of sensitive variables has also been proposed to evaluate a trained model within some (small) conidence bound.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [165] propose to preferentially sample instances close to a decision boundary as these are most likely to be discriminated or favored due to underlying biases within the training data.

    “[165], propose to preferentially sample (similar to oversampling) instances łclosež to a decision boundary (based on an initial model prototype to approximate a decision boundary) as these are most likely to be discriminated or favored due to underlying biases within the training data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [165] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    These evaluation approaches present fairness as a dichotomous outcome: a model is fair, or it isn't, as opposed to quantifying how (un)fair a model is.

    “Similar to other evaluation approaches (e.g.[10]), these approaches present fairness as a dichotomous outcome: a model is fair, or it isn't, as opposed to quantifying how (un)fair a model is.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [148] extend the decoupled classifier approach by creating an ensemble of ensembles to operate on the protected groups.

    “[148] extend this approach creating an ensemble of ensembles to operate on the protected groups.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [148]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Statistical hypothesis testing is employed to reveal whether models are significantly different with respect to fairness quantities or denote variable instability.

    “Statistical hypothesis testing is employed to reveal whether models are signiicantly diferent with respect to fairness quantities or denote variable instability, i.e., when a model is not robustly fair when a given variable or set of variables are included.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Different candidate groupings are often evaluated via in- or post-processing methods to inhibit overfitting, maximize some fairness metric(s), and/or prevent other theoretical violations.

    “Thus, diferent candidate groupings are often evaluated via in-or post-processing methods to inhibit overitting, maximize some fairness metric(s), and/or prevent other theoretical violations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [98] discuss transforming the distribution of SAT scores towards the median to degender the distribution into one which retains only the rank order for individuals.

    “To illustrate transformation, [98] discuss transforming the distribution of SAT scores towards the median to łdegenderž the distribution into one which retains only the rank order for individuals, i.e, they remove information about protected variables from a set of covariates.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [98] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    A balance is needed to ensure groups are as atomic as possible but robust against gerrymandering.

    “Similarly, and especially for decoupled classiiers, is the challenge of selecting groups: some can be rarer than others [283] and as such a balance is needed to ensure groups are as atomic as possible but robust against gerrymandering [140,170].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [140]

      Not checked in the free run.

    • cites [170]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Good clusters well approximate underlying data distributions and subgroups.

    “Common approaches in group formation are recursive partitioning (e.g.[170,298]) and clustering (e.g.: [70,148]) as (good) clusters well approximate underlying data distributions and subgroups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    There are parallels between blinding (in the immunity sense) and independence mappings, as both share a common goal of creating independence from one or more specific sensitive variables.

    “There are parallels between blinding (in the immunity sense) and independence mappings, as in many ways these two approaches share a common goal: creating independence from one or more speciic sensitive variables.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [148] used clustering as a means to build stratified samples of different subgroups within the data as an exercise in bagging for the training of fair ensembles.

    “[148] used clustering as a means to build stratiied samples of diferent subgroups within the data as an exercise in bagging for the training of fair ensembles.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [148]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Current transformation approaches operate mainly on numeric data, which is a significant limitation.

    “Current transformation approaches operate mainly on numeric data, which is a signiicant limitation [98], yet approaches for other scenarios such as image classiications have been proposed [132].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [98] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Causal methods may be a useful precursor to transformation techniques.

    “2) Understanding the relationship(s) between sensitive and potential proxy variables is hard [98], thus causal methods (subsection 4.2) may be useful precursor to transformation techniques;”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    An alternative to retaining rank order is the use of distortion constraints which seek to prevent mapping high values to low values and vice versa.

    “An alternative to retaining rank order is the use of distortion constraints (e.g.[88]) which seek to prevent mapping łhighž values to łlowž values and vice versa.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [88] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    [52,202] define a similar approach yet model the transformation process with different assumptions and objectives.

    “[52,202] deine a similar approach yet model the transformation process with diferent assumptions and objectives.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [52] source

      Not checked in the free run.

    • cites [202]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    [65,173,219] transform the output of a classifier in accordance to the identification of unfair causal pathways, either by averaging, constraining the conditional distribution of the decision variable or through counterfactual correction.

    “[65,173,219] transform the output of a classiier in accordance to the identiication of unfair causal pathways, either by averaging [219], constraining the conditional distribution of the decision variable [173] or through counterfactual correction [65].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [65] source

      Not checked in the free run.

    • cites [173]

      Not checked in the free run.

    • cites [219] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The transformed data should not be significantly different from the original data, otherwise the extent of repair can diminish the utility of the produced classifier through data loss.

    “There are a number of challenges when applying transformation techniques: 1) the transformed data should not be signiicantly diferent from the original data, otherwise the extent of łrepairž can diminish the utility of the produced classiier [88,98,202] through data loss [125].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [88] source

      Not checked in the free run.

    • cites [98] source

      Not checked in the free run.

    • cites [202]

      Not checked in the free run.

    • cites [125]

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Although largely a pre-processing method, transformation can also be applied within a post-processing phase.

    “Although largely a pre-processing method, transformation can also been applied within a post-processing phase.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Transformation as an approach is similar to calibration and thresholding.

    “As an approach, this is similar to the idea of calibration (see subsection 4.10) and thresholding (see subsection 4.11) .”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Perturbation has been proposed as a mechanism to detect proxy variables, influential variables and counterfactual distributions.

    “It has been proposed as a mechanism to detect proxy variables, inluential variables [292] and counterfactual distributions [291].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [292]

      Not checked in the free run.

    • cites [291]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Often perturbation-based approaches are applied as a pre-processing step to prepare for an in-processing approach; often reweighing and/or regularisation/optimization.

    “Often perturbation-based approaches are applied as a pre-processing step to prepare for an in-processing approach; often reweighing (e.g.[152]; see subsection 4.6), and/or regularisation / optimization (e.g.[283,291]; see subsection 4.7).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [152]

      Not checked in the free run.

    • cites [283]

      Not checked in the free run.

    • cites [291]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Many handle missing data in transformation by simply removing these samples, yet this may raise other methodological issues.

    “Many handle this by simply removing these samples, yet this may raise other methodological issues;”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Missing data provides specific problems for transformation approaches, as it is unclear how to deal with such data samples.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Relabelling can also be applied to the test data upon the basis of strategies or probabilities learned on the training data.

    “It can also be applied to the test data upon the basis of strategies or probabilities learned on the training data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Relabelling involves the modification of the labels of training data instances so that the proportion of positive instances are equal across all protected groups.

    “Referred to as data-massaging by [165,313], relabelling involves the modiication of the labels of training data instances so that the proportion of positive instances are equal across all protected groups.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [165] source

      Not checked in the free run.

    • cites [313]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Often, but not always, relabelling approaches seek to retain the overall class distribution.

    “Often, but not always, approaches seek to retain the overall class distribution.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [204] relabel the dependent variable if the data instance is determined as being discriminated against with respect to the observed outcome.

    “For example, [204] relabel the dependent variable (lip it from positive to negative or vice versa) if the data instance is determined as being discriminated against with respect to the observed outcome.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [204] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Transformation makes the model less interpretable, which may be at odds with data protection legislation.

    “6) transformation makes the model less interpretable [192,202], which may be at odds with data protection legislation;”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [192] source

      Not checked in the free run.

    • cites [202]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Relabelling is often used in counterfactual studies which investigate if flipping the (in)dependent variable(s) affect the classification outcome.

    “Relabelling is also often used in counterfactual studies (e.g.[142,161,291]) which investigate if lipping the (in)dependent variable(s) afect the classiication outcome.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [142] source

      Not checked in the free run.

    • cites [161]

      Not checked in the free run.

    • cites [291]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    [181] note that some classifiers may be unaffected by the presence or specific nuances of some biases, and others may be negatively affected by altering the training data in an attempt to mitigate them.

    “[181] also note that some classiiers may be unafected by the presence or speciic nuances of some biases, and others may be negatively afected by altering the training data in an attempt to mitigate them.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [181] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Sensitivity analysis is a relatively under-addressed area in the fairness literature.

    “This is a relatively under-addressed area in the fairness literature, perhaps due to the introduction of SHAP valuesfoot_3 [203], which has seen an increase in attention from fairness researchers (e.g.[126]).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    [114,211] investigated visual mechanisms to better display issues with data to users, yet these approaches generally lack support for novice users.

    “[114,211] investigated visual mechanisms to better display łissuesž with data to users, yet these approaches generally lack support for novice users [24].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [114]

      Not checked in the free run.

    • cites [211]

      Not checked in the free run.

    • cites [24] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The introduction of SHAP values has seen an increase in attention from fairness researchers.

    “This is a relatively under-addressed area in the fairness literature, perhaps due to the introduction of SHAP valuesfoot_3 [203], which has seen an increase in attention from fairness researchers (e.g.[126]).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [203]

      Not checked in the free run.

    • cites [126] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Changes to the data should be minimised.

    “As for transformation, modiication of the data via relabelling and perturbation is not always legally permissible [17] and changes to the data should be minimised [136,204].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [136] source

      Not checked in the free run.

    • cites [204] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The application of sensitivity analysis in ML is often to measure the stability of a model.

    “The application of sensitivity analysis in ML is often to measure the stability of a model [242].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [242] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Sensitivity analysis (and explainable fairness in general) has been well motivated.

    “Yet sensitivity analysis (and explainable fairness in general) has been well motivated (although perhaps indirectly): [124] called for a better understanding of bias stemming from uncertainty, and [139] who stressed that assessment of data reliability is needed.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [124]

      Not checked in the free run.

    • cites [139]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Relabelling and perturbation are not far from the principles of sensitivity analysis.

    “Yet, relabelling and perturbation are not far from the principles of sensitivity analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Sensitivity analysis is closely related to perturbation approaches.

    “Closely related to perturbation approaches is the use of sensitivity analysis (see [254]) to explore how various aspects of the feature vector afect a given outcome.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [254]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Perturbation appears more prevalent in the discrimination-aware data mining literature where it is often used as a means of privacy preservation.

    “Whilst there are a number of papers that harness perturbation (it is not always referred to as perturbation) in the ML literature, this approach appears more prevalent in the discrimination-aware data mining literature where it is often used as a means of privacy preservation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [80] proposed the perturbation of feature vectors to measure the effect on model performance of specific interventions.

    “[80] proposed the perturbation of feature vectors to measure the efect on model performance of speciic interventions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [80]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Weights can be introduced to improve classifier stability.

    “Weights can be introduced for multiple purposes: 1) to indicate a frequency count for an instance type (e.g.[52]), 2) to place lower/higher importance on łsensitivež training samples (e.g.[53,152,165]), or 3) to improve classiier stability (e.g.[181]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [181] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Weights can be introduced to indicate a frequency count for an instance type.

    “Weights can be introduced for multiple purposes: 1) to indicate a frequency count for an instance type (e.g.[52]), 2) to place lower/higher importance on łsensitivež training samples (e.g.[53,152,165]), or 3) to improve classiier stability (e.g.[181]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [165] seek to assign weights that take into consideration the likelihood of an instance with a specific class and sensitive value pairing (pre-processing).

    “For example, [165] seek to assign weights that take into consideration the likelihood of an instance with a speciic class and sensitive value pairing (pre-processing).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [165] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [181] first build an unweighted classifier, learn the weights of samples, then retrain their classifier using these weights (in-processing).

    “Whereas, [181] irst build an unweighted classiier, learn the weights of samples, then retrain their classiier using these weights (in-processing).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [181] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    [152] identify sensitive training instances (pre-processing), but then learn weights for these instances (in-processing) to optimize for the chosen fairness metric.

    “A similar approach is taken by [152] who identify sensitive training instances (pre-processing), but then learn weights for these instances (in-processing) to optimize for the chosen fairness metric.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [152]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some notable exceptions emphasise TPR/FPR of protected groups.

    “Some notable exceptions emphasise: 1) empirical risk subject to fairness constraints or welfare conditions (e.g.[87,142]), 2) TPR/FPR of protected groups (e.g.[19]), 3) stability of fairness (e.g.[145]), or 4) counterfactual terms (e.g.[85]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Applied to fairness, regularization means adding penalty terms to penalize the classifier for discriminatory practices.

    “Applied to fairness, regularization means adding penalty terms to penalize the classiier for discriminatory practices [168].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [168] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Fairness regularization is not hypothesis (or model) driven, but data driven and based upon the notion(s) of fairness considered.

    “Thus, it is not hypothesis (or model) driven, but data driven [19] and based upon the notion(s) of fairness considered.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some notable exceptions emphasise stability of fairness.

    “Some notable exceptions emphasise: 1) empirical risk subject to fairness constraints or welfare conditions (e.g.[87,142]), 2) TPR/FPR of protected groups (e.g.[19]), 3) stability of fairness (e.g.[145]), or 4) counterfactual terms (e.g.[85]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [145]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some notable exceptions emphasise counterfactual terms.

    “Some notable exceptions emphasise: 1) empirical risk subject to fairness constraints or welfare conditions (e.g.[87,142]), 2) TPR/FPR of protected groups (e.g.[19]), 3) stability of fairness (e.g.[145]), or 4) counterfactual terms (e.g.[85]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [85]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [218] approached constraint optimization via reinforcement learning.

    “[218] also approached this via reinforcement learning.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [218]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Constraint optimization approaches can also include other constraints, and/or reduce the problem to a cost-sensitive classification problem.

    “Yet, these approaches can also include other constraints, and/or reduce the problem to a cost-sensitive classiication problem (e.g.[3,120,221]).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [3]

      Not checked in the free run.

    • cites [120]

      Not checked in the free run.

    • cites [221]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Different regularization terms and penalties have diverse results on different data sets, i.e. this choice can have qualitative effects on the trade-off between accuracy and fairness.

    “3) diferent regularization terms and penalties have diverse results on diferent data sets, i.e. this choice can have qualitative efects on the trade-of between accuracy and fairness [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [79,120,221] summarizes additional constraint types as precision or budget constraints, quantification or coverage constraints, churn constraints, and stability constraints.

    “[79,120,221] summarizes a number of additional constraint types as precision or budget constraints to address the accuracy-fairness trade-of (often expressed as utility or risk functions e.g.[77,135]); quantiication or coverage constraints to capture disparities in class or population frequencies; churn constraints capturing online learning scenarios and enforcing that classiiers do not difer sign”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [79]

      Not checked in the free run.

    • cites [120]

      Not checked in the free run.

    • cites [221]

      Not checked in the free run.

    • cites [109] source

      Not checked in the free run.

    • cites [145]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The idea of robustness of fairness is often not considered by many works.

    “This idea of łrobustness of fairnessž is often not considered by many works and would warrant better inclusion in future fairness studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The robustness framework can also be used to quantify the robustness of fairness.

    “With the goal of better fairness generalization, the approach can also be used to quantify the robustness of fairness.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [103]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [103] suggest a robustness framework for fairness that ensures fairness over different training samples.

    “In light of this, [103] suggest a robustness framework for fairness that ensures fairness over diferent training samples.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [103]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Regularization as a mechanism is fairly generic, and can lead to a lack of model robustness and generalizability.

    “Aside from this, [117,313] note that regularization as a mechanism is fairly generic, and can lead to a lack of model robustness and generalizability.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [117] source

      Not checked in the free run.

    • cites [313]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [145] present empirical evidence for addressing stability of fairness through regularization via extensions of [168,307].

    “[145] argue that stability of fairness can be addressed through regularization and present corresponding empirical evidence through extensions of [168,307].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [145]

      Not checked in the free run.

    • cites [168] source

      Not checked in the free run.

    • cites [307] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Stability of fairness can be addressed through regularization.

    “[145] argue that stability of fairness can be addressed through regularization and present corresponding empirical evidence through extensions of [168,307].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [145]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Regularization approaches are often non-convex in nature or achieve convexity at the cost of probabilistic interpretation.

    “Key challenges for regularization approaches are: 1) they are often non-convex in nature or achieve convexity at the cost of probabilistic interpretation [119];”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [119] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    When applied to fairness in ML, an adversary seeks to determine whether the training process is fair, and when not, feedback from the adversary is used to improve the model.

    “When applied to applications of fairness in ML, an adversary instead seeks to determine whether the training process is fair, and when not, feedback from the adversary is used to improve the model [58].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In a Deep Neural Network setting, [115] show that different trade-offs between accuracy and fairness can be achieved by selective dropout of neurons.

    “Additionally, in a Deep Neural Network setting, [115] show that diferent trade-ofs between accuracy and fairness can be achieved by selective dropout of neurons.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [115]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    An advantage of adversarial approaches is that they can consider multiple fairness constraints, often treating the model as a blackbox.

    “An advantage of adversarial approaches is that they can consider multiple fairness constraints [289], often treating the model as a blackbox [206].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [289]

      Not checked in the free run.

    • cites [206]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Adversarial fairness approaches are often formulated as a multi-constraint optimization problem considering many of the constraints discussed for regularization.

    “This is often formulated as a multi-constraint optimization problem considering many of the constraints as discussed in subsection 4.7.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Work has started applying the notions of causal and counterfactual fairness to adversarial learning.

    “Work has also started applying the notions of causal and counterfactual fairness to adversarial learning (e.g.[299]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [299]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Decision makers may not be able to define what it means to be fair but may recognize unfairness when they see it.

    “As a reinforcement learning framework, bandits are motivated on the need for decisions to be made in an online manner [156] (although not always, e.g.[215]), and that decision makers may not be able to deine what it means to be łfairž but that they may recognize łunfairnessž when they see it [116].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [116]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    This allows researchers to frame the fairness problem as a stochastic multi-armed bandit, assigning either individuals or groups of similar individuals to arms, and represent fairness as regret.

    “This allows researchers to frame the fairness problem as a stochastic multi-armed bandit, assigning either individuals or groups of łsimilarž individuals to arms, and represent fairness as regret [156,201].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [156]

      Not checked in the free run.

    • cites [201]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Bandits are motivated on the need for decisions to be made in an online manner.

    “As a reinforcement learning framework, bandits are motivated on the need for decisions to be made in an online manner [156] (although not always, e.g.[215]), and that decision makers may not be able to deine what it means to be łfairž but that they may recognize łunfairnessž when they see it [116].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [156]

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    The use of generative adversarial networks (GAN) with fairness considerations permits applications within unstructured data or as a generative process of creating an unbiased dataset using a number of samples.

    “The use of generative adversarial networks (GAN) with fairness considerations also permit applications within unstructured (for example multimedia) data or more generally as a generative process of creating an łunbiasedž dataset using a number of samples.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The difference between the application of bandits to fair classification and fair recommendation is often hard to distinguish.

    “It is important to highlight that the diference between the application of bandits to fair classiication and fair recommendation (subsection 5.2) is often hard to distinguish.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The application of bandits to ML fairness has proliferated recently.

    “The application of bandits to ML fairness has proliferated recently.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The main notions of fairness applied in the application of bandits are meritocratic fairness (group agnostic), subjective fairness (emphasises fairness in each time period) and counterfactual fairness.

    “The main notions of fairness applied in the application of bandits are meritocratic fairness [127,156,157] (group agnostic), subjective fairness [201] (emphasises fairness in each time period ) and counterfactual fairness (e.g.[146]).”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [127]

      Not checked in the free run.

    • cites [156]

      Not checked in the free run.

    • cites [157] source

      Not checked in the free run.

    • cites [201]

      Not checked in the free run.

    • cites [146]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The literature has proposed various approaches to handle the impasse of achieving calibration and other fairness measures.

    “The literature has proposed various approaches to handle the impasse of achieving calibration and other fairness measures.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Calibrating a ML model for multiple protected groups and/or using multiple fairness criteria at once has been shown to be impossible.

    “Calibrating a ML model for multiple protected groups and/or using multiple fairness criteria at once has been shown to be impossible [69,140,174,178,200,201,223,239].”
    Quote in the paperNot a substantial claim0 of 8 sources checked
    • cites [69] source

      Not checked in the free run.

    • cites [140]

      Not checked in the free run.

    • cites [174]

      Not checked in the free run.

    • cites [178]

      Not checked in the free run.

    • cites [200]

      Not checked in the free run.

    • cites [201]

      Not checked in the free run.

    • cites [223]

      Not checked in the free run.

    • cites [239]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    One approach has been to apply a randomization post-processing process to try and achieve a balance between accuracy and fairness.

    “One approach has been to apply a randomization post-processing process to try and achieve a balance between accuracy and fairness, yet [76,139,178,223,239] discuss a number of shortcomings of this approach.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    A calibrated model does not inhibit biases of decision makers, but ensures that risk estimates for various (protected) subgroups carry the same meaning.

    “As such, a calibrated model does not inhibit biases of decision makers, but ensures that risk estimates for various (protected) subgroups carry the same meaning [239].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [239]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The goals of low error and calibration are competing objectives for a model.

    “[239] even note that the goals of low error and calibration are competing objectives for a model.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [239]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [223] propose a cost-based approach to balance calibration and error parity.

    “[223] also note that this approach is Pareto sub-optimal, and instead propose a cost-based approach to balance calibration and error parity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [223]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The individuals who are randomized are not necessarily positively impacted, and the overall accuracy of the model can be adversely affected.

    “Notably, the individuals who are randomized are not necessarily positively impacted, and the overall accuracy of the model can be adversely afected.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [76]

      Not checked in the free run.

    • cites [139]

      Not checked in the free run.

    • cites [178]

      Not checked in the free run.

    • cites [223]

      Not checked in the free run.

    • cites [239]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    [170,175] undertake a similar approach under different settings.

    “[170,175] undertake a similar approach under diferent settings.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [170]

      Not checked in the free run.

    • cites [175] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [140] seek to achieve approximate calibration using a multi-calibration approach that operates on identifiable subgroups to balance individual and group fairness measures even for small samples.

    “[140] instead seek to achieve approximate calibration (i.e. to guarantee calibration with high probability) using a multi-calibration approach that operates on identiiable subgroups to balance individual and group fairness measures even for small samples: a speciic challenge for achieving calibration [200].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [140]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The randomization post-processing approach is Pareto sub-optimal.

    “[223] also note that this approach is Pareto sub-optimal, and instead propose a cost-based approach to balance calibration and error parity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [223]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [201] have proposed a bandit-based approach to calibration.

    “[201] have proposed a bandit-based approach to calibration.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [201]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [200] suggest that calibration is sufficient as a fairness criterion if the model is unconstrained.

    “[200] suggest that calibration is suicient as a fairness criterion if the model is unconstrained.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [200]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Computing threshold values by hand can introduce new biases without training, as fairness is typically not monotonic, thus assigning a threshold value may be quite arbitrary.

    “Computing by hand can introduce new biases without training [286], as fairness is typically not monotonic, thus assigning a threshold value may be quite arbitrary [270].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [286] source

      Not checked in the free run.

    • cites [270] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The underlying idea of thresholding is to incentivize good performance (in terms of both fairness and accuracy) across all classes and groups.

    “The underlying idea here is to incentivize good performance (in terms of both fairness and accuracy) across all classes and groups.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Such instances are considered to be ambiguous, and therefore potentially influenced by bias.

    “Such instances are considered to be ambiguous, and therefore potentially inluenced by bias [167].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [167] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Threshold values are often computed with respect to accuracy, but cases of class imbalance would invalidate this.

    “Threshold values are often computed with respect to accuracy, but cases of class imbalance would invalidate this [148].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [148]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    There is a sufficient need for fairness researchers to also focus on other ML problems.

    “note that there is a suicient need for fairness researchers to also focus on other ML problems.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Fairness in recommender systems can be seen as a multi-sided concept that can be relevant for both users and items.

    “An aspect that distinguishes fairness considerations in recommender systems is that fairness can be seen as multi-sided concept that can be relevant for both users (who consume the recommendations) and items.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The motivation for using transfer learning is typically in response to an observable covariate shift between the source (training) and target distributions.

    “The motivation for using transfer learning is typically in response to an observable covariate shift between the source (training) and target distributions.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [31]

      Not checked in the free run.

    • cites [240]

      Not checked in the free run.

    • cites [273]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    When implementing fairness measures, we must emphasize either fairness or model performance as improving one can often detriment the other.

    “However, when implementing fairness measures, we must emphasize either fairness or model performance as improving one can often detriment the other [27,54,77,89,135,139,319].”
    Quote in the paperSubstantial claim0 of 7 sources checked
    • cites [27]

      Not checked in the free run.

    • cites [54]

      Not checked in the free run.

    • cites [77]

      Not checked in the free run.

    • cites [89] source

      Not checked in the free run.

    • cites [135]

      Not checked in the free run.

    • cites [139]

      Not checked in the free run.

    • cites [319]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ML practitioners can measure performance in a multitude of ways, and there has been much discussion concerning the choice of different performance measures and approaches.

    “ML practitioners can measure performance in a multitude of ways, and there has been much discussion concerning the choice of diferent performance measures and approaches [84,267].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [84]

      Not checked in the free run.

    • cites [267] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Unlike other approaches to fairness in ML, fair clustering has significant computational costs associated to it.

    “Yet, there is a cost here, unlike other approaches to fairness in ML, fair clustering has signiicant computational costs associated to it.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A reduction in accuracy may in fact be the desired result, if it was discrimination that raised accuracy in the first place.

    “As noted by [98], however, a reduction in accuracy may in fact be the desired result, if it was discrimination that raised accuracy in the irst place.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [98] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Approaches that utilize transfer learning do so in combination with other methods.

    “Approaches that utilize transfer learning do so in combination with other methods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Women have differing views in the inclusion / exclusion of gender as a protected variable to men.

    “E.g., [237] notes that women have difering views in the inclusion / exclusion of gender as a protected variable to men.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [237]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The choice of performance measure(s) itself may even harbor, disguise, or create new underlying ethical concerns.

    “The choice of performance measure(s) itself may even harbor, disguise, or create new underlying ethical concerns.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Improving fairness in ML is challenging and simultaneously there are many different notions for researchers and practitioners to navigate.

    “Thus, improving fairness in ML is challenging and simultaneously there are many diferent notions for researchers and practitioners to navigate.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Often approaches to group fairness tackle between-group issues, worsening within-group issues through this choice.

    “[270] also note that often approaches to group fairness tackle between-group issues, worsening within-group issues through this choice.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [270] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    There is an over-emphasis in the literature on specific measures of fairness and insufficient dialogue between researchers and affected communities.

    “This is corroborated by [266] who note an over-emphasis in the literature on speciic measures of fairness and insuicient dialogue between researchers and afected communities.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [266]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Currently, there is little runtime benchmarking of methods outside of clustering approaches.

    “We also note that currently, there is little runtime benchmarking of methods outside of clustering approaches (see [15,258]).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15]

      Not checked in the free run.

    • cites [258]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    With a reliance on expressing fairness mathematically, these definitions often do not map to normative social, economic, or legal understandings of fairness.

    “To further complicate things, [76,128] argue that with a reliance on expressing fairness mathematically these deinitions often do not map to normative social, economic, or legal understandings of fairness.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [76]

      Not checked in the free run.

    • cites [128]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation

    Critics of current research argue that efforts will fail unless contextual, sociocultural, and social policy challenges are better understood.

    “However, beyond this, critics of current research [50,193,266,286,287,297,297,306] argue that eforts will fail unless contextual, sociocultural, and social policy challenges are better understood.”
    Quote in the paperNot a substantial claim0 of 7 sources checked
    • cites [50] source

      Not checked in the free run.

    • cites [193] source

      Not checked in the free run.

    • cites [266]

      Not checked in the free run.

    • cites [286] source

      Not checked in the free run.

    • cites [287] source

      Not checked in the free run.

    • cites [297]

      Not checked in the free run.

    • cites [306] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    This suggests a disconnect between organizational realities and current research, which undermines advancements.

    “This suggests a disconnect between organizational realities and current research, which undermines advancements [197,287].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [197] source

      Not checked in the free run.

    • cites [287] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The literature typically hints toward 'optimizing' fairness without transparency of the root(s) of (un)fairness, rarely extending beyond '(un)fair', typically to mirror current legal thought.

    “The literature typically hints toward łoptimizingž fairness without transparency of the root(s) of (un)fairness [192] rarely extending beyond ł(un)fairž [80,270] typically to mirror current legal thought [98].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [192] source

      Not checked in the free run.

    • cites [80]

      Not checked in the free run.

    • cites [270] source

      Not checked in the free run.

    • cites [98] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Addressing one form of bias results in another (bias in, bias out property of ML).

    “[164] have noted that this dilemma can be articulated as a bias in, bias out property of ML: i.e. addressing one form of bias results in another.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [164]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    There is the prevalent assumption that 'unfairness' has a uniform context-agnostic egalitarian valuation function for decision makers when considering different (sub)populations.

    “There is the prevalent assumption that łunfairnessž has a uniform context-agnostic egalitarian valuation function for decision makers when considering diferent (sub)populations [33,76,77].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [33]

      Not checked in the free run.

    • cites [76]

      Not checked in the free run.

    • cites [77]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ML application development follows a traditional software development model: it is modular, sequential, and based on large collections of (often) open source libraries, but methods to highlight bias, fairness, or ethical issues assume high expertise in ML development and do not consider 'on-the-street' practitioners.

    “The challenge is that ML application development follows a traditional software development model: it is modular, sequential, and based on large collections of (often) open source libraries, but methods to highlight bias, fairness, or ethical issues assume high expertise in ML development and do not consider łon-the-streetž practitioners [192,287].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [192] source

      Not checked in the free run.

    • cites [287] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Policy makers are generally not considered or involved in the ML fairness domain.

    “[287] notes that policy makers are generally not considered or involved in the ML fairness domain.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [287] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Data used to train models represent past decisions, and as such inherent bias(es) in these decisions are amplified.

    “[163,172,186,187,298] note that there is an additional challenge in the datasets used to train models: data represent past decisions, and as such inherent bias(es) in these decisions are ampliied.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [163]

      Not checked in the free run.

    • cites [172]

      Not checked in the free run.

    • cites [186] source

      Not checked in the free run.

    • cites [187]

      Not checked in the free run.

    • cites [298]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The fairness domain has relatively few open source tools available for practitioners and there is little accommodation for varying levels of technical proficiency, which undermines current advancement.

    “However, the fairness domain has relatively few open source tools available for practitioners (see online appendix) and there is little accommodation for varying levels of technical proiciency, which undermines current advancement [50,124,266,286,287].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [50] source

      Not checked in the free run.

    • cites [124]

      Not checked in the free run.

    • cites [266]

      Not checked in the free run.

    • cites [286] source

      Not checked in the free run.

    • cites [287] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Today, ML technologies are more accessible than ever.

    “Today, ML technologies are more accessible than ever.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Lowering the entry barrier to ML through democratization means an increase in (un)intentional socially insensitive uses of ML technologies.

    “Lowering the entry barrier to ML through democratization means an increase in (un)intentional socially insensitive uses of ML technologies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Approaches tend to operate on too small a subset of features raising stability concerns.

    “More realistic datasets are needed: [242] argue that approaches tend to operate on too small a subset of features raising stability concerns.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [242] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    There is worryingly little literature to discuss: we cannot comprehend intervention robustness under real world data challenges.

    “There is worryingly little literature to discuss: we cannot comprehend intervention robustness under real world data challenges.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Tackling fairness from the perspective of protected variables or groups needs methodological care, as 'fixing' one set of biases may inflate another, rendering the model as intrinsically discriminatory as a random model.

    “Tackling fairness from the perspective of protected variables or groups needs methodological care, as łixingž one set of biases may inlate another [33,76] rendering the model as intrinsically discriminatory as a random model [90,239].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [33]

      Not checked in the free run.

    • cites [76]

      Not checked in the free run.

    • cites [90]

      Not checked in the free run.

    • cites [239]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Realistically, data will never be both complete and clean.

    “Realistically this will never be the case.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The literature almost unilaterally focuses on supervised learning with an overwhelming emphasis on binary classification.

    “The literature almost unilaterally focuses on supervised learning with an overwhelming emphasis on binary classiication [26]: diversiication is needed.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given

    With very few exceptions, the approaches discussed in this article operate on the assumption of some set of (usually a priori known) 'protected variables'.

    “Linked to this, and with very few exceptions, the approaches discussed in this article operate on the assumption of some set of (usually a priori known) łprotected variablesž.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    There is a current trend in the literature to create new fairness measures as well as interventions.

    “We also note that there is a current trend in the literature to create new fairness measures as well as interventions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    Uptake of in-processing approaches requires better integration with standard ML libraries to overcome porting challenges.

    “Uptake of in-processing approaches requires better integration with standard ML libraries to overcome porting challenges.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Often test-time access to protected attributes is needed, which may not be legally permissible, and have other undesirable effects.

    “with [3] noting that often test-time access to protected attributes is needed, which may not be legally permissible, and have other undesirable efects [61].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [3]

      Not checked in the free run.

    • cites [61]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    It may often be better to fix the underlying data sample (e.g. collect more data, which better represents minority or protected groups and delay the modeling phase) than try to fix a discriminatory ML model.

    “Perhaps, as [61,96,199,298] note, it may often be better to ix the underlying data sample (e.g.collect more data, which better represents minority or protected groups and delay the modeling phase) than try to ix a discriminatory ML model.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [61]

      Not checked in the free run.

    • cites [96]

      Not checked in the free run.

    • cites [199]

      Not checked in the free run.

    • cites [298]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationcitation

    Many approaches to reduce discrimination may themselves be unethical or impractical in settings where model accuracy is critical such as in healthcare, or criminal justice scenarios.

    “As a closing thought many approaches to reduce discrimination may themselves be unethical or impractical in settings where model accuracy is critical such as in healthcare, or criminal justice scenarios [61].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [61]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Pre-processing methods assume sufficient knowledge of the data, and make assumptions over its veracity.

    “They also assume suicient knowledge of the data, and make assumptions over its veracity [76].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [76]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    [9] provides an overview of commonly used datasets in Algorithmic Fairness publications, e.g. for the evaluation of novel mitigation strategies.

    “Additionally, [9] provide an overview of commonly used datasets in Algorithmic Fairness publications, e.g. for the evaluation of novel mitigation strategies.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Pre-processing methods, which modify the training data, are at odds with policies like GDPR's right to an explanation, and can introduce new subjectivity biases.

    “Pre-processing methods, which modify the training data, are at odds with policies like General Data Protection Regulation (GDPR)'s right to an explanation, and can introduce new subjectivity biases [286].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [286] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Aequitas is an open source bias audit toolkit that focuses on standard ML metrics and their evaluation for different subgroups of a protective attribute.

    “Aequitas [13] Open source bias audit toolkit.Focuses on standard ML metrics and their evaluation for diferent subgroups of a protective attribute.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    FairTest is a generic framework that provides measures and statistical tests to detect unwanted associations between the output of an algorithm and a sensitive attribute.

    “FairTest [14] Generic framework that provides measures and statistical tests to detect unwanted associations between the output of an algorithm and a sensitive attribute.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Fairlean implements several parity-based fairness measures and algorithms for binary classification and regression, as well as a dashboard to visualize disparity in accuracy and parity.

    “Fairlean 1 Implements several parity-based fairness measures and algorithms [1,2,6] for binary classiication and regression as well as a dashboard to visualize disparity in accuracy and parity.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [1] source

      Not checked in the free run.

    • cites [2] source

      Not checked in the free run.

    • cites [6] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Audit AI implements various statistical significance tests to detect discrimination between groups and bias from standard machine learning procedures.

    “Audit AI 4 Implements various statistical signiicance tests to detect discrimination between groups and bias from standard machine learning procedures.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    AIF360 provides several pre-, in-, and post-processing approaches for binary classification and several pre-implemented datasets commonly used in fairness research.

    “AIF360 [3] Set of tools that provides several pre-, in-, and post-processing approaches for binary classiication as well as several pre-implemented datasets that are commonly used in Fairness research”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Responsibly provides datasets, metrics, and algorithms to measure and mitigate bias in classification as well as NLP (bias in word embeddings).

    “Responsibly [11] Provides datasets, metrics, and algorithms to measure and mitigate bias in classiication as well as NLP (bias in word embeddings).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ML Fairness Gym is part of Google's Open AI project, a simulation toolkit to study long-run impacts of ML decisions, and analyzes how algorithms that take fairness into consideration change the underlying data (previous classifications) over time.

    “ML Fairness Gym Part of Google's Open AI project, a simulation toolkit to study long-run impacts of ML decisions.5 Analyzes how algorithms that take fairness into consideration change the underlying data (previous classiications) over time (see e.g.[4,5,8,10,12]).”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [4]

      Not checked in the free run.

    • cites [5] source

      Not checked in the free run.

    • cites [8]

      Not checked in the free run.

    • cites [10] source

      Not checked in the free run.

    • cites [12] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Fairness provides commonly used fairness metrics (e.g., statistical parity, equalized odds) for R projects.

    “Fairness 2 Tool that provides commonly used fairness metrics (e.g., statistical parity, equalized odds) for R projects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    Fairness Measures considers quantitative definitions of discrimination in classification and ranking scenarios and provides datasets, measures, and algorithms (for ranking) that investigate fairness.

    “Fairness Measures 3 Project that considers quantitative deinitions of discrimination in classiication and ranking scenarios.Provides datasets, measures, and algorithms (for ranking) that investigate fairness.”
    Quote in the paperNot a substantial claim
190 of 190

  • The paper relies on this without support.

    The area of fairness in ML is complex and hard to penetrate for newcomers to the domain.

    “There is signiicant literature on approaches to mitigate bias and promote fairness, yet the area is complex and hard to penetrate for newcomers to the domain.”

  • The paper relies on this without support.

    Current ML libraries often do not (yet) accommodate means to ascertain social accountability.

    “All of this results in an array of challenges that can be overwhelming, and current ML libraries often do not (yet) accommodate means to ascertain social accountability.”

  • The paper relies on this without support.

    Fairness is a complex topic that has to try and balance many different contexts, multi-faceted sociocultural concepts, aspects of equality and diversity, and ethical principles.

    “However, fairness is a complex topic that has to try and balance many diferent contexts, multi-faceted sociocultural concepts (beyond race, gender, age etc.), aspects of equality and diversity, and ethical principles such as whether it is even appropriate to use ML at all.”

  • The paper relies on this without support.

    Sensitive variables often are correlated with other variables in the data.

    “For instance, sensitive variables often are correlated with other variables in the data, and approaches focusing on immunity aim to prevent these indirect efects from resulting in discrimination measured through the sensitive variable.”

  • The paper relies on this without support.

    Training data often reflects some form of underlying discrimination.

    “Approaches using causal methods recognise that the training data often relects some form of underlying discrimination.”

  • The paper relies on this without support.

    Missing data provides specific problems for transformation approaches, as it is unclear how to deal with such data samples.

    “5) missing data provides speciic problems for transformation approaches, as it is unclear how to deal with such data samples.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Techniques such as SHAP and Grad-CAM have been used in federated learning-based models to help clinicians with visual insights to find the most significant features.

    “Techniques, such as SHAP (SHapley Additive exPlanations) and Grad-CAM have been used in FL-based models, helping clinicians with visual insights to find the most significant features.”

    From Federated Learning in Smart Healthcare: A Comprehensive Review on Privacy, Security, and Predictive Analytics with IoT Integration (2024) · cites this paper as [62]

    Section 6.6 (page not given) explains that SHAP measures feature importance for specific predictions. The paper does not report SHAP or Grad-CAM being used in federated learning-based models, or providing visual insights to clinicians; searches for Grad-CAM and federated learning with clinicians found no supporting passage. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • SHAP and Grad-CAM have been used in diabetic retinopathy detection systems by producing interpretable outputs and enhancing clinician confidence.

    “It has been used in diabetic retinopathy detection systems by producing interpretable outputs, enhancing the clinician’s confidence [62].”

    From Federated Learning in Smart Healthcare: A Comprehensive Review on Privacy, Security, and Predictive Analytics with IoT Integration (2024) · cites this paper as [62]

    The passage labeled 6.6 Concluding Remarks describes SHAP as measuring feature importance for specific predictions, which supports the general point that it can produce interpretable outputs. It does not address Grad-CAM, use in diabetic retinopathy detection systems, or enhanced clinician confidence; searches for those terms found no supporting passage. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Studies such as Wan et al. [4] and Caton and Haas [5] provide comprehensive reviews of processing techniques and fairness in ML but often lack a specific focus on how these techniques impact underserved populations.

    “Studies such as Wan et al. [4] and Caton and Haas [5] provide comprehensive reviews of processing techniques and fairness in ML but often lack a specific focus on how these techniques impact underserved populations.”

    From Advancing Health Equity through Multi-Level Fairness in Health Informatics (2026) · cites this paper as [5]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Immunity to sensitive variables both addresses and overlooks proxies

    Minor

    The synopsis says immunity addresses indirect effects of sensitive variables through correlated input variables. The commentary then says immunity overlooks relationships with proxy variables, which are those correlated inputs.

1 of 1

Cites

Flagged first, then by citations

3 of the 366 references couldn’t be matched to a known paper.

Research works printed but not linked: 21

  1. Sara Ahmadian, Alessandro Epasto, Ravi Kumar, and Mohammad Mahdian. 2020. Fair correlation clustering. In Proceedings of the International Conference on Artificial Intelligence and Statistics. PMLR, 4195–4205.
  2. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica May 23 (2016).
  3. Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2019. Fairness and Machine Learning. fairmlbook.org.
  4. Suman Bera, Deeparnab Chakrabarty, Nicolas Flores, and Maryam Negahbani. 2019. Fair algorithms for clustering. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 4954–4965.
  5. Amanda Bower, Sarah N. Kitchen, Laura Niss, Martin J. Strauss, Alexander Vargas, and Suresh Venkatasubramanian. 2017. Fair pipelines. arXiv preprint arXiv:1707.00391 (2017). · arXiv
  6. Danah Boyd and Kate Crawford. 2011. Six provocations for big data. In A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society, Vol. 21. Oxford Internet Institute Oxford, UK.
  7. R. Wirth and J. Hipp. 2000. CRISP-DM: Towards a standard process model for data mining. In Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and Data Mining Vol. 1 29–39.
  8. Anshuman Chhabra, Adish Singla, and Prasant Mohapatra. 2022. Fair clustering using antidote data. In Proceedings of the Algorithmic Fairness through the Lens of Causality and Robustness Workshop. PMLR, 19–39.
  9. 10.2759/177365 · DOI
  10. Bo Cowgill and Catherine Tucker. 2017. Algorithmic bias: A counterfactual perspective. NSF Trustwor. Algor. (2017).
  11. Michele Donini, Luca Oneto, Shai Ben-David, John S. Shawe-Taylor, and Massimiliano Pontil. 2018. Empirical risk minimization under fairness constraints. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2791–2801.
  12. Bo Gao. 2015. Exploratory Visualization Design towards Online Social Network Privacy and Data Literacy. Ph.D. Dissertation. KU Leuven.
  13. Bryce W. Goodman. 2016. Economic models of (algorithmic) discrimination. In Proceedings of the 29th Conference on Neural Information Processing Systems, Vol. 6.
  14. Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2018. Recommendation independence. In Proceedings of the Conference on Fairness, Accountability and Transparency. PMLR, 187–201.
  15. Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2018. Delayed impact of fair machine learning. In Proceedings of the International Conference on Machine Learning. PMLR, 3150–3158.
  16. Bruce McKeown and Dan B. Thomas. 2013. Q methodology. Quantit. Applic. Soc. Sci. 66 (2013).
  17. 10.5555/1462129 · DOI
  18. Hao Wang, Berk Ustun, and Flavio P. Calmon. 2018. Avoiding disparate impact with counterfactual distributions. In Proceedings of the NeurIPS Workshop on Ethical, Social and Governance Issues in AI.
  19. Lequn Wang, Yiwei Bai, Wen Sun, and Thorsten Joachims. 2021. Fairness of exposure in stochastic bandits. In Proceedings of the International Conference on Machine Learning. PMLR, 10686–10696.
  20. Lauren Weber and Elizabeth Dwoskin. 2014. Are workplace personality tests fair. Wall Street J. 29 (2014).
  21. 10.5555/3367032.3367238 · DOI

Cited by

Sorted by citations

Related

Authors
Caton, S., & Haas, C.
Affiliations
  • University College Dublin (IE)
  • Vienna University of Economics and Business (AT)
ORCID
Venue
ACM Computing Surveys
Publisher
Association for Computing Machinery (ACM)
Published
2024-04-09
Volume
56
Issue
7
Pages
1-38
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
DOI
10.1145/3616865
OpenAlex
W3092541244
Semantic Scholar
fee8f63972906214b77f16cfeca0b93ee8f36ba2
Cited by
646
References
366
Citations by year
2026: 140 · 2025: 205 · 2024: 145 · 2023: 62 · 2022: 46 · 2021: 46 · 2020: 1
FWCI
137.81
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Computing Surveys (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
algorithmic fairness, algorithmic bias, post-processing, demographic parity, equal opportunities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review