CoChat
Sign Up
Cited by 117Ethics and Social Impacts of AIAI and HR TechnologiesNames, Identity, and Discrimination Research

Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey

Open access
Fabris, Alessandro, Baranowska, Nina, Dennis, Matthew J., Graus, David, Hacker, Philipp, Saldivar, Jorge, Zuiderveen Borgesius, Frederik, Biega, Asia J.·2025·ACM Transactions on Intelligent Systems and Technology
DOI 10.1145/3696457
Open this paper in CoChat — free account

Abstract

Employers are adopting algorithmic hiring technology throughout the recruitment pipeline. Algorithmic fairness is especially applicable in this domain due to its high stakes and structural inequalities. Unfortunately, most work in this space provides partial treatment, often constrained by two competing narratives, optimistically focused on replacing biased recruiter decisions or pessimistically pointing to the automation of discrimination. Whether, and more importantly what types of , algorithmic hiring can be less biased and more beneficial to society than low-tech alternatives currently remains unanswered, to the detriment of trustworthiness. This multidisciplinary survey caters to practitioners and researchers with a balanced and integrated coverage of systems, biases, measures, mitigation strategies, datasets, and legal aspects of algorithmic hiring and fairness. Our work supports a contextualized understanding and governance of this technology by highlighting current opportunities and limitations, providing recommendations for future work to ensure shared benefits for all stakeholders.

AI snapshot

Written from the paper’s full text.

TL;DR

This multidisciplinary survey synthesizes computer science, legal, and social-science literature on fairness and bias in algorithmic hiring, covering pipeline stages, bias-conducive factors, fairness measures, mitigation strategies, datasets, and EU/US law. It argues that algorithmic hiring can either mitigate or amplify structural inequalities, and that contextualized, multidisciplinary evaluation is needed to ensure shared benefits.

Key findings

  • The survey maps four hiring stages—sourcing, screening, selection, and evaluation—and identifies bias-conducive factors across institutional biases, individual preferences, and technology blindspots, including direct discrimination, horizontal/vertical segregation, cultural fit, ad delivery optimization, accessibility issues, and biased psychological or background-check tools.
  • It reports market scale: over 250 AI tools for HR, an average job posting yielding more than 100 candidates, and a 2023 survey in which 98% of 300 US HR leaders said software and algorithms would assist them with layoff decisions.
  • It unifies fairness measures (outcome, accuracy, impact, process, representational) and mitigation strategies (pre-, in-, post-processing), noting that most measures focus on outcome fairness and that proxy reduction is popular but yields limited improvements when inequity is less significant.
  • Dataset review: 20 datasets, mostly early pipeline stages—11 sourcing, 8 screening, 0 selection, 1 evaluation; gender is the most common sensitive attribute, race/ethnicity appears in 6 of 20, and disability, religion, and sexual orientation are largely missing; English and US/Netherlands contexts dominate.

Methods

  • Multidisciplinary survey carried out by a team with mixed computer science, law, and philosophy backgrounds, with input from practitioners; Sections 4–6 result from a systematic literature review, while other sections draw on technical reports, industry white papers, and legal literature.
  • Systematic review used IEEE Xplore, ACM Digital Library, and Google Scholar; query `algorithmic ∧ hiring ∧ fairness` with expanded terms; top 100 results per engine; manual inclusion of articles treating fairness in algorithmic hiring quantitatively; forward/backward snowballing and related surveys.
  • Analysis provides unified notation and dimensions for fairness measures, a taxonomy of mitigation methods, and a compilation of datasets in Table 4.

Limitations

  • Paper-stated: fairness measures often focus narrowly on local algorithmic outcomes and miss the broader socio-technical context; outcome fairness at every stage does not guarantee fair outcomes across the whole pipeline; data have reduced geographical and linguistic coverage and missing sensitive attributes; legal and privacy constraints limit sensitive-data use; most fairness approaches are restricted to proxy reduction.
  • Paper-stated: video-based systems yield the largest disparities and are unreliable under varying conditions; target variables such as employability, motivation, and job performance have questionable validity; there are no datasets for selection and only one for evaluation.
  • Evident from the text: as a survey, it does not present new empirical experiments; the systematic review is limited to top 100 results per engine and specific inclusion criteria, which may miss relevant work; dataset coverage is skewed toward English, US, and Netherlands contexts.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3696457

Cites

232 of 232 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

9 of 304 cited sources checked · 5 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 3.2 Individual preferences · p. 7

    Self-promotion gaps related to gender have been documented in the hiring domain and beyond.

    “Self-promotion gaps related to gender have been documented in the hiring domain [5] and beyond; men tend to self-evaluate higher than women even without intrinsic incentives [87].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [5] [5] Kristen M. Altenburger, Rajlakshmi De, Kaylyn Frazier, Nikolai Avteniev, and Jim Hamilton. 2017. Are There Gender Differences in Professional Self-Promotion? An Empirical Case Study of LinkedIn Profiles Among Recent MBA Graduates. In Proc. of the Eleventh International Conf. on Web and Social Media, ICWSM 2017, Montréal, Québec, Canada, May 15-18, 2017 . AAAI Press, 460-463. https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15615 - source
  • Premisecitation
    § 3.3 Technology blindspots · p. 8

    Maximization of cost-effectiveness based on ad delivery metrics, such as impressions or clicks, makes delivery skewed in accordance with gender and race stereotypes in jobs.

    “Multiple studies have shown that maximization of cost-effectiveness based on ad delivery metrics, such as impressions or clicks, makes delivery skewed in accordance with gender and race stereotypes in jobs [4, 10, 61, 158].”
    Quote in the paperNot a substantial claim1 of 4 sources checked
    • cites [4] [4] Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination through Optimization: How Facebook's Ad Delivery Can Lead to Biased Outcomes. Proc. ACM Hum. Comput. Interact. 3, CSCW (2019), 199:1-199:30. https://doi.org/10.1145/3359301 - source
    • cites [10] [10] Julia Angwin, Noam Scheiber, and Ariana Tobin. 2017. Dozens of Companies Are Using Facebook to Exclude Older Workers From Job Ads . Machine Bias. ProPublica, New York, NY, USA. https://www.propublica.org/article/facebookads-age-discrimination-targeting -

      Not checked in the free run.

    • cites [61]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [158]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingcitation
    § 4.3 Measures · p. 15

    Measures to quantify stereotypes and representational harms are relatively understudied.

    “Measures to quantify stereotypes and representational harms are relatively understudied [1, 92], despite being a fundamental driver in the reinforcement of societal biases.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [1] [1] Mohsen Abbasi, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2019. Fairness in representation: quantifying stereotyping as a representational harm. In Proc. of the 2019 SIAM International Conf. on Data Mining, SDM 2019, Calgary, Alberta, Canada, May 2-4, 2019 , Tanya Y. Berger-Wolf and Nitesh V. Chawla (Eds.). SIAM, 801-809. https://doi.org/10.1137/1.9781611975673.90 - source
    • cites [92]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation
    § 3.3 Technology blindspots · p. 8

    Ad text and images can further skew the audience.

    “Ad text and images can further skew the audience [4].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [4] [4] Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination through Optimization: How Facebook's Ad Delivery Can Lead to Biased Outcomes. Proc. ACM Hum. Comput. Interact. 3, CSCW (2019), 199:1-199:30. https://doi.org/10.1145/3359301 - source

      The ‘Individual components’ impact on ad delivery’ section (p. 15) finds that adding an image dramatically changed the gender composition of the audience. It also reports that adding text without an image did not appreciably change delivery, so the claim is not fully supported for text. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 8.3 Operational challenges · p. 36

    Direct discrimination is generally much more difficult to legally justify than indirect discrimination.

    “Direct discrimination, in turn, is generally much more difficult to legally justify [2].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [2] [2] Jeremias Adams-Prassl, Reuben Binns, and Aislinn Kelly-Lyth. 2023. Directly discriminatory algorithms. The Modern Law Review 86, 1 (2023), 144-175. - source
  • Classified as probably not a substantial claim. 161
  • May combine two claims. 130
  • The quote only partly states this claim. 2

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.3 Measures · p. 13

    Representation in Positive Predicted (RPP) is used in noncooperative audits where the overall population of interest for an algorithm is unknown.

    “Representation in Positive Predicted (RPP) [4, 132, 199] is a measure used in noncooperative audits, where the overall population of interest for an algorithm is unknown.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [4] [4] Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination through Optimization: How Facebook's Ad Delivery Can Lead to Biased Outcomes. Proc. ACM Hum. Comput. Interact. 3, CSCW (2019), 199:1-199:30. https://doi.org/10.1145/3359301 - source

      The auditing section describes outside auditors treating unavailable algorithms as black boxes, and the measuring-statistical-significance section (p. 199:13) measures representation as the fraction of an attribute group among people reached by an ad. The paper does not call this measure “Representation in Positive Predicted (RPP)” or say it is used specifically when the overall population of interest is unknown. (LLM, conf=0.80)

      Experimental — accuracy not yet measured
    • cites [132]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [199]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 5.4 Discussion · p. 25

    Legislation against disparate treatment (US) and direct discrimination (EU) prohibits basing hiring decisions on protected attributes.

    “which prohibit basing hiring decisions on protected attributes [2, 158]”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [2] [2] Jeremias Adams-Prassl, Reuben Binns, and Aislinn Kelly-Lyth. 2023. Directly discriminatory algorithms. The Modern Law Review 86, 1 (2023), 144-175. - source
    • cites [158]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 5.4 Discussion · p. 25

    Proxy reduction gained popularity because it aligns with legislation against disparate treatment and direct discrimination and because it does not require run-time access to sensitive attributes.

    “This approach probably gained popularity because it aligns with legislation against disparate treatment (US) and direct discrimination (EU), which prohibit basing hiring decisions on protected attributes [2, 158] and because it does not require run-time access to sensitive attributes.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [2] [2] Jeremias Adams-Prassl, Reuben Binns, and Aislinn Kelly-Lyth. 2023. Directly discriminatory algorithms. The Modern Law Review 86, 1 (2023), 144-175. - source

      The Introduction (page unavailable) says that treating people less favourably because of a protected characteristic constitutes direct discrimination, supporting the legal premise. The Direct Algorithmic Discrimination section (page unavailable) adds that using a proxy can also constitute direct discrimination. Our copy does not establish why proxy reduction gained popularity or whether it avoids run-time access to sensitive attributes. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [158]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § 4.1 Dimensions of Fairness Measures · p. 11

    The same model can lead to different results if used in different ways, including different thresholds for classifiers or different cutoffs for rankers.

    “The same model can lead to different results if used in different ways, including the use of different thresholds for classifiers or different cutoffs for rankers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation
    § 4.3 Measures · p. 15

    Salary Difference (SD) is the only measure of impact fairness proposed in the literature.

    “Salary Difference (SD) [222] is the only measure of this family proposed in the literature.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation
    § 4.4 Discussion · p. 18

    The field of algorithmic hiring fairness is strongly influenced by the 80% rule, which is appreciated as a quantitative rule of thumb but also often criticized.

    “Most of the fairness measures considered in hiring focus on outcome equity; the field is strongly influenced by the 80% rule (Section 8.2), which is appreciated as a quantitative rule of thumb, but also often criticized [74, 267].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [74]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [267]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning
    § 4.4 Discussion · p. 17

    Measures of accuracy fairness such as BCRD and MID are especially difficult to read in absolute terms.

    “This is especially true for measures of accuracy fairness such as BCRD and MID.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given
    § 4.4 Discussion · p. 19

    The version of DI reported in Equation (3) computes the ratio between the acceptance rates of the worse-off group (min) and the best-off group (max), ignoring prior knowledge of structural inequality and affected communities.

    “For example, the version of DI reported in Equation (3) computes the ratio between the acceptance rates of the worse-off group (min) and the best-off group (max), ignoring prior knowledge of structural inequality and affected communities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given
    § 4.4 Discussion · p. 19

    Although it is commonly held in fairness research that disparities against disadvantaged groups should be mitigated, there is less consensus on how to treat algorithms that favor historically disadvantaged communities.

    “Although it is commonly held, especially in fairness research, that disparities against disadvantaged groups should be mitigated, there is less consensus on how to treat algorithms that happen to favor historically disadvantaged communities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation
    § 4.3 Measures · p. 15

    Sensitive AUC (sAUC) is a widely used measure of the information about sensitive attributes stored in (proxy) non-sensitive attributes.

    “Sensitive AUC (sAUC) [33, 119, 120, 194, 196, 222] is a widely used measure of the information about sensitive attributes stored in (proxy) non-sensitive attributes.”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [33]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [119]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [194]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [196]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [222]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § 4.3 Measures · p. 17

    Mutual Information Amplification (MIA) is an ex-post, model-dependent measure.

    “Differently from sAUC and GTR, MIA is an ex-post , model-dependent measure.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § 4.3 Measures · p. 15

    In algorithmic hiring, the literature focuses on the amount of information on sensitive attributes s encoded by non-sensitive features x and the target variable y, with the goal of minimizing it.

    “In algorithmic hiring, the literature focuses on the amount of information on sensitive attributes 𝑠 encoded by non-sensitive features 𝑥 and the target variable 𝑦 , with the goal of minimizing it.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § 4.4 Discussion · p. 18

    Process fairness is operationalized with reference to the information on sensitive attributes encoded in the remaining variables.

    “Departing from this approach, process fairness is operationalized with reference to the information on sensitive attributes encoded in the remaining variables.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § 4.4 Discussion · p. 18

    Measures of process fairness such as sAUC and MIA detect the presence of sensitive attribute proxies.

    “Measures of process fairness such as sAUC and MIA detect the presence of sensitive attribute proxies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning
    § 6.4 Discussion · p. 30

    The target variables in Table 4 are very diverse.

    “the target variables in Table 4 are very diverse”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The datasets collected in Table 4 are representative of the research on fairness in algorithmic hiring.

  • May combine two claims.

    Findingnone given
    § 6.4 Discussion · p. 30

    Focusing on screening datasets, we find different constructs (e.g. communication skills vs. commitment) annotated by people with different competencies (AMT workers vs. experts) from disparate data sources (YouTube videos vs. mock interviews).

    “Focusing on screening datasets, we find different constructs (e.g. communication skills vs. commitment) annotated by people with different competencies (AMT workers vs. experts) from disparate data sources (YouTube videos vs. mock interviews)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The datasets collected in Table 4 are representative of the research on fairness in algorithmic hiring.

  • May combine two claims.

    Findingown data
    § 6.4 Discussion · p. 29

    Datasets with geographical information primarily represent US citizens (6 datasets) and the Netherlands (3 datasets).

    “datasets with geographical information primarily represent US citizens (6 datasets) and the Netherlands (3 datasets)”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The datasets collected in Table 4 are representative of the research on fairness in algorithmic hiring.

  • May combine two claims.

    Findingnone given
    § 6.4 Discussion · p. 29

    Gender is by far the most common sensitive attribute, overwhelmingly encoded as binary.

    “Gender is by far the most common sensitive attribute, overwhelmingly encoded as binary”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The datasets collected in Table 4 are representative of the research on fairness in algorithmic hiring.

  • May combine two claims.

    Findingown data
    § 6.4 Discussion · p. 29

    There are no datasets for selection, and only one for evaluation (IBM HR Analytics).

    “We found no datasets (and consequently no studies) for selection, and only one for evaluation ( IBM HR Analytics )”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The datasets collected in Table 4 are representative of the research on fairness in algorithmic hiring.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    The additional search steps found one additional dataset (IBM HR Analytics) and no additional article.

    “finding one additional dataset (IBM HR Analytics) and no additional article.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3.1 Institutional biases · p. 7

    Biased employee evaluation has different forms and derives from multiple causes.

    “Biased employee evaluation has different forms and derives from multiple causes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Discrimination has shaped residential patterns and influenced public transportation, so workplace proximity may have disproportionate impacts along racial and ethnic lines.

    “Since discrimination has shaped residential patterns and influenced public transportation, this factor may have disproportionate impacts along racial and ethnic lines [144].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [144]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Wage gaps have several concurrent causes, including expectations reinforced by the status quo and lower success in salary negotiation.

    “This BCF has several concurrent causes, including expectations reinforced by the status quo and lower success in salary negotiation [68, 110, 163].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [68]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [110]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [163]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 3.1 Institutional biases · p. 7

    In the hiring domain, agency is deemed important for leadership positions and it is harder to demonstrate for women without giving an impression of low communion.

    “This is especially problematic in the hiring domain, where agency is deemed important for leadership positions and it is harder to demonstrate for women without giving an impression of low communion [160].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [160]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 2

    Algorithmic hiring runs the risk of reinforcing and amplifying biases, causing harm and hindering trustworthiness.

    “While algorithmic hiring represents an opportunity to mitigate these biases, it also runs the risk of reinforcing and amplifying them, causing harm and hindering trustworthiness [168].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [168]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 3.1 Institutional biases · p. 6

    When translated into data and dignified with a ground truth status, vertical segregation leads to models that reinforce wage gaps and lack of diversity in high-status positions.

    “When translated into data and dignified with a ground truth status, vertical segregation leads to models that reinforce wage gaps [222] and lack of diversity in high-status positions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [222]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey · p. 1

    Whether and what types of algorithmic hiring can be less biased and more beneficial to society than low-tech alternatives currently remains unanswered, to the detriment of trustworthiness.

    “Whether, and more importantly what types of , algorithmic hiring can be less biased and more beneficial to society than low-tech alternatives currently remains unanswered, to the detriment of trustworthiness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.1 Institutional biases · p. 6

    Prior experience is considered a fundamental predictor of suitability for a given position.

    “Prior experience is considered a fundamental predictor of suitability for a given position [101].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [101]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § 3 BIAS CONDUCIVE FACTORS · p. 5

    Hiring datasets display measurement errors whose severity varies with protected groups, most critically in target variables related to employability, reflecting current biases in human ratings.

    “First, hiring datasets display measurement errors whose severity varies with protected groups, most critically in target variables related to employability, reflecting current biases in human ratings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3 BIAS CONDUCIVE FACTORS · p. 5

    There are higher-order effects caused by technological blindspots and biases in external tools integrated into the hiring pipeline.

    “Finally, there are some higher-order effects caused by technological blindspots and biases in external tools integrated into the hiring pipeline.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3 BIAS CONDUCIVE FACTORS · p. 5

    Biased decisions at early stages result in downstream samples misrepresenting certain groups.

    “Moreover, biased decisions at early stages result in downstream samples misrepresenting certain groups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 INTRODUCTION · p. 2

    Algorithmic hiring represents an opportunity to mitigate biases in hiring.

    “While algorithmic hiring represents an opportunity to mitigate these biases, it also runs the risk of reinforcing and amplifying them, causing harm and hindering trustworthiness [168].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [168]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3.1 Institutional biases · p. 7

    Models that predict the likelihood that candidates will accept an offer can reinforce existing wage gaps.

    “Models that predict the likelihood that candidates will accept an offer can reinforce existing wage gaps.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 3.1 Institutional biases · p. 7

    Two key drivers of biased employee evaluation are stereotypes and employee-manager relationships.

    “Two key drivers are stereotypes and employee-manager relationships.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Gender and race stereotypes about competence drive people's perception on the workplace and potentially bias supervisor evaluations against female and black employees.

    “Gender and race stereotypes about competence drive people's perception on the workplace and potentially bias supervisor evaluations against female and black employees [116, 229, 233, 242].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [116]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [229]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [233]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [242]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § 3 BIAS CONDUCIVE FACTORS · p. 5

    These patterns put some job seekers at a systematic disadvantage based on sensitive attributes such as age, disability, gender, religion or belief, racial or ethnic origin, or sexual orientation.

    “These patterns put some job seekers at a systematic disadvantage based on sensitive attributes such as age, disability, gender, religion or belief, racial or ethnic origin, or sexual orientation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Elitism in recruiter evaluations favors candidates educated at prestigious institutions who can also list specific extracurricular accomplishments correlated with socioeconomic status, further entrenching family status and social class.

    “Elitism in recruiter evaluations favors candidates educated at prestigious institutions who can also list specific extracurricular accomplishments correlated with socioeconomic status [218], further entrenching family status and social class [51, 184].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [218]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [51]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [184]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3 BIAS CONDUCIVE FACTORS · p. 6

    Video interviews and resumes contain a wealth of information on gender, race, and other sensitive attributes, which can lead algorithms to learn stereotypical associations between sensitive and target variables encoded in the data.

    “For instance, video interviews and resumes contain a wealth of information on gender, race, and other sensitive attributes [63, 69, 189, 218, 222], which can lead algorithms to learn stereotypical associations between sensitive and target variables encoded in the data.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [63]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [69]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [189]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [218]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [222]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3.1 Institutional biases · p. 6

    Job segregation plays a fundamental role in hiring decisions.

    “Job segregation (i.e. the world today) plays a fundamental role in hiring decisions (i.e. the world tomorrow).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 6

    Recent studies have highlighted glass ceiling effects for non-binary workers, racial minorities, and intersectional identities.

    “Predominantly analyzed with respect to binary gender [53, 82], recent studies have also highlighted glass ceiling effects for non-binary workers [62], racial minorities [117], and intersectional identities [273].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [62]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [117]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [273]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 3 BIAS CONDUCIVE FACTORS · p. 5

    Disparities may exist in data sample composition and feature values across sensitive groups that reflect and amplify problematic structural differences in society.

    “Overall, disparities may exist in data sample composition and feature values across sensitive groups that reflect and amplify problematic structural differences in society.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Workplace proximity and availability of reliable commutes can influence job satisfaction and candidate-employer interactions, with amplifying effects introduced by technology.

    “Workplace proximity and availability of reliable commutes can influence job satisfaction [201] and candidate-employer interactions [262], with amplifying effects introduced by technology [142].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [201]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [262]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [142]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 INTRODUCTION · p. 2

    The average job posting yields more than 100 candidates.

    “The average job posting yields more than 100 candidates [101, 206].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [101]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [206]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3.1 Institutional biases · p. 6

    Cultural fit is often considered predictive of a candidate's ability to conform and adapt to the core values and collective behaviors of an organization.

    “Cultural fit is often considered predictive of a candidate's ability to conform and adapt to the core values and collective behaviors of an organization.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 3 BIAS CONDUCIVE FACTORS · p. 6

    Sensitive attribute proxies allow models to learn and reflect the diverse BCFs described in the following section.

    “Sensitive attribute proxies allow models to learn and reflect the diverse BCFs described in the following section.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3 BIAS CONDUCIVE FACTORS · p. 5

    Agency is stereotypically associated with men, while communion is frequently associated with women.

    “For example, agency (orientation towards leadership and goal attainment) is stereotypically associated with men, while communion (warmth and propensity for care) is frequently associated with women [72, 118], with far-reaching effects of gender roles and expectations in employment [118, 215].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [72]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [118]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey · p. 1

    Most work in algorithmic hiring fairness provides partial treatment, constrained by two competing narratives: optimistically focused on replacing biased recruiter decisions or pessimistically pointing to the automation of discrimination.

    “Unfortunately, most work in this space provides partial treatment, often constrained by two competing narratives, optimistically focused on replacing biased recruiter decisions or pessimistically pointing to the automation of discrimination.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 INTRODUCTION · p. 2

    The debate on algorithmic hiring fairness is often polarized between techno-enthusiasm and pessimism due to partial perspectives on a field that is large and complex.

    “The debate on this topic is often polarized between techno-enthusiasm [139, 177] and pessimism [9, 132] due to partial perspectives on a field that is large and complex.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [139]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [177]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9] [9] Lori Andrews and Hannah Bucher. 2022. Automating Discrimination: AI Hiring Practices and Gender Inequality. Cardozo L. Rev. 44 (2022), 145. -
    • cites [132]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: The field of algorithmic hiring is large and complex.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 INTRODUCTION · p. 2

    There are over 250 Artificial Intelligence (AI) tools for HR on the market.

    “By one count, there are over 250 Artificial Intelligence (AI) tools for HR on the market [99]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [99]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation
    § 4.1 Dimensions of Fairness Measures · p. 11

    The conditional fairness dimension is strongly connected to world views and to the extent to which differences in variables between sensitive groups are influenced by measurement errors and unjust social structures.

    “This dimension is strongly connected to world views [100, 123], and the extent to which differences in variables between sensitive groups are influenced by measurement errors and unjust social structures.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [100]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [123]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 4.3 Measures · p. 13

    Demographic Disparity (DD) may be easier to interpret than Disparate Impact (DI), but fails to capture large disparities when acceptance rates are small.

    “This approach may be easier to interpret [286], but fails to capture large disparities when acceptance rates are small.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [286]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 3.4 Overlaps and Interactions · p. 10

    Increased representation of a minority group in a company can improve job satisfaction, increase their importance and visibility, and encourage other minority applicants in future hiring rounds.

    “For example, an increased representation of a minority group in a company can improve job satisfaction, increase their importance and visibility, and encourage other minority applicants in future hiring rounds.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [155]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [174]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 3.2 Individual preferences · p. 7

    This fact may be reflected in data sets as a lower tenure for these groups, which can be penalized by algorithms trained to maximize tenure in order to reduce hiring costs and retain human capital.

    “This fact may be reflected in data sets as a lower tenure for these groups, which can be penalized by algorithms trained to maximize tenure in order to reduce hiring costs and retain human capital.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 3.3 Technology blindspots · p. 8

    The platform(s) chosen to run a campaign can introduce a further bias in favor of its predominant demographics; campaigns run on platforms that cater to younger users are less likely to reach older segments of the population.

    “It should be noted that the platform(s) chosen to run a campaign can introduce a further bias in favor of its predominant demographics; campaigns run on platforms that cater to younger users, for example, are less likely to reach older segments of the population.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.3 Technology blindspots · p. 8

    Asynchronous video interviews can produce specific patterns from candidates with speech impairment or mismeasure input from candidates with sight impairment, which are judged unfavorably by algorithmic recruitment models.

    “Asynchronous video interviews can produce specific patterns from candidates with speech impairment (e.g. short answers) or mismeasure input from candidates with sight impairment (e.g. eye contact), which are judged unfavorably by algorithmic recruitment models [189].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [189]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.3 Measures · p. 14

    Log Rank Regression (LRR) is typical of situations with limited access to the model(s) responsible for outcomes, a common setting for external noncooperative audits.

    “Fitting models to quantify the influence of a sensitive feature on an outcome variable is typical of situations with limited access to the model(s) responsible for the outcomes. This is a common setting for external noncooperative audits.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.2 Individual preferences · p. 8

    Different interpretations have been advanced for salary negotiation differences, including lower risk aversion and the perceived chance of success.

    “Different interpretations have been advanced, including lower risk aversion and the perceived chance of success [110, 122].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [110]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [122]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.2 Individual preferences · p. 8

    Job descriptions with unrealistic requirements discourage job seekers who lack one or more requirements, with a repulsive effect that can be stronger for candidates from vulnerable groups.

    “On the contrary, job descriptions with unrealistic requirements discourage job seekers who lack one or more requirements, with a repulsive effect that can be stronger for candidates from vulnerable groups [101, 179].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [101]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [179]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.3 Technology blindspots · p. 8

    Skew happens even though advertisers design neutral campaigns and is exacerbated if they target specific attributes.

    “This happens even though advertisers design neutral campaigns and is exacerbated if they target specific attributes [258], increasing the opportunity for bias and opacity at the sourcing stage.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [258]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.2 Individual preferences · p. 8

    Explicitly mentioning requirements such as community outreach can signal the posture of an employer and act as a pull factor for minority job seekers.

    “Explicitly mentioning requirements such as community outreach can signal the posture of an employer and act as a pull factor for minority job seekers [192, 234].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [192]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [234]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 4.3 Measures · p. 13

    True Positive Rate Difference (TPRD) is closely related to equal opportunity and separation and presupposes the availability of a ground-truth variable y to condition on.

    “This measure is closely related to equal opportunity [114] and separation [17] and presupposes the availability of a ground-truth variable 𝑦 to condition on:”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [114]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [17] [17] Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2019. Fairness and Machine Learning: Limitations and Opportunities . fairmlbook.org. http://www.fairmlbook.org. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.2 Individual preferences · p. 8

    Salary negotiation differences between men and women are documented, including in propensity and strategy.

    “Salary negotiation differences between men and women are documented, including in propensity [163] and strategy [110].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [163]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [110]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4.1 Dimensions of Fairness Measures · p. 11

    A measure providing a contextualized and accurate operationalization of a construct defined in the law displays normativity but inherits all the limitations of the underlying law.

    “For example, while a measure providing a contextualized and accurate operationalization of a construct defined in the law displays normativity, the measure will inherit all the limitations of the underlying law.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Well-connected candidates have an inherent advantage over candidates with lower centrality; this advantage may be amplified by algorithms for link prediction.

    “Well-connected candidates have an inherent advantage over candidates with lower centrality; this advantage may be amplified by algorithms for link prediction [264].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [264]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.3 Technology blindspots · p. 9

    Automation bias leads to over-reliance on technology and reduced human oversight.

    “More general examples include position bias that causes underexposure risks for candidates that are not at the top of a ranking [60] and automation bias leading to over-reliance on technology and reduced human oversight [106, 156].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [106]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [156]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given
    § 4.1 Dimensions of Fairness Measures · p. 10

    Impact fairness measures require a broader understanding and modeling of the sociotechnical system around the decision-making algorithm.

    “These measures are rare in the literature, as they require a broader understanding and modeling of the sociotechnical system around the decision-making algorithm.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 3.2 Individual preferences · p. 7

    Self-promotion gaps reflect on the visibility and perceived competence of candidates at the different stages of the hiring pipeline; it can be especially difficult to subvert for women due to the social and economic penalties incurred for violating female gender stereotypes.

    “This reflects on the visibility and perceived competence of candidates at the different stages of the hiring pipeline; it can be especially difficult to subvert for women due to the social and economic penalties incurred for violating female gender stereotypes [182].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [182]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4.1 Dimensions of Fairness Measures · p. 10

    Accuracy fairness measures lack the notion of preferable outcomes and candidate benefits.

    “Measures in this family lack the notion of preferable outcomes and candidate benefits.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3.3 Technology blindspots · p. 9

    Platforms are more likely to have rich profiles and metadata for their common users, while lacking information for less engaged job seekers, who are penalized due to missing data.

    “Moreover, platforms are more likely to have rich profiles and metadata for their common users, while lacking information for less engaged job seekers, who are penalized due to missing data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.2 Individual preferences · p. 8

    People with a migration background are less likely to own a motor vehicle.

    “Furthermore, people with a migration background are less likely to own a motor vehicle [146].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [146]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.3 Technology blindspots · p. 9

    These databases contain a wealth of pre-conviction information, such as arrest data whose validity as a proxy for crime is questionable.

    “These databases contain a wealth of pre-conviction information, such as arrest data [157] whose validity as a proxy for crime is questionable [97].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [157]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [97]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 3.4 Overlaps and Interactions · p. 10

    With time, increased minority representation contributes to reducing vertical segregation, with minority employees in key positions shaping company culture and attracting more candidates with similar background.

    “With time, this contributes to reducing vertical segregation, with minority employees in key positions shaping company culture, and attracting more candidates with a similar background.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [155]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [174]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3.4 Overlaps and Interactions · p. 9

    Intersectional identities are particularly vulnerable as they are likely to be affected by multiple BCFs.

    “Intersectional identities are particularly vulnerable as they are likely to be affected by multiple BCFs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 4 MEASURES · p. 10

    The next three sections are based on a systematic review of the literature summarized in Appendix A.

    “It is worth noting that the next three sections are based on a systematic review of the literature summarized in Appendix A.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 3.4 Overlaps and Interactions · p. 9

    BCFs are interlocking and overlapping insofar as they are mutually reinforced by the structure of institutions, individual preferences, and technological blind spots.

    “It is important to note that these factors are interlocking and overlapping insofar as BCFs are mutually reinforced by the structure of institutions (Section 3.1), individual preferences (Section 3.2), and technological blind spots (Section 3.3).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.3 Technology blindspots · p. 9

    This causes an overrepresentation of the former in the training data, leading to rich-get-richer dynamics in job platforms.

    “This causes an overrepresentation of the former in the training data, leading to rich-get-richer dynamics in job platforms [185].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [185]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3.2 Individual preferences · p. 8

    This difference may be related to gender roles with respect to household and childcare responsibilities.

    “This difference may be related to gender roles with respect to household and childcare responsibilities.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1 Institutional biases · p. 7

    Social networks topology influences job seekers' awareness of openings and their likelihood of being successfully screened by recruiters due to a successful referral.

    “Social networks topology influences job seekers' awareness of openings [96], and their likelihood of being successfully screened by recruiters due to a successful referral [136, 284].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [96]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [136]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [284]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 3.3 Technology blindspots · p. 8

    Accessibility issues and ableist norms can discourage disabled people from job applications and tilt evaluations against them.

    “Accessibility issues and ableist norms can discourage disabled people from job applications [226] and tilt evaluations against them [245, 246].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [226]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [245]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [246]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 3.3 Technology blindspots · p. 8

    Off-the-shelf algorithms from these domains integrated into hiring pipelines are likely to underserve minority candidates for feature extraction and negatively affect algorithms that are based on these features.

    “Off-the-shelf algorithms from these domains integrated into hiring pipelines [141] are likely to underserve minority candidates for feature extraction and negatively affect algorithms that are based on these feature.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [141]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 3.3 Technology blindspots · p. 9

    Minority job seekers may suffer a lower quality of service, leading to disengagement and triggering a negative feedback loop by iteratively lowering training representation, quality of service, and engagement.

    “Note that minority job seekers may suffer a lower quality of service, leading to disengagement and triggering a negative feedback loop by iteratively lowering training representation, quality of service, and engagement.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.3 Technology blindspots · p. 9

    Background check tools allow employers to obtain additional information on candidates from public records.

    “Background check tools allow employers to obtain additional information on candidates from public records [244].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [244]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 4.3 Measures · p. 14

    Accuracy fairness measures are all conditional on the target variable y, inheriting the biases encoded in the ground truth.

    “They are all conditional on the target variable 𝑦 , inheriting the biases encoded in the ground truth.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.4 Discussion · p. 18

    Accuracy fairness shares the criticism of outcome fairness and has the additional limitations of being less interpretable and less aligned with the desiderata of job seekers.

    “Similar criticism can be moved to accuracy fairness, with the additional limitation that it is less interpretable and less aligned with the desiderata of job seekers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § 4.4 Discussion · p. 18

    Dedicated proxy reduction approaches can mitigate the risk of discrimination resulting from sensitive attribute proxies.

    “Dedicated proxy reduction approaches (Section 5) can mitigate the resulting risk of discrimination.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § 5.1 Pre-processing · p. 20

    Most intrinsically gendered information in word embeddings is contained within a small gender subspace.

    “this method is based on the observation that most intrinsically gendered information in word embeddings, such as the difference between the vectors for 'mother' and 'father', is contained within a small gender subspace.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 5.2 In-processing · p. 22

    Name Decorrelation improves TPRD and RMS with respect to gender but not ethnicity.

    “In terms of outcome fairness, Name Decorrelation is found to improve TPRD and RMS with respect to gender but not ethnicity.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [119]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationnone given
    § 5.4 Discussion · p. 25

    Proxy reduction achieves mitigation on a held-out test set during development but provides no guarantee at deployment.

    “this approach achieves mitigation on a held-out test set during development, but provides no guarantee at deployment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.4 Discussion · p. 24

    Balanced sampling can counter the effect of under-representation in training sets caused by horizontal and vertical job segregation.

    “Balanced sampling, for example, can counter the effect of under-representation in training sets caused by factors such as horizontal and vertical job segregation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.1 Text-based datasets · p. 26

    Chinese Bios consists of short resumes containing a binary gender indicator and a two-sentence description of job skills for IT, finance, and administration.

    “It consists of short resumes containing a binary gender indicator (he/she) and a two-sentence description of job skills for IT, finance, and administration.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [283]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4.4 Discussion · p. 18

    Impact fairness can highlight nuanced aspects of hiring beyond selection rates; SD captures wage gaps and salary negotiation differences prompting further analysis into remuneration packages.

    “Impact fairness can highlight nuanced aspects of hiring beyond selection rates; SD, for instance, captures wage gaps and salary negotiation differences prompting further analysis into remuneration packages.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 5.4 Discussion · p. 25

    Proxy reduction can mitigate large outcome disparities but does not produce convincing improvements when the inequity is less significant.

    “While this method can mitigate large outcome disparities, it does not produce convincing improvements when the inequity is less significant [33, 119, 120].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [33]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [119]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.1 Text-based datasets · p. 26

    IT Resumes comprises approximately 900,000 resumes in the historical hiring records of eight IT firms based in the US, relevant to just over 6,000 job postings from the IT sector.

    “The dataset comprises approximately 900,000 resumes (without names, emails, and URLs) in the historical hiring records of eight IT firms based in the US, relevant to just over 6,000 job postings from the IT sector.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [196]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 4.4 Discussion · p. 18

    A combination of job segregation, work gaps, and disparate performance of language processing tools, even if relatively weak on their own, may trigger a very low (unfair) value of DI when they compound.

    “To exemplify, a combination of job segregation, work gaps, and disparate performance of language processing tools, even if relatively weak on their own, may trigger a very low (unfair) value of DI when they compound.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5.4 Discussion · p. 25

    Post-processing is less popular and hybrid approaches combining different types of fairness interventions remain under-explored.

    “These are likely two key factors explaining why post-processing is less popular and why hybrid approaches combining different types of fairness interventions, e.g. mitigating via preand post-processing, remain under-explored [11].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] [11] Adam Mehdi Arafan, David Graus, Fernando P Santos, and Emma Beauxis-Aussalet. 2022. End-to-End Bias Mitigation in Candidate Recommender Systems with Fairness Gates. In 2nd Workshop on Recommender Systems for Human Resources, RecSys-in-HR 2022 . CEUR-WS, 1-8. -

      Not checked in the free run.

  • The quote only partly states this claim.

    Premisecitation
    § 5.2 In-processing · p. 22

    Face Decorrelation's sAUC is on par with adversarial methods explicitly trained to predict gender and ethnicity.

    “notably, their sAUC is on par with adversarial methods explicitly trained to predict gender and ethnicity,”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • The authors state this as an assumption: Face data contain enough information on sensitive features such as gender and race that successful debiasing can be achieved without explicit sensitive attribute information.

  • May combine two claims.

    Premisecitation
    § 6.1 Text-based datasets · p. 26

    DPG Resumes contains over 10 million vacancies and just under 1 million resumes augmented with job categories of interest and gender inferred from first names.

    “DPG Resumes [222] contains over 10 million vacancies (including salary range, working hours, and job descriptions) and just under 1 million resumes augmented with job categories of interest and gender inferred from first names.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [222]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.1 Text-based datasets · p. 26

    Bias in Bios is composed of textual biographies written in English and extracted from the Common Crawl dataset.

    “Bias in Bios , [63] is composed of textual biographies written in English and extracted from the Common Crawl dataset.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4.4 Discussion · p. 18

    Representational fairness (GBS) captures biases in representations; when applied to job descriptions, extreme values highlight stereotypically gendered language, signaling a risk of culture-based avoidance.

    “Representational fairness (GBS) captures biases in representations; when applied to job descriptions, extreme values highlight stereotypically gendered language, signaling a risk of culture-based avoidance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 5.4 Discussion · p. 25

    Having access to sensitive attributes at runtime enables precise and reliable interventions, whereas strategies that do not rely on this data must be continuously monitored to ensure they are meeting their intended goals.

    “Having access to sensitive attributes at runtime enables precise and reliable interventions, whereas strategies that do not rely on this data must be continuously monitored to ensure they are meeting their intended goals.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given
    § 5.4 Discussion · p. 24

    Proxy reduction methods target sensitive attribute proxies, reducing the overreliance of models on sensitive information.

    “Proxy reduction methods (e.g. decorrelation, adversarial inference) target sensitive attribute proxies, reducing the overreliance of models on sensitive information.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Resumes and biographies contain strong proxies for sensitive attributes such as race and gender, including names and addresses.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given
    § 5.1 Pre-processing · p. 21

    Group Norming represents an intermediate view on conditional discrimination: it prohibits inter-group discrimination based on a specific feature while allowing it as a basis for intra-group discrimination.

    “This approach represents an intermediate view on conditional discrimination: it prohibits inter-group discrimination based on a specific feature, while allowing it as a basis for intra-group discrimination.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5.4 Discussion · p. 25

    Visual signals have been removed from several products because they inevitably encode sensitive information such as race and gender while lacking a solid foundation to justify their use in the hiring domain.

    “Visual signals have been removed from several products [175] because they inevitably encode sensitive information such as race and gender while lacking a solid foundation to justify their use in the hiring domain.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [175]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 5.4 Discussion · p. 25

    Hiring algorithms based on face analysis are unlikely to predictably generalize and maintain accuracy or fairness under variable conditions, or at least less likely than algorithms based on more established data modalities and better-understood correlations with job performance.

    “Even if found to be accurate and fair in a specific evaluation, hiring algorithms based on face analysis are unlikely to predictably generalize and maintain accuracy or fairness under variable conditions, or at least they are less likely to do so than algorithms based on more established data modalities and better-understood correlations with job performance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.4 Discussion · p. 24

    Output re-ranking can mitigate the joint effect of different bias conducive factors.

    “Finally, output re-ranking can mitigate the joint effect of different bias conducive factors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5.2 In-processing · p. 22

    Face Decorrelation shows positive effects on DI for very unfair models based on the video modality and limited effects on more equitable systems based on language and audio.

    “The effectiveness of Face Decorrelation for outcome fairness is more nuanced, showing positive effects on DI for very unfair models based on the video modality and limited effects on more equitable systems based on language and audio.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • The authors state this as an assumption: Face data contain enough information on sensitive features such as gender and race that successful debiasing can be achieved without explicit sensitive attribute information.

  • May combine two claims.

    Interpretationreasoning
    § 5.2 In-processing · p. 23

    Targeting process fairness (proxy reduction) can improve outcome fairness in highly imbalanced situations, while only providing limited benefits in situations with lower inequity.

    “this result suggests that targeting this type of process fairness can improve outcome fairness in highly imbalanced situations, while only providing limited benefits in situations with lower inequity.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [119]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisereasoning
    § 4.4 Discussion · p. 19

    Gender and race are considered sensitive attributes due to the recurrent structural disadvantages incurred by women and black people.

    “Gender and race, for example, are considered sensitive attributes due to the recurrent structural disadvantages incurred by women and black people.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 5.4 Discussion · p. 25

    Subspace projection and face decorrelation function without sensitive attribute information about data subjects.

    “Furthermore, specific data modalities, such as text and vision, facilitate specialized methods like subspace projection and face decorrelation, which function without sensitive attribute information about data subjects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.4 Discussion · p. 24

    Group norming seems applicable to counter systematic biases encoded in input features against protected groups, such as biased employee evaluations.

    “Group norming seems applicable to counter systematic biases encoded in input features against protected groups, such as biased employee evaluations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 5.4 Discussion · p. 25

    Pre-processing and in-processing approaches require access to sensitive attributes only during training, not testing.

    “Conversely, pre-processing and in-processing approaches, such as importance-based scraping and adversarial inference, require access to sensitive attributes only during training, not testing.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § 4.4 Discussion · p. 18

    No single fairness measure provides a complete picture; different fairness flavors have complementary strengths and weaknesses.

    “No single measure provides a complete picture. Different fairness flavors have complementary strengths and weaknesses.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4.4 Discussion · p. 18

    Outcome fairness is based on a narrow view of algorithmic hiring as a single decision point abstracted away from its context, where equity is purely a function of algorithmic estimates and (sometimes) their similarity to target variables approximating a ground truth.

    “Indeed, outcome fairness is based on a narrow view of algorithmic hiring as a single decision point abstracted away from its context, where equity is purely a function of algorithmic estimates and (sometimes) their similarity to target variables approximating a ground truth.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.2 In-processing · p. 22

    Adversarial inference demonstrates SKL improvements in a multimodal setting based on synthetic resume ranking.

    “while Peña et al. [199] demonstrate SKL improvements in a multimodal setting based on synthetic resume ranking .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [199]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4.4 Discussion · p. 18

    Impact fairness quantifies the downstream impacts of algorithmic outcomes on different populations and models important aspects of the surrounding socio-technical system, going beyond brittle algorithmic abstraction.

    “Impact fairness quantifies the downstream impacts of algorithmic outcomes on different populations; modeling important aspects of the surrounding socio-technical system, it goes beyond brittle algorithmic abstraction.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.1 Text-based datasets · p. 26

    Engineers & Scientists is composed of applications, i.e. pairs of resumes and job postings for engineers (software and hardware) and technical scientists.

    “It is composed of applications, i.e. pairs of resumes and job postings for engineers (software and hardware) and technical scientists.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation
    § 5.4 Discussion · p. 25

    Post-processing explicitly takes into account sensitive attributes to change algorithmic outcomes for job candidates, which may be critical for disparate treatment and affirmative action (US) as well as direct discrimination and positive action (EU).

    “It is worth noting that post-processing explicitly takes into account sensitive attributes to change algorithmic outcomes for job candidates, which may be critical for disparate treatment and affirmative action (US) as well as direct discrimination and positive action (EU) [22, 112].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [22]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [112]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • The quote only partly states this claim.

    Premisecitation
    § 5.2 In-processing · p. 22

    Face Decorrelation variants reduce the sensitive information encoded in latent representations.

    “Both variants of Adversarial Removal (NS, MSE) are shown reduce the sensitive information encoded in latent representations:”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • The authors state this as an assumption: Face data contain enough information on sensitive features such as gender and race that successful debiasing can be achieved without explicit sensitive attribute information.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.3 Measures · p. 17

    Both sAUC and GTR are ex-ante measures that can be computed from the data before the final model is trained; therefore, the granularity dimension does not apply to them.

    “Both sAUC and GTR are ex-ante measures, i.e. they can be computed from the data before the final model is trained; therefore, the granularity dimension does not apply to them.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 7.3 Scalability and efficiency · p. 32

    Geyik et al. decouple their post-processing method from specific model choices and properties of input data, which means their method can naturally scale across different ranking systems at LinkedIn.

    “Geyik et al. [103] decouple their post-processing method from specific model choices and properties of input data, which means their method can naturally scale across the different (ranking) systems at LinkedIn”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [103]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 6.4 Discussion · p. 29

    Datasets that target employee tenure and performance, such as Walmart Employees and IBM HR Analytics, signal an interest from companies in understanding factors that predict future productivity and loyalty.

    “datasets that target employee tenure and performance, such as Walmart Employees and IBM HR Analytics , signal an interest from companies in understanding the factors that predict future productivity and loyalty”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6.2 Visual datasets · p. 27

    FairCVs has artificial target scores generated as a linear combination of numerical features; biased scores are derived with an ethnicity- and gender-dependent additive penalty emulating biases in the data.

    “Artificial target scores are generated for each CV, as a linear combination of numerical features; biased scores are derived from these with an ethnicity- and gender-dependent additive penalty emulating biases in the data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [199]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 6.4 Discussion · p. 30

    This can be especially problematic when using conditional fairness measures whose very definition hinges on this so-called ground truth.

    “This can be especially problematic when using conditional fairness measures whose very definition hinges on this so-called ground truth”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 7.1 Data modalities · p. 30

    GPT-4 is found to more frequently use female pronouns in reference letters generated for female-dominated occupations (such as nannies), and male pronouns for male-dominated occupations (such as plumbers).

    “GPT-4 is found to more frequently use female pronouns in reference letters generated for female-dominated occupations (such as nannies), and male pronouns for male-dominated occupations (such as plumbers) [35]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 7.5 Fairness definitions and intervention targets · p. 33

    This challenge may explain in part the popularity of disparate impact as a fairness metric, due to its ease of interpretation across a broad and diverse stakeholder group, or the popularity of complying with the EEOC's 80% rule as a fairness target (adopted e.g. by Pymetrics), as it appears, at least superficially, a legally grounded target.

    “This challenge may explain in part the popularity of disparate impact as a fairness metric, due to its ease of interpretation across a broad and diverse stakeholder group, or the popularity of complying with the EEOC's 80% rule [77] as a fairness target (adopted e.g. by Pymetrics [271]), as it appears, at least superficially, a legally grounded target”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [77]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [271]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 7.4 Sensitive attribute data availability and usage · p. 32

    This allows these methods to be deployed and run in isolation from production systems in controlled batch scenarios, which can be another important practical benefit.

    “this allows these methods to be deployed and run in isolation from production systems in controlled batch scenarios, which can be another important practical benefit”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning
    § 7.4 Sensitive attribute data availability and usage · p. 33

    A third family of bias mitigation methods that can operate without requiring access to sensitive attributes at all are the rule-based approaches described above, which are commonly used for mitigating bias in textual data such as job descriptions or resumes.

    “a third family of bias mitigation methods that can operate without requiring access to sensitive attributes at all are the rule-based approaches described above, which are commonly used for mitigating bias in textual data such as job descriptions or resumes”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 7.5 Fairness definitions and intervention targets · p. 33

    Defining fairness and formulating an intervention target is not a one-off task, as algorithmic hiring components will be deployed in different, product-specific contexts and stages of the hiring pipeline.

    “Defining fairness and formulating an intervention target is not a one-off task, as algorithmic hiring components will be deployed in different, product-specific contexts and stages of the hiring pipeline”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 7.6 Fairness vs. utility trade-offs · p. 33

    Geyik et al. find their DetGreedy method improves their fairness metric, with no significant impact on utility.

    “Geyik et al. [103] find their DetGreedy method improves their fairness metric, with no significant impact on utility”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [103]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 8.2 US law · p. 35

    The EEOC issues nonbinding but practically important guidelines to interpret Title VII.

    “Significantly, the Equal Employment Opportunity Commission (EEOC) issues nonbinding, but practically important guidelines to interpret Title VII.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 7.4 Sensitive attribute data availability and usage · p. 32

    Examples of these bias mitigation methods used by practitioners include adversarial inference, and re-balancing training data with synthetic data generation.

    “Examples of these bias mitigation methods used by practitioners include adversarial inference [222], and re-balancing training data with synthetic data generation [11]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [222]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [11] [11] Adam Mehdi Arafan, David Graus, Fernando P Santos, and Emma Beauxis-Aussalet. 2022. End-to-End Bias Mitigation in Candidate Recommender Systems with Fairness Gates. In 2nd Workshop on Recommender Systems for Human Resources, RecSys-in-HR 2022 . CEUR-WS, 1-8. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 7.4 Sensitive attribute data availability and usage · p. 33

    These methods rely solely on domain and task-specific gazetteers or dictionaries, and hence can be used when access to sensitive attributes of individuals is not available or desirable.

    “These methods rely solely on domain and task-specific gazetteers or dictionaries, and hence can be used when access to sensitive attributes of individuals is not available or desirable”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 7.1 Data modalities · p. 31

    This type of adversarial bias mitigation can be applied over a wider set of data modalities.

    “This type of adversarial bias mitigation can, however, be applied over a wider set of data modalities”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 7.5 Fairness definitions and intervention targets · p. 33

    The challenge in defining fairness and formulating intervention targets in algorithmic hiring is exacerbated by the multi-stakeholder nature of the hiring domain.

    “the challenge in defining fairness and formulating intervention targets in algorithmic hiring is exacerbated by the multi-stakeholder nature of the hiring domain”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 7.4 Sensitive attribute data availability and usage · p. 32

    Pre-processing bias mitigation methods may prove more useful, as they only require access to sensitive attributes at training time, and can operate on attributes even when available for only a subset of job seekers.

    “pre-processing bias mitigation methods may prove more useful, as they (only) require access to sensitive attributes at training time, and can operate on attributes even when available for only a subset of job seekers”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 7.1 Data modalities · p. 31

    Tabular data is a common data modality in algorithmic hiring and machine learning more broadly.

    “tabular data is a common data modality in algorithmic hiring and machine learning more broadly”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 6.4 Discussion · p. 30

    The validity of these estimates and their connection with job performance has been called into question.

    “the validity of these estimates and their connection with job performance has been called into question [19, 214]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [19] [19] Murray R Barrick and Michael K Mount. 1991. The big five personality dimensions and job performance: a metaanalysis. Personnel psychology 44, 1 (1991), 1-26. - source
    • cites [214]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.2 Visual datasets · p. 27

    Oil Company Interviews comprises AVIs with 154 students from Utrecht University carrying out a mock interview with a fictitious oil company.

    “It comprises AVIs with 154 students from Utrecht University carrying out a mock interview with a fictitious oil company.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [139]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisereasoning
    § 7.2 Tasks · p. 31

    For classification tasks, getting started with bias mitigation is relatively straightforward, as there exist several open source or freely available software packages and libraries that provide different implementations of bias measurement methods and mitigation algorithms, specifically designed for classification tasks.

    “For classification tasks, getting started with bias mitigation is relatively straightforward, as there exist several open source or otherwise freely available software packages and libraries, as mentioned above, that provide different implementations of bias measurement methods and mitigation algorithms, specifically designed for classification tasks”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6.4 Discussion · p. 29

    The findings are consistent with previous research reporting that most efforts to improve fairness in AI are influenced by the Global North, particularly the US, and focus on English language resources.

    “they are consistent with previous research reporting that most efforts to improve fairness in artificial intelligence are influenced by the Global North [187, 221], particularly the US [135], and focus on English language resources [210]”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [187]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [221]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [135]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [210]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 7.5 Fairness definitions and intervention targets · p. 33

    Once data modality, task, infrastructural and technical choices, and access to sensitive data are set, one important practical challenge is that of defining 'fairness' and formulating the intervention target of a bias mitigation method.

    “Once data modality, task, infrastructural and technical choices, and access to sensitive data are set, one important practical challenge is that of defining 'fairness' and formulating the intervention target of a bias mitigation method”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 8.2 US law · p. 35

    Disparate treatment requires an adverse action based on a protected attribute and proof of intent, unlike EU direct discrimination.

    “Importantly, disparate treatment requires not only an adverse action based on a protected attribute, but also the proof of intent on the part of the discriminating individual - unlike the EU variety of direct discrimination [254].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [254]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 7.5 Fairness definitions and intervention targets · p. 33

    Candela et al. present the (evolving) framework used at LinkedIn that guides definitions and operationalizations of AI fairness across their different products.

    “Candela et al. [39] present the (evolving) framework used at LinkedIn that guides definitions and operationalizations of AI fairness across their different (types of) products”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 7.3 Scalability and efficiency · p. 32

    Adversarial inference methods even require training multiple neural networks simultaneously, which means they can incur substantially higher costs compared to their non-mitigated counterparts.

    “Adversarial inference methods, which have been applied to a variety of data modalities and tasks, even require training multiple neural networks simultaneously, which means they can incur substantially higher costs compared to their non-mitigated counterparts”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 7.2 Tasks · p. 31

    The two most common tasks in algorithmic hiring are classification and ranking.

    “The two most common tasks in algorithmic hiring are classification and ranking”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 8.1 European law · p. 35

    Developers of high-risk AI systems must ensure that training data are appropriate and do not lead to unlawful discrimination.

    “Developers of high-risk AI systems must, for instance, ensure that the training data are appropriate and do not lead to unlawful discrimination.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [84]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 7.3 Scalability and efficiency · p. 32

    In general, post-processing approaches offer an advantage in terms of system integration, but they should be applied with care due to antidiscrimination law.

    “In general, post-processing approaches offer an advantage in terms of system integration, but they should be applied with care due to antidiscrimination law”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 7.2 Tasks · p. 31

    Ranking is a common task in the sourcing and screening stages of hiring, performed by search engines or recommender systems.

    “Ranking is a common task in the sourcing and screening stages of hiring, performed by search engines or recommender systems”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning
    § 7.1 Data modalities · p. 31

    Video-based systems tend to yield more disadvantageous effects for protected groups, also due to the strong encoding of sensitive information.

    “video-based systems tend to yield more disadvantageous effects for protected groups, also due to the strong encoding of sensitive information”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6.2 Visual datasets · p. 27

    SHL Interviews is a proprietary AVI dataset with more than 5,000 videos from 810 real job seekers from the US, UK, India, and Europe.

    “SHL Interviews [231] is a proprietary AVI dataset with more than 5,000 videos from 810 real job seekers from the US, UK, India, and Europe answering behavioral and domain knowledge questions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [231]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 8.1 European law · p. 34

    EU law distinguishes two categories of prohibited discrimination: direct and indirect.

    “EU law distinguishes two categories of prohibited discrimination: direct and indirect”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 7.1 Data modalities · p. 31

    Incoming regulation proposals prohibit inference of emotions or intentions from facial data in workplace contexts.

    “incoming regulation proposals that prohibit inference of emotions or intentions from facial data in workplace contexts”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 7.1 Data modalities · p. 31

    These types of methods are widely available in open-source bias mitigation software packages such as Fairlearn or AI Fairness 360.

    “These types of methods are widely available in open-source bias mitigation software packages such as the Fairlearn Python package [268] or the AI Fairness 360 toolkit (available in Python and R) [21]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [268]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [21] [21] Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. 2018.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 7.1 Data modalities · p. 30

    These mitigation strategies tend to be technically straightforward to implement, as they can be developed as standalone components and need not be integrated in complex algorithmic hiring pipelines.

    “These mitigation strategies tend to be technically straightforward to implement, as they can be developed as standalone components and need not be integrated in complex algorithmic hiring pipelines”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 7.5 Fairness definitions and intervention targets · p. 33

    Alignment with the company's goals and values, interpretability, and collaboration across different stakeholders were mentioned as key factors in guiding the eventual target distribution.

    “alignment with the company's goals and values, interpretability (or ability to explain), and collaboration across different stakeholders were mentioned as key factors in guiding the eventual target distribution”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [103]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 7.3 Scalability and efficiency · p. 32

    Algorithmic efficiency and computational costs will vary across bias mitigation methods in nature and magnitude, with simple rule-based scraping methods on the cheap end and bias mitigation methods that involve re-training LLMs at the resource-intensive end.

    “algorithmic efficiency and computational costs will vary across bias mitigation methods in nature and magnitude, with simple rule-based scraping methods that involve dictionary look-ups on the cheap end of the spectrum, and bias mitigation methods that involve re-training LLMs at the resource-intensive end [167]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [167]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation
    § 7.3 Scalability and efficiency · p. 32

    Such a method can be developed and deployed as a standalone component or micro-service, which decreases development time, effort, and alignment, when compared to pre-processing or in-processing methods that may need to be designed and integrated in complex algorithmic hiring pipelines.

    “This also means that such a method can be developed and deployed as a standalone component or micro-service, which decreases development time, effort, and alignment, when compared to pre-processing or in-processing methods that may need to be designed and integrated in complex algorithmic hiring pipelines”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 6.4 Discussion · p. 30

    Companies interested in algorithmic hiring solutions pick one or more target variables, balancing different priorities such as efficiency and quality of hire, with unpredictable effects on algorithmic fairness, a phenomenon called multi-target multiplicity.

    “Companies interested in algorithmic hiring solutions pick one or more of these variables, balancing different priorities such as efficiency and quality of hire, with unpredictable effects on algorithmic fairness, a phenomenon called multi-target multiplicity [266]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [266]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.2 Visual datasets · p. 27

    Student Interviews consists of video interviews with 733 upper-level undergraduate students.

    “Student Interviews [33] consists of video interviews with 733 upper-level undergraduate students,”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 8.1 European law · p. 35

    There is a proposal for an EU AI Act with many requirements for high-risk systems, including algorithmic systems for HR.

    “There is a proposal for an AI Act in the EU, with many requirements for 'high-risk' systems, including algorithmic systems for HR [84].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [84]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 7.2 Tasks · p. 31

    The prevalence of classification-based bias metrics in algorithmic hiring is reflected in open source packages, which means that measuring and mitigating bias in ranking systems tends to follow the practice of re-purposing classification methods through thresholding rankings.

    “the prevalence of classification-based bias metrics in algorithmic hiring (discussed in more detail in Section 4) is reflected in the aforementioned open source packages, which means that measuring and mitigating bias in ranking systems tends to follow the practice of re-purposing classification methods through, e.g., thresholding rankings”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 7.4 Sensitive attribute data availability and usage · p. 32

    Access to sensitive attributes may be at odds with privacy regulations such as the GDPR.

    “access to sensitive attributes may be at odds with privacy regulations such as the GDPR”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given
    § 8.1 European law · p. 34

    Direct discrimination is always prohibited, with a few narrowly defined and specific examples.

    “Direct discrimination is always prohibited, with a few narrowly defined and specific examples”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 7.4 Sensitive attribute data availability and usage · p. 32

    Many other sensitive attributes (e.g., religious beliefs) will not be easily available, and will have to be explicitly requested for bias measurement and mitigation purposes.

    “Many other sensitive attributes (e.g., religious beliefs) will not be easily available, and will have to be explicitly requested for bias measurement and mitigation purposes”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 7.1 Data modalities · p. 30

    Textual data is a common data modality encountered in the sourcing stage, typically in the form of textual CV or job description data, or in the screening stage, e.g. transcripts of video interviews.

    “Textual data is a common data modality encountered in the sourcing stage, typically in the form of textual CV or job description data, or in the screening stage, e.g. transcripts of video interviews”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 7.1 Data modalities · p. 30

    Different data modalities enable different bias mitigation strategies.

    “different data modalities enable different bias mitigation strategies”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given
    § 7.1 Data modalities · p. 31

    Mitigating bias in tabular data can be done through pre-processing approaches that directly intervene on the features, e.g. by removing features highly correlated with sensitive attributes via importance-based scraping, or increasing representation of vulnerable populations with balanced sampling.

    “Mitigating bias in tabular data can be done through pre-processing approaches that directly intervene on the features, e.g. by removing features that are highly correlated with sensitive attributes via importance-based scraping, or increasing the representation of vulnerable populations in training sets with balanced sampling”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 7.6 Fairness vs. utility trade-offs · p. 33

    In the sourcing or screening stages, depending on the intervention target, outcome fairness may come at the cost of utility; this can happen, for example, when optimizing for gender parity in heavily male or female-dominated industries.

    “In the sourcing or screening stages, depending on the intervention target, outcome fairness may come at the cost of utility; this can happen, for example, when optimizing for gender parity in heavily male or female-dominated industries”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 8.1 European law · p. 35

    The AI Act proposals contain a new exception to the GDPR to enable the use of sensitive data for debiasing algorithmic systems.

    “The proposals in the AI Act also contain a new exception to the GDPR, to enable the use of sensitive data for debiasing algorithmic systems.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 8.2 US law · p. 35

    Disparate impact closely resembles indirect discrimination and does not require intent.

    “Disparate impact, in turn, closely resembles indirect discrimination. Intent is not required [249].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [249]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 7.5 Fairness definitions and intervention targets · p. 33

    The desired distribution of the ranking is set to be equivalent to that of the underlying population of job seekers.

    “the desired distribution of the ranking is set to be equivalent to that of the underlying population of job seekers”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [103]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § 6.4 Discussion · p. 30

    Many target variables can seem reasonable at face value; therefore, initial data curation and design choices have a prominent role.

    “Many target variables can seem reasonable at face value; therefore, initial data curation and design choices have a prominent role”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation
    § 6.4 Discussion · p. 29

    The importance of both countries in the recruitment industry reflects these findings.

    “they reflect the importance of both countries in the recruitment industry”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] [3] Ifeoma Ajunwa. 2019. The paradox of automation as anti-bias intervention. Cardozo L. Rev. 41 (2019), 1671. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 8.2 US law · p. 35

    The EEOC suggested that a disadvantage usually occurs if the chance of a protected group member being positively evaluated is 80% or less than that of a privileged group member.

    “In its guidelines, the EEOC suggested that such a disadvantage usually occurs if the chance of a member of the protected group being positively evaluated is 80% or less than that of a member of the privileged group (so-called 80% rule or 4/5 rule) [77].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [77]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 7.1 Data modalities · p. 31

    Video-based algorithms are unreliable in the face of varying conditions.

    “due to video-based algorithms unreliability in the face of varying conditions”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given
    § 8.1 European law · p. 34

    For indirect discrimination, the law focuses on the effects of a practice; the intention of the alleged discriminator is not relevant.

    “For indirect discrimination, the law focuses on the effects of a practice; the intention of the alleged discriminator is not relevant”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 7.6 Fairness vs. utility trade-offs · p. 33

    Desired utility levels are just one dimension, and often not the most restrictive.

    “desired utility levels are just one dimension, and often not the most restrictive”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 7.4 Sensitive attribute data availability and usage · p. 32

    Some sensitive attributes, such as age and gender, may have to be recorded in the hiring process for identification purposes, and can hence be assumed to be available for all job seekers.

    “some sensitive attributes, such as age and gender, may have to be recorded in the hiring process for identification purposes, and can hence be assumed to be available for all job seekers”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6.2 Visual datasets · p. 27

    ChaLearn First Impression contains 10,000 YouTube video clips of people facing a camera and speaking in English.

    “ChaLearn First Impression [79] contains 10,000 YouTube video clips of people facing a camera and speaking in English.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [79]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 7.1 Data modalities · p. 30

    Collecting the required data (e.g. constructing task-specific dictionaries) leans heavily on domain and context-specific knowledge, which is typically available to practitioners in the HR domain.

    “collecting the required data (e.g. constructing task-specific dictionaries) leans heavily on domain and context-specific knowledge, which is typically available to practitioners in the HR domain”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 8.2 US law · p. 35

    US law distinguishes between two fundamental types of discrimination: disparate treatment and disparate impact.

    “For all these acts, US law distinguishes between two fundamental types of discrimination: disparate treatment and disparate impact.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 8.2 US law · p. 35

    Title VII of the Civil Rights Act of 1964 prohibits employment discrimination based on race, color, religion, sex, and national origin.

    “Title VII of the Civil Rights Act of 1964 constitutes a federal law prohibiting employment discrimination based on race, color, religion, sex, and national origin.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 6.4 Discussion · p. 30

    This reflects the length of hiring pipelines and the diversity of data journeys.

    “this reflects the length of hiring pipelines and the diversity of data journeys”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 10.1 Bias and Validity · p. 38

    Some algorithms are explicitly trained and evaluated to predict the competency scores candidates would have been given by trained human reviewers, inheriting individual biases from recruiters.

    “Some algorithms are explicitly trained and evaluated to 'predict the competency scores candidates would have been given by trained human reviewers' [189], inheriting individual biases from recruiters.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [189]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 8.2 US law · p. 36

    The Illinois AI Video Interview Act requires employers to notify and obtain consent before AI analysis of video interviews, allows candidates to request deletion, and requires employers relying solely on AI to collect and report race and ethnicity.

    “In force since the year 2020, the law requires employers to put candidates on notice, and obtain their consent, before subjecting their video interview to AI analysis. Candidates may request deletion, and employers that rely solely on AI need to collect and report candidates' race and ethnicity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [131]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    For each included article, the review performed forward and backward snowballing, pre-filtering article titles with a 'hiring ∧ fairness' query and applying the inclusion criteria from step (3).

    “For each included article, we perform forward and backward snowballing [272], pre-filtering article titles with a ' hiring ∧ fairness ' query and applying the inclusion criteria in (3).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [272]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    The review used three scholarly search engines (IEEE Xplore, ACM Digital Library, and Google Scholar) to ensure broad coverage of CS-centered scientific literature.

    “To ensure a broad coverage of the scientific literature centered on CS, we leverage three scholarly search engines: IEEE Xplore, ACM Digital Library, and Google Scholar.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    To guarantee precision, the review manually analyzed each article and selected only those that treat fairness in algorithmic hiring from a quantitative perspective (performing a fairness audit or introducing a novel method).

    “To guarantee precision, we manually analyze each article and only select the ones that treat fairness in algorithmic hiring from a quantitative perspective, i.e. performing a fairness audit or introducing a novel method.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    Some fairness measures are completely symmetric and consider advantages for vulnerable and privileged groups as equally problematic, which is generally required by law.

    “Some of these measures are completely symmetric and consider advantages for vulnerable and privileged groups as equally problematic (which is, however, generally required by the law).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 10.1 Bias and Validity · p. 38

    Algorithms cloaked in objectivity can promote bias while targeting and legitimizing ill-defined quantities.

    “Algorithms cloaked in objectivity can promote bias while targeting and legitimizing ill-defined quantities.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 10.2 Broader Context · p. 39

    Effective and equitable hiring can result from a fruitful interaction between technology and recruiters, leveraging complementary strengths.

    “Effective and equitable hiring can result from a fruitful interaction between technology and recruiters, leveraging complementary strengths.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning
    § 10.4 Law · p. 40

    Data protection law restricts analyses of hiring practices and should consider exceptions for algorithmic fairness, as suggested in the EU AI Act.

    “Data protection law restricts these analyses and should consider exceptions for algorithmic fairness, as now suggested in the EU AI Act.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given
    § 10.4 Law · p. 40

    The exact contours of legally compliant algorithmic fairness remain contested.

    “The exact contours of legally compliant algorithmic fairness remain contested.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation
    § 10.2 Broader Context · p. 39

    Predictive models for job performance and tenure risk exploitation for termination decisions and raise stronger ethical and social concerns.

    “Finally, we invite special caution in the development of predictive models for job performance and tenure, due to a risk of exploitation for termination decisions [285]; such an application of algorithms raises even stronger ethical and social concerns, which are only beginning to be discussed [223].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [285]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [223]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    Improving fairness does not require completely removing bias.

    “It is worth highlighting that improving fairness does not require completely removing bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.4 Law · p. 40

    The EU AI Act may offer a limited solution by allowing certain types of sensitive data processing to remove biases in high-risk scenarios.

    “The EU AI Act may offer a (limited) solution by allowing certain types of sensitive data processing to remove biases in high-risk scenarios.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 9 THE REAL WORLD IS MESSY · p. 37

    Non-technological factors have been shown, on occasion, to strongly influence decisions and amplify technological bias.

    “This is because non-technological factors have been shown, on occasion, to strongly influence decisions and amplify technological bias, as we will explore in this section.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § 10 OPPORTUNITIES AND LIMITATIONS · p. 38

    Algorithmic hiring benefits from and contributes to fairness and anti-discrimination work.

    “Algorithmic hiring benefits from, and contributes to, fairness and anti-discrimination work, as the previous sections have shown.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 9.2 Algorithmic Fairness · p. 37

    Changes in the socio-technical context may be difficult to detect and quantify.

    “On the other hand, these changes may be difficult to detect and quantify.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    The review used the query algorithmic ∧ hiring ∧ fairness on article titles, with each term expanded to include synonyms for algorithmic, hiring, and fairness.

    “We use the query algorithmic ∧ hiring ∧ fairness on article titles, where each term is expanded as follows:”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    Most fairness measures are focused on narrow algorithmic outcomes, neglecting the wider socio-technical context around these algorithms.

    “Most fairness measures are focused on narrow algorithmic outcomes, neglecting the wider socio-technical context around these algorithms.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    To ensure high recall, the review considered the top 100 results.

    “To ensure high recall, we consider the top 100 results.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 9.1 Algorithmic Uptake · p. 37

    Tech companies have invested heavily in digital hiring technology as suppliers or customers and are highly motivated to retain it even after the need passes.

    “These companies have invested heavily in their digital hiring technology as suppliers or customers, so are highly motivated to retain it, even once there is no longer a strong need for it.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 10.3 Data · p. 40

    Important sensitive attributes are missing from the data, making it difficult or impossible to evaluate algorithms for specific vulnerable groups.

    “In addition, important sensitive attributes are missing from the data, making it difficult or impossible to evaluate algorithms for specific vulnerable groups.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 8.3 Operational challenges · p. 36

    An algorithmic system does not cause indirect discrimination or disparate impact if it pursues legitimate aims through appropriate means.

    “In general terms, an algorithmic system does not cause indirect discrimination or disparate impact if it pursues legitimate aims through appropriate means .”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 9.1 Algorithmic Uptake · p. 37

    During the COVID-19 pandemic, delivery companies such as Amazon, Deliveroo, and DoorDash rolled out algorithmic recruitment systems to avoid viral contagion in HR teams and to experiment with technology that could save millions in HR bills.

    “During the COVID-19 pandemic, delivery companies such as Amazon, Deliveroo, and DoorDash rolled out algorithmic recruitment systems to avoid the danger of viral contagion in their HR teams, as well as experiment with a new technology that had the potential to save millions of dollars in HR bills [281].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [281]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.4 Law · p. 40

    In practice, binding regulation shapes algorithmic development more than ethical guidelines or self-regulation.

    “In practice, binding regulation shapes algorithmic development more than ethical guidelines or self-regulation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 9.2 Algorithmic Fairness · p. 37

    Changing incentive structures, the complexity of handling sensitive data, and the frequent delay between decisions and feedback in hiring further complicate data representativeness.

    “This is further complicated by changing incentive structures, by the complexity of handling sensitive data, and by the frequent delay between decisions and feedback in hiring.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 9.2 Algorithmic Fairness · p. 37

    In the presence of an exogenous shock such as the COVID-19 pandemic, workplaces are heavily affected with large consequences on hiring and recruitment.

    “However, in the presence of an exogenous shock, such as the COVID-19 pandemic, workplaces are heavily affected, with large consequences on hiring and recruitment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 8.2 US law · p. 36

    Federal legislation specifically addressing discrimination in AI systems is unlikely to emerge anytime soon in the US.

    “Apart from the acts mentioned, federal legislation specifically addressing discrimination in AI systems is unlikely to emerge anytime soon.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Algorithmic Accountability Act is stalled in a gridlocked Congress.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.1 Bias and Validity · p. 39

    Individual fairness measures can surface problematic situations for individuals that may go unnoticed when studying group fairness.

    “Individual fairness measures can surface problematic situations for individuals that may go unnoticed when studying group fairness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.3 Data · p. 40

    More data entails more scrutiny and reflection, which can inform organizational frameworks such as DEI and scholarly fields such as applied psychology and economics.

    “More data entails more scrutiny and reflection, which can inform organizational frameworks, such as Diversity, equity, and inclusion , and scholarly fields, such as applied psychology and economics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § A SYSTEMATIC REVIEW METHODOLOGY · p. 55

    The review considered articles and datasets presented in related surveys [90, 91, 173, 280] or published at dedicated venues.

    “Finally, we consider articles and datasets presented in related surveys [90, 91, 173, 280] or published at dedicated venues”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [90]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [91]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [173]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [280]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 8.3 Operational challenges · p. 36

    The answers about compliance of bias mitigation depend on affirmative action (US) and positive action (EU) law, which are in flux after the US Supreme Court's decision against Harvard's and UNC's affirmative action programs.

    “Here, the answers depend on affirmative action (US) and positive action (EU) law, which are currently in flux after the US Supreme Court's decision against Harvard's and UNC's affirmative action programs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 9.1 Algorithmic Uptake · p. 37

    Hiring technology can be more ethical than humans and reduce bias in decision-making.

    “Although hiring technology can be more ethical than humans and reduce bias in decision-making [138], it often reinforces bias and results in distinctly non-ethical outcomes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [138]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 10.4 Law · p. 40

    Precise requirements set out in future regulation and case law have the potential to influence the algorithmic hiring industry positively and profoundly.

    “Although clear guidance on algorithmic hiring is currently missing, precise requirements set out in future regulation and case law, informed by research and practice on fair algorithmic hiring, have the potential to influence the industry positively and profoundly.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 9.1 Algorithmic Uptake · p. 37

    Hiring technology often reinforces bias and results in distinctly non-ethical outcomes.

    “Although hiring technology can be more ethical than humans and reduce bias in decision-making [138], it often reinforces bias and results in distinctly non-ethical outcomes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 10.3 Data · p. 40

    Data and research are constrained by a dual tension with the privacy of data subjects and the intellectual property of companies.

    “Data and research are constrained by a dual tension with the privacy of data subjects, on one side, and the intellectual property (IP) of companies, on the other side.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    Fairness for employers is currently missing; two-sided platforms would be well-positioned to study performance across employers, especially small businesses.

    “Furthermore, fairness for employers is currently missing; two-sided platforms would be well-positioned to study the performance of their algorithms across employers, devoting special attention to small businesses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    Outcome fairness at every stage does not guarantee fair outcomes for job seekers throughout the hiring pipeline.

    “Zooming out from single-algorithm evaluations, it is worth noting that outcome fairness at every stage does not guarantee fair outcomes for job seekers throughout the hiring pipeline.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.3 Data · p. 40

    Algorithmic fairness research is contributing additional analysis into hiring practices from a perspective of diversity and non-discrimination.

    “Algorithmic fairness research is contributing additional analysis into hiring practices from a perspective of diversity and non-discrimination.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 10.4 Law · p. 40

    Special categories of personal data are often lacking and difficult to process, particularly in the EU under current data protection law.

    “Furthermore, special categories of personal data are often lacking and difficult to process, particularly in the EU under the current data protection law.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 10.4 Law · p. 40

    Promising technological approaches include multiparty computation, sample-level estimators, and noise injection mechanisms.

    “Promising technological approaches include multiparty computation [143], sample-level estimators [89], and noise injection mechanisms [137].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [143]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [89]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [137]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.1 Bias and Validity · p. 39

    Exploratory policies based on partially stochastic mechanisms can provide new information in repeated decision-making scenarios.

    “Exploratory policies based on partially stochastic mechanisms can provide new information in repeated decision-making scenarios.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § 10.4 Law · p. 40

    The restriction of fairness approaches to proxy reduction or removal is most likely due to concerns of infringing regulation on disparate treatment and direct discrimination.

    “This is most likely due to concerns of infringing regulation on disparate treatment and direct discrimination.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 9.2 Algorithmic Fairness · p. 37

    Strong shocks in the hiring domain are an issue for data representativeness more broadly, making fresh data necessary.

    “Strong shocks in the hiring domain are an issue for data representativeness more broadly. Fresh data becomes necessary.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 10.3 Data · p. 40

    Research on fairness in algorithmic hiring is based on data with reduced geographical and linguistic coverage.

    “Research on fairness in algorithmic hiring is based on data with reduced geographical and linguistic coverage.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 8.2 US law · p. 36

    NYC Local Law 144 applies from July 5, 2023 to employers using AI to substantially assist or replace discretionary decision-making in hiring, requiring impartial bias evaluations by an independent auditor.

    “From July 5, 2023, NYC Local Law 144 applies to employers using AI to substantially assist or replace discretionary decision-making in hiring. They are required to conduct and publish impartial bias evaluations by an independent auditor.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [186]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation
    § 10.2 Broader Context · p. 39

    HR professionals are better suited to assess special cases and operate under changing conditions.

    “For example, HR professionals are better suited to assess special cases and operate under changing conditions [166].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [166]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    The discovery of patterns that predict job performance for hiring can open the way to models for termination decisions.

    “Finally, it is worth noting that the discovery of patterns that predict job performance for hiring can open the way to models for termination decisions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 10.1 Bias and Validity · p. 38

    Fair and trustworthy algorithms can lead to a positive form of automation bias and attract minority candidates.

    “Fair and trustworthy algorithms can lead to a positive form of automation bias and attract minority candidates.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationcitation
    § 10.2 Broader Context · p. 39

    The prevalent human vs. algorithm evaluation framework is of limited utility; more research on recruiter-machine interaction is required.

    “The prevalent human vs. algorithm evaluation framework is of limited utility; to overcome it, more research on recruitermachine interaction is required [240], including candidate screening models [6] leading to more granular measures.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [240]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [6] [6] Jose M Alvarez and Salvatore Ruggieri. 2023. The Initial Screening Order Problem. arXiv preprint arXiv:2307.15398 (2023). - source

      Checked before — its earlier verdict is held back until a new check runs.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 10.2 Broader Context · p. 39

    Algorithmic hiring can reduce certain disparities and trigger deflating feedback loops across bias conducive factors.

    “Algorithmic hiring can reduce certain disparities and trigger deflating feedback loops across bias conducive factors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § 10.4 Law · p. 40

    Most fairness approaches developed so far are restricted to proxy reduction or removal, neglecting a wealth of solutions developed by the algorithmic fairness community.

    “Most of the fairness approaches developed so far are restricted to proxy reduction or removal, neglecting a wealth of solutions developed by the algorithmic fairness community.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 10.4 Law · p. 40

    Algorithmic fairness can conflict with privacy and non-discrimination doctrine.

    “Algorithmic fairness can conflict with privacy and non-discrimination doctrine.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 11 CONCLUSIONS · p. 40

    The social, technological, and legal landscape around algorithmic hiring is rapidly evolving, and algorithmic fairness has become a necessary component for both business-as-usual product development and frontier research.

    “The social, technological, and legal landscape around algorithmic hiring is rapidly evolving; algorithmic fairness has become a necessary component for both business-as-usual product development and frontier research.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 9.1 Algorithmic Uptake · p. 37

    Once the danger of the pandemic passed, non-technological hiring procedures could be reintroduced.

    “The recent pandemic necessitated social distancing, but once this danger had passed, non-technological hiring procedures could be reintroduced.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 10.1 Bias and Validity · p. 38

    Job performance is famously difficult to define and measure, let alone predict.

    “Job performance is famously difficult to define and measure, let alone predict [220, 263].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [220]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [263]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 9.1 Algorithmic Uptake · p. 37

    One reason delivery companies could easily roll out algorithmic recruitment systems was a regulatory environment that was laxer than usual due to emergency conditions.

    “One reason they were able to do this with ease was due to the regulatory environment, which was laxer than usual due to the ongoing emergency conditions.”
    Quote in the paperNot a substantial claim
239 of 239

  • The paper relies on this without support.

    Employers are adopting algorithmic hiring technology throughout the recruitment pipeline.

    “Employers are adopting algorithmic hiring technology throughout the recruitment pipeline.”

  • The paper relies on this without support.

    Algorithmic fairness is especially applicable in algorithmic hiring due to its high stakes and structural inequalities.

    “Algorithmic fairness is especially applicable in this domain due to its high stakes and structural inequalities.”

  • The paper relies on this without support.

    The field of algorithmic hiring is large and complex.

    “due to partial perspectives on a field that is large and complex.”

  • The authors state this as an assumption.

    Face data contain enough information on sensitive features such as gender and race that successful debiasing can be achieved without explicit sensitive attribute information.

    “This approach is based on the key assumption that face data contain enough information on sensitive features such as gender and race, so that successful debiasing can be achieved without explicit sensitive attribute information.”

  • The paper relies on this without support.

    Resumes and biographies contain strong proxies for sensitive attributes such as race and gender, including names and addresses.

    “Resumes and biographies contain strong proxies for sensitive attributes such as race and gender, including names and addresses.”

  • The paper relies on this without support.

    The datasets collected in Table 4 are representative of the research on fairness in algorithmic hiring.

    “Table 4. Datasets used in research on fairness in algorithmic hiring.”

  • The paper relies on this without support.

    The European Convention on Human Rights and the Charter of Fundamental Rights of the European Union protect the right to non-discrimination in Europe.

    “The right to non-discrimination is protected as a human right in Europe by the European Convention on Human Rights (1950) and the Charter of Fundamental Rights of the European Union (2020).”

  • The paper relies on this without support.

    The word 'disadvantage' in indirect discrimination must be interpreted in a wide way.

    “The word disadvantage must be interpreted in a wide way.”

  • The paper relies on this without support.

    The organization using an algorithmic system is responsible if the system discriminates illegally.

    “The organization is also responsible if it uses an algorithmic system provided by, for instance, a company.”

  • The paper relies on this without support.

    The Algorithmic Accountability Act is stalled in a gridlocked Congress.

    “The Algorithmic Accountability Act is stalled in a gridlocked Congress.”

  • The paper relies on this without support.

    Previous experience is a preeminent feature for assessing candidates.

    “Previous experience is a preeminent feature for assessing candidates.”

  • The paper relies on this without support.

    Current job segregation exists.

    “In conjunction with current job segregation, this means that the most important features are inevitably skewed against historically disadvantaged groups.”
12 of 12

Cites

Flagged first, then by citations

Research works printed but not linked: 26

  1. Julia Angwin, Noam Scheiber, and Ariana Tobin. 2017. Dozens of companies are using facebook to exclude older workers from job ads. Machine Bias. ProPublica, New York, NY. Retrieved from https://www.propublica.org/article/facebook-ads-age-discrimination-targeting
  2. Adam Mehdi Arafan, David Graus, Fernando P. Santos, and Emma Beauxis-Aussalet. 2022. End-to-end bias mitigation in candidate recommender systems with fairness gates. In Proceedings of the 2nd Workshop on Recommender Systems for Human Resources (RecSys-in-HR ’22). CEUR-WS, 1–8.
  3. Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2019. Fairness and Machine Learning: Limitations and Opportunities. Retrieved from http://www.fairmlbook.org
  4. Miranda Bogen and Aaron Rieke. 2018. Help Wanted: An Examination of Hiring Algorithms, Equity, and Bias. Technical Report. Upturn.
  5. 10.2308/accr-10099 · DOI
  6. Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016. Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.), 4349–4357. Retrieved from https://proceedings.neurips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html
  7. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the Conference on Fairness, Accountability and Transparency (FAT ’18). Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81, PMLR, 77–91. Retrieved from http://proceedings.mlr.press/v81/buolamwini18a.html
  8. Ian Burke, Robin Burke, and Goran Kuljanin. 2021. Fair candidate ranking with spatial partitioning: Lessons from the SIOP ML competition. In Proceedings of the 1st Workshop on Recommender Systems for Human Resources (RecSys in HR ’21) Co-located with the 15th ACM Conference on Recommender Systems (RecSys ’21), Vol. 2967.
  9. Bo Cowgill. 2018. Bias and Productivity in Humans and Algorithms: Theory and Evidence from Resume Screening. Columbia Business School, Columbia University, Vol. 29.
  10. Harrison Edwards and Amos J. Storkey. 2016. Censoring representations with an adversary. In Proceedings of the 4th International Conference on Learning Representations (ICLR ’16). Yoshua Bengio and Yann LeCun (Eds.). arXiv:1511.05897. Retrieved from http://arxiv.org/abs/1511.05897 · arXiv
  11. Ben Eubanks. 2022. Artificial Intelligence for HR: Use AI to Support and Develop a Successful Workforce. Kogan Page Publishers.
  12. A. S. Fokkens, C. J. Beukeboom, and E. Maks. 2018. Leeftijdsdiscriminatie in vacatureteksten: Een geautomatiseerde inhoudsanalyse naar verboden leeftijd-gerelateerd taalgebruik in vacatureteksten: Rapport in opdracht van het College voor de Rechten van de Mens.
  13. Joseph Fuller, Manjari Raman, Eva Sage-Gavin, and Kristen Hines. 2021. Hidden Workers: Untapped Talent. Technical Report. Harvard Business School.
  14. Jobvite. 2021. 2021 Recruiter Nation Report. Technical Report. Retrieved from https://www.jobvite.com/lp/2021-recruiter-nation-report/
  15. Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein. 2021. Noise: A Flaw in Human Judgment. Hachette UK.
  16. Nicolas Kayser-Bril. 2023. LinkedIn Automatically Rates “Out-of-Country” Candidates as “Not Fit” in Job Applications. Technical Report. AlgorithmWatch.
  17. Niki Kilbertus, Adrià Gascón, Matt J. Kusner, Michael Veale, Krishna P. Gummadi, and Adrian Weller. 2018. Blind justice: Fairness with encrypted sensitive attributes. In Proceedings of the 35th International Conference on Machine Learning (ICML ’18). Jennifer G. Dy and Andreas Krause (Eds.), Vol. 80, PMLR, 2635–2644. Retrieved from http://proceedings.mlr.press/v80/kilbertus18a.html
  18. Pauline T. Kim. 2022. Race-aware algorithms: Fairness, nondiscrimination and affirmative action. California Law Review 110 (2022), 1539.
  19. Eve A. Levin. 2018. Gender-normed physical-ability tests under Title VII. Columbia Law Review 118, 2 (2018), 567–604.
  20. ORCAA. 2020. Description of Algorithmic Audit: Pre-built Assessments. Technical Report. Retrieved from https://techinquiry.org/HireVue-ORCAA.pdf
  21. Mary-Ann Russon. 2020. Uber sued by drivers over ’automated robo-firing’. BBC News 26 (2020).
  22. Lawrence B. Solum. 2004. Procedural justice. Southern California Law Review 78 (2004), 181.
  23. Keith E. Sonderling, Bradford J. Kelley, and Lance Casimir. 2022. The promise and the peril: Artificial intelligence and employment discrimination. University of Miami Law Review 77 (2022), 1.
  24. Elmira van den Broek, Anastasia V. Sergeeva, and Marleen Huysman. 2019. Hiring algorithms: An ethnography of fairness in practice. In Proceedings of the 40th International Conference on Information Systems (ICIS ’19). Helmut Krcmar, Jane Fedorowicz, Wai Fong Boh, Jan Marco Leimeister, and Sunil Wattal (Eds.), Association for Information Systems. Retrieved from https://aisel.aisnet.org/icis2019/future_of_work/future_work/6
  25. Sarah-Jane van Els, David Graus, and Emma Beauxis-Aussalet. 2022. Improving fairness assessments with synthetic data: A practical use case with a recommender system for human resources. In Proceedings of the 1st International Workshop on Computational Jobs Marketplace (CompJobs ’22), 5 pages.
  26. Hilde Weerts Miroslav Dudík Richard Edgar Adrin Jalali Roman Lutz and Michael Madaio. 2023. Fairlearn: Assessing and improving fairness of AI systems 8 pages. Retrieved from http://jmlr.org/papers/v24/23-0389.html

Cited by

Sorted by citations

Related

Authors
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J.
Affiliations
  • Max Planck Institute for Security and Privacy (DE)
  • Radboud University Nijmegen (NL)
  • Eindhoven University of Technology (NL)
  • Randstad, Diemen, The Netherlands
  • European University Viadrina (DE)
  • Universitat Pompeu Fabra (ES)
  • Radboud University, Nijmegen, The Netherlands
ORCID
Venue
ACM Transactions on Intelligent Systems and Technology
Publisher
Association for Computing Machinery (ACM)
Published
2025-01-03
Volume
16
Issue
1
Pages
1-54
ISSN
  • 2157-6904 (ISSN-L)
  • 2157-6912
DOI
10.1145/3696457
OpenAlex
W4402747641
Semantic Scholar
768a0b0c53681a2fe901660dc1596d4dc11cb125
Cited by
117
References
213
Citations by year
2026: 75 · 2025: 39 · 2024: 3
FWCI
48.63
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Transactions on Intelligent Systems and Technology (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
  • Radboud Repository (Radboud University) (submittedVersion)
  • ArTS Archivio della ricerca di Trieste (University of Trieste https://www.units.it/) (submittedVersion, cc-by)
  • TU/e Research Portal (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • FINDHR (101070212)
  • Alexander von Humboldt-Stiftung
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
  • Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
Keywords
automated hiring, algorithmic fairness, algorithmic bias, recruitment automation, discrimination, fairness metrics, bias mitigation, technology governance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-30.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-30.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review