CoChat
Sign Up
Cited by 10Ethics and Social Impacts of AIAI and HR TechnologiesExplainable Artificial Intelligence (XAI)

Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits

Open access
Gerchick, Marissa Kumar, Encarnación, Ro, Tanigawa-Lau, Cole, Armstrong, Lena, Gutiérrez, Ana, Metaxa, Danaé·2025·Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency
DOI 10.1145/3715275.3732004
Open this paper in CoChat — free accountOpen PDF

Abstract

In this work, we "audit the audits," analyzing the documents produced pursuant to one of the United States' first enacted laws regulating the use of artificial intelligence in employment: New York City's Local Law 144.This law requires employers and employment agencies using certain types of automated tools to publish "bias audits" with statistics about how different sex and racial groups fare in the hiring process when the tools are used.We collect and conduct a comprehensive analysis of all Local Law 144 bias audits (N=116) made publicly available to our knowledge from the law taking effect in July 2023 until early November 2024, and describe the extensive challenges we faced in identifying, archiving, extracting information from, and ultimately analyzing these bias audits.We identify several ways that bias audits produced in accordance with Local Law 144 are incomplete evaluations of algorithmic bias, despite news coverage and characterizations by employers and vendors suggesting otherwise.We show that Local Law 144 bias audits are significantly hampered by several issues, including missing demographic data, opaque data aggregation, problematic uses of "test data, " and reliance on metrics that do not represent how automated hiring tools are used in practice.We analyze the reported results in Local Law 144 bias audits alongside the four-fifths rule often used as a measure for assessing adverse impact in employment contexts.Most audits do not report results that would not suggest violations of the four-fifths rule.Crucially, however, we show that these tools could often be in violation of the four-fifths rule when considering potential impacts of missing demographic data.We offer ten practical recommendations to strengthen future legislative efforts that mandate algorithm auditing in hiring and other areas, and contribute an open dataset and codebase for extracting and combining bias audit results to support future auditing efforts.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper audits New York City Local Law 144 bias audits, analyzing all 116 publicly identified audits from July 2023 through November 2024. It finds the audits are often incomplete, uninterpretable, and potentially inaccurate evaluations of algorithmic bias, and offers ten recommendations for future audit-mandating legislation.

Key findings

  • The dataset covers 116 audits in 44 audit reports, involving 31 deploying entities, 22 tool vendors, and 11 auditors; audits were found for approximately 2% of Fortune 500 companies despite over 98% reportedly using applicant tracking software.
  • Missing demographic data was pervasive: 83% of audits reported missing race and/or sex information for some applicants, and 86% did not identify the date of first AEDT use; 62% did not identify job roles.
  • Reported impact ratios often appeared compliant, but 53% of audits included at least one impact ratio below 0.8; at the group level, 23% of results included at least one impact ratio below 0.8. When accounting for missing demographic data, 70% of impact ratios had possible lower bounds below 0.8 under the paper’s main bounding assumption, falling to 28% under an alternative assumption.
  • The paper also found 54% of audits had at least one impact ratio above 1, and identified “silent duplicates” in audits all conducted by DCI Consulting Group, all but one describing HireVue tools.

Methods

  • The authors collected and analyzed all Local Law 144 bias audits publicly available to their knowledge from July 2023 through early November 2024, yielding 116 audits across 44 audit reports.
  • A team of six researchers spent hundreds of hours searching, archiving, extracting, and validating audits; they used a common data schema, automated table extraction with R libraries, manual data entry, and manual validation against source audits.
  • They conducted mixed quantitative and qualitative analysis, including impact ratio analysis, comparisons of 2023 and 2024 audits, and calculation of uncertainty bounds for impact ratios under assumptions about missing demographic data.

Limitations

  • The paper states its analysis covered less than two years of audits, that there are likely published audits they did not find, that many recommendations would be challenging to implement, and that even full implementation would likely be insufficient to ensure effective algorithm audits.
  • The text also makes evident that the sample depends on public availability and searchability, that missing data and reporting inconsistencies limited analysis, that one collected audit report was removed before validation and excluded, and that the authors were unaware of any NYC DCWP enforcement actions related to Local Law 144 at the time of writing.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3715275.3732004

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 34 cited sources checked · 0 supported · 0 flagged

  • Findingnone given

    To our knowledge as of the time of writing, NYC DCWP does not maintain a public-facing repository of LL144 bias audits.

    “To our knowledge as of the time of writing, NYC DCWP does not maintain a public-facing repository of LL144 bias audits, so locating bias audits was very time-consuming and challenging.”
    Quote in the paperSubstantial claim
  • Findingown data

    We found audits for approximately 2% of the Fortune 500 companies.

    “We found audits for approximately 2% of the Fortune 500 companies”
    Quote in the paperSubstantial claim
  • Findingown data

    We did not find any audits focused only on tools used for promotion.

    “We did not find any audits focused only on tools used for promotion, so we focus our analyses here on hiring.”
    Quote in the paperSubstantial claim
  • Findingown data

    Most Local Law 144 bias audits do not report results that would not suggest violations of the four-fifths rule.

    “Most audits do not report results that would not suggest violations of the four-fifths rule.”
    Quote in the paperSubstantial claim
  • Findingown data

    We identified several instances where results from the same underlying analyses were reported multiple times, a phenomenon we dubbed silent duplicates.

    “We identified several instances where it appeared that results from the same underlying analyses (e.g., calculations of scoring or selection rates and impact ratios for a particular tool and demographic category) were reported multiple times, a phenomenon we dubbed silent duplicates”
    Quote in the paperSubstantial claim
  • May combine two claims. 30
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 26

  • May combine two claims.

    Findingown data

    In our sample, 75% of audit reports used historical data, 16% used test data, and for 9% we could not identify the type of data used.

    “The use of test data was relatively rare in our sample; 75% of audit reports in our sample used historical data, 16% used test data, and for 9% we were not able to identify the type of data used.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    The audits we reviewed did not typically report their level of access, and LL144 does not specify an access requirement.

    “The audits we reviewed did not typically report their level of access, and LL144 does not specify an access requirement.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    23% of results across all audits included at least one impact ratio below 0.8.

    “When analyzing impact ratios at the group level (e.g., considering impact ratios for applicants broken down by sex and broken down by race for the same tool as separate sets of results), 23% of results across all audits included at least one impact ratio below 0.8.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The most common auditors in our dataset were Holistic AI (20% of audit reports), DCI Consulting Group (20%), and BABL AI (16%).

    “The most common auditors were Holistic AI (20% of audit reports), DCI Consulting Group (20%), and BABL AI (16%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All impact ratios below 0.8 resulted from analyses broken down by race or intersectional categories; none were from analyses broken down solely by sex.

    “All of these impact ratios below 0.8 resulted from analyses broken down by race or intersectional categories (none were from analyses broken down solely by sex).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Almost all 2024 audit reports used the same auditor and data type (historical or test data) as their 2023 predecessors.

    “almost all the 2024 audit reports used the same auditor and data type-historical or test data-as their 2023 predecessors”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    As of the time of writing, we are unaware of any enforcement actions taken by the NYC DCWP related to LL144.

    “unaware of any enforcement actions taken by the NYC DCWP related to LL144.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The public facing version of one vendor's bias audit did not report the number of applicants in each group or whether any candidates evaluated by the tool were missing demographic data, and thus not included in the audit calculations.

    “the public facing version of this bias audit did not (to our knowledge) report the number of applicants in each group or whether any candidates evaluated by the tool were missing demographic data, and thus not included in the audit calculations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    One audit report with silent duplicates was published by Wells Fargo and its tool vendor could not be identified.

    “there was one audit report with silent duplicates published by Wells Fargo where we could not identify the tool vendor [54].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    A dominant theme of our data collection and analysis was missingness, including missing information required to be reported under LL144, missing demographic data on large swaths of applicants, and missing basic details about auditors or tool vendors.

    “A dominant theme of our data collection and analysis was missingness-including missing information that is required to be reported under LL144, missing demographic data on large swaths of applicants, and even missing basic details about auditors or tool vendors.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the other 59% of cases, there were fewer clear trends; both the comparator group and group with a selection or scoring rate less than 0.8 varied.

    “In the other 59% of cases, there were fewer clear trends; both the comparator group and group with a selection or scoring rate less than 0.8 varied.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Two tools in the sample (HackerRank's plagiarism detection and image analysis tools, audited in both 2023 and 2024) had selection not the preferred outcome, changing the interpretation of selection rates and impact ratios, in contrast to the vast majority of AEDTs for which the paper had bias audits.

    “there were two tools in our sample (HackerRank's plagiarism detection [69,73] and image analysis tools [68,72], audited in both 2023 and 2024) for which selection was not the preferred outcome, changing the interpretation of selection rates and impact ratios, in contrast to the vast majority of AEDTs for which we had bias audits in our sample.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [69]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [73]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [68]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [72]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    All but one of the silent duplicates identified described HireVue tools.

    “all but one described HireVue tools”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Some audit reports appeared to use data from the same time period as the previous audit, while others used more recent data sometimes along with older data.

    “some audit reports (e.g., [69,73]) appeared to use data from the same time period as the previous audit, while others used more recent data (e.g., [67,71]) sometimes along with older data (e.g., [7,11]).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All silent duplicates we observed appeared in audits conducted by DCI Consulting Group, and all but one described HireVue tools.

    “all of them appeared in audits conducted by DCI Consulting Group, and all but one described HireVue tools.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    For scoring tools, LL144 audits must include the rate at which individuals in a race and/or sex category received a score above the sample's median score, when those applicants' scores were assigned by an AEDT.

    “For scoring tools, the audits must include the rate at which individuals in a race and/or sex category received a score above the sample's median score, when those applicants' scores were assigned by an AEDT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We conducted a thorough search process totaling hundreds of hours over more than a year, beginning when the law took effect in July 2023 and continuing periodically over the next 18 months.

    “We conducted a thorough search process totaling hundreds of hours over more than a year; we began searching for bias audits as soon as the law took effect in July 2023 and continued to search periodically over the course of the next 18 months.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    LL144 requires that bias audits be conducted, but only requires that bias audit summaries be published on employers' websites, and does not require that selection/scoring rates or impact ratios adhere to any particular threshold.

    “Importantly, LL144 requires that bias audits be conducted, but only requires that bias audit summaries be published on employers' websites, and LL144 does not require that selection/scoring rates or impact ratios adhere to any particular threshold [94].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [94]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The four-fifths rule is not the legal standard for discrimination; the EEOC has stated it is 'merely a rule of thumb'.

    “even though the rule is not the legal standard for discrimination (as the EEOC has stated, the rule is”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    An industry report found that more than 98% of Fortune 500 companies use applicant tracking software, which often leverages automated systems or artificial intelligence to process candidate materials.

    “despite an industry report finding more than 98% of Fortune 500 companies use applicant tracking software, which often leverages automated systems or artificial intelligence to process candidate materials [100].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [100]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    For selection tools, LL144 audits must include the rate at which individuals in a race and/or sex category are automatically selected to move forward or assigned a classification by an AEDT.

    “For selection tools, the audits must include the rate at which individuals in a race and/or sex category are automatically selected to move forward or assigned a classification by an AEDT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    For both selection and scoring tools, the impact ratio must be reported in the audit, defined as the selection or scoring rate for each race and/or sex category divided by the selection or scoring rate of the most selected race and/or sex category.

    “For both selection and scoring tools, the impact ratio must also be reported in the audit, defined as the selection or scoring rate for each race and/or sex category divided by the selection or scoring rate of the most selected race and/or sex category [94].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [94]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The four-fifths rule states that a selection rate for any race, sex, or ethnic group less than four-fifths of the rate for the group with the highest rate will generally be regarded by Federal enforcement agencies as evidence of adverse impact.

    “It states that”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [96]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    We manually reviewed all entries in our final dataset, cross-referencing them with the underlying audits to check the accuracy of our extraction.

    “For data validation, we manually reviewed all entries in our final dataset, cross-referencing them with the underlying audits to check the accuracy of our extraction.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    LL144 does not specify what filetype or format audits should take.

    “LL144 also does not specify what filetype or format audits should take”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    If the impact ratio is below 0.8, the four-fifths rule is violated.

    “if the impact ratio is below 0.8, the four-fifths rule is violated.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Local Law 144 bias audits are significantly hampered by missing demographic data, opaque data aggregation, problematic uses of 'test data,' and reliance on metrics that do not represent how automated hiring tools are used in practice.

    “We show that Local Law 144 bias audits are significantly hampered by several issues, including missing demographic data, opaque data aggregation, problematic uses of”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisenone given

    LL144 defines the impact ratio as comparing a given group's selection or scoring rate against the selection or scoring rate of the highest scoring or most selected group.

    “LL144 also defines the impact ratio as comparing a given group's selection or scoring rate against the selection or scoring rate of the highest scoring or most selected group”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Enforcement of NYC anti-discrimination law falls to the city's Commission on Human Rights.

    “Enforcement of NYC anti-discrimination law falls to the city's Commission on Human Rights [97].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [97]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    While most audits that disclose the number of applications missing demographic information generally report impact ratios above 0.8, the impact ratios could often be below 0.8 depending on the distribution of the missing demographic data.

    “while most audits that disclose the number of applications missing demographic information generally report impact ratios above 0.8, the impact ratios could often be below 0.8 depending on the distribution of the missing demographic data”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: For each group a in a demographic category, the number of applicants missing demographic information could all belong to group a.
    • The authors state this as an assumption: The selection rates for other groups are the same (holding all else equal).

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationnone given

    We are not aware of examples of silent duplicates beyond LL144 bias audits, but we believe this issue is relevant more generally for algorithm audits in contexts where the same automated tools may be used by different entities.

    “While we are not aware of examples of silent duplicates beyond LL144 bias audits, we believe this issue is relevant more generally for algorithm audits in contexts where the same automated tools may be used by different entities.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Missingness made it challenging to analyze individual audit reports and complicated comparative analysis for 2023 and 2024 audit reports.

    “This missingness made it challenging to analyze individual audit reports and also complicated our comparative analysis for 2023 and 2024 audit reports.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    If the LL144 definition of impact ratio is followed, impact ratios should only take on values in [0, 1].

    “a standard which, if followed, would mean that impact ratios should only take on values in [0, 1].”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    Our analysis revealed a need for greater accessibility, transparency, and rigor in algorithm auditing than mandated by LL144.

    “Our analysis revealed a need for greater accessibility, transparency, and rigor in algorithm auditing than mandated by LL144.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    While NYC DCWP is charged with enforcing audit and reporting requirements, LL144 does not mandate any particular action based on the results of a bias audit.

    “while NYC DCWP is charged with enforcing audit and reporting requirements, LL144 does not mandate any particular action based on the results of a bias audit.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    These issues affect the integrity, interpretability, and ultimate usefulness of these algorithm audits, falling short of the legislation's goals.

    “Our analysis demonstrated that these issues affect the integrity, interpretability, and ultimate usefulness of these algorithm audits, falling short of the legislation's goals.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Our analysis covered less than two years' worth of bias audits.

    “Our analysis covered less than two years' worth of bias audits”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    We encountered numerous issues at each stage of searching for, archiving, extracting information from, and analyzing bias audits produced pursuant to Local Law 144, painting a picture of these audits as incomplete, misleading, and potentially even inaccurate.

    “We encountered numerous issues at each stage of our process of searching for, archiving, extracting information from, and analyzing bias audits produced pursuant to Local Law 144, painting a picture of these audits as incomplete, misleading, and potentially even inaccurate.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Study design and conductown data

    Our analysis was limited given available data.

    “Our analysis was limited given available data”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The choice of the sample median as the threshold for scoring rates under LL144 is unrealistic; in practice, employers set and use different thresholds to determine whether an applicant moves forward.

    “But this choice of the sample median is unrealistic; in practice, employers set and use different thresholds to determine whether an applicant moves forward in their process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Many of our recommendations would be challenging to implement, in part because they may require consensus among policymakers about open challenges in the field of algorithm auditing.

    “many of our recommendations would be challenging to implement, in part because they may require consensus among policymakers about open challenges in the field of algorithm auditing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The assumption that all missing data comes from one group is unlikely to hold in practice.

    “we recognize that the assumption that all missing data comes from one group is unlikely to hold in practice.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: For each group a in a demographic category, the number of applicants missing demographic information could all belong to group a.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    We were unable to verify data for one audit report we collected during this time period because it was removed from the website where it had been posted and we had not properly preserved it, so we removed this audit report from our dataset.

    “We were unable to verify data for one audit report we collected during this time period-between our extraction and validation processes, it was removed from the website where it had been posted, and we had not properly preserved it, so we removed this audit report from our dataset.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Evaluating internal governance is a crucial part of algorithm audits.

    “Evaluating internal governance is a crucial part of algorithm audits [81,102].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [81]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [102]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The shortcomings we found speak to problems in nearly every stage of the audit and reporting process.

    “The shortcomings we found speak to problems in nearly every stage of the audit and reporting process”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Without a version number or other information identifying whether a given tool had substantially changed between 2023 and 2024 audits, it was difficult to tell how changes in selection rates and impact ratios across years should be interpreted, complicating comparisons over time.

    “without a version number or other information identifying whether a given tool had substantially changed between 2023 and 2024 audits, it was difficult to tell how changes in the selection rates and impact ratios across years should be interpreted, complicating comparisons over time.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Employers may have little incentive to conduct audits given the lack of enforcement and because employers have wide latitude to determine whether their tools are subject to the law.

    “Employers may have little incentive to conduct audits given this lack of enforcement (and because employers have wide latitude to determine whether their tools are subject to the law).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Missing applicant demographic data is a much more pervasive issue than impact ratio definition nuances and adds even more complexity to interpreting bias audit results.

    “However, as we discuss in Section 4.2.4,a much more pervasive issue-missing applicant demographic data-adds even more complexity to interpreting the bias audit results.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    There are many ways these bias audits can miss signs of bias, so sweeping conclusions of any nature about these bias audits can be misleading.

    “there are many ways these bias audits can miss signs of bias, so sweeping conclusions of any nature about these bias audits can be misleading.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Even if implemented fully, our recommendations are likely insufficient to ensure algorithm audits are effective.

    “even if implemented fully, our recommendations are likely insufficient to ensure algorithm audits are effective.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Researchers have highlighted the importance of evaluating automated systems using metrics that represent how thresholds are used in practice.

    “Researchers have highlighted the importance of evaluating automated systems using metrics that represent how thresholds are used in practice [79].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [79]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

51 of 51

  • The authors state this as an assumption.

    For each group a in a demographic category, the number of applicants missing demographic information could all belong to group a.

    “For each group 𝑎 ∈ 𝐴, we assume that the number of applicants missing demographic information could all belong to group 𝑎”

  • The authors state this as an assumption.

    The selection rates for other groups are the same (holding all else equal).

    “holding all else equal (e.g., assuming the selection rates for other groups are the same)”

  • The authors state this as an assumption.

    The distribution of missing demographic data is the same as the demographic distribution of applicants with known demographics.

    “assuming that the distribution of missing data is the same as the demographic distribution of applicants with known demographics”

  • The authors state this as an assumption.

    All missing demographic data comes from one group, and other selection rates are held equal when computing impact ratio bounds.

    “we assume that all missing demographic data comes from one group and hold other selection rates equal when computing impact ratio bounds.”
4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • 2024 Morgan Stanley audit cites a 2023 reference

    Minor

    Table 2 cites reference [85] for the 2024 Morgan Stanley companion audit, but reference 85 is explicitly dated 2023. The references list the 2024 Morgan Stanley audit as reference 86.

  • Silent-duplicate definition excludes duplicates within an audit

    Minor

    The formal definition requires silent duplicates to be rows in different audits, but the paper also calls identical results within the same audit "intra-audit silent duplicates." The definition does not cover that stated case.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. Ifeoma Ajunwa. 2020. An Auditing Imperative for Automated Hiring Systems. Harvard Journal of Law & Technology (Harvard JOLT) 34 2 (2020) 621–700. https://jolt.law.harvard.edu/assets/articlePDFs/v34/5.-Ajunwa-An-Auditing-Imperative-for-Automated-Hiring-Systems.pdf
  2. Lydia XZ Brown, Ridhi Shetty, and Michelle Richardson. 2020. Report-Algorithm-driven Hiring Tools: Innovative Recruitment or Expedited Disability Discrimination. Technical Report. Center for Democracy & Technology. https://cdt.org/insights/report-algorithm-driven-hiring-tools-innovative-recruitment-or-expedited-disability-discrimination/
  3. Center for Democracy & Technology (CDT); American Association for People with Disabilities (AAPD); American Civil Liberties Union (ACLU); The Leadership Conference on Civil and Human Rights; National Women’s Law Center; Upturn. 2022. Civil Rights Standards for 21st Century Employment Selection Procedures. https://cdt.org/wp-content/uploads/2022/12/updated-2022-12-05-Civil-Rights-Standards-for-21st-Century-Employment-Selection-Procedures.pdf
  4. Divij Joshi. 2021. Algorithmic Accountability for the Public Sector: Learning from the First Wave of Policy Implementation. Technical Report. Ada Lovelace Institute, AI Now Institute and Open Government Partnership. https://www.opengovpartnership.org/documents/ algorithmic-accountability-public-sector/
  5. Hilke Schellmann. 2024. The Algorithm: How AI Can Hijack Your Career and Steal Your Future. Hurst Publishers,  .
  6. 10.1007/16495202361 · DOI
  7. Akshaj Kumar Veldanda, Fabian Grob, Shailja Thakur, Hammond Pearce, Benjamin Tan, Ramesh Karri, and Siddharth Garg. 2023. Investigating hiring bias in large language models. In R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models.  ,  ,  .

Cited by

Sorted by citations

Related

Authors
Gerchick, M. K., Encarnación, R., Tanigawa-Lau, C., Armstrong, L., Gutiérrez, A., & Metaxa, D.
Affiliations
  • American Civil Liberties Union (US)
  • University of Pennsylvania (US)
  • Stanford University (US)
  • Harvard University (US)
  • University of Pennsylvania, Philadelphia, PA, USA
ORCID
Venue
Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2025-06-23
Pages
29-44
DOI
10.1145/3715275.3732004
OpenAlex
W4411550541
Cited by
10
References
47
Citations by year
2026: 10
FWCI
10.33
Type
conference paper
Language
en
Open access
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic accountability, algorithmic auditing, bias auditing, algorithmic discrimination, adverse impact, data aggregation, test data, employment discrimination, AI regulation, New York City

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-30.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-30.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review