Cited by 887Names, Identity, and Discrimination ResearchExperimental Behavioral Economics StudiesSocial and Intergroup Psychology

The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; none of the 6 sources its claims cite is checked for support yet.

Meta-analysis of field experiments shows no change in racial discrimination in hiring over time

Open access
Quillian, Lincoln, Pager, Devah, Hexel, Ole, Midtbøen, Arnfinn H.·2017·Proceedings of the National Academy of Sciences

Abstract

Significance: Many scholars have argued that discrimination in American society has decreased over time, while others point to persisting race and ethnic gaps and subtle forms of prejudice. The question has remained unsettled due to the indirect methods often used to assess levels of discrimination. We assess trends in hiring discrimination against African Americans and Latinos over time by analyzing callback rates from all available field experiments of hiring, capitalizing on the direct measure of discrimination and strong causal validity of these studies. We find no change in the levels of discrimination against African Americans since 1989, although we do find some indication of declining discrimination against Latinos. The results document a striking persistence of racial discrimination in US labor markets.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

A meta-analysis of all available US field experiments of hiring discrimination found no change over time in discrimination against African Americans since 1989, though there was modest evidence of declining discrimination against Latinos. Whites received substantially more callbacks than equally qualified African American and Latino applicants.

Key findings

  • Across studies since 1989, white applicants received on average 36% more callbacks than equally qualified African Americans (95% CI: 25–47% more) and 24% more callbacks than Latinos (95% CI: 15–33% more).
  • For African Americans, the paper found no change in hiring discrimination over the past 25 years; the 95% CI of the annual slope from 1989–2015 was −0.007 to 0.015, ruling out all but a slow decline, with the point estimate indicating no decline.
  • For Latinos, the trend suggested a possible decline, from whites receiving 30% more callbacks than Latinos in 1990 to 15% more in 2010, but this was only marginally significant (P = 0.099) and based on only 9 Latino field experiments.
  • Accounting for applicant education, gender, study method, occupational groups, local labor market conditions, and other controls did little to alter the African American result; including unpublished studies also did little to affect estimates.

Methods

  • The authors conducted a meta-analysis of every available US field experiment of hiring discrimination, including resume audits and in-person audits, with fieldwork through December 2015.
  • The full search located 28 studies representing 55,842 applications for 26,326 positions; the core analysis since 1989 used 24 studies with 30 estimates, representing 54,318 applications for 25,517 positions.
  • They used random-effects meta-regression, regressing the log discrimination ratio—the ratio of white callback rates to African American or Latino callback rates—on year of fieldwork, with controls for study and applicant characteristics.

Limitations

  • The paper states that results pertain only to discrimination at the point of hire, not later employment outcomes such as wage setting or termination, and do not address possible declines during the 1960s or early 1970s.
  • The paper states that the small number of Latino field experiments (n = 9) limits controls and precision, and that the overall sample of African American studies since 1990 (n = 21) limits the ability to detect statistical significance for covariates.
  • The paper notes publication bias as a potential concern but reports that including 11 unpublished field experiments did little to affect estimates; it also states that studies focused on racial discrimination showed no more discrimination than studies where race was secondary.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.1706255114

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

33 Own result · 9 Own method · 6 Sources not found · 4 Own interpretation · 3 No basis given

Show all 34 claims

  • It may state two separate claims as one. 13
  • Probably not a substantial claim. 12
21 of 21
  • The paper relies on this without support.

    The racial identifiers used in the original field experiments (e.g., racially identifiable names or tester race) validly signal race to employers and do not differ systematically on other employment-relevant characteristics.

    “In the typical resume audit, clues indicating race (such as a racially identifiable name) are randomly assigned to otherwise similar resumes, allowing for treatment and control groups to be equated through randomization.”
  • The paper relies on this without support.

    The year in which data were gathered is a valid and comparable indicator of the time trend in hiring discrimination.

    “We analyze the relationship of discrimination ratios to years in which the data were gathered to provide an estimate of the trend in discrimination.”
  • The authors state this as an assumption.

    The true effect of race on callback rates varies across studies and situations, such as by year, occupation, and applicant skill level.

    “This is surely the case in our analysis, as we expect that the level of racial discrimination may depend on the year of the study, the situation the study considers (e.g., the occupational categories), the skill level of the applicants, and so on.”
  • The authors state this as an assumption.

    The true effects of race on callback rate in each situation estimated by each study vary between studies with a normal distribution around a mean effect.

    “More formally, random-effects meta-analysis allows the true effects of race on the callback rate in each situation estimated by each study, θ i , to vary between studies by assuming that they have a normal distribution around a mean effect, θ.”
  • The authors state this as an assumption.

    Study-level heterogeneity follows a normal distribution around the linear predictor.

    “The model assumes the study-level heterogeneity follows a normal distribution around the linear predictor:”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 36 references

Research works printed but not linked: 1

  1. D Wessel Fear of bias suits may be affecting hiring decisions. Wall Street Journal section A p 2. (September 11 2003).

Cited by

Sorted by citations

Items 501–600 of 886

Metadata

Authors
Quillian, L., Pager, D., Hexel, O., & Midtbøen, A. H.
Affiliations
  • Northwestern University (US)
  • Harvard University (US)
  • Institute for Social Research (NO)
Venue
Proceedings of the National Academy of Sciences
Published
2017-09-12
Volume
114
Issue
41
Pages
10870-10875
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W2756054281
Cited by
887
References
36
Citations by year
2026: 88 · 2025: 121 · 2024: 111 · 2023: 138 · 2022: 130 · 2021: 104 · 2020: 76 · 2019: 79 · 2018: 29 · 2017: 4
FWCI
13.97
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • BIBSYS Brage (BIBSYS (Norway)) (acceptedVersion)
  • HAL (submittedVersion, cc-by)
  • NORA - Norwegian Open Research Archives (submittedVersion, other-oa)
Funders
  • Russell Sage Foundation (RSF) (88-15-06)
  • Northwestern Institute for Policy Research (N/A)
Topics
  • Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Experimental Behavioral Economics Studies
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
Keywords
racial discrimination in hiring, callbacks, African Americans, Hispanic Americans, labor market discrimination, field experiments, applicant gender
MeSH terms
Humans, Personnel Selection, Racism, Social Discrimination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review