CoChat
Sign Up
Cited by 249Names, Identity, and Discrimination ResearchExperimental Behavioral Economics StudiesSocial and Intergroup Psychology

The state of hiring discrimination: A meta-analysis of (almost) all recent correspondence experiments

Open access
Lippens, Louis, Vermeiren, Siel, Baert, Stijn·2023·European Economic Review
DOI 10.1016/j.euroecorev.2022.104315
Open this paper in CoChat — free accountOpen PDF

Abstract

Notwithstanding the improved integration of various minority groups in the workforce, unequal treatment in hiring still hinders many individuals’ access to the labour market. To tackle this inaccessibility, it is essential to know which and to what extent minority groups face hiring discrimination. This meta-analysis synthesises a quasi-exhaustive register of correspondence experiments on hiring discrimination published between 2005 and 2020. Using a random-effects model, we computed pooled discrimination ratios concerning ten discrimination grounds upon which unequal treatment in hiring is forbidden by law. Our meta-analysis shows that hiring discrimination against candidates with disabilities, older candidates, and less physically attractive candidates seems equally severe as the unequal treatment of candidates with salient racial or ethnic characteristics. Moreover, hiring discrimination against older applicants is more prominent in Europe than in the United States. Last, while we initially find a significant decrease in ethnic hiring discrimination in (Western) Europe, we find no structural evidence of recent temporal changes in hiring discrimination when controlling for the minority groups considered, at the country level, or based on the various other grounds within the scope of this review.

AI snapshot

Written from the paper’s full text.

TL;DR

This meta-analysis synthesizes 306 correspondence experiments from 169 studies published between 2005 and 2020, comprising almost 965,000 fictitious job applications, to quantify hiring discrimination across ten legally protected grounds. It finds substantial discrimination against racial/ethnic minorities, people with disabilities, older applicants, and less physically attractive applicants, with limited robust evidence of temporal change and some regional differences.

Key findings

  • Pooled discrimination ratios indicate applicants with disabilities receive about 41% fewer positive responses (DR = 0.5885, 95% CI [0.5277; 0.6563]), less physically attractive applicants about 37% fewer (DR = 0.6308, CI [0.4738; 0.8397]), older applicants about 31% fewer (DR = 0.6867, CI [0.6503; 0.7250]), and ethnic minority candidates about 29% fewer (DR = 0.7113, CI [0.6924; 0.7307]).
  • Among specific groups, Arab/Maghrebi/Middle Eastern applicants face about 41% fewer positive responses (DR = 0.5937, CI [0.5548; 0.6353]), Eastern/South-Eastern Asian applicants about 37% fewer (DR = 0.6286, CI [0.5368; 0.7361]), Southern European applicants about 33% fewer (DR = 0.6673, CI [0.5711; 0.7798]), and African/African American/Black applicants about 32% fewer (DR = 0.6845, CI [0.6444; 0.7270]).
  • Age discrimination is more severe in Europe (DR = 0.6288, CI [0.5349; 0.7392]) than the Americas (DR = 0.6881, CI [0.6438; 0.7354]); older applicants in several European countries face about 50% fewer positive call-backs (DR = 0.5152, CI [0.4258; 0.6234]) versus about 30% fewer in the United States (DR = 0.6916, CI [0.6342; 0.7541]).
  • An initial decline in ethnic hiring discrimination in Europe becomes statistically insignificant when controlling for minority groups considered or country-level analyses; no structural temporal changes are found for other grounds. Potential publication bias is found for race/ethnicity/national origin and sexual orientation, but not structurally for other grounds.

Methods

  • Systematic review and meta-analysis of correspondence experiments first published in English between 2005 and 2020, limited to discrimination grounds forbidden under United States federal or state law; sources included Baert’s (2018) register plus Web of Science and Google Scholar searches in spring 2021.
  • The final dataset contains 169 studies and 306 units of observation, with almost 965,000 fictitious applications; two researchers independently reviewed studies, and missing outcome data for 32 experiments were partly completed, leaving 20 excluded.
  • Random-effects models pooled log-transformed discrimination ratios (risk ratios) by discrimination ground and treatment group, using Knapp-Hartung confidence intervals and Mantel-Haenszel weights adjusted for between-study variance; heterogeneity was assessed with I² and weighted least squares meta-regressions for call-back classification, region, and period, with publication-bias and robustness checks.

Limitations

  • Stated in the paper: the review includes only correspondence experiments, not in-person audits; it excludes unpublished and non-English research, so publication bias may remain; marital status, wealth, and military service/affiliation have too few experiments for convincing conclusions; most variability is unexplained, and meta-regression results are not causal because many relevant covariates were unavailable.
  • Evident from the text: statistical heterogeneity is very high, with I² ranging from 82.36% for age to 98.53% for sexual orientation, and several pooled estimates rely on fewer than ten experiments, making uncertainty large; some regional findings rest on very few studies, such as the Africa gender comparison and the Asia sexual-orientation estimate based on one Cyprus study.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.euroecorev.2022.104315

Cites

99 of 99 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

17 of 44 cited sources checked · 11 supported · 0 flagged · accuracy not yet measured

  • Premisereasoning

    The correspondence testing method is still considered the reference method for measuring hiring discrimination at present.

    “the correspondence testing method is still considered the reference method for measuring hiring discrimination at present (Baert, 2018;Neumark, 2018;Verhaeghe, 2022).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Baert, 2018

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Neumark, 2018 source
    • cites Verhaeghe, 2022 source
  • Premisecitation

    The regional difference in age discrimination is in line with Lahey (2010) in that legislation and social norms around working at an advanced age are on average more lenient in the United States compared to European countries.

    “This finding is also in line with the analysis of Lahey (2010) in that the legislation and social norms around working at an advanced age are on average more lenient in the United States compared to European countries.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Lahey, 2010 source

      The Abstract says European age-discrimination laws generally provide less enforcement and more exemptions than corresponding US laws, and the European Case section notes that many European countries permit mandatory retirement ages. This supports a legal comparison favorable to continued work at older ages in the US. The Conclusion discusses social norms around older employment but does not establish that US social norms are, on average, more permissive than European ones. Page numbers are not available in our copy. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Premisecitation

    A recent large-scale correspondence audit study in Australia found that males received substantially more (less) positive responses than females in occupations dominated by male (female) gender workers.

    “Similar findings arise from another recent large-scale correspondence audit study in Australia (with more than 12,000 applications) where males received substantially more (less) positive responses than females in occupations dominated by male (female) gender workers (Adamovic and Leibbrandt, 2022).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Adamovic and Leibbrandt, 2022 source
  • Findingnone given

    The meta-analysis synthesises a quasi-exhaustive register of correspondence experiments on hiring discrimination published between 2005 and 2020.

    “This meta-analysis synthesises a quasi-exhaustive register of correspondence experiments on hiring discrimination published between 2005 and 2020.”
    Quote in the paperSubstantial claim
  • Findingown data

    The meta-analysis finds no structural evidence of recent temporal changes in hiring discrimination when controlling for the minority groups considered, at the country level, or based on the various other grounds within the scope of this review.

    “we find no structural evidence of recent temporal changes in hiring discrimination when controlling for the minority groups considered, at the country level, or based on the various other grounds within the scope of this review.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.
    • The authors state this as an assumption: The included discrimination ratios follow a normal-like distribution.
  • May combine two claims. 46
  • Classified as probably not a substantial claim. 33
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation

    The Knapp-Hartung method assumes a t-distribution of the pooled effect rather than a normal distribution, which reduces the chance of obtaining false-positive results.

    “This method assumes a t-distribution of the pooled effect rather than a normal distribution, which reduces the chance of obtaining false-positive results (Langan et al., 2019).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Langan et al., 2019 source
    • The authors state this as an assumption: The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    This underutilisation of talent could result in needless economic costs for firms and society.

    “This underutilisation of talent could result in needless economic costs for firms and society (Baert, 2021;OECD, 2020a;Pager, 2016).”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites Baert, 2021 source

      The paper’s Abstract (page unavailable) links labour underutilisation to a societal economic concern: when more working-age people are inactive, fewer workers support non-working people, making social-security financing a greater problem. It does not address economic costs to firms or specifically describe the underutilised resources as talent. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites OECD, 2020a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Pager, 2016 source

      The Abstract says racial preferences unrelated to productivity can be costly to employers, and the section ‘What Does the Relationship between Discrimination and Business Failure Represent?’ describes discrimination as prioritizing worker characteristics irrelevant to productivity. These passages support possible costs to firms, but the paper does not establish economic costs to society from underutilised talent. The paper also cautions that its results cannot establish discrimination as the direct cause of firm failure. Page numbers were not provided in our copy. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Some experiments only look at a 6-year age gap (e.g. Baert et al., 2016), while others go as far as examining a 35-year age gap (e.g. Neumark et al., 2019).

    “some experiments only look at a 6-year age gap (e.g.Baert et al., 2016), while others go as far as examining a 35-year age gap (e.g.Neumark et al., 2019).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Baert et al., 2016 source
    • cites Neumark et al., 2019 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Bias statistics were only calculated if the total number of correspondence audits for a given analysis equalled or exceeded ten, because otherwise statistical power could be too low to detect asymmetry.

    “we only calculated bias statistics if the total number of correspondence audits for a given analysis equalled or exceeded ten, otherwise, the statistical power could be too low to detect asymmetry.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Harrer et al., 2021 source
    • cites Sterne et al., 2011 source

      Box 2 recommends not testing funnel plot asymmetry when a meta-analysis has fewer than 10 studies, because test power is usually too low. The citing sentence applies that threshold to correspondence audits rather than studies; the source does not establish that it applies in that setting. No page number is provided in our copy. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    In the reported models, standard errors related to the model coefficients were clustered at the study level.

    “In the reported models, standard errors related to the model coefficients were clustered at the study level (Viechtbauer, 2010).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Viechtbauer, 2010 source

      The paper discusses standard errors for model coefficients (Raw and standardized residuals, p. 22) and describes fitting meta-analytic models (Model fitting, p. 5). It does not say that coefficient standard errors were clustered at the study level or establish how the citing authors handled their reported models. Searches for “cluster,” “clustered standard errors,” and “sandwich robust standard errors” found no such passage. (LLM, conf=0.75)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    Higher levels of hiring discrimination against Western Asian applicants in Asia could be explained by the relatively large local presence of these minority groups in Asian labour markets and negative connotations associated with these groups in those regions.

    “These higher levels of hiring discrimination could be explained by (i) the relatively large local presence of these minority groups in Asian labour markets compared to European labour markets and (ii) the negative connotations associated with these groups within these particular regions, which do not necessarily exist in European countries (Asali et al., 2018;Maurer-Fazio, 2013).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Asali et al., 2018 source

      The Introduction (p. 3) says Armenian and Azeri people are Georgia’s largest distinct ethnic minorities, and the Conclusions (p. 16) report lower callback rates for non-Georgian applicants. The paper does not establish that these groups have a larger presence in Asian than European labour markets, or that negative connotations explain an Asia–Europe difference in hiring discrimination. Its Conclusions (p. 17) instead leave statistical and employer-preference explanations equally likely. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Maurer-Fazio, 2013

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The study paid special attention to the reporting guidelines for meta-analyses in economics of Havránek et al. (2020).

    “In this process, we paid special attention to the reporting guidelines for meta-analyses in economics of Havránek et al. (2020)-we refer the reader to Appendix A (Table A2) for the corresponding checklist.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Havránek et al. (2020) source

  • May combine two claims.

    Premisecitation

    A recent large-scale correspondence audit study in the United States found that contact rates for male and female gender applicants differed significantly between firms, with some employers favouring male and others favouring female candidates.

    “a recent large-scale correspondence audit study conducted in the United States (with more than 80,000 applications) in which the authors found that contact rates for male gender and female gender applicants differed significantly between firms: some employers favoured male gender candidates, while others favoured female gender candidates (Kline et al., 2021).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Kline et al., 2021 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Van Borm and Baert (2022) found women to be perceived as more social and supportive but less assertive or physically strong than men.

    “Van Borm and Baert (2022), for example, found women to be perceived as more social and supportive but less assertive or physically strong than men.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Van Borm and Baert (2022) source

  • May combine two claims.

    Study design and conductcitation

    Our research is based on a synthesis of only correspondence experiments, whereas some earlier meta-studies also included in-person audits.

    “First of all, our research is based on a synthesis of only correspondence experiments, whereas some earlier meta-studies that focused on specific discrimination grounds have also included in-person audits to paint a broader picture of hiring discrimination (e.g.Quillian et al., 2017).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Quillian et al., 2017 source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    To assess robustness, the authors measured and controlled for outliers, evaluated potential statistical dependence between sampled discrimination ratios, and recalculated p-values of meta-regression coefficients via a permutation test.

    “To further assess the robustness of our results, we (i) measured and controlled for outliers (i.e.influential cases that substantially affect the pooled discrimination ratio), (ii) evaluated potential statistical dependence between the sampled discrimination ratios, and (iii) recalculated p-values of the meta-regression coefficients via a permutation test (Borenstein et al., 2009;Higgins et al., 2”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Borenstein et al., 2009 source
    • cites Higgins et al., 2019

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Viechtbauer et al., 2015 source

  • May combine two claims.

    Premisecitation

    In many studies where LGB+ orientation status is signalled through activism or affiliation, there is a similar non-LGB+ signal attributed to the control group, which might 'wash out' the activism or affiliation effect.

    “Moreover, in many of the studies where LGB+ orientation status is signalled through activism or affiliation, there is a similar non-LGB+ signal that is attributed to the control group, which might 'wash out' the activism or affiliation effect (e.g.Drydakis, 2009;Tilcsik, 2011).”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites Drydakis, 2009 source
    • cites Tilcsik, 2011 source

  • May combine two claims.

    Findingown data

    The 20 units with no data were excluded from the meta-analysis, resulting in 306 valid units.

    “These units were excluded from the meta-analysis, resulting in 306 valid units (cfr.supra).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After contacting corresponding authors, 12 cases could be completed, leaving no data for 20 units of observation.

    “After contacting the corresponding authors of the respective studies to retrieve these data, 12 cases could be completed (37.50% of cases with missing data), meaning that we had no data for the remaining 20 units of observation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The study gathered 306 correspondence experiments from 169 separate correspondence audit studies.

    “Altogether, we gather 306 correspondence experiments (i.e. units of observation in our analysis) originating from 169 separate correspondence audit studies.”
    • Also stated in: § Timing · plain
      “We retained a total of 169 studies, of which 90 were already included in Baert's (2018) book chapter, resulting in 306 units of observation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Older applicants (vis-à-vis young to middle-aged applicants), but not younger applicants (vis-à-vis middle-aged applicants), are strongly discriminated against in correspondence experiments, with a pooled discrimination ratio of 0.6646 (95% CI [0.6292; 0.7020]).

    “Older applicants (vis-à-vis young to middle-aged applicants), but not younger applicants (vis-à-vis middle-aged applicants), are strongly discriminated against in correspondence experiments ( ̂ DR = 0.6646, CI 95% = [0.6292;0.7020]).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random-effects model is an appropriate model to account for between-study heterogeneity.

  • May combine two claims.

    Findingown data

    Only one experiment related to political orientation and one experiment related to union affiliation were found, from which pooled discrimination ratios could not be calculated.

    “we found only one experiment related to political orientation and one experiment related to union affiliation, from which we could not calculate pooled discrimination ratios.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The decline becomes statistically insignificant when controlling for the considered treatment groups or when restricting the analysis to each country separately.

    “The decline becomes statistically insignificant when controlling for the considered treatment groups or when restricting the analysis to each country for which we have data separately (see Table A16, Table A18 and Figure A1-4).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Of the full texts excluded after appraisal, 10 (17.24%) were excluded because the study did not use the correspondence testing method.

    “the study did not use the correspondence testing method (e.g.in-person audit; N = 10, 17.24%)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Of the full texts excluded after appraisal, 1 (1.72%) was excluded because the study was solely related to housing discrimination instead of hiring discrimination.

    “the study was solely related to housing discrimination instead of hiring discrimination (N = 1, 1.72%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    People with disabilities are on average approximately 41% less likely to receive a positive response to a job application, with a pooled discrimination ratio of 0.5885 (95% CI [0.5277; 0.6563]).

    “people with disabilities are on average approximately 41% less likely to receive a positive response to a job application ( ̂ DR = 0.5885, CI 95% = [0.5277;0.6563])”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random-effects model is an appropriate model to account for between-study heterogeneity.

  • May combine two claims.

    Findingown data

    Although the original estimate related to physical disability is not statistically significant, this discrimination ratio becomes statistically significant at the 5% level after excluding one outlier, with a k-adjusted ratio of 0.7494 (95% CI [0.6259; 0.8974]).

    “Although the original estimate related to physical disability is not statistically significant, this discrimination ratio becomes statistically significant at the 5% level after excluding one outlier from the analysis (k-adj.̂ DR = 0.7494, CI 95% = [0.6259;0.8974]; see Table A6 and Table A12).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The ages in the treatment groups of the European correspondence experiments range from 37 to 56 years, while the ages used in the American studies range from 50 to 66 years.

    “This finding is exceptional given that the ages in the treatment groups of the European correspondence experiments range from 37 to 56 years, while the ages used in the American studies are generally higher, ranging from 50 to 66 years.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Estimates based on age indicate reduced positive responses by approximately 31%, with a pooled discrimination ratio of 0.6867 (95% CI [0.6503; 0.7250]).

    “estimates based on physical appearance and age indicate reduced positive responses by approximately 37% ( ̂ DR = 0.6308, CI 95% = [0.4738;0.8397]) and 31% ( ̂ DR = 0.6867, CI 95% = [0.6503;0.7250]), respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    I2 estimates range from 82.36% (for age) to 98.53% (for sexual orientation), not considering wealth, military service or affiliation, and marital status, which are based on too few experiments to meaningfully interpret statistical heterogeneity.

    “I 2 estimates range from 82.36% (for age) to 98.53% (for sexual orientation)-not considering the exceptional cases of wealth, military service or affiliation, and marital status, which are based on a too low number of correspondence experiments to meaningfully interpret the statistical heterogeneity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Hiring discrimination based on religion seems to be mainly driven by the unequal treatment of Muslims, with a pooled discrimination ratio of 0.7730 (95% CI [0.7069; 0.8452]).

    “Hiring discrimination based on religion seems to be mainly driven by the unequal treatment of this former group (i.e.Muslims; ̂ DR = 0.7730, CI 95% = [0.7069;0.8452]).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random-effects model is an appropriate model to account for between-study heterogeneity.

  • May combine two claims.

    Findingown data

    There is empirical evidence for unequal treatment in hiring concerning the discrimination grounds of race, ethnicity, and national origin, age, religion, disability, physical appearance, wealth, and marital status.

    “we find empirical evidence for unequal treatment in hiring concerning the discrimination grounds of race, ethnicity, and national origin, age, religion, disability, physical appearance, wealth, and marital status.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random-effects model is an appropriate model to account for between-study heterogeneity.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    After initial screening, the full text of 137 articles was reviewed.

    “After an initial screening of the titles and abstracts, we reviewed the full text of the remaining 137 articles.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Estimates based on physical appearance indicate reduced positive responses by approximately 37%, with a pooled discrimination ratio of 0.6308 (95% CI [0.4738; 0.8397]).

    “estimates based on physical appearance and age indicate reduced positive responses by approximately 37% ( ̂ DR = 0.6308, CI 95% = [0.4738;0.8397]) and 31% ( ̂ DR = 0.6867, CI 95% = [0.6503;0.7250]), respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The discrimination ratio for marital status is 0.8846 (95% CI [0.8109; 0.9650]), which is significantly different from but closer to one.

    “the discrimination ratios for marital status ( ̂ DR = 0.8846, CI 95% = [0.8109;0.9650]) and wealth ( ̂ DR = 0.8806, CI 95% = [0.8081;0.9596]), which are significantly different from but closer to one.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After excluding two clear outliers, the unequal treatment based on affiliation with an LGB+ organisation becomes statistically insignificant at the 5% level, with a k-adjusted ratio of 0.7924 (95% CI [0.6203; 1.0122]).

    “After excluding two clear outliers from the analysis, we also observe that the unequal treatment based on affiliation with an LGB+ organisation becomes statistically insignificant at the 5% level (k-adj.̂ DR = 0.7924, CI 95% = [0.6203;1.0122]; see Table A6 and Table A12).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There is overall hiring discrimination against African, African American, or Black (DR = 0.6845, 95% CI [0.6444; 0.7270]), Southern Asian or Indian (DR = 0.7004, 95% CI [0.6352; 0.7723]), and Western Asian (DR = 0.7508, 95% CI [0.6977; 0.8080]) applicants.

    “We also find overall hiring discrimination against African, African American, or Black ( ̂ DR = 0.6845, CI 95% = [0.6444;0.7270]), Southern Asian or Indian ( ̂ DR = 0.7004, CI 95% = [0.6352;0.7723], and Western Asian ( ̂ DR = 0.7508, CI 95% = [0.6977;0.8080]) applicants.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Unfavourable treatment in hiring is highest for applicants belonging to the groups of Arab, Maghrebi, or Middle Eastern (DR = 0.5937, 95% CI [0.5548; 0.6353]), Eastern Asian or South-Eastern Asian (DR = 0.6286, 95% CI [0.5368; 0.7361]), and Southern European (DR = 0.6673, 95% CI [0.5711; 0.7798]) applicants.

    “unfavourable treatment in hiring is highest for applicants belonging to the groups of Arab, Maghrebi, or Middle Eastern ( ̂ DR = 0.5937, CI 95% = [0.5548;0.6353]), Eastern Asian or South-Eastern Asian ( ̂ DR = 0.6286, CI 95% = [0.5368;0.7361]), and Southern European ( ̂ DR = 0.6673, CI 95% = [0.5711;0.7798]) applicants.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random-effects model is an appropriate model to account for between-study heterogeneity.
    • Rests on an assumption: The pooled discrimination ratios enable comparison of the severity of unequal treatment in hiring across different discrimination grounds.

  • May combine two claims.

    Findingown data

    There is no evidence of hiring discrimination related to the treatment groups 'transgender' or 'mother'.

    “we find no evidence of hiring discrimination related to the treatment groups 'transgender' or 'mother'.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When the only identified outlier from the analysis of the Hispanic, Latin American, or Caribbean group is excluded, the pooled discrimination ratio becomes statistically significant at the 5% level, with a k-adjusted ratio of 0.8175 (95% CI [0.7095; 0.9420]).

    “when the only identified outlier from the analysis of this minority group is excluded, the pooled discrimination ratio becomes statistically significant at the 5% level (k-adj.̂ DR = 0.8175, CI 95% = [0.7095;0.9420]; see Table A6 and Table A12).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There is no evidence for overall hiring discrimination against Hispanic, Latin American, or Caribbean applicants, with a pooled discrimination ratio of 0.9220 (95% CI [0.8091; 1.0507]).

    “we do not find evidence for overall hiring discrimination against Hispanic, Latin American, or Caribbean applicants ( ̂ DR = 0.9220, CI 95% = [0.8091;1.0507])”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingown data

    Hiring discrimination against LGB+ applicants is higher in Asia than in the Americas or Europe.

    “Last, we initially observe that hiring discrimination against applicants who are affiliated with an LGB+ organisation or who signal to be LGB+ orientated is higher in Asia ( ̂ DR = 0.2489, CI 95% = [0.2218;0.2793]) than in the Americas ( ̂ DR = 0.7650, CI 95% = [0.4181;1.400]; β = -1.1651,p = 0.019) or Europe ( ̂ DR = 0.7735, CI 95% = [0.5378;1.1126]; β = -1.0876,p = <0.001;see Table 3, Table 4, a”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Of the full texts excluded after appraisal, 27 (46.55%) were excluded because unequal treatment based on the discrimination ground was not forbidden under United States law.

    “unequal treatment based on the discrimination ground in the scope of the study was not forbidden under United States law (N = 27, 46.55% of the total number of excluded full texts)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The discrimination ratio for wealth is 0.8806 (95% CI [0.8081; 0.9596]), which is significantly different from but closer to one.

    “the discrimination ratios for marital status ( ̂ DR = 0.8846, CI 95% = [0.8109;0.9650]) and wealth ( ̂ DR = 0.8806, CI 95% = [0.8081;0.9596]), which are significantly different from but closer to one.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Ethnic minority candidates face on average approximately 29% fewer positive responses, with a pooled discrimination ratio of 0.7113 (95% CI [0.6924; 0.7307]) based on 143 experiments (46.73% of total units of observation).

    “Ethnic minority candidates face on average approximately 29% fewer positive responses (k = 143, 46.73% of total units of observation; ̂ DR = 0.7113, CI 95% = [0.6924;0.7307]).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random-effects model is an appropriate model to account for between-study heterogeneity.

  • May combine two claims.

    Findingown data

    After excluding outliers, the discrimination ratio for female gender (vis-à-vis male gender) applicants remains statistically significant and positive, with a k-adjusted ratio of 1.0663 (95% CI [1.0221; 1.1124]).

    “After excluding outliers from the analysis, the discrimination ratio for female gender (vis-à-vis male gender) applicants remains statistically significant and positive (k-adj.̂ DR = 1.0663,CI 95% = [1.0221;1.1124]; see Table A6 and Table A12).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In cases where we suspected publication bias, we reported the results with the necessary caveats but, in general, there were few.

    “In cases where we suspected publication bias, we reported the results with the necessary caveats but, in general, there were few.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Of the full texts excluded after appraisal, 20 (34.48%) were excluded because the correspondence experiment was entirely based on data used in a previously published and already included study.

    “the correspondence experiment was entirely based on data used in a previously published (and already included) study (N = 20, 34.48%)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Based on estimates from just four experiments, there is no overall evidence of hiring discrimination based on military service or affiliation, with a pooled discrimination ratio of 0.9983 (95% CI [0.7766; 1.2834]).

    “estimates from just four experiments, we find no overall evidence of hiring discrimination based on military service or affiliation ( ̂ DR = 0.9983, CI 95% = [0.7766;1.2834]).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The uncovered unequal treatment based on gender and motherhood status is very small, with a pooled discrimination ratio of 1.0413 (95% CI [1.0151; 1.0682]).

    “the uncovered unequal treatment based on gender and motherhood status is very small ( ̂ DR = 1.0413,CI 95% = [1.0151;1.0682]; see also Section 3.3.2)”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The search evaluated the titles and abstracts of 933 studies against the eligibility criteria.

    “In total, we evaluated the titles and abstracts of 933 studies against our eligibility criteria (see Section 2.1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    For RNO, the intercept coefficient is -35.1630 (SE 15.4015), significant at p<0.05.

    “Intercept | -35.1630* (15.4015)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For RNO, the period year coefficient is 0.0173 (SE 0.0077), significant at p<0.05.

    “Period | Year | 0.0173* (0.0077)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For SEO, the Asia region coefficient is -1.1651 (SE 0.2440), significant at p<0.05.

    “Asia | -0.3906 (0.2133) | 0.0743 (0.1293) | N/A | -0.1604 (0.6872) | N/A | -1.1651* (0.2440)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For AGE, the Europe region coefficient is -0.2839 (SE 0.0974), significant at p<0.05.

    “Europe | -0.0833 (0.0687) | 0.0356 (0.1192) | -0.2839* (0.0974)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For GMO, the Africa region coefficient is 0.7105 (SE 0.1268), significant at p<0.001.

    “Africa | N/A | 0.7105*** (0.1268)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The risk of reviewer bias was reduced by having two researchers independently review the selected articles.

    “The risk of reviewer bias was reduced by having two researchers independently review the selected articles.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The accuracy of the outcome measure variables was independently assessed and verified by at least two authors.

    “The accuracy of these variables was independently assessed and verified by at least two authors.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To generate more balanced weights, the weights were adjusted for between-study variance (τ2).

    “To generate more balanced weights, the weights were adjusted for between-study variance (τ 2 ).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Knapp-Hartung adjustments were applied when calculating confidence intervals around pooled discrimination ratio estimates.

    “We applied Knapp-Hartung adjustments when calculating the confidence intervals around the pooled discrimination ratio estimates (Knapp and Hartung, 2003).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Knapp and Hartung, 2003 source
    • The authors state this as an assumption: The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Reporting bias was formally evaluated when testing for publication bias.

    “Reporting bias, which could (partly) originate from missing data, was formally evaluated when we tested for publication bias (see Section 2.4.3).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Knapp-Hartung adjustments produce more conservative (wider) confidence interval estimates than when these adjustments would not be applied.

    “The Knapp-Hartung adjustments produce more conservative (i.e.wider) confidence interval estimates than when these adjustments would not be applied.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Other studies focusing on different discrimination grounds could face the same criticism that the signal used may not be externally valid.

    “As an anonymous reviewer has pointed out, other studies focusing on different discrimination grounds could face the same criticism-i.e. the signal used may not be externally valid.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Meta-regressions were only performed on groups of studies for which the total number of included studies was equal to or greater than ten.

    “Following Schwarzer et al., (2015) guidelines, we only performed meta-regressions on the groups of studies for which the total number of included studies was equal to or greater than ten.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Schwarzer et al., (2015) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    In correspondence experiments where the effects of origin and religion interact, confounding might lead to a wrong estimation of the true discrimination.

    “Potentially, in correspondence experiments where the effects of origin and religion interact, confounding might lead to a wrong estimation of the true discrimination.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    These individuals are at elevated risk of long-term unemployment and labour market inactivity.

    “these individuals are at elevated risk of long-term unemployment and labour market inactivity”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites OECD, 2020a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites OECD, 2020b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisereasoning

    Compared with in-person audits, the perceived differences between applicants produced by minute differences in their behaviour during the interview are nullified.

    “Compared with in-person audits, the perceived differences between applicants, produced by minute differences in their behaviour during the interview, are nullified.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Discrimination based on LGBT+ status has been made illegal at the US federal level.

    “One noteworthy change, however, is that discrimination based on LGBT+ status has been made illegal at the US federal level (Boystock v. Clayton County, 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Boystock v. Clayton County, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    The standardised meta-analytical approach enables comparisons of levels of hiring discrimination across discrimination grounds and minority groups.

    “The standardised meta-analytical approach enables comparisons of levels of hiring discrimination across discrimination grounds and minority groups.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.
    • The authors state this as an assumption: The included discrimination ratios follow a normal-like distribution.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The discrepancy with prior findings of higher ethnic hiring discrimination in Europe than the Americas is presumably because we focused on (sub-)regional level differences and not country-level differences.

    “This discrepancy is presumably because we primarily focused on differences at the (sub-)regional level and not on differences at the country level.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    To measure and counter the impact of publication bias, the authors applied three analytic techniques: graphical inspection of funnel plot asymmetry, calculation of a bias statistic of funnel plot asymmetry, and calculation of bias-adjusted hiring discrimination estimates through limit meta-analyses.

    “To measure and counter the impact of publication bias, we applied three analytic techniques: (i) a graphical inspection of funnel plot asymmetry, (ii) the calculation of a 'bias statistic' of funnel plot asymmetry, and (iii) the calculation of bias-adjusted hiring discrimination estimates through 'limit' meta-analyses (Harrer et al., 2021).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Harrer et al., 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductown data

    Several of these country-level results are based on a small number of correspondence experiments and should be interpreted with caution.

    “Note that several of these results are based on a small number of correspondence experiments and thus should be interpreted with caution.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In the study by Riach (2015), older candidates were 47 years old, while younger candidates were 27 years old, creating a 20-year age gap between the treatment group and the control group.

    “A representative example is the study by Riach (2015), where older candidates were 47 years old, while younger candidates were 27 years old, creating a 20-year age gap between the treatment group and the control group.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Riach (2015) source

  • May combine two claims.

    Study design and conductown data

    The estimate for younger applicants is based on a very small number of tests (k = 2), which warrants caution in interpreting this discrimination ratio.

    “the estimate for younger applicants is based on a very small number of tests (k = 2), which warrants caution in interpreting this discrimination ratio.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    We did not explain most of the variability around the hiring discrimination estimates, in part because the scope of our review was broad and many relevant covariates were not retrievable at the study level across discrimination grounds.

    “Finally, we did not explain most of the variability around the hiring discrimination estimates, in part because the scope of our review was already very broad but also because many relevant covariates were not retrievable at the study level across discrimination grounds.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Discrepancies in legislation and resulting social norms might have played a role in regional age discrimination differences, but we cannot rule out alternative explanations.

    “Discrepancies in legislation and the resulting social norms might have played a role but we cannot rule out alternative explanations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Meta-regression analyses were only performed for discrimination grounds for which k ≥ 10.

    “meta-regression analyses were only performed for discrimination grounds for which k ≥ 10”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Schwarzer et al. (2015) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Standard errors were clustered at the study level.

    “Standard errors were clustered at the study level.”
    Quote in the paperNot a substantial claim
73 of 73

  • The authors state this as an assumption.

    The true level of discrimination varies across studies due to differences in the definition and conceptualisation of treatment and control groups, measurement of responses, and overall experimental design and process.

    “We assumed that there was at least some variation in these levels caused by (subtle) differences in the (i) definition and conceptualisation of the treatment and control groups, (ii) measurement of the responses (or call-backs), and (iii) overall experimental design and process.”

  • The authors state this as an assumption.

    The included discrimination ratios follow a normal-like distribution.

    “Since our estimation strategy assumed that the included discrimination ratios follow a normal-like distribution, we logtransformed these ratios before pooling them in our meta-analysis (see Section 2.4.1).”

  • The authors state this as an assumption.

    The random-effects model is an appropriate model to account for between-study heterogeneity.

    “This confirms our priors that (i) there is between-study variation that is driven by differences in the operationalisation of the experimental groups and, more broadly, how the correspondence experiments are designed, and (ii) the random-effects model is an appropriate model to account for such between-study heterogeneity.”

  • The paper relies on this without support.

    Publication bias could adversely impact the results from the meta-analysis, resulting in either an over- or underestimation of hiring discrimination in certain cases.

    “There are several reasons why publication bias could adversely impact the results from our meta-analysis, resulting in either an over-or underestimation of hiring discrimination in certain cases.”

  • The paper relies on this without support.

    The pooled discrimination ratios enable comparison of the severity of unequal treatment in hiring across different discrimination grounds.

    “The pooled discrimination ratios enable us to compare the severity of unequal treatment in hiring across different discrimination grounds.”

  • The authors state this as an assumption.

    Hiring discrimination may be lower if the call-back is measured in the broad sense compared to the narrow sense.

    “We assume that hiring discrimination may be lower if the call-back is measured in the broad sense compared to the narrow sense.”

  • The paper relies on this without support.

    Correspondence testing is the reference method of measurement that allows for a causal interpretation of the empirical evidence of unequal treatment in hiring.

    “In this meta-analysis, we extensively documented and synthesised the recent hiring discrimination literature grounded in the correspondence testing method-i.e. the reference method of measurement that allows for a causal interpretation of the empirical evidence of unequal treatment in hiring.”

  • The paper relies on this without support.

    The Journal of Ethnic and Migration Studies compiled a special issue on ethnic discrimination in the labour market that was first published online in 2019.

    “the Journal of Ethnic and Migration Studies compiled a special issue on ethnic discrimination in the labour market that was first published online in 2019”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A recent meta-analysis of hiring practices shows that discrimination against candidates with disabilities, older candidates, less physically attractive candidates, and candidates with salient racial or ethnic traits is still an issue in Europe and the United States.

    “A recent meta-analysis of hiring practices34 shows that discrimination against candidates with disabilities, older candidates, less physically attractive candidates, and candidates with salient racial or ethnic traits is still an issue in Europe and the United States.”

    From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [34]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Lippens, L., Vermeiren, S., & Baert, S.
Affiliations
  • Vrije Universiteit Brussel (BE)
  • Ghent University (BE)
  • University of Antwerp (BE)
Corresponding author
Lippens, Louis
ORCID
Venue
European Economic Review
Publisher
Elsevier BV
Published
2023-01
Volume
151
Pages
104315
ISSN
0014-2921 (ISSN-L)
DOI
10.1016/j.euroecorev.2022.104315
OpenAlex
W4287460468
Cited by
0
References
91
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • European Economic Review (publishedVersion, cc-by-nc-nd)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Ghent University Academic Bibliography (Ghent University) (submittedVersion, cc-by-nc-nd)
  • Econstor (Econstor) (submittedVersion)
Funders
Fonds Wetenschappelijk Onderzoek (S004119N)
Topics
  • Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
  • Social Sciences › Safety Research › Experimental Behavioral Economics Studies
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
Keywords
hiring discrimination, correspondence experiment, labor market access, minority groups, racial discrimination, ethnic discrimination, disability discrimination, age discrimination, physical attractiveness, discrimination ratio

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-11-04.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-04.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review