Getting a Foot in the Door: A Meta-Analysis of U.S. Audit Studies of Gender Bias in Hiring
Open accessAbstract
For the past three decades, scholars have conducted field experiments to examine gender-based hiring discrimination in the United States. However, these studies have produced mixed results. To further interpret these findings, we performed a meta-analysis of 37 audit studies conducted between 1990 and 2022. Using an aggregated sample of 243,202 fictitious job applications, the study finds no evidence of statistically significant gender discrimination at the study level. However, a series of more focused meta-analyses reveal important variations in the extent of discrimination by occupation type and applicant race. First, the gender composition of an occupation predicts gender bias in hiring. Second, the intersection of gender and race is critical—in female-dominated jobs, White female applicants receive more callbacks than their male counterparts, but Black female applicants experience no such benefit. The study contributes to the literature on labor market and gender (in)equality by synthesizing the findings of field experiments.
AI snapshot
Written from the paper’s full text.
TL;DR
This meta-analysis of 37 U.S. audit studies finds no statistically significant overall gender discrimination in hiring callbacks, but gender bias varies by occupation and race. Female-dominated occupations favor female applicants, yet this advantage appears for White women but not Black women.
Key findings
- At the study level, the weighted average male/female discrimination ratio (logged risk ratio) was -0.05, indicating a slight preference for female applicants, but it was not statistically significant (95% CI: -0.10 to 0.01).
- At the occupation level, the aggregate discrimination ratio was -0.07 (p < 0.05), again favoring female applicants; the female-to-male ratio of an occupation significantly predicted gender bias (p < 0.05), while skill level, median hourly wage, and occupational prestige did not.
- For White applicants, the occupation-level discrimination ratio was -0.08 (p < 0.1), with White women receiving more callbacks than White men in female-dominated occupations; for Black applicants, the ratio was -0.03 and not statistically significant, and the interaction between race and gender composition was significant at p < 0.05.
- The full data set included 66,145 White applicants and 34,328 Black applicants; the occupational-level data set comprised 85 data points from 23 studies.
Methods
- The authors compiled 37 U.S. correspondence audit studies conducted between 1990 and 2022, covering 243,202 fictitious job applications, and constructed a separate occupation-level data set of 85 data points.
- They used random-effect meta-regressions with the logged discrimination ratio as the dependent variable, calculated from callback rates for men relative to women, and included study-level demographic characteristics and occupation-level characteristics (skill level, female ratio, median hourly wage, prestige).
- They ran pooled models, occupation-level models, and separate models for White and Black applicants; Hispanic and Asian applicants were excluded because of small sample sizes.
Limitations
- The paper states that the data reflect only the initial screening stage, mostly entry-level positions, and do not follow up on callbacks, so the results may underrepresent discrimination in later hiring stages.
- The paper notes that evidence for Black applicants is limited, with wide confidence intervals, and that Hispanic and Asian applicants were excluded due to small samples; it also calls for more research on AI-based screening.
- The text makes evident additional constraints: most studies were conducted after 2010, limiting trend assessment; two studies lacked data-collection years and were imputed using publication year minus two; and occupation categories required crosswalking and matching, with some missing wage data filled by similar occupations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
5 of 24 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured
- Premisecitation
Field experiments that explore gender-based discrimination have produced mixed results.
“Notably, field experiments that explore gender-based discrimination have produced mixed results.”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Deming et al. 2016 source
The paper’s Main Results (page unavailable) report higher callback rates for women than men, while its callback tables show that gender differences are not significant in every job category. This supports variation in gender-related findings within this experiment, but the paper does not establish that field experiments generally have produced mixed results on gender-based discrimination. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites Pedulla 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Botelho and Chang, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
The finding that there is an interplay between occupational gender composition and race parallels results of ongoing studies that investigate both race and gender to explain hiring discrimination.
“Second, our finding that there is an interplay between occupational gender composition and race parallels the results of ongoing studies that investigate both race and gender to explain hiring discrimination (Bertrand and Mullainathan 2004;Chavez, Weisshaar, and Cabello-Hutt 2022;Deming et al. 2016).”
Quote in the paperSubstantial claim2 of 3 sources checked- cites Bertrand and Mullainathan 2004 source
- cites Chavez, Weisshaar, and Cabello-Hutt 2022 source
- cites Deming et al. 2016 source
The Work section says the study randomized applicants’ race and gender, and the Results report callback differences by both characteristics. It also examines differences across occupations, but the passages do not establish an interplay between occupational gender composition and race. The citation supports the comparison to research on race and gender in hiring, not a parallel finding of that specific interplay. (LLM, conf=0.80)
Experimental — accuracy not yet measured
- Findingown data
Only the ratio of females in the occupation predicts gender bias in callbacks.
“In sum, based on field experiments conducted in the U.S. labor market, we find that only the ratio of females in the occupation predicts gender bias.”
- Also stated in: § Discussion · plain
“we do find a gender compositional effect, where the ratio of female employees predicts gender bias in the initial hiring stage.”
- Also stated in: § The Intersection of Race and Gender · hedged
“In sum, the findings indicate that occupational characteristics, specifically the gender composition of an occupation, influence gender discrimination in hiringthe more female-dominated an occupation, the more female applicants are preferred.”
Quote in the paperSubstantial claim - Also stated in: § Discussion · plain
- Findingown data
In female-dominated jobs, White female applicants receive more callbacks than their male counterparts.
“in female-dominated jobs, White female applicants receive more callbacks than their male counterparts”
Quote in the paperSubstantial claim - Findingown data
Occupational prestige does not predict gender bias in hiring.
“Prestige scores do not predict gender bias.”
Quote in the paperSubstantial claim
- Findingown data
The interaction term between race and gender in the model of occupational gender composition and hiring outcomes is statistically significant at p < 0.05.
“The interaction term was statistically significant at the p < 0.05 level.”
Quote in the paperSubstantial claim - Findingown data
Skill level does not predict gender bias in hiring.
“First, skill level does not predict gender bias, as demonstrated in the top-left graph of Figure 5.”
- Also stated in: § Patterns at the Occupation Level · plain
“Both meta-regression analyses controlling and not controlling for other occupational characteristics produced statistically insignificant results for the skill-level difference variable.”
Quote in the paperSubstantial claim - Also stated in: § Patterns at the Occupation Level · plain
- Findingown data
The median hourly wage of an occupation does not predict gender discrimination in callbacks.
“Other occupational-level characteristics measured by the average skill level, prestige score, and median hourly wage do not predict gender discrimination in callbacks.”
Quote in the paperSubstantial claim - Findingown data
A higher proportion of female workers in an occupation is associated with a higher likelihood that female job applicants receive a callback (p < 0.05).
“This variable is statistically significant (p < 0.05) in the meta-regression.”
Quote in the paperSubstantial claim - Findingown data
In female-dominated jobs, Black female applicants experience no such callback benefit relative to their male counterparts.
“but Black female applicants experience no such benefit.”
Quote in the paperSubstantial claim - Findingown data
Occupational prestige scores do not predict gender bias in callbacks.
“Finally, prestige scores do not predict gender bias.”
- Also stated in: § Patterns at the Occupation Level · plain
“Additional analysis that does not control for the ratio of females in the occupation also shows that the prestige level does not predict gender bias.”
Quote in the paperSubstantial claim - Also stated in: § Patterns at the Occupation Level · plain
- Findingown data
The discrimination ratio varies across occupation groups.
“Figure 4, which depicts a forest plot with results aggregated by the target occupations in the selected audit studies, reveals that the discrimination ratio varies across occupation groups.”
Quote in the paperSubstantial claim - Findingown data
The average skill level of an occupation does not predict gender discrimination in callbacks.
“Other occupational-level characteristics measured by the average skill level, prestige score, and median hourly wage do not predict gender discrimination in callbacks.”
Quote in the paperSubstantial claim - Findingown data
The meta-regression results for changes in the level of gender bias over fieldwork years are not statistically significant, regardless of whether applicant and occupation characteristics are controlled.
“Using fieldwork years instead of publication years revealed a similar pattern: the meta-regression results for changes in the level of gender bias over fieldwork years are not statistically significant, regardless of whether applicant and occupation characteristics are controlled.”
Quote in the paperSubstantial claim - Findingown data
Wage level does not predict gender bias in hiring.
“Third, wage level does not predict gender bias.”
- Also stated in: § Patterns at the Occupation Level · plain
“These variables still did not significantly predict gender bias.”
Quote in the paperSubstantial claim - Also stated in: § Patterns at the Occupation Level · plain
- Findingown data
The aggregate occupation-level discrimination ratio is -0.07 (p < 0.05), indicating a preference for female over male applicants.
“The aggregate results for the occupation-level data show a statistically significant discrimination ratio of -0.07 (p < 0.05), signaling a preference for female over male applicants.”
Quote in the paperNot a substantial claim - Findingown data
The meta-analysis of 37 U.S. audit studies using 243,202 fictitious job applications found no statistically significant gender discrimination at the study level.
“Using an aggregated sample of 243,202 fictitious job applications, the study finds no evidence of statistically significant gender discrimination at the study level.”
- Also stated in: § Patterns at the Study Level · plain
“However, the results are not statistically significant when studies are the unit of analysis (95 percent confidence interval: -0.10 and 0.01).”
Quote in the paperSubstantial claim - Also stated in: § Patterns at the Study Level · plain
- Findingown data
The study included 37 individual studies that analyzed 243,202 job applications.
“In total, we included 37 individual studies that analyzed 243,202 job applications in the study-level data set.”
Quote in the paperSubstantial claim - Findingown data
The gender composition of an occupation predicts gender bias in hiring.
“First, the gender composition of an occupation predicts gender bias in hiring.”
- Also stated in: § Patterns at the Occupation Level · plain
“Second, as shown in the top-right figure, the female-to-male ratio predicts gender bias.”
Quote in the paperSubstantial claim - Also stated in: § Patterns at the Occupation Level · plain
- Findingown data
For White applicants, as occupations shift from relatively more female to male-dominated, there is a negative-to-positive trend in discrimination ratios.
“As in the pooled data, there is a negative-to-positive trend across occupations as they shift from relatively more female to male-dominated.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The initial screening process has implications for hiring discrimination because of its impact on later stages of the hiring process.
“Nonetheless, the initial screening process has implications for hiring discrimination because of its impact on the later stages of the hiring process.”
Quote in the paperSubstantial claim - Premisereasoning
Audit studies offer a rare opportunity to assess discrimination in the initial stage of hiring.
“Unlike surveys, which focus on individuals who have already been hired, audit studies offer a rare opportunity to assess discrimination in the initial stage of hiring.”
Quote in the paperSubstantial claim - Interpretationreasoning
The patterns observed at the initial screening stage may underrepresent the extent of hiring discrimination throughout the process.
“Therefore, the patterns observed at the initial screening stage may underrepresent the extent of hiring discrimination throughout the process.”
Quote in the paperSubstantial claim - Study design and conductnone given
Occupations were comparable across Black and White applicants.
“Occupations were also comparable across Black and White applicants.”
Quote in the paperNot a substantial claim - Premisereasoning
This study is the first to perform a meta-analysis that solely situates gender discrimination within the U.S. labor market and policy context.
“In particular, this study is the first to perform a meta-analysis that solely situates gender discrimination within the U.S. labor market and policy context.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Theories of employer biases based on a single category of gender do not sufficiently explain patterns of hiring demonstrated through field experiments.
“This indicates that theories of employer biases based on a single category of gender do not sufficiently explain patterns of hiring demonstrated through field experiments.”
Quote in the paperSubstantial claim - Interpretationreasoning
The meta-analysis results reflect only the initial screening stage of hiring, not later stages.
“The data set is composed of field experiments, and because of this, the results of meta-analyses can only reflect the results of the field experiments that take place at the initial point of the screening stage.”
- Also stated in: § Discussion · plain
“Because most field experiments do not include later stages of the hiring process, meta-analyses using the experiments cannot produce information on the patterns of actual hiring outcomes.”
Quote in the paperSubstantial claim - Also stated in: § Discussion · plain
- Interpretationown data
The significant interaction term indicates a meaningful difference in the relationship between occupational gender composition and hiring outcomes across Black and White applicants.
“This finding reinforces our main analysis, indicating a meaningful difference in the relationship between the occupational gender composition and hiring outcomes across these racial groups”
Quote in the paperSubstantial claim - Study design and conductown data
The analysis included only White and Black applicants because the number of applications from candidates of other races was negligible.
“In addition, the analysis included only White and Black applicants because the number of applications from candidates of other races was negligible.”
Quote in the paperSubstantial claim - Premisecitation
Limited studies that examine the final point of the screening stage-the job offer-present more substantial discrimination than when only examining the callback stage.
“Limited studies that do examine the final point of the screening stage-the job offer-present more substantial discrimination than when only examining the callback stage (Quillian, Lee, and Oliver 2020).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Quillian, Lee, and Oliver 2020 source
- Premisereasoning
Pooling multiple countries with divergent labor market structures, workplace conditions, and policies makes it challenging to sort out the impacts of occupation and group-based differences.
“However, these studies pool multiple countries with divergent labor market structures, workplace conditions, and policies.This-using international settings and heterogeneous macro-level contexts-makes it challenging for scholars to adequately sort out the impacts of occupation and group-based differences.”
Quote in the paperSubstantial claim - Premisecitation
There are few experimental studies on hiring discrimination resulting from AI-based screening processes.
“thus far, there are few experimental studies on hiring discrimination-based on race, gender, sexual orientation, and parental statusresulting from AI-based screening processes (for more discussion on this gap in the literature, see Pedulla 2018: 1498-1499).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Pedulla 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
The findings are aligned with empirical studies arguing that persisting sex segregation in the workforce shapes and is shaped by gender-based stereotypes.
“Our findings further support these theories and are aligned with empirical studies that argue that persisting sex segregation in the workforce shapes and is shaped by genderbased stereotypes (Breda et al. 2020;Charles and Bradley 2002).”
Quote in the paperNot a substantial claim0 of 2 sources checked - Study design and conductown data
The overall results remained the same when the analyses were rerun with only studies that randomized or varied the gender of job applicants.
“We conducted a robustness check by rerunning the analyses with only those studies that randomized or varied the gender of job applicants.The overall results remained the same.”
Quote in the paperNot a substantial claim - Interpretationreasoning
It is not possible to clearly explain the hiring patterns for Black applicants based on the limited existing evidence.
“However, clearly explaining the hiring patterns for Black applicants is not possible based on the limited, existing evidence.”
Quote in the paperSubstantial claim - Interpretationreasoning
The results do not answer whether the pattern that occupational gender composition predicts gender bias applies only to White women.
“Overall, the results do not answer the question of whether the gender composition of an occupation predicting gender bias pattern only applies to the White female group.”
Quote in the paperSubstantial claim - Interpretationown data
Based on the data and available evidence, the pattern that occupational gender composition predicts gender bias is likely found only among White women.
“Based on our data and the available evidence, this pattern is likely to be only found among White women.”
Quote in the paperSubstantial claim - Interpretationown data
The wide confidence intervals make it difficult to draw firm conclusions about the relationship between occupational characteristics and gender discrimination patterns for Black applicants.
“The wide confidence intervals suggest that it is difficult to draw firm conclusions about the relationship between occupational characteristics and gender discrimination patterns for Black applicants.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 19
- May combine two claims. 23
- The quote only partly states this claim. 2
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationcitationThe findings align with existing work on the persistence of gender segregation and racial inequality in the U.S. labor market.
“Both of these findings align with existing work on the persistence of gender segregation (Zhu and Grusky 2022) and racial inequality (Quillian et al. 2017) in the U.S. labor market.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites Zhu and Grusky 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Quillian et al. 2017 source
May combine two claims.
Study design and conductcitationFollowing prior literature, rates of 0 were changed to 0.5 to avoid dropping cases, reduce bias in the estimator, and prevent division by zero.
“Following an approach used in prior literature (Schaerer et al. 2023), we changed rates of 0 to 0.5 to avoid dropping cases; this methodological choice also reduces bias in the estimator and prevents division by zero.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Schaerer et al. 2023 source
Section 3.1.4 (Statistical analyses; page not supplied) says the authors added 0.5 to every cell of a 2×2 frequency table before calculating log odds ratios. It explicitly gives reducing estimator bias and avoiding division by zero as reasons. It does not say they changed zero rates to 0.5 or that the adjustment was made to avoid dropping cases. (LLM, conf=0.88)
Experimental — accuracy not yet measured
May combine two claims.
Findingown dataFor Black applicants, the occupational-level discrimination ratio is -0.03 and is not statistically significant.
“ratio calculated from the occupational-level data for Black applicants (-0.03) is not statistically significant.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe association between percent female in occupation and gender bias is statistically significant for White applicants but not for Black applicants.
“As shown in Figure 6, a statistically significant outcome was observed for White applicants in contrast to the non-significant results for Black applicants.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThere is a non-statistically significant trend such that female candidates are more likely to receive callbacks for lower-prestige jobs, and as job prestige increases, the preference for female applicants declines.
“This result means that female candidates are more likely to receive callbacks for lower-prestige jobs, and as job prestige increases, the preference for female applicant declines.However, despite this overall trend, the results of the meta-regression are not statistically significant.”
Quote in the paperNot a substantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataAt the study level, the male/female discrimination ratio was -0.03 (95% CI -0.10 to 0.03) for White applicants and 0.01 (95% CI -0.10 to 0.12) for Black applicants, and neither was statistically significant.
“The White applicant group sample yielded a male/female discrimination ratio of -0.03 (95 percent confidence interval: -0.10 and 0.03), whereas the Black applicant group sample was 0.01 (95 percent confidence interval: -0.10 and 0.12).”
- Also stated in: § Additional information · plain
“The results for both White and Black applicant groups were not statistically significant.”
Quote in the paperNot a substantial claim- Also stated in: § Additional information · plain
May combine two claims. The quote only partly states this claim.
Findingown dataFor White applicants, the occupational-level discrimination ratio is -0.08 and statistically significant at p < 0.1, indicating a preference for female job applicants.
“the results from the occupational-level data for White applicants yield a statistically significant (p < 0.1) discrimination ratio of -0.08.”
- Also stated in: § Additional information · plain
“Thus, a weighted average of the occupation-specific-effect sizes among White job candidates across the different studies signals a preference for female job applicants.”
Quote in the paperSubstantial claim- Also stated in: § Additional information · plain
May combine two claims.
Findingnone givenAt the individual study level, some studies identified gender discrimination, some did not, and others observed gender bias only in conjunction with variables such as race/ethnicity, credentials, or parenthood status.
“At the individual study level, some studies identified gender discrimination, some did not, and others observed gender bias only in conjunction with variables such as race/ethnicity, credentials, or parenthood status.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe result that only the ratio of females predicts gender bias holds in meta-regressions both when the ratio of females in occupations is controlled for and when it is not controlled for.
“Such a result holds for both meta-regressions when the ratios of females in occupations are controlled for and are not controlled for.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe meta-analysis found two significant patterns of gender bias: an occupational characteristic significantly predicted gender bias in job callback rates and findings varied by race.
“Importantly, however, the meta-analysis found two significant patterns of gender bias: an occupational characteristic significantly predicted gender bias in job callback rates and findings varied by race.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn more female-dominated occupations, White women received more callbacks than their male counterparts, whereas Black women did not experience a similar gender preference.
“we find that in more female-dominant occupations, White women received more callbacks than their male counterparts, whereas Black women did not experience a similar gender preference.”
- Also stated in: § The Intersection of Race and Gender · plain
“White female applicants in more female-dominated occupations benefit from a preference in hiring, whereas Black female applicants in female-dominated occupations do not experience a similar benefit.”
- Also stated in: § Discussion · plain
“in female-dominated occupations, White women applicants received more callbacks than White men, but Black women did not experience a parallel benefit.”
Quote in the paperSubstantial claim- Also stated in: § The Intersection of Race and Gender · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe remaining 24 percent of studies did not manipulate gender but controlled for gender by sending resumes from only female or only male applicants to each job opening.
“The remaining 24 percent of the studies did not manipulate gender but did control for gender by sending resumes from either only female applicants or only male applicants to each job opening.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe trend across occupations for Black applicants is less clear-cut: some more female-dominated occupations have positive discrimination ratios, whereas some more male-dominated occupations have negative discrimination ratios.
“Furthermore, the trend across occupations is less clear-cut.Some more female-dominated occupations have positive discrimination ratios (male candidates received more callback offers than female candidates), whereas some more male-dominated occupations have negative discrimination ratios (women received more callbacks than men).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOccupations at the top of the forest plot (e.g., nursing and administrative positions) tend to be female-dominated, and their log discrimination ratios are mostly negative, indicating that resumes from women received more callbacks than resumes from men.
“Occupations at the top of the forest plot (e.g., nursing and administrative positions) tend to be female-dominated.The log discrimination ratios for these occupations are mostly negative, indicating that resumes from women received more callbacks than resumes from men.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOccupations at the bottom of the plot (e.g., maintenance work and manufacturing) tend to be more male-dominated, and their log discrimination ratios are mostly positive, indicating that resumes from men received more callbacks than resumes from women.
“Occupations at the bottom of the plot (e.g., maintenance work and manufacturing) tend to be more male-dominated.The log discrimination ratios for these occupations are mostly positive, indicating that resumes from men received more callbacks than resumes from women.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe education levels used in each study and occupation group of applications were comparable across White and Black applicants.
“The education levels used in each study and occupation group of applications were comparable, with an average of 14.82 years of education for White applicants and 14.75 years of education for Black applicants.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
PremisecitationThe intersectional perspective emphasizes that status beliefs are not based solely on the applicant's gender but represent compounded beliefs about multiple groups.
“A second conceptual framework concerning status beliefs is the intersectional perspective, which emphasizes that status beliefs are not based solely on the applicant's gender but rather represent compounded beliefs about multiple groups (Choo and Ferree 2010;Crenshaw 1989).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Choo and Ferree 2010 source
- cites Crenshaw 1989
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationown dataThe findings support theories of tokenism and statistical discrimination.
“Our findings further support these theories and are aligned with empirical studies that argue that persisting sex segregation in the workforce shapes and is shaped by genderbased stereotypes (Breda et al. 2020;Charles and Bradley 2002).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe authors independently cross-coded the occupations and went through multiple rounds of calibration.
“The authors independently cross-coded the occupations and went through multiple rounds of calibration.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationHiring discrimination against women in male-dominant jobs is the result of employers' rational choices, given that they seek to overcome uncertainty around applicant performance.
“Statistical discrimination theory argues that hiring discrimination against women in male-dominant jobs is the result of employers' rational choices, given that they seek to overcome uncertainty around applicant performance (Arrow 1973).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Arrow 1973
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe two researchers cross-coded the data by reading each article to extract and code the variables necessary for the analysis.
“After compiling the study list, the two researchers cross-coded the data by reading each article to extract and code the variables necessary for the analysis.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe results of existing field experiments on gender bias generalize predominantly to White female applicants, and Black female applicants follow somewhat different patterns.
“These nuanced dynamics highlight that the results of existing field experiments on gender bias generalize predominantly to White female applicants and Black female applicants follow somewhat different patterns.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisereasoningThe proportion of women in an organization or institution (i.e., occupation) is an important factor associated with employers' hiring bias.
“A third key perspective is a composition-based status perspective, which posits that the proportion of women in an organization or institution (i.e., occupation) is an important factor associated with employers' hiring bias.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataGender discrimination occurs across the horizontal sex segregation of occupations in both female- and male-dominant occupations.
“Such results suggest gender discrimination occurs across the horizontal sex segregation of occupations in both female-and male-dominant occupations.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWhen women hold a proportional minority of positions within an organization, it induces gender inequality.
“Organizational scholars also emphasize the composition of the workforce as a factor driving gender discrimination, arguing that when women hold a proportional minority of positions within an organization, it induces gender inequality (Kanter 1977).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kanter 1977 source
Classified as probably not a substantial claim.
PremisereasoningThe experimental research design allows researchers to explore decisions made by real employers, thus offering a high internal validity level.
“The experimental research design allows researchers to explore decisions made by real employers, thus offering a high internal validity level.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe results suggest a mixed pattern of gender-biased hiring decisions, which may stem from the diversity in institutional contexts and occupational characteristics across studies.
“In sum, the results suggest a mixed pattern of gender-biased hiring decisions, which we posit may stem from the diversity in institutional contexts and occupational characteristics across studies.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation98.8 percent of Fortune 500 companies use applicant tracking systems.
“Research conducted by Jobscan (Myers 2023), a recruitment solutions company, showed that 98.8 percent of Fortune 500 companies use applicant tracking systems-software that makes hiring decisions by using algorithms to automatically filter submitted resumes.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Myers 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable).
PremisereasoningDemand-side explanations of hiring discrimination are particularly useful in explaining the results of prior field experiments.
“Thus, demand-side explanations of hiring discrimination, which focus on employers' decisions to hire, promote, and evaluate employees, are particularly useful in explaining the results of prior field experiments”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
PremisecitationAudit studies use fictitious job applications with randomly assigned characteristics to audit employment outcomes and assess discrimination in hiring based on manipulated characteristics.
“In audit studies, researchers use fictitious job applications with randomly assigned characteristics (e.g., gender, race/ethnicity, and criminal record) to audit employment outcomes and thereby assess discrimination in hiring based on manipulated characteristics (Gaddis 2018).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gaddis 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationStatistical discrimination theory posits that employers make hiring decisions based on their prior knowledge about average productivity levels across different groups.
“Economists have introduced the idea of statistical discrimination, which posits that employers make hiring decisions based on their prior knowledge about average productivity levels across different groups (e.g., women and other identifiable classes of job applicants) (Bielby and Baron 1986;McPherson, Smith-Lovin, and Cook 2001).”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningOnly when the multidimensionality of gender and racial stereotypes is considered can we adequately assess the changes and continuities of hiring inequality at the initial screening stage.
“Only when the multidimensionality of gender and racial stereotypes is considered can we adequately assess the changes and continuities of hiring inequality at the initial screening stage.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe random-effect model was used to account for potential variability across studies due to different geographic areas and occupation categories and variation in applicant characteristics.
“we used a randomeffect model to account for potential variability across studies that may result from two factors: individual studies targeting different geographic areas and occupation categories and variation across studies in the characteristics of job applicants.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The random-effect model assumes that there is not a single true effect size but rather a distribution of true effect sizes.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationCrenshaw (1989:139) first coined the term intersectionality based on her evidence from legal cases and her argument that Black women are legally, policywise, and theoretically erased when scholars and policymakers depend on the single-axis analysis that distorts the experiences of Black women.
“Crenshaw (1989: 139) first coined the term intersectionality based on her evidence from legal cases and her argument that Black women are legally, policywise, and theoretically erased when scholars and policymakers depend on the”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Crenshaw (1989: 139)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductcitationThe occupational skill-level variable is an index of occupational education based on the 1989 General Social Survey and 1990 U.S. Census, used following Yu and Kuo (2017) as a measure of overall skill levels of workers within occupations.
“The occupational skill-level variable is an index of occupational education, which is a measure of the percentage of occupational incumbents' education level (one or more years of post-secondary education completed) created using the 1989 General Social Survey and the 1990 U.S. Census.Following Yu and Kuo (2017), we use this as a measure of the overall skill levels of workers within occupations.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yu and Kuo (2017) source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenElements of the typical design of field experiments (focusing on entry-level positions or mixing fictitious resumes with real applicants) reduce the risk that an employer will become aware of the experiment.
“Although elements of the typical design of field experiments (focusing on entry-level positions or mixing fictitious resumes with real applicants) reduce the risk that an employer will become aware of the experiment”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationMost audit studies involve sending fictitious resumes for consideration for entry-level positions but do not follow up on the callbacks received.
“Most audit studies involve sending fictitious resumes for consideration for entry-level positions but do not follow up on the callbacks received (Gaddis 2018).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gaddis 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationcitationPolicies that aim to solve issues of gender- and race-based discrimination separately often result in reinforcing existing marginalizations (i.e., Black women) who fall into a crack neither noticed nor protected by the policies.
“In particular, policies that aim to solve issues of gender-and race-based discrimination (separately) often result in reinforcing existing marginalizations (i.e., Black women) who will fall into the crack that is neither noticed nor protected by the policies.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
The random-effect model assumes that there is not a single true effect size but rather a distribution of true effect sizes.
“The random-effect model assumes that there is not a single true effect size but rather a distribution of true effect sizes.”
The paper relies on this without support.
Gender and race are context-specific social constructs.
“considering the fact that gender and race are context-specific social constructs.”
The paper relies on this without support.
Fictitious names reliably signal race to employers.
“racial background was indicated by fictitious names”
The paper relies on this without support.
Callback rates from audit studies are a valid indicator of hiring discrimination at the initial screening stage.
“Most audit studies involve sending fictitious resumes for consideration for entry-level positions but do not follow up on the callbacks received (Gaddis 2018).”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Gender-manipulation descriptions conflict
MinorThe text divides all studies into 35 percent focused primarily on gender and 65 percent including gender as part of the manipulation. That implies every study manipulated gender, contrary to the note that 24 percent did not.
“Among all the selected studies, 35 percent focus primarily on gender, whereas 65 percent include gender as part of the manipulation.”
“The remaining 24 percent of the studies did not manipulate gender but did control for gender by sending resumes from either only female applicants or only male applicants to each job opening.”
Data-collection date range excludes a year shown in the results
MinorThe study-level description says the included experiments were conducted from 2002 through 2021, but the results list 2001 as a year of data collection.
“our data span field experiments that were conducted between 2002 and 2021”
“Year of Data Collection 2001 2005 2006 2010”
Cites
Birds of a Feather: Homophily in Social Networks
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.415 · Cited by 18,567 · see citing works
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793 · see citing works
Bertrand, M., & Mullainathan, S. (2004) · American Economic Review · doi:10.1257/0002828042002561 · Cited by 4,504 · see citing works
Some Effects of Proportions on Group Life: Skewed Sex Ratios and Responses to Token Women
Kanter, R. M. (1977) · American Journal of Sociology · doi:10.1086/226425 · Cited by 3,154 · see citing works
Pager, D. (2003) · American Journal of Sociology · doi:10.1086/374403 · Cited by 2,840 · see citing works
Getting a Job: Is There a Motherhood Penalty?
Correll, S. J., Benard, S., & Paik, I. (2007) · American Journal of Sociology · doi:10.1086/511799 · Cited by 2,525 · see citing works
Choo, H. Y., & Ferree, M. M. (2010) · Sociological Theory · doi:10.1111/j.1467-9558.2010.01370.x · Cited by 1,698 · see citing works
Why Status Matters for Inequality
Ridgeway, C. L. (2013) · American Sociological Review · doi:10.1177/0003122413515997 · Cited by 979 · see citing works
4. Socioeconomic Indexes for Occupations: A Review, Update, and Critique
Hauser, R. M., & Warren, J. R. (1997) · Sociological Methodology · doi:10.1111/1467-9531.271028 · Cited by 913 · see citing works
Men and Women at Work: Sex Segregation and Statistical Discrimination
Bielby, W. T., & Baron, J. N. (1986) · American Journal of Sociology · doi:10.1086/228350 · Cited by 907 · see citing works
Meta-analysis of field experiments shows no change in racial discrimination in hiring over time
Quillian, L., Pager, D., Hexel, O., & Midtbøen, A. H. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1706255114 · Cited by 887 · see citing works
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845 · see citing works
Sex Discrimination in Restaurant Hiring: An Audit Study
Neumark, D., Bank, R. J., & Nort, K. D. V. (1995) · The Quarterly Journal of Economics · doi:10.2307/2946676 · Cited by 677 · see citing works
Charles, M., & Grusky, D. B. (2004) · Stanford University Press eBooks · doi:10.1515/9781503618183 · Cited by 570 · see citing works
Equal but Separate? A Cross-National Study of Sex Segregation in Higher Education
Charles, M., & Bradley, K. (2002) · American Sociological Review · doi:10.2307/3088946 · Cited by 491 · see citing works
Progress toward gender equality in the United States has slowed or stalled
England, P., Levine, A., & Mishel, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1918891117 · Cited by 480 · see citing works
Budig, M. J., & Hodges, M. J. (2010) · American Sociological Review · doi:10.1177/0003122410381593 · Cited by 424 · see citing works
Eaton, A. A., Saunders, J. F., Jacobson, R. K., & West, K. (2019) · Sex Roles · doi:10.1007/s11199-019-01052-w · Cited by 401 · see citing works
Class Advantage, Commitment Penalty
Rivera, L. A., & Tilcsik, A. (2016) · American Sociological Review · doi:10.1177/0003122416668154 · Cited by 380 · see citing works
Lamont, M., Silva, G. M., Welburn, J. S., Guetzkow, J., Mizrachi, N., Herzog, H., & Reis, E. (2016) · Princeton University Press eBooks · doi:10.2307/j.ctv346qr9 · Cited by 375 · see citing works
The Mark of a Woman’s Record: Gender and Academic Performance in Hiring
Quadlin, N. (2018) · American Sociological Review · doi:10.1177/0003122418762291 · Cited by 342 · see citing works
Male Advantage and the Gender Composition of Jobs: Who Rides the Glass Escalator?
Budig, M. J. (2002) · Social Problems · doi:10.1525/sp.2002.49.2.258 · Cited by 337 · see citing works
The Value of Postsecondary Credentials in the Labor Market: An Experimental Study
Deming, D. J., Yuchtman, N., Abulafi, A., Goldin, C., & Katz, L. F. (2016) · American Economic Review · doi:10.1257/aer.20141757 · Cited by 278 · see citing works
Gender stereotypes can explain the gender-equality paradox
Breda, T., Jouini, E., Napp, C., & Thebault, G. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2008704117 · Cited by 233 · see citing works
Cotter, D. A., Hermsen, J. M., & Vanneman, R. D. (2004) · Cited by 212 · see citing works
Weisshaar, K. (2018) · American Sociological Review · doi:10.1177/0003122417752355 · Cited by 192 · see citing works
Is the Motherhood Penalty Larger for Low-Wage Women? A Comment on Quantile Regression
Killewald, A. A., & Bearak, J. (2014) · American Sociological Review · doi:10.1177/0003122414524574 · Cited by 189 · see citing works
Ridgeway, C. L. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448895 · Cited by 148 · see citing works
Women’s Work Pathways Across the Life Course
Damaske, S., & Frech, A. (2016) · Demography · doi:10.1007/s13524-016-0464-z · Cited by 142 · see citing works
Nunley, J. M., Pugh, A., Romero, N., & Seals, A. (2015) · The B E Journal of Economic Analysis & Policy · doi:10.1515/bejeap-2014-0082 · Cited by 119 · see citing works
Hirsh, C. E. (2009) · American Sociological Review · doi:10.1177/000312240907400205 · Cited by 117 · see citing works
Quillian, L., Lee, J. J., & Oliver, M. (2020) · Social Forces · doi:10.1093/sf/soaa026 · Cited by 105 · see citing works
Yu, W.-H., & Kuo, J. C.-L. (2017) · American Sociological Review · doi:10.1177/0003122417712729 · Cited by 101 · see citing works
Schaerer, M., du Plessis, C., Nguyen, M. H. B., van Aert, R. C. M., Tiokhin, L., Lakens, D., Giulia Clemente, E., Pfeiffer, T., Dreber, A., Johannesson, M., et al. (2023) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2023.104280 · Cited by 87 · see citing works
Galos, D. R., & Coppock, A. (2023) · Science Advances · doi:10.1126/sciadv.ade7979 · Cited by 84 · see citing works
How Race and Unemployment Shape Labor Market Opportunities: Additive, Amplified, or Muted Effects?
Pedulla, D. S. (2018) · Social Forces · doi:10.1093/sf/soy002 · Cited by 82 · see citing works
Discrimination against gays and lesbians in hiring decisions: a meta-analysis
Flage, A. (2019) · International Journal of Manpower · doi:10.1108/ijm-08-2018-0239 · Cited by 78 · see citing works
Gender Inequality Across Local Wage Hierarchies
Huffman, M. L. (2004) · Work and Occupations · doi:10.1177/0730888404266384 · Cited by 65 · see citing works
The Labor Market Value of Taste: An Experimental Study of Class Bias in U.S. Employment
Thomas, K. (2018) · Sociological Science · doi:10.15195/v5.a24 · Cited by 44 · see citing works
The Evaluation of Founder Failure and Success by Hiring Firms: A Field Experiment
Botelho, T. L., & Chang, M. H. (2022) · Organization Science · doi:10.1287/orsc.2022.1592 · Cited by 40 · see citing works
Gender Composition of Labor Queues and Gender Disparities in Hiring
Campero, S., & Fernandez, R. M. (2018) · Social Forces · doi:10.1093/sf/soy097 · Cited by 39 · see citing works
Valentino, L. (2019) · Social Forces · doi:10.1093/sf/soz145 · Cited by 36 · see citing works
Gaddis, S. M., Larsen, E. N., Crabtree, C. D., & Holbein, J. B. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3975770 · Cited by 33 · see citing works
Educational Authority in the ‘‘Open Door’’ Marketplace
Deterding, N. M., & Pedulla, D. S. (2016) · Sociology of Education · doi:10.1177/0038040716652455 · Cited by 32 · see citing works
Chavez, K., Weisshaar, K., & Cabello-Hutt, T. (2022) · Work and Occupations · doi:10.1177/07308884221094539 · Cited by 29 · see citing works
Mihut, G. (2021) · Studies in Higher Education · doi:10.1080/03075079.2020.1870949 · Cited by 29 · see citing works
The intergenerational sources of the U-turn in gender segregation
Ling, Z., & Grusky, D. B. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2121439119 · Cited by 18 · see citing works
Mai, Q. D. (2022) · Work and Occupations · doi:10.1177/07308884221134470 · Cited by 9 · see citing works
Blacklist or Short List: Do Employers Discriminate against Union Supporter Job Applicants?
Kreisberg, N., & Wilmers, N. (2021) · Industrial and Labor Relations Review · doi:10.1177/00197939211036444 · Cited by 7 · see citing works
Lippens, L., Vermeiren, S., & Baert, S. (2023) · European Economic Review · doi:10.1016/j.euroecorev.2022.104315
Flagged first, then by citations
Research works printed but not linked: 4
- Arrow, Kenneth J. 1973. "The Theory of Discrimination." Pp. 3-33 in Discrimination in Labor Markets. Princeton University Press.
- Cotter, David, Joan Hermsen, and Paula England. 2008. "Moms and Jobs: Trends in Mothers' Employment and Which Mothers Stay Home." Pp. 379-86 in, American families: A multicultural reader. New York: Routledge.
- Crenshaw, Kimberlé. 1989. "Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Anti-Discrimination Doctrine, Feminist Theory and Anti-Racist Politics." Pp. 139-167 in University of Chicago Legal Forum.
- Damaske, Sarah. 2011b. For the Family? How Class and Gender Shape Women's Work. New York: Oxford University Press.
Cited by
Sorted by citations
Rozado, D. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3628 · Cited by 8 · see citing works
Habicht, I. M., Zschirnt, E., & Daikeler, J. (2025) · doi:10.31235/osf.io/7zxcy_v1
Why Treatment Prevalence Matters: Overcoming a Blind Spot in Experimental Inequality Research
Pietrzyk, I., & Jacob, M. (2026) · KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie · doi:10.1007/s11577-026-01068-7
Gender discrimination in hiring in older age? Experimental evidence from employer evaluations
Qvist, J. Y., & Larsen, C. A. (2026) · Acta Sociologica · doi:10.1177/00016993261448810
Is Self-Employment a Career Trap? A Large-Scale Field Experiment in the Labor Market
Asanov, I., & Mavlikeeva, M. (2026) · Entrepreneurship Theory and Practice · doi:10.1177/10422587261440312
Are Occupations Associated with Women Devalued? A Causal Test
Hart, C. G. (2026) · Gender & Society · doi:10.1177/08912432261472510
Ben-Menachem, J., Galper, A., & Fishman, N. (2026) · Sociological Methods & Research · doi:10.1177/00491241261490850
Buttler, D., Smoter, M., & Imdorf, C. (2026) · Research in Globalization · doi:10.1016/j.resglo.2026.100399
- Authors
- Park, S. Y., & Oh, E.
- Affiliations
- University of Wisconsin–Madison (US)
- ORCID
- Oh, Eunsil
- Venue
- Sociological Science
- Publisher
- Society for Sociological Science
- Published
- 2025
- Volume
- 12
- Pages
- 26-50
- ISSN
- 2330-6696 (ISSN-L)
- OpenAlex
- W4406006721
- Cited by
- 7
- References
- 42
- Citations by year
- 2026: 7
- FWCI
- 1.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Sociological Science (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University of Wisconsin-Madison
- Division of Graduate Education
- Topics
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Social Sciences › Gender Studies › Labour Market and Migration
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Labor market dynamics and wage inequality
- Keywords
- gender discrimination in hiring, audit studies, callbacks, job applications, gender composition of occupations, intersectionality, race and gender, female-dominated occupations, Black women, labor market inequality, field experiments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-11-04.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-04.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review