On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions
Open accessAbstract
A preregistered meta-analysis, including 244 effect sizes from 85 field audits and 361,645 individual job applications, tested for gender bias in hiring practices in female-stereotypical and gender-balanced as well as male-stereotypical jobs from 1976 to 2020. A “red team” of independent experts was recruited to increase the rigor and robustness of our meta-analytic approach. A forecasting survey further examined whether laypeople (n = 499 nationally representative adults) and scientists (n = 312) could predict the results. Forecasters correctly anticipated reductions in discrimination against female candidates over time. However, both scientists and laypeople overestimated the continuation of bias against female candidates. Instead, selection bias in favor of male over female candidates was eliminated and, if anything, slightly reversed in sign starting in 2009 for mixed-gender and male-stereotypical jobs in our sample. Forecasters further failed to anticipate that discrimination against male candidates for stereotypically female jobs would remain stable across the decades.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Following prior literature, rates of 0 were changed to 0.5 to avoid dropping cases, reduce bias in the estimator, and prevent division by zero.
“Following an approach used in prior literature (Schaerer et al. 2023), we changed rates of 0 to 0.5 to avoid dropping cases; this methodological choice also reduces bias in the estimator and prevents division by zero.”
From Getting a Foot in the Door: A Meta-Analysis of U.S. Audit Studies of Gender Bias in Hiring (2025) · cites this paper as Schaerer et al. 2023
Section 3.1.4 (Statistical analyses; page not supplied) says the authors added 0.5 to every cell of a 2×2 frequency table before calculating log odds ratios. It explicitly gives reducing estimator bias and avoiding division by zero as reasons. It does not say they changed zero rates to 0.5 or that the adjustment was made to avoid dropping cases. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 1× · see citing works
“…value of future investigations, we recommend researchers adopt open science best practices such as direct replication (Klein et al., 2014; Open Science Collaboration, 2015; Simons, 2014), pre-registration (Wagenmakers et al., 2012), red teams (Lakens, 2020), registered reports (Chambers…”
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053 · see citing works
Are leader stereotypes masculine? A meta-analysis of three research paradigms
Koenig, A. M., Eagly, A. H., Mitchell, A. A., & Ristikari, T. (2011) · Psychological Bulletin · doi:10.1037/a0023557 · Cited by 1,748 · see citing works
Evaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314 · cited 1× · see citing works
“Previous research has demonstrated that academics sometimes perform well in anticipating the results of scientific studies, based on limited information such as article abstracts and study materials (Camerer et al., 2016; Dreber et al., 2015; Forsell et al., 2019).”
Self-citationEagly, A. H., Nater, C., Miller, D. I., Kaufmann, M., & Sczesny, S. (2020) · American Psychologist · doi:10.1037/amp0000494 · Cited by 1,136 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
Methodological issues and advances in biological meta-analysis
Nakagawa, S., & Santos, E. S. A. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9555-5 · Cited by 1,032 · see citing works
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916 · see citing works
Meta-analysis of field experiments shows no change in racial discrimination in hiring over time
Quillian, L., Pager, D., Hexel, O., & Midtbøen, A. H. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1706255114 · Cited by 887 · see citing works
The Value of Direct Replication
Simons, D. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514755 · Cited by 630 · see citing works
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588 · see citing works
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548 · cited 2× · see citing works
“In light of the strong need to enhance objectivity, increase trust, and generally maximize the informational value of our meta-analysis, we leveraged emerging best practices of open science, including pre-registration of analyses and open data (Nelson et al., 2018; Wagenmakers et al., 2012).”
Fixed effects and variance components estimation in three-level meta-analysis
Konstantopoulos, S. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.35 · Cited by 521 · see citing works
Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration
Mellers, B., Hertwig, R., & Kahneman, D. (2001) · Psychological Science · doi:10.1111/1467-9280.00350 · Cited by 448 · see citing works
Not So Subtle: A Meta-Analytic Investigation of the Correlates of Subtle and Overt Discrimination
Jones, K. P., Peddie, C. I., Gilrane, V. L., King, E. B., & Gray, A. L. (2016) · Journal of Management · doi:10.1177/0149206313506466 · Cited by 445 · see citing works
Williams, M. J., & Tiedens, L. Z. (2016) · Psychological Bulletin · doi:10.1037/bul0000039 · Cited by 374 · cited 1× · see citing works
“…they were part of a scientific study, the effects of year can be used to assess stability or change in labor market biases over time (Eagly, Makhijani, & Klonsky, 1992; Koch, D ’ Mello, & Sackett, 2015; Quillian & Lee, 2023; Quillian et al., 2017; Stanley & Jarrell, 1998; Williams & Tiedens, 2016).”
Jost, J. T., Rudman, L. A., Blair, I. V., Carney, D. R., Dasgupta, N., Glaser, J., & Hardin, C. D. (2009) · Research in Organizational Behavior · doi:10.1016/j.riob.2009.10.001 · Cited by 353 · see citing works
Using prediction markets to estimate the reproducibility of scientific research
Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek, B. A., & Johannesson, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516179112 · Cited by 331 · see citing works
Self-citationCultural Change: The How and the Why
Varnum, M. E. W., & Grossmann, I. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617699971 · Cited by 294 · see citing works
Gender and leadership emergence: A meta-analysis and explanatory model
Badura, K. L., Grijalva, E., Newman, D. A., Yan, T. T., & Jeon, G. (2018) · Personnel Psychology · doi:10.1111/peps.12266 · Cited by 288 · cited 1× · see citing works
“…it comes to deciding who to hire (Tetlock & Mitchell, 2009), and given empirical evidence of changes in at least some gender norms and behaviors (Badura, Grijalva, Newman, Yan, & Jeon, 2018; Hora, Badura, Lemoine, & Grijalva, 2021; Koenig, Eagly, Mitchell, & Ristikari, 2011) may also be less…”
What mediates sex discrimination in hiring decisions?
Glick, P., Zion, C., & Nelson, C. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.55.2.178 · Cited by 284 · see citing works
Why Should Women Get Less? Evidence on the Gender Pay Gap from Multifactorial Survey Experiments
Auspurg, K., Hinz, T., & Sauer, C. (2017) · American Sociological Review · doi:10.1177/0003122416683393 · Cited by 269 · see citing works
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219 · see citing works
Comparative Perspectives on Racial Discrimination in Hiring: The Rise of Field Experiments
Quillian, L., & Midtbøen, A. H. (2021) · Annual Review of Sociology · doi:10.1146/annurev-soc-090420-035144 · Cited by 180 · see citing works
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.
Landy, J. F., Jia, M. (., Ding, I. L., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q. F., et al. (2020) · Psychological Bulletin · doi:10.1037/bul0000220 · Cited by 168 · see citing works
Self-citationRich, J. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2517887 · Cited by 134 · see citing works
Clark, C. J., & Winegard, B. M. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1721233 · Cited by 128 · see citing works
Self-citationLippens, L., Vermeiren, S., & Baert, S. (2023) · European Economic Review · doi:10.1016/j.euroecorev.2022.104315
Flagged first, then by citations
Cited by
Sorted by citations
Getting a Foot in the Door: A Meta-Analysis of U.S. Audit Studies of Gender Bias in Hiring
Park, S. Y., & Oh, E. (2025) · Sociological Science · doi:10.15195/v12.a2 · Cited by 7 · see citing works
- Authors
- Schaerer, M., du Plessis, C., Nguyen, M. H. B., van Aert, R. C. M., Tiokhin, L., Lakens, D., Giulia Clemente, E., Pfeiffer, T., Dreber, A., Johannesson, M., Clark, C. J., & Luis Uhlmann, E.
- Affiliations
- Singapore Management University (SG)
- Eindhoven University of Technology (NL)
- Massey University (NZ)
- Stockholm School of Economics (SE)
- Corresponding author
- Schaerer, Michael
- ORCID
- Venue
- Organizational Behavior and Human Decision Processes
- Publisher
- Elsevier BV
- Published
- 2023-11
- Volume
- 179
- Pages
- 104280
- ISSN
- 0749-5978 (ISSN-L)
- OpenAlex
- W4388565334
- Cited by
- 87
- References
- 29
- Citations by year
- 2026: 18 · 2025: 33 · 2024: 31 · 2023: 2 · 2022: 2 · 2020: 1
- FWCI
- 3.61
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Organizational Behavior and Human Decision Processes (publishedVersion, cc-by)
- Multilingual Matters (Channel View Publications) (publishedVersion, cc-by)
- TU/e Research Portal (publishedVersion, cc-by)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (publishedVersion, cc-by)
- ePrints Soton (University of Southampton) (acceptedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Open Access at Essex (University of Essex) (acceptedVersion)
- Massey Research Online (Massey University) (acceptedVersion, cc-by)
- Deposito Adademico Digital Universidad De Navarra (University of Navarra) (submittedVersion, cc-by)
- eScholarship (California Digital Library) (submittedVersion)
- Veřejné služby Informačního systému (Masarykiana Brunensis Universitas) (submittedVersion, cc-by)
- PEARL (University of Plymouth) (submittedVersion, cc-by)
- St Andrews Research Repository (St Andrews Research Repository) (submittedVersion, cc-by)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, cc-by)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Research Online (University of Wollongong) (submittedVersion)
- UNI ScholarWorks (University of Northern Iowa) (submittedVersion, cc-by)
- Research portal (Tilburg University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- BOA (University of Milano-Bicocca) (submittedVersion)
- BORIS (University Library Bern) (submittedVersion, cc-by)
- Research Portal (King's College London) (submittedVersion, cc-by)
- IE Repository (submittedVersion, cc-by)
- CBS Research Portal (Copenhagen Business School) (submittedVersion, cc-by)
- Open Research (University of Surrey) (submittedVersion, cc-by)
- Open Access CRIS of the University of Bern
- Funders
- Knut and Alice Wallenberg Foundation
- Marianne and Marcus Wallenberg Foundation
- Jan Wallander and Tom Hedelius Foundation
- Government of Singapore Ministry of Education
- Netherlands Organisation for Scientific Research
- Topics
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Social Sciences › Gender Studies › Gender Diversity and Inequality
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Keywords
- gender discrimination in hiring, gender bias, job applications, field experiments, gender stereotypes, hiring decisions, forecast bias, overconfidence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review