Potential publication bias in the stereotype threat literature: Comment on Nguyen and Ryan (2008).
Abstract
Stereotype threat is a widely cited psychological phenomenon with purported important real-world consequences. Reanalysis of data from the Nguyen and Ryan (2008) stereotype threat meta-analysis indicated the presence of small study effects in which the effect size for less precise studies was larger than the effect size for more precise studies. Four methods to adjust the meta-analysis effect size for potential publication bias produced divergent estimates, from essentially no change, to a 50% decrease, to a reduction of the estimated effect size to near zero. Caution is therefore warranted both for citing Nguyen and Ryan (2008) as evidence of a meaningful stereotype threat effect and for claiming that the stereotype threat effect size is negligible based on these adjustments, given that the detected small study effects might be due to unexplored moderators instead of publication bias. (PsycINFO Database Record
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Initial studies on stereotype threat found statistically significant effects, but subsequent re-analyses and other studies failed to replicate them.
“even though the initial studies on stereotype threat found statistically significant effects (Nguyen and Ryan [2008]), subsequent re-analyses (Zigerell [2017]), and other studies failed to replicate them (Finnigan and Corker [2016]; Flore et al. [2019]).”
From Science Communication and the Problematic Impact of Descriptive Norms (2023) · cites this paper as Zigerell [2017]
Cites
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · see citing works
Brown, R. P., & Day, E. A. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.4.979 · Cited by 120 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Shewach, O. R., Sackett, P. R., & Quint, S. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000420 · Cited by 97 · see citing works
Science Communication and the Problematic Impact of Descriptive Norms
Peters, U. (2023) · The British Journal for the Philosophy of Science · doi:10.1086/715001 · Cited by 13 · see citing works
- Authors
- Zigerell, L. J.
- Affiliations
- Illinois State University (US)
- Corresponding author
- Zigerell, L. J.
- ORCID
- Zigerell, L. J.
- Venue
- Journal of Applied Psychology
- Published
- 2017-08
- Volume
- 102
- Issue
- 8
- Pages
- 1159-1168
- ISSN
- 0021-9010 (ISSN-L)
- 1939-1854
- OpenAlex
- W2744784989
- Cited by
- 49
- References
- 2
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 5 · 2023: 6 · 2022: 9 · 2021: 8 · 2020: 2 · 2019: 8 · 2018: 5 · 2017: 1
- FWCI
- 3.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Applied Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- stereotype threat, publication bias, small-study effects, meta-analysis, reanalysis, moderators
- MeSH terms
- Female, Humans, Minority Groups, Stereotyping, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
