Publication bias as a threat to the validity of meta-analytic results
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Publication bias presents a strong challenge to any review of evaluation studies (Rothstein, 2008).”
From Problem-oriented policing for reducing crime and disorder: An updated systematic review and meta-analysis (2020) · cites this paper as Rothstein, 2008
In the citing paper
“Trim‐and‐fill procedures do suffer from some well‐known limitations that could result in the underestimation or overestimation of publication bias (Rothstein, 2008; Simonsohn, Nelson, & Simmons, 2014).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Rothstein, 2008
In the citing paper
“Following the advice of Rothstein (2008), we apply two of the conventional methods that address publication bias.”
From A Meta-analysis of Attachment to Parents and Delinquency (2012) · cites this paper as Rothstein 2008
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1×
“( Rosenthal 1979 ). He was concerned that metaanalytic results could be wrong, due to non-significant studies that remained in researchers’ file drawers.”
ResultPerceived organizational support: a review of the literature.
Rhoades, L., & Eisenberger, R. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.698 · Cited by 7,592 · cited 1×
“Mullen et al. (2001) proposed Rosenthal’s guideline as a formal rule, and the authors of several recent psychology meta-analyses (Del Vecchio and O’Leary 2004; Rhoades and Eisenberger 2002 ) have used this formula to assess the results of their file drawer analyses.”
MethodJournal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Research works printed but not linked: 2
- Ioannidis, J. (2005). Differentiating biases from genuine heterogeneity. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.). Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
- Wall Street Journal, (2004, November 1). E-Mails Suggest Merck Knew Vioxx’s Dangers at Early Stage.
Cited by
Items 201–228 of 228
Empirical Benchmarks for Interpreting Effect Sizes in Child Counseling Research
Weisberger, A. G. (2017) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc984168 · Cited by 1
Duggan, M., & Tressoldi, P. E. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3097702 · Cited by 1
Jin, L., Yu, D., Jin, Z., & Li, H. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/63sjt · Cited by 1
Cognitive functioning in criminal offenders: A multi-level meta-analysisPreprint
Hutten, J. C., Ziermans, T. B., Geurts, H. M., Stams, G. J. J. M., & van Horn, J. E. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/s48rn · Cited by 1
Causal Meta-Analysis : Methodology and Applications
Bax, L. (2009) · Utrecht University Repository (Utrecht University)
ELEKTRONISCH TOEZICHT EN 'WHAT WORKS': Een meta-analyse over recidive na elektronisch toezicht
der Rechtsgeleerdheid, F. (2013)
Am I Luckier Than You? Individual Differences in Precognition Performance
Zdrenka, M. (2018) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.19314362 · Cited by 1
“However, without a metaanalysis, researchers will likely impose their own synthesis of the data, and a meta-analysis can provide greater clarity in this regard—even if it only incorporates two or three studies, as Valentine, Pigott, and Rothstein (2010) argue that all other synthesis techniques are less transparent an…”
BackgroundSchmidt, F. L. (2012) · Wiley eBooks · doi:10.1002/9781118133880.hop202020
Ponce, M. F. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3119644
Ponce, M. F. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3164259
Duggan, M., & Tressoldi, P. E. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/5u48e
陈, 俊. (2024) · Advances in Social Sciences · doi:10.12677/ass.2024.139870
Cognitive functioning in criminal offenders: A multi-level meta-analysisPreprint
Hutten, J. C., Ziermans, T. B., Geurts, H. M., Stams, G. J. J. M., & van Horn, J. E. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/s48rn_v2
Schwaighofer, M. (2015) · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · doi:10.5282/edoc.17906
Barbeau-Grégoire, M. (2022) · Papyrus : Institutional Repository (Université de Montréal) · doi:10.71781/3692 · Cited by 1
Emotional Regulation Skills and Depression in Adolescence
Thompson, Z. (2024) · UEA Digital Repository (University of East Anglia) · Cited by 1
Mapping the role of attachment in eating behaviors
Faber, A. (2015) · eScholarship@McGill (McGill) · doi:10.82308/22146
Chowdhury, M., Chang, R., & Choi, Y. (2023) · OakTrust (Texas A&M University Libraries)
Chen, K., Yang, Y., Shao, J., & Wang, Z. (2026) · Aggression and Violent Behavior · doi:10.1016/j.avb.2026.102155
Wang, D., Wang, X., Jaapar, S. Z. S., & Razak, A. A. (2026) · BMC Psychology · doi:10.1186/s40359-026-04527-0
“…and should be interpreted cautiously, the significant Begg’s test and the asymmetry observed in the funnel plot suggest that studies with small or null effects may be underrepresented in the published literature, a pattern commonly interpreted as indicative of potential publication bias [63, 85].”
The relationship between family factors and children’s executive function: A series of meta-analyses
Jie, X. U., Huang, Y., & Gai, X. (2026) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2026.1170
A Contemporary Meta-Analysis of Goal Setting and Performance
Zhou, Y. (2025) · University of Minnesota Digital Conservancy (University of Minnesota)
Father Involvement and Children’s Social Behavior: A Three-Level Meta-Analysis
Li, Y., & Hao, J. (2026) · Behavioral Sciences · doi:10.3390/bs16081426
“In terms of the funnel plot, when the effect results were all at the top of the funnel plot and were relatively evenly distributed on both sides of the average effect size, it indicated that the meta-analysis was less likely to be affected by publication bias (Rothstein, 2008).”
Meta-analysis reconceived from a finite population sampling perspective
Forte, M., & Tipton, E. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10115
Predicting Symptom Change in Mental Health Chatbot Interventions: A Meta-Analysis
Saucier, C. J., Li, H., Calabrese, C. R., Baker, A. M., & Park, J. (2026) · Health Communication · doi:10.1080/10410236.2026.2734165
Pitfalls, common errors, and false interpretation
Gumaa, M., Lewis, D. I., & Swellam, M. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45595-7.00006-6
Family functioning and meaning in life among Chinese adolescents and young adults: a meta-analysis
Mingcheng, H., Wang, Y., Li, J., Pan, Z., & Min, Z. (2026) · Current Psychology · doi:10.1007/s12144-026-10271-w
Metadata
- Authors
- Rothstein, H. R.
- Affiliations
- Baruch College (US)
- Corresponding author
- Rothstein, Hannah R.
- ORCID
- Rothstein, Hannah R.
- Venue
- Journal of Experimental Criminology
- Published
- 2007-12-19
- Volume
- 4
- Issue
- 1
- Pages
- 61-81
- ISSN
- 1572-8315 (ISSN-L)
- 1573-3750
- OpenAlex
- W2010026440
- Cited by
- 216
- References
- 51
- Citations by year
- 2026: 11 · 2025: 23 · 2024: 23 · 2023: 12 · 2022: 20 · 2021: 10 · 2020: 10 · 2019: 13 · 2018: 20 · 2017: 11 · 2016: 9 · 2015: 8 · 2014: 16 · 2013: 4 · 2012: 9
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Journal of Experimental Criminology (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- publication bias, trim and fill, cumulative meta-analysis, effect size estimation, drug courts, recidivism, criminology research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.