Publication bias as a threat to the validity of meta-analytic results
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Publication bias presents a strong challenge to any review of evaluation studies (Rothstein, 2008).”
From Problem-oriented policing for reducing crime and disorder: An updated systematic review and meta-analysis (2020) · cites this paper as Rothstein, 2008
In the citing paper
“Trim‐and‐fill procedures do suffer from some well‐known limitations that could result in the underestimation or overestimation of publication bias (Rothstein, 2008; Simonsohn, Nelson, & Simmons, 2014).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Rothstein, 2008
In the citing paper
“Following the advice of Rothstein (2008), we apply two of the conventional methods that address publication bias.”
From A Meta-analysis of Attachment to Parents and Delinquency (2012) · cites this paper as Rothstein 2008
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1×
“( Rosenthal 1979 ). He was concerned that metaanalytic results could be wrong, due to non-significant studies that remained in researchers’ file drawers.”
ResultPerceived organizational support: a review of the literature.
Rhoades, L., & Eisenberger, R. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.698 · Cited by 7,592 · cited 1×
“Mullen et al. (2001) proposed Rosenthal’s guideline as a formal rule, and the authors of several recent psychology meta-analyses (Del Vecchio and O’Leary 2004; Rhoades and Eisenberger 2002 ) have used this formula to assess the results of their file drawer analyses.”
MethodJournal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Research works printed but not linked: 2
- Ioannidis, J. (2005). Differentiating biases from genuine heterogeneity. In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.). Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley.
- Wall Street Journal, (2004, November 1). E-Mails Suggest Merck Knew Vioxx’s Dangers at Early Stage.
Cited by
Items 101–200 of 228
Goh, X. X., Tang, P. Y., & Tee, S. F. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114429 · Cited by 25
Petersen, K., Davis, R. C., Weisburd, D. L., & Taylor, B. G. (2022) · Campbell Systematic Reviews · doi:10.1002/cl2.1217 · Cited by 27
“638) problem, or the potential for selective publication of studies based on their findings is a major concern associated with systematic reviews (Rothstein, 2008).”
BackgroundThe effectiveness of 3D holographic technology on students’ learning performance: a meta-analysis
Yu, Q., Li, B., & Wang, Q. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2022.2124424 · Cited by 25
“If the plot is asymmetric, which suggests the presence of publication bias (Rothstein, 2008).”
Estimating the Prevalence of Copycat Crime: A Research Note
Surette, R. (2013) · Criminal Justice Policy Review · doi:10.1177/0887403413499579 · Cited by 33
“A meta-analysis concern termed publication bias is the possibility that additional unpublished but unknown studies exist that, if included, would substantially alter the meta-analysis conclusions (Rothstein, 2008).”
BackgroundCognitive Processing Therapy: A Meta-analytic Review Among Veterans and Military Personnel with PTSD
Raines, A. M., Clauss, K. E., Schafer, K. M., Shapiro, M. O., Houtsma, C., Boffa, J. W., Ennis, C. R., O'Neil, M. E., & Franklin, C. L. (2023) · Cognitive Therapy and Research · doi:10.1007/s10608-023-10429-x · Cited by 26
“Moderator analyses were conducted using the template by Lipsey and Wilson (2001). problem; Rothstein, 2008).”
The effects of hot spots policing on violence: A systematic review and meta-analysis
Turchan, B. S., & Braga, A. A. (2024) · Aggression and Violent Behavior · doi:10.1016/j.avb.2024.102011 · Cited by 27
Duggan, M., & Tressoldi, P. E. (2018) · F1000Research · doi:10.12688/f1000research.14330.2 · Cited by 24
Zhu, L.-L., Zhou, B.-J., Hou, J.-L., Wang, J., & Zhou, Y.-N. (2024) · Psychology of Sport And Exercise · doi:10.1016/j.psychsport.2024.102762 · Cited by 13
Suárez, M. I., Stackhouse, E. W., Keese, J., & Thompson, C. G. (2022) · Journal of Family Studies · doi:10.1080/13229400.2022.2060121 · Cited by 21
“Therefore, we concluded that there may be publication bias (Rothstein, 2008).”
BackgroundTahamont, S., Jelveh, Z., Chalfin, A. J., Yan, S., & Hansen, B. (2020) · Journal of Quantitative Criminology · doi:10.1007/s10940-020-09461-x · Cited by 25
“Moreover, since null findings may be less likely to be published (Braga and Apel 2016; Gerber and Malhotra 2008; Rothstein 2008), Type II errors can have dire consequences not only for individual papers but also for the entire research literature.”
BackgroundBlushing and Social Anxiety: A Meta-Analysis
Nikolić, M., Colonnesi, C., de Vente, W., Drummond, P. D., & Bögels, S. M. (2015) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12102 · Cited by 21
“The individuals attend to their own internal impression of themselves which is influ-enced by autonomic symptoms of anxiety (particularly those that might be visible, such as blushing) and assume that this is how others see them (Schultz & Heimberg, 2008).”
BackgroundHow Strong is the Evidence-Base for Crime Reduction Professionals?
Tompson, L., Belur, J., Thornton, A. E., BOWERS, K. J., Johnson, S. D., Sidebottom, A., Tilley, N., & Laycock, G. (2020) · Justice Evaluation Journal · doi:10.1080/24751979.2020.1818275 · Cited by 19
Chung, H., & Ahn, S. H. (2019) · Journal of Korean Academy of Nursing Administration · doi:10.11111/jkana.2019.25.3.157 · Cited by 19
“있는 경우에는 메타분석 연구의 표본이 모집단의 특성을 반영 했다고 볼 수 없고, 분석 결과의 신뢰성과 타당성에 문제가 발생한다[22].”
Can we return good for evil? A meta‐analysis of social exclusion and prosocial behaviour
Lin, S., Su, W., Wang, Y., & Bai, L. (2025) · British Journal of Social Psychology · doi:10.1111/bjso.12879 · Cited by 21
What shapes learners’ trust in AI? A meta-analytic review of its antecedents and consequences
Cui, Y. L., Lin Zeng, M., Ke Du, X., & Miao He, W. (2025) · IEEE Access · doi:10.1109/access.2025.3611367 · Cited by 12
“A commonly used robustness threshold is 5k + 10, where k denotes the number of included studies [77].”
Portilla, I., Rubio‐Aparicio, M., Fuster‐RuizdeApodaca, M. J., Portilla-Tamarit, J., Reus, S., & Portilla, J. (2024) · AIDS and Behavior · doi:10.1007/s10461-024-04298-y · Cited by 19
“Furthermore, Egger’s regression test was also applied [57].”
MethodPositive Youth Development and Prosocial Behavior: A Systematic Review and Multilevel Meta‐Analysis
Meulenbeek, K., Loheide‐Niesmann, L., Spagnuolo, F., & Cima, M. (2025) · Journal of Adolescence · doi:10.1002/jad.70040 · Cited by 19
When School Doesn’t Start at Age 5: Elementary Principal Leadership of Pre-K Programs in Schools
Little, M., Drake, T. A., Cohen‐Vogel, L., & Eagle, J. L. (2022) · The Elementary School Journal · doi:10.1086/720563 · Cited by 18
“This is an important step in any systematic review because a publication bias toward studies with statistically significant effects can inadvertently lead reviewers to conclude that programs have larger impacts than they do (Rothstein, 2008).”
BackgroundRost, M., Clercq, E. D., Arnold, L. S., & Rakic, M. (2023) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2023.102277 · Cited by 17
Fitriyah, I. J., Erlangga, S. Y., Muqoyyidin, A. W., & Aprilia, S. (2025) · Discover Education · doi:10.1007/s44217-025-00756-4 · Cited by 19
Association between childhood maltreatment and empathy: A three-level meta-analytic review
Meng, X., Yu, D., Yijing, C. H. E. N., Zhang, L., & Fu, X. (2023) · Acta Psychologica Sinica · doi:10.3724/sp.j.1041.2023.01285 · Cited by 22
The Identification and Prevention of Publication Bias in the Social Sciences and Economics
Weiß, B., & Wagner, M. A. (2011) · Jahrbücher für Nationalökonomie und Statistik · doi:10.1515/jbnst-2011-5-608 · Cited by 23
A Trim and Fill Examination of the Extent of Publication Bias in Communication Research
Carpenter, C. J. (2012) · Communication Methods and Measures · doi:10.1080/19312458.2011.651347 · Cited by 16
“Borenstein, Hedges, Higgins, and Rothstein (2009) note that there are generally three possible outcomes of this estimate: the estimate of the effect size is not changed at all, the estimate is changed trivially (as is the case in the example in Table 1), or the effect size estimate is affected so greatly it changes th…”
BackgroundMethodInfluentialAdolescence as a peak period of borderline personality features? A meta-analytic approach
Aleva, A., Laceulle, O. M., Denissen, J. J. A., Hessels, C. J., & van Aken, M. A. G. (2022) · European Journal of Personality · doi:10.1177/08902070221134652 · Cited by 16
“Although the validity of Fail-Safe N is questioned (Rothstein, 2008), it is regularly used in the fi eld (e.g. Bschor et al., 2018; Byrne & Coetzer, 2016; McClain et al., 2021).”
Hill, A. P., Kim, H., Ashdown‐Franks, G., & Pratt, V. B. (2025) · Canadian Psychology/Psychologie canadienne · doi:10.1037/cap0000450 · Cited by 15
Surin, S., & Damrongpanit, S. (2024) · European Journal of Educational Research · doi:10.12973/eu-jer.13.3.1391 · Cited by 17
Learning effectiveness of virtual technologies in chemistry using a three-level meta-analysis
Song, Q., Li, M., Zhang, Z., & Zhou, Q. W. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13321-1 · Cited by 15
“Publication bias may reduce the reliability of meta-analysis results (Rothstein, 2008).”
Crosby, E. S., Spitzer, E. G., & Kavookjian, J. (2022) · Behavioral Sleep Medicine · doi:10.1080/15402002.2022.2108033 · Cited by 14
“Given the well-established “file drawer phenomenon” in which studies with significant results are more likely to be published (Rothstein, 2008), two methods were used to assess publication bias.”
MethodKelekatan dan Kesejahteraan Psikologis Anak dan Remaja: Studi Meta-Analisis
Wahyuningsih, H., Novitasari, R., & Kusumaningrum, F. A. (2021) · Psympathic Jurnal ilmiah Psikologi · doi:10.15575/psy.v7i2.6426 · Cited by 20
Zuriaga, A., Kaplan, M. S., Choi, N. G., Hodkinson, A., Storman, D., Brudasca, N. I., Hirani, S. P., & Brini, S. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.05.005 · Cited by 10
Vriezen, R., Sargeant, J. M., Vriezen, E. R., Winder, C. B., & O'Connor, A. M. C. (2019) · Animal Health Research Reviews · doi:10.1017/s146625231900029x · Cited by 12
“Publication bias occurs when the published research on a topic is not representative of all completed studies on that topic for a variety of reasons, including but not limited to: outcome bias (reporting of the results, and the selection of results for reporting, of a primary study and the willingness of journal edito…”
BackgroundMODERATOR EFFECT OF MOBILE LEARNING ON STUDENTS’ ACHIEVEMENT IN PHYSICS: A META-ANALYSIS
Abdullah, W. D., Afikah, A., Apino, E., Supahar, S., & Jumadi, J. (2024) · Journal of Baltic Science Education · doi:10.33225/jbse/24.23.187 · Cited by 11
Partnerships in Policing: How Third Parties Help Police to Reduce Crime and Disorder
MAZEROLLE, L. G., Petersen, K., Sydes, M. L., & Ransley, J. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009472029 · Cited by 13
Welsh, B. C., Braga, A. A., & Hollis-Peel, M. E. (2012) · Journal of Experimental Criminology · doi:10.1007/s11292-012-9153-0 · Cited by 10
“The meta-analysis of effect sizes suggests a highly significant effect in favor of these police crime prevention strategies (p .001), with an overall effect size of .171.4 Publication bias Publication bias presents a strong challenge to any review of evaluation studies (Rothstein 2008).”
BackgroundInfluentialAssociations between Microaggression and Adjustment Outcomes: A Meta-Analytic and Narrative ReviewPreprint
Lui, P. P., & Quezada, L. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/xj8n3 · Cited by 10
Petersen, K., Weisburd, D. L., Hinkle, J. C., Telep, C. W., & Fay, S. (2024) · Aggression and Violent Behavior · doi:10.1016/j.avb.2024.101988 · Cited by 12
Problematic Sexual Behavior Among Children: A Meta-Analysis of Demographic and Clinical Correlates
Allen, B., Wamser‐Nanney, R. A., Pistone, L. F., & Campbell, C. L. (2025) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-025-01300-6 · Cited by 9
“Despite the best of efforts to locate all suitable studies, it remains likely some studies were missed and that unpub-lished data exist, offering threats to the validity of this meta-analysis (Rothstein, 2008).”
CSCL in STEM Education: Preliminary Findings from a Meta-Analysis
Jeong, H., Hmelo‐Silver, C. E., Jo, K., & Shin, M. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.11 · Cited by 10
“Studies reporting statistically significant results are more likely to be published and cited [36].”
BackgroundPark, S., Kim, K., Park, M., Lee, U., Sim, H.-S., Shin, I., & Song, Y. (2017) · Pain Management Nursing · doi:10.1016/j.pmn.2017.10.012 · Cited by 11
“Publication bias was represented using a graphical diagnostic (a funnel plot with Trim and Fill) (Rothstein, 2007), which addressed the question of whether there is evidence for publication bias in a given meta-analysis (Rothstein, 2007).”
InfluentialBeyond sequential presentation: Misconceptions and misrepresentations of sequential lineups
Lindsay, R. C. L., Mansour, J. K., Beaudry, J. L., Leach, A., & Bertrand, M. I. (2009) · Legal and Criminological Psychology · doi:10.1348/135532508x382104 · Cited by 9
“Again, this issue has been discussed at length in the meta-analysis area as the issue of “file-drawer studies” and new techniques for dealing with the issue continue to be developed (e.g., Rothstein, 2008).”
Personality and Multisource Feedback Improvement: A Longitudinal Investigation
Walker, A. G., Smither, J. W., ATWATER, L. E., Dominick, P. G., Brett, J. F., & Reilly, R. B. (2010) · Journal of Behavioral and Applied Management · Cited by 16
The association between childhood maltreatment and internet addiction: A three-level meta-analysis
Xianxin, M., Chen, Y., Delin, Y., Shuling, G., & Fu, X. (2024) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2024.01087 · Cited by 10
The relationship between media multitasking and attention: a three-level meta-analysis
Chen, H., Peng, L., Peng, J., Liu, C., Yin, L., Zhang, Y., Cheng, Y., & Shi, Z. (2025) · Current Psychology · doi:10.1007/s12144-025-07624-2 · Cited by 8
“The presence of this phenomenon may result in the published literature failing to comprehensively represent the full body of research in a given field, thereby reducing the reliability of meta-analytic conclusions (Rothstein, 2008 ).”
The Relationship Between Sexual Minority Stress and Sexual Satisfaction: A Meta-Analytic Review
Croteau, T. A., & Morrison, T. G. (2024) · The Journal of Sex Research · doi:10.1080/00224499.2024.2390092 · Cited by 9
“…publication status is one way that researchers can gauge the presence of publication bias (Dowdy et al., 2021), which occurs when studies that contain statistically significant findings are more likely to be published than studies with statistically non-significant findings (Rothstein, 2008).”
The relationship between narcissism and prosocial behavior: a three-level meta-analysis
Gao, M., Qiu, X. Y., Zhu, H., & Zhang, K. (2024) · Current Psychology · doi:10.1007/s12144-024-07082-2 · Cited by 8
“This phenomenon can result in a published literature that does not accurately represent the entirety of completed research within a given field, thereby potentially compromising the reliability of meta-analytic outcomes (Rothstein, 2008).”
Bolarinwa, O. A., Odimegwu, C. O., Ajayi, K. V., Oni, T. O., Sah, R. K., & Akinyemi, A. I. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-12028-2 · Cited by 9
“The contour-enhanced option col.contour of the funnel.meta() function with significance thresholds at 0.99, 0.95 and 0.0 (equivalent to p -value at 0.01, 0.05, 0.1) was applied to the studies to prevent a wrong attribution of publication bias to the asymmetric pattern of the funnel plot [50, 51].”
InfluentialFurther improving reporting in crime and justice: an addendum to Perry, Weisburd and Hewitt (2010)
Sidebottom, A., & Tilley, N. (2011) · Journal of Experimental Criminology · doi:10.1007/s11292-011-9128-6 · Cited by 9
“It is well known that much meaningful research is never published, producing the so-called publication bias (in relation to criminology see Rothstein 2008).”
BackgroundLens model studies: Revealing teachers’ judgements for teacher education
Kaufmann, E. (2022) · Journal of Education and Teaching · doi:10.1080/02607476.2022.2061336 · Cited by 8
“Notably, the methods of estimating and controlling for publication bias are controversial (Rothstein 2008).”
Positive Outcome Bias in the Influential Otolaryngology Clinical Trial Literature
Yang, H., Tsai, M., Mukdad, L. A., & John, M. A. S. (2023) · Otolaryngology · doi:10.1002/ohn.562 · Cited by 8
Family Resilience and Perceived Stress: A Three‐Level Meta‐Analysis
Li, R., Mi, T., Wu, D., Guo, Z., & Ren, Z. (2025) · Family Process · doi:10.1111/famp.70045 · Cited by 7
Meulen, P., Koning, N., Assink, M., van Hooren, S., Kolthoff, E. W., & Heynen, E. J. E. (2025) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x251344620 · Cited by 9
“…non-significant or negative results are less likely to be published than studies producing significant and positive results (Beelmann & Lösel, 2021; Lipsey & Wilson, 2001; Olsson et al., 2021; Rothstein, 2008), which is referred to as publication bias or the file drawer problem (Rosenthal, 1995).”
Systematische Reviews und Meta-Analysen
Beelmann, A. (2021) · Springer eBooks · doi:10.1007/978-3-658-31434-7_25 · Cited by 8
Park, S., Lee, Y. R., Nelson, G., & Tipton, E. (2023) · Learning Disability Quarterly · doi:10.1177/07319487231185133 · Cited by 7
“The bias of selected publications can threaten the validity of meta-analysis results because the average effect size is likely to be positively biased (Rothstein, 2008).”
Clauss, K. E., Schafer, K. M., Somohano, V. C., O'Neil, M. E., Constans, J. I., Raines, A. M., & Bardeen, J. R. (2023) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-023-10077-1 · Cited by 7
“The file drawer problem, wherein studies with significant results are more likely to be published than those with null results, has drawn increasing attention in recent years (Rothstein, 2008).”
Park, H., & 김상아 (2018) · Locality and Globality Korean Journal of Social Sciences · doi:10.33071/ssricb.42.3.201812.5 · Cited by 7
Lin, Y.-W. D. (2011) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc68001 · Cited by 11
Duggan, M., & Tressoldi, P. E. (2018) · F1000Research · doi:10.12688/f1000research.14330.1 · Cited by 6
“All the traditional tests, like the Fail-Safe, the Trim-and-Fill, the Funnel Plot have been criticized for their limitations (Jin et al ., 2015; Rothstein, 2008).”
MethodPignotti, M. G., & Albright, D. L. (2010) · Clinical Social Work Journal · doi:10.1007/s10615-010-0297-y · Cited by 6
“Rothstein (2008) has criticized the fail-safe N or file drawer method.”
MethodOutcome Reporting Bias in Randomized Experiments on Substance Use Disorders
Wooditch, A., Sloas, L. B., Wu, X., & Key, A. (2018) · Journal of Quantitative Criminology · doi:10.1007/s10940-018-9388-7 · Cited by 8
Offerman, E., Asselman, M. W., Bolling, F., Stams, G. J. J. M., Bosman, R. J., Lindauer, R., & Hoeve, M. (2025) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2025.101871 · Cited by 5
The impact of anxiety on Chinese as a second language achievement: a meta-analytic perspective
Qiao, P., Yuan, Z., & Li, S. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1620275 · Cited by 5
“If N significantly exceeds this benchmark, the results of the meta-analysis can be considered robust and unlikely to be overturned by unpublished studies; the higher the N , the lower the likelihood of bias affecting the results (Rothstein, 2008).”
The Impact of AI-Assisted Learning on the Agency of Foreign Language Learners: A Meta-Analysis
Zai, F., & Zhou, X. (2026) · Behavioral Sciences · doi:10.3390/bs16030379 · Cited by 6
“…refers to the phenomenon where studies with significant results are more readily accepted and published, making it difficult to obtain research that does not yield significant results during the literature collection process, thus affecting the accuracy of meta-analysis results (Rothstein, 2005).”
Teachers’ judgment accuracy: A replication check by psychometric meta-analysis
Kaufmann, E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307594 · Cited by 5
“It is important to note that publication bias is also associated with additional biases, such as language bias and availability bias, as underscored by Rethlefsen et al. [36], Rothstein [35], and Song et al. [37].”
BackgroundMeta-analyses of lightweight versus conventional (heavy weight) mesh in inguinal hernia surgery
Memon, M. A., Khan, S., & Osland, E. (2012) · Hernia · doi:10.1007/s10029-012-0987-1 · Cited by 6
The effectiveness of virtual reality media on student learning outcomes: A meta-analysis
Setiawan, R., Yuliantri, R. D. A., Aman, A., Fitri, E. S. M., Kurniawati, I., & Ramadan, S. (2024) · Jurnal Pembangunan Pendidikan Fondasi dan Aplikasi · doi:10.21831/jppfa.v12i1.67764 · Cited by 5
Ramadan, S., Istiyono, E., Farisiyah, U., Setyaningsih, K., & Mustakim, S. S. (2025) · Harmoni Sosial Jurnal Pendidikan IPS · doi:10.21831/hsjpi.v12i2.91592 · Cited by 5
“According to (Rothstein, 2008), when the value of Safe N Files is greater than 5K + 10 (with K being the number of studies), it can be concluded that there is no publication bias.”
PROTOCOL: Broken Windows Policing to Reduce Crime: A Systematic Review
Braga, A. A., & Welsh, B. C. (2016) · Campbell Systematic Reviews · doi:10.1002/cl2.158 · Cited by 4
“Publication bias presents a strong challenge to any review of evaluation studies (Rothstein, 2008).”
BackgroundBeyond PRISMA 2020 and AMSTAR 2: Further Actions Are Needed to Deal With Problematic Meta‐Analyses
Baveye, P. C. (2025) · Learned Publishing · doi:10.1002/leap.1666 · Cited by 4
“…shortcomings, so that after several decades a consensus still does not appear to exist among specialists about the best assessment approach (e.g., Rothstein 2008; Fragkos, Tsagris, and Frangos 2014; McShane, Böckenholt, and Hansen 2016; Lin and Chu 2018; Shi and Lin 2019; Mathur and VanderWeele…”
InfluentialStallard, P. (2014) · Routledge eBooks · doi:10.4324/9780203098714-6 · Cited by 4
Empirical likelihood meta-analysis with publication bias correction under Copas-like selection model
Li, M., Liu, Y., Li, P., & Qin, J. (2021) · Annals of the Institute of Statistical Mathematics · doi:10.1007/s10463-021-00793-4 · Cited by 4
“…basis of meta-analysis, published scientific results may not be representative of those from all relevant studies, both published and unpublished, due to the so-called publication bias, which is a well-recognized threat to the validation of the results of a meta-analysis (Rothstein 2008; Jin et al.”
What’s in Your File Drawer? The Case of the Missing Null in Criminology and Criminal Justice
Crow, M. S., Smykla, J. O., O’Brien, H., Cerna, T., Johnson, A., Pisaris, S., Suarez, M., & Wilder, J. (2022) · Crime & Delinquency · doi:10.1177/00111287221090959 · Cited by 4
“Little is known, however, about the existence, role, and impact of publication bias in the CCJ literature (Kim, 2022; Rothstein, 2008).”
BackgroundETHICS IN MEDICAL RESEARCH AND PUBLISHING
Ghaderi, H., Gooshki, E. S., & Kruger, E. (2022) · Central Asian Journal of Medical Hypotheses and Ethics · doi:10.47316/cajmhe.2022.3.3.02 · Cited by 5
“It has been reported that studies with positive findings are significantly more represented in literature than studies with negative/no effect [44].”
BackgroundSandberg, B. (2011) · Nordic Studies on Alcohol and Drugs · doi:10.2478/v10199-011-0022-3 · Cited by 4
Andersen, S. N., & Hyatt, J. M. (2018) · European Journal of Criminology · doi:10.1177/1477370818788015 · Cited by 5
Prasetya, F., Fortuna, A., Samala, A. D., Rawas, S., Mystakidis, S., Syahril, S., Waskito, W., Primawati, P., Wulansari, R. E., & Kassymova, G. K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4849673 · Cited by 6
Ignoring criticisms of meta-analyses and systematic reviews is a risky strategy
Baveye, P. C. (2025) · Plant and Soil · doi:10.1007/s11104-025-07646-6 · Cited by 3
“As Rothstein (2008) wrote already 16 years ago, “Over the long run, the best way to deal with publication bias is to prevent it”.”
Retail Metaverse acceptance: A meta-analysis with Hofstede’s cultural moderation
Fares, O. H., Haines, S. J., Lee, S. H. (., Azmy, A., Mohan, M., & Le, S. (2025) · Journal of Retailing · doi:10.1016/j.jretai.2025.12.003 · Cited by 4
Currie, E. P. (2010) · Criminology & Public Policy · doi:10.1111/j.1745-9133.2010.00604.x · Cited by 3
The Relationship Between Intellectual Capital and Competitive Advantage: A Meta-Analysis Study
KILIÇLI, Y. (2022) · Akademik Araştırmalar ve Çalışmalar Dergisi (AKAD) · doi:10.20990/kilisiibfakademik.1149041 · Cited by 3
Factors Affecting Trainees’ Performance in Organic Chemistry in Colleges of Education in Ghana
Amoako, S. K., Oppong, K., Tabi, J. A., & Ossei-Anto, T. A. (2022) · European Journal of Education and Pedagogy · doi:10.24018/ejedu.2022.3.6.403 · Cited by 3
Gallagher, K. M. (2014) · OhioLink ETD Center (Ohio Library and Information Network) · Cited by 3
Are federal security efforts evidence-based?
Lum, C. M., Cave, B., & Nichols, J. (2017) · Security Journal · doi:10.1057/s41284-017-0092-3 · Cited by 4
“…publication bias may be a limitation in our research; especially with situational crime prevention measures, unsuccessful evaluations may not have been published, which may lead to overly optimistic conclusions about particular crime prevention mechanisms (see further discussion by Rothstein 2007).”
BackgroundAdministrative Data Linking and Statistical Power Problems in Randomized Experiments
Tahamont, S., Jelveh, Z., Chalfin, A. J., Yan, S., & Hansen, B. (2019) · National Bureau of Economic Research · doi:10.3386/w25657 · Cited by 4
A meta-analysis of the attitudes of university students in Türkiye toward ageism
Güzel, B. (2024) · Educational Gerontology · doi:10.1080/03601277.2024.2385204 · Cited by 2
“As it is widely recognized, publication bias is an important concern in meta-analyses because biased or incomplete selection of literature reviews can affect the results of meta-analysis (Rothstein, 2008; Rothstein et al., 2005).”
SAĞLIK OKURYAZARLIĞI VE AKILCI İLAÇ KULLANIMI ARASINDAKİ İLİŞKİ: BİR META ANALİZ ÇALIŞMASI
Fidan, C. (2025) · Hacettepe sağlık idaresi dergisi · doi:10.61859/hacettepesid.1580137 · Cited by 4
Yıldırım, Ö., & Demir, S. B. (2026) · Psychological Reports · doi:10.1177/00332941261416397 · Cited by 2
Cognitive functioning in criminal offenders: A multi-level meta-analysis
Hutten, J. C., Ziermans, T. B., Geurts, H. M., Stams, G. J. J. M., & van Horn, J. E. (2026) · Aggression and Violent Behavior · doi:10.1016/j.avb.2026.102159 · Cited by 2
이옥진, & HongBaegEui (2016) · Health and Social Welfare Review · doi:10.15709/hswr.2016.36.1.175 · Cited by 2
Data Aggregation: A Proposed Psychometric IPD Meta-Analysis
Kaufmann, E. (2018) · Open Journal of Statistics · doi:10.4236/ojs.2018.81004 · Cited by 2
“Different types of estimates are introduced to the field and, although publication bias is often required by journal editors to have a manuscript published, we note that there is still a critical discussion on the estimation of publication bias (see [9]).”
BackgroundKILIÇLI, Y. (2023) · Journal of Gastronomy Hospitality and Travel (joghat) · doi:10.33083/joghat.2023.305 · Cited by 3
Kaur, A., & Krishnan, V. V. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae327 · Cited by 2
“In addition, financial, political, or ideological reasons might contribute to selective reporting of the data or studies (16).”
BackgroundÜstün‐Yavuz, M. S., Brooks, R., Fricke, S., & Thomson, J. (2024) · Journal of Research in Reading · doi:10.1111/1467-9817.12471 · Cited by 4
작업치료실, 건., Oh, Y.-J., Cho, B.-H., & Lee, J. (2014) · The Journal of Korean Society of Occupational Therapy · doi:10.14519/jksot.2014.22.3.06 · Cited by 2
Uso problemático del smartphone y resiliencia: un meta-análisis
Hidalgo-Fuentes, S., Correa-Iriarte, S., & Vilar, M. M. (2025) · Anuario de Psicología/The UB Journal of Psychology · doi:10.1344/anpsic2025.55.2.2 · Cited by 1
Zhang, Y., Gao, F., Gagnon, A., Fan, Y., Martinez, G., Gillet, V., Michael, A., Zhu, J., Posner, J., Bellenger, J., et al. (2025) · Environmental Science & Technology · doi:10.1021/acs.est.5c08031 · Cited by 1
Influence of sales technologies on B2B salesforce performance: a meta-analytic investigation
Romoli, A., Oduro, S., Cardinali, S., Mainolfi, G., & Nisco, A. D. (2026) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-01-2025-0024 · Cited by 1
Meta-analysis on the Effect of Information Literacy Instruction
Kim, S., & Hong, S. (2016) · Journal of the Korean BIBLIA Society for library and Information Science · doi:10.14699/kbiblia.2016.27.1.059 · Cited by 1
Building a Collaborative Culture in a Middle School: A Case Study
Olson, A. (2019) · Digital Commons - University of South Florida (University of South Florida) · Cited by 5
Uther, M. (2019) · MDPI eBooks · doi:10.3390/books978-3-03897-661-5 · Cited by 6
Shenderovich, Y. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.26471 · Cited by 1
Metadata
- Authors
- Rothstein, H. R.
- Affiliations
- Baruch College (US)
- Corresponding author
- Rothstein, Hannah R.
- ORCID
- Rothstein, Hannah R.
- Venue
- Journal of Experimental Criminology
- Published
- 2007-12-19
- Volume
- 4
- Issue
- 1
- Pages
- 61-81
- ISSN
- 1572-8315 (ISSN-L)
- 1573-3750
- OpenAlex
- W2010026440
- Cited by
- 216
- References
- 51
- Citations by year
- 2026: 11 · 2025: 23 · 2024: 23 · 2023: 12 · 2022: 20 · 2021: 10 · 2020: 10 · 2019: 13 · 2018: 20 · 2017: 11 · 2016: 9 · 2015: 8 · 2014: 16 · 2013: 4 · 2012: 9
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Journal of Experimental Criminology (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- publication bias, trim and fill, cumulative meta-analysis, effect size estimation, drug courts, recidivism, criminology research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.