The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.
Open accessPublished version of a preprint
Abstract
Selective reporting of results based on their statistical significance threatens the validity of meta-analytic findings. A variety of techniques for detecting selective reporting, publication bias, or small-study effects are available and are routinely used in research syntheses. Most such techniques are univariate, in that they assume that each study contributes a single, independent effect size estimate to the meta-analysis. In practice, however, studies often contribute multiple, statistically dependent effect size estimates, such as for multiple measures of a common outcome construct. Many methods are available for meta-analyzing dependent effect sizes, but methods for investigating selective reporting while also handling effect size dependencies require further investigation. Using Monte Carlo simulations, we evaluate three available univariate tests for small-study effects or selective reporting, including the trim and fill test, Egger's regression test, and a likelihood ratio test from a three-parameter selection model (3PSM), when dependence is ignored or handled using ad hoc techniques. We also examine two variants of Egger's regression test that incorporate robust variance estimation (RVE) or multilevel meta-analysis (MLMA) to handle dependence. Simulation results demonstrate that ignoring dependence inflates Type I error rates for all univariate tests. Variants of Egger's regression maintain Type I error rates when dependent effect sizes are sampled or handled using RVE or MLMA. The 3PSM likelihood ratio test does not fully control Type I error rates. With the exception of the 3PSM, all methods have limited power to detect selection bias except under strong selection for statistically significant effects. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Publication bias was assessed both visually, using funnel plots, and statistically, using a multilevel Egger-type regression test, in which the standard error of the effect size was included as a predictor in a mixed-effects model (39).”
From Carbohydrate ingestion during prolonged exercise and net skeletal muscle glycogen utilization: A meta-analysis. (2026) · cites this paper as [39]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
Items 101–200 of 490
Chen, Z., Zeng, J., Liao, J., & Guo, C. (2025) · Journal of Youth and Adolescence · doi:10.1007/s10964-025-02151-6 · Cited by 35
Breastfeeding Support Provided by Lactation Consultants
D’Hollander, C. J., McCREDIE, V. A., Uleryk, E. M., Kucab, M., Le, R. M., Hayosh, O., Keown‐Stoneman, C. D. G., Birken, C. S., & Maguire, J. L. (2025) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2024.6810 · Cited by 39
The Cognitive Profile of Math Difficulties: A Meta-Analysis Based on Clinical Criteria
Haberstroh, S., & Schulte‐Körne, G. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.842391 · Cited by 44
Murphy, L. D., Cobb, H. R., Rudolph, C. W., & Zacher, H. (2022) · Organizational Psychology Review · doi:10.1177/20413866221131404 · Cited by 42
“Given concerns regarding omitted studies, we conducted a formal test of funnel plot asymmetry following the suggestions of Rodgers and Pustejovsky (2021). Although multiple means exist for assessing and correcting for the presence of omitted studies, not all are well-geared for the multilevel/multivariate meta-analyti…”
MethodInfluentialPoppe, A., Ritter, F. D. E., Bais, L., Pustejovsky, J. E., van Tol, M., Ćurĉić‐Blake, B., Pijnenborg, G. H. M., & van der Meer, L. (2023) · Psychological Bulletin · doi:10.1037/bul0000406 · Cited by 34
Self-citationViesel-Nordmeyer, N., Reuber, J., Kuhn, J., Moll, K., Holling, H., & Dobel, C. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09735-3 · Cited by 36
He, G., Chen, S. H. A., Lin, H., & Su, A. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09922-w · Cited by 43
“According to the suggestions and improvements by Rodgers and Pustejovsky (2021), this study adopts the robust variance estimation (RVA) method to improve the Egger test, known as the Egger sandwich test.”
Kang, H., Miksche, M. S., & Ellingsen, D. (2022) · Pain · doi:10.1097/j.pain.0000000000002753 · Cited by 36
Siegel, M., Legler, M., Neziraj, F., Goldberg, A. E., & Zemp, M. (2022) · Children · doi:10.3390/children9091364 · Cited by 35
Roest, J. J., de Poll, M. J. W.-V., van der Helm, P. G., Stams, G. J. J. M., & Hoeve, M. (2022) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-022-00986-2 · Cited by 35
Hsu, T., Adamowicz, J. L., & Thomas, E. B. K. (2023) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2023.10.006 · Cited by 31
A Meta-Analysis of Green Supply Chain Management Practices and Firm Performance
Holling, H., & Backhaus, L. (2023) · Sustainability · doi:10.3390/su15064730 · Cited by 28
A systematic and meta-analytic review of the impact of sleep restriction on memory formation
Crowley, R., Alderman, E., Javadi, A., & Tamminen, J. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105929 · Cited by 51
“Note however that the 3-PSM does not account for dependence between effect sizes which can inflate the Type I error rate (Rodgers and Pustejovsky, 2020).”
BackgroundAre metacognition interventions in young children effective? Evidence from a series of meta-analyses
Eberhart, J., Ingendahl, F., & Bryce, D. (2024) · Metacognition and Learning · doi:10.1007/s11409-024-09405-x · Cited by 60
“There are several possible reasons that effect sizes and their precision might be correlated, which would result in a significant Egger’s test, such as high between-study heterogeneity, low training fidelity, or even chance (Harrer et al., 2021; Page et al., 2021; Rodgers & Pustejovsky., 2021).”
A meta‐analytic review of experimental tests of the interrogation technique of Hanns Joachim Scharff
Luke, T. J. (2020) · Applied Cognitive Psychology · doi:10.1002/acp.3771 · Cited by 26
The validity of situation awareness for performance: a meta-analysis
Bakdash, J. Z., Marusich, L. R., Cox, K. R., Geuss, M. N., Zaroukian, E. G., & Morris, K. (2021) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2021.1921310 · Cited by 29
Lo, A., Lo, C. C. W., & Oliver‐Williams, C. (2023) · European Heart Journal Open · doi:10.1093/ehjopen/oead061 · Cited by 32
“Funnel plots were produced, and Egger’s test was conducted using a robust variance estimation variant for correlated and hierarchal meta-analysis models.(19)”
MethodFong, C. J., Patall, E. A., Snyder, K. E., Hoff, M. A., Jones, S. J., & Zuniga-Ortega, R. E. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100566 · Cited by 43
The Role of Parent–Child Attachment, Hostile Attribution Bias in Aggression: A Meta-Analytic Review
Xu, X., Wu, Y., Xu, Y., Ding, M., Zhou, S., & Long, S. (2023) · Trauma Violence & Abuse · doi:10.1177/15248380231210920 · Cited by 37
“Thus, a Meta-Analysis of Multilevel data using the Maximum Likelihood (MAML Egger’s regression) method, a variant of Egger’s regression test considering the clustered data structure, was used to test the publication bias (Rodgers & Pustejovsky, 2021).”
MethodRöhl, S., Bijlsma, H. J. E., & Schwichow, M. (2025) · Educational Assessment Evaluation and Accountability · doi:10.1007/s11092-024-09450-9 · Cited by 53
“We deal with the dependency between effect sizes in the Eggert test by using the multi-level meta-regression models instead of simple meta-regressions (Fernández-Castilla et al., 2021; Rodgers & Pustejovsky, 2021).”
How Does Aging Affect Social Attention? A Test of Competing Theories Using Multilevel Meta-Analysis
McKay, K. T., Talipski, L. A., Grainger, S. A., Alister, M., & Henry, J. D. (2022) · The Journals of Gerontology Series B · doi:10.1093/geronb/gbac052 · Cited by 26
Jung, M., Ryu, S., Kang, M., Javadi, A., & Loprinzi, P. D. (2021) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218211048807 · Cited by 30
“This method augments the observed data, which in turn makes the funnel plot more symmetric and adjusts the observed average ES (Rodgers & Pustejovsky, 2021).”
MethodFamily Function and Post-Traumatic Stress Disorder in Children and Adolescents: A Meta-Analysis
Ye, Y., Li, Y., Jin, S., Huang, J., Ma, R., Wang, X., & Zhou, X. (2022) · Trauma Violence & Abuse · doi:10.1177/15248380221126182 · Cited by 29
Ma, T., Tellegen, C. L., Hodges, J., & Sanders, M. R. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09937-3 · Cited by 66
“Given that conducting publication bias tests with multilevel meta-analysis is still an emerging area of research with no software adaptation available for the fail-safe N analysis (Rodgers & Pustejovsky, 2020), the issue of data dependency was ignored in the fail-safe N analysis.”
Ortiz, J. A., Nolasco, J. M., Huang, Y. T., & Chow, J. C. (2024) · Journal of Speech Language and Hearing Research · doi:10.1044/2024_jslhr-24-00212 · Cited by 31
“In addition, we conducted a variant of Egger ’ s test that used a multilevel model with robust variance estimation to account for dependent effect sizes (Rodgers & Pustejovsky, 2021).”
Zhang, L., Carter, R. A., Liu, Y., & Peng, P. (2024) · Review of Educational Research · doi:10.3102/00346543241293424 · Cited by 52
“Further, the results of the regression test showed that the standard errors of correlations significantly predicted effect sizes among studies, β = 2.837, df = 237, p .001, indicating the presence of publication bias (Rodgers & Pustejovsky, 2021).”
InfluentialDoes parochial cooperation exist in childhood and adolescence? A meta‐analysis
Lazić, A., Purić, D., & Krstić, K. (2021) · International Journal of Psychology · doi:10.1002/ijop.12791 · Cited by 23
Fishstrom, S., Wang, H.-H., Bhat, B. H., Daniel, J., Dille, J. T., Capin, P., & Vaughn, S. R. (2022) · Journal of School Psychology · doi:10.1016/j.jsp.2022.03.011 · Cited by 45
A meta-analysis on learning from 360° video
Schroeder, N. L., Siegle, R. F., & Craig, S. D. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104901 · Cited by 25
Assessing robustness to worst case publication bias using a simple subset meta-analysis
Mathur, M. B. (2024) · BMJ · doi:10.1136/bmj-2023-076851 · Cited by 25
“10 Also, these methods can require a large number of studies to perform well—this is especially the case for selection models that accommodate more complex forms of publication bias.”
BackgroundLee, A. H., Kitagawa, Y., Mirhashem, R., Rodriguez, M., Hilerio, R., & Bernard, K. (2024) · Development and Psychopathology · doi:10.1017/s0954579424000737 · Cited by 26
Digital Media Use and Child Health and Development
Teague, S. J., Somoray, K., Shatte, A. B. R., Miller, D., Moss, K., Crawford, A., Wildman, H., Kayal, D., & Hutchinson, D. C. S. (2026) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2026.0085 · Cited by 47
Kamel, A. M., Sabry, N. A., & Farid, S. F. (2022) · BMC Cardiovascular Disorders · doi:10.1186/s12872-022-02845-w · Cited by 26
“We used the method suggested by Pustejovsky and Rodgers when testing for the funnel plot asymmetry as the effect is dependent on the standard error [30].”
MethodA meta‐analysis on the effects of just‐below versus round prices
Troll, E. S., Frankenbach, J., Friese, M., & Loschelder, D. D. (2023) · Journal of Consumer Psychology · doi:10.1002/jcpy.1353 · Cited by 26
Thurston, C., Murray, A. L., Franchino-Olsen, H., Silima, M., Hemady, C. L., & Meinck, F. (2026) · Trauma, Violence, & Abuse · doi:10.1177/15248380251358223 · Cited by 25
“…for all five models (Table 7), but we recommend notable caution when seeking meaningful interpretation of the three-level models where noninde-pendence of effect sizes has been ignored in line with extant recommendations (Peters et al., 2007; Rodgers & Pustejovsky, 2021; Terrin et al., 2003).”
Yang, Y., Lagisz, M., Williams, C., Noble, D. W. A., Pan, J., & Nakagawa, S. (2024) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14377 · Cited by 35
“Extensive simula - tions have shown that it maintains close- to-nominal Type I error rates in the presence of dependent effect sizes (Fernández-Castilla et al., 2021 ; Rodgers & Pustejovsky, 2021 ).”
Rahapsari, S., & Levita, L. (2024) · Trauma Violence & Abuse · doi:10.1177/15248380241286812 · Cited by 31
“This method determines the effect sizes’ nonindependence (Rodgers & Pustejovsky, 2021).”
Shen, J., & Wu, H. (2024) · Educational Administration Quarterly · doi:10.1177/0013161x241286527 · Cited by 49
“& Fill test also does not directly assess publication bias (Borenstein et al., 2021; Rodgers & Pustejovsky, 2021).”
Girdwood, M., Crossley, K. M., Rio, E., Patterson, B., Haberfield, M. J., Couch, J. L., Mentiplay, B. F., Hedger, M., & Culvenor, A. G. (2024) · Sports Medicine · doi:10.1007/s40279-024-02121-1 · Cited by 21
Tian, X., Huebner, E. S., & Tian, L. (2025) · Journal of Youth and Adolescence · doi:10.1007/s10964-025-02176-x · Cited by 22
“Then, an adapted Egger ’ s test was performed, which explained the dependence between effect sizes (Rodgers & Pustejovsky, 2021).”
Long-Term Effects of Early Communication Interventions: A Systematic Review and Meta-Analysis
Pak, N. S., Chow, J. C., Dillehay, K. M., & Kaiser, A. P. (2023) · Journal of Speech Language and Hearing Research · doi:10.1044/2023_jslhr-22-00711 · Cited by 24
Manipulating belief in free will and its downstream consequences: A meta-analysisPreprint
Genschow, O., Cracco, E., Schneider, J., Protzko, J., Wisniewski, D., Braß, M., & Schooler, J. W. (2021) · doi:10.31234/osf.io/quwgr · Cited by 21
Jennissen, S., Huber, J., Ditzen, B., & Dinger, U. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2423662 · Cited by 28
Growth mindset in Chinese culture: A meta-analysis
Wang, Y., & Sun, X. (2025) · Social Psychology of Education · doi:10.1007/s11218-024-09955-7 · Cited by 22
Murdoch, E. M., Lines, R. L. J., Crane, M. F., Ntoumanis, N., Brade, C. J., Quested, E., Ayers, J., & Gucciardi, D. F. (2021) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1977974 · Cited by 19
“…the multicomponent publication bias tests because simulations show that it works best with large numbers of e ff ects and sample sizes (Fernández-Castilla et al., 2021), something which was uncharacteristic of our dataset, and has limited power to detect selection bias (Rodgers & Pustejovsky, 2020).”
BackgroundSuicidal thoughts, behaviors, and event‐related potentials: A systematic review and meta‐analysis
Gallyer, A. J., Dougherty, S. P., Burani, K., Albanese, B. J., Joiner, T. E., & Hajcak, G. (2021) · Psychophysiology · doi:10.1111/psyp.13939 · Cited by 20
Nie, X., Tian, Y., Liu, M., Wu, D., & Guo, Y. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13735-x · Cited by 20
“The results of Egger’s regression test were reported for both the two-level and three-level models, with the three-level model including 2√N as a predictor (Cui et al., 2024 ; Rodgers & Pustejovsky, 2021; Zhang et al., 2024).”
Xie, J., Cho, K. W., Wei, T., Xu, J., & Fan, M. (2025) · Learning and Motivation · doi:10.1016/j.lmot.2025.102109 · Cited by 22
Reuveni, N., Carlson, C. A., Schwartz, S., Meter, D. J., Barrett, T. S., & Freeman, S. M. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-01967-1 · Cited by 20
“Currently, there is no single method to evaluate publication bias while addressing sample dependency [61].”
MethodResultInfluentialThe impact of open educational resources on student achievement: A meta-analysis
Cho, K. W., & Permzadian, V. (2024) · International Journal of Educational Research · doi:10.1016/j.ijer.2024.102365 · Cited by 29
Lund, C., Orkin, K., Witte, M., Walker, J., Davies, T., Haushofer, J., Murray, S. M., Bass, J., Murray, L. K., Tol, W. A., et al. (2024) · National Bureau of Economic Research · doi:10.3386/w32423 · Cited by 19
The impact of formative assessment on K-12 learning: a meta-analysis
Yao, Y., Amos, M., Snider, K. A., & Brown, T. (2024) · Educational Research and Evaluation · doi:10.1080/13803611.2024.2363831 · Cited by 39
Dahl‐Leonard, K., Hall, C., Cho, E., Capin, P., Roberts, G. J., Kehoe, K. F., Haring, C. D., Peacott, D., & Demchak, A. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-09985-3 · Cited by 40
Bell, C. N., George, R. S., Honan, C. A., Bell, L. J., Jolly, A. T. W., & Matthews, A. J. (2025) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2025.102632 · Cited by 25
Reading Interventions for Students in Grades 3–12 With Significant Word Reading Difficulties
Boucher, A. N., Bhat, B. H., Clemens, N. H., Vaughn, S. R., & O’Donnell, K. (2023) · Journal of Learning Disabilities · doi:10.1177/00222194231207556 · Cited by 25
“To examine potential publication bias, we used Egger’s regression with CHE working model to account for dependent effect sizes in our sample (Rodgers & Pustejovsky, 2021), step-function selection model tests (Hedges & Vevea, 2005), and contour-enhanced funnel plots (Peters et al., 2008).”
MethodEllis, L. B., Molina, K., Robbins, C., Freisthler, M. S., Sgargi, D., Mandrioli, D., & Perry, M. J. (2023) · Environmental Health Perspectives · doi:10.1289/ehp12678 · Cited by 20
“93 ffiffiffi p A sample size-based precision estimate (1 = n ) was assessed as the predictor rather than the standard error to avoid known distortions in funnel plot assessments of standardized mean di ff erences and standard errors.”
MethodZhang, C., Zhou, Y., Wijaya, T. T., Chen, J., & Ning, Y. (2024) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2024.101587 · Cited by 18
Creativity in the West and the East: A Meta-Analysis of Cross-Cultural Differences
Barth, P., & Stadtmann, G. (2024) · Creativity Research Journal · doi:10.1080/10400419.2024.2369442 · Cited by 25
“In this regard, we applied an additional variant of Egger’s regression test (Egger et al., 1997) that accounts for the dependency between effect sizes in this meta-analysis, namely the “Egger sandwich test” (Rodgers & Pustejovsky, 2021).”
InfluentialLahiri, S., Bingenheimer, J. B., Evans, W. D., Wang, Y., Cislaghi, B., Dubey, P., & Snowden, B. (2024) · Addiction · doi:10.1111/add.16685 · Cited by 19
Relationships Between Experiences of Autonomy and Well(Ill)-Being for K-12 Youth: A Meta-Analysis
Vite, A., Patall, E. A., & Chen, M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09967-x · Cited by 23
“Finally, we examined the possibility of publication bias and funnel plot asymmetry by conducting an Egger’s regression test that accounted for dependent effect sizes and by examining publication status as a moderator in meta-regression models (Egger et al., 1997; Rodgers & Pustejovsky, 2021).”
Body image and risk of exercise addiction in adults: A systematic review and meta-analysis
Guo, S., Kamionka, A., Xue, Q., Izydorczyk, B., Lipowska, M., & Lipowski, M. (2025) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00085 · Cited by 22
“This approach is particularly effective when individual studies contribute only a limited number of effect sizes (Rodgers & Pustejovsky, 2020).”
Myers, J. A., Hughes, E. M., Witzel, B. S., Anderson, R. D., & Owens, J. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2080131 · Cited by 30
“Further, they used a traditional approach for addressing dependent ESs (i.e., selection of a single outcome), which may introduce bias (Rodgers & Pustejovsky, 2021).”
BackgroundMethodInfluentialEndogenous oxytocin and human social interactions: A systematic review and meta-analysis.
Burenkova, O. V., Dolgorukova, T. A., An, I., Kustova, T. A., Podturkin, A. A., Shurdova, E. M., Talantseva, O. I., Zhukova, M., & Grigorenko, E. L. (2023) · Psychological Bulletin · doi:10.1037/bul0000402 · Cited by 17
Zecchinato, F., Ahmadzadeh, Y. I., Kreppner, J., & Lawrence, P. J. (2024) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2024.04.005 · Cited by 19
Genetic nurture effects on education: a systematic review and meta-analysisPreprint
Wang, B.-Y., Baldwin, J., Schoeler, T., Cheesman, R., Barkhuizen, W., Dudbridge, F., Bann, D., Morris, T. T., & Pingault, J.-B. (2021) · bioRxiv · doi:10.1101/2021.01.15.426782 · Cited by 10
Coles, N. A., Wyatt, M., & Frank, M. C. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/uw85a · Cited by 16
Pappa, E., Baah, F., Lynch, J., Shiel, L., Blackman, G., Raihani, N., & Bell, V. (2025) · Schizophrenia Bulletin · doi:10.1093/schbul/sbae225 · Cited by 19
“Publication bias tests were selected to be specific for multilevel meta-analysis and included Egger’s regression test 27 and a multilevel funnel plot test.”
Bourke, M., Wang, H., Fortnum, K., Thomas, G., O’Flaherty, M., Mulcahy, S. K., Gomersall, S. R., Alsop, T., Trost, S. G., Bruijns, B. A., et al. (2025) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/sms.70120 · Cited by 22
“Funnel plot asymmetry was assessed using the “Egger sandwich test” [54].”
On the role of linguistic features for comprehension and learning from STEM texts. A meta-analysis
Strohmaier, A., Ehmke, T., Härtig, H., & Leiß, D. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100533 · Cited by 32
Feng, S., Zhang, L., Wang, S., & Cai, Z. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104864 · Cited by 19
Health Care Utilization Following Interventions to Improve Social Well-Being
HaGani, N., Surkalim, D. L., Clare, P. J., Merom, D., Smith, B. J., & Ding, D. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.21019 · Cited by 17
Meta-analysis of Cognitive Function Following Non-severe SARS-CoV-2 Infection
Austin, T. A., Thomas, M. L., Lü, M., Hodges, C. B., Darowski, E. S., Bergmans, R. S., Parr, S., Pickell, D., Catazaro, M., Lantrip, C., et al. (2024) · Neuropsychology Review · doi:10.1007/s11065-024-09642-6 · Cited by 17
“This technique has been suggested to be an extension of Egger’s regression test when meta-ana-lyzing dependent effect sizes (Rodgers & Pustejovsky, 2021).”
Burghart, M., Schmidt, S., & Mier, D. U. (2024) · Psychological Medicine · doi:10.1017/s0033291724001259 · Cited by 31
Pulliam, G., Feldman, J. I., & Woynaroski, T. G. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105130 · Cited by 15
Decline effects are rare in ecology: Comment
Yang, Y., Lagisz, M., & Nakagawa, S. (2023) · Ecology · doi:10.1002/ecy.4069 · Cited by 13
Li, W., Kuang, Z., Leng, X., Mayer, R. E., & Wang, F. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09910-0 · Cited by 14
“Because publication bias 67 Page 24 of 48 tests for conventional meta-analysis do not perform well in three-level meta-analyses (Rodgers & Pustejovsky, 2021), the Egger sandwich test was used to improve the Egger test.”
Müller, V. I., Cieslik, E. C., Ficco, L., Tyralla, S., Sepehry, A. A., Aziz-Safaie, T., Feng, C., Eickhoff, S. B., & Langner, R. (2024) · Neuropsychology Review · doi:10.1007/s11065-024-09647-1 · Cited by 17
“Sampling bias was examined by calculating an RVE meta-regression model with the standard error of Hedges g effect size (SEg) as a moderator (Rodgers & Pustejovsky, 2020), which is similar to Egger’s regression but adjusted for dependent effect sizes.”
Fang, X., Ng, D. T. K., & Yuen, M. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13052-9 · Cited by 23
“The criteria of the effect size were adopted in previous studies (e.g., Rodgers & Pustejovsky, 2021): 0.2–0.5 (small effect), 0.5–0.8 (moderate effect), and > 0.8 (large effect).”
InfluentialOakman, J., Macdonald, W. A., McCredie, K., & Clune, S. (2025) · Applied Ergonomics · doi:10.1016/j.apergo.2025.104481 · Cited by 14
Zhu, H., Jia, F., Zhang, Y., & Wang, R. (2025) · Behavioral Sciences · doi:10.3390/bs15091143 · Cited by 14
“When effect sizes are not independent of one another, Egger–MLMA regression is more effective in controlling for Type I error compared to the traditional Egger test (Rodgers & Pustejovsky, 2021).”
Association between childhood maltreatment and empathy: A three-level meta-analytic review
Meng, X., Yu, D., Yijing, C. H. E. N., Zhang, L., & Fu, X. (2023) · Acta Psychologica Sinica · doi:10.3724/sp.j.1041.2023.01285 · Cited by 16
A multi-level meta-analytic review of attachment and fears of compassion
Varley, D., Sherwell, C. S., Fu, M., & Kirby, J. N. (2023) · Personality and Individual Differences · doi:10.1016/j.paid.2023.112477 · Cited by 18
The effect of mere presence of smartphone on cognitive functions: A four-level meta-analysis.
Hartanto, A., Lua, V. Y. Q., Kasturiratna, K. T. A. S., Koh, P. S., Tng, G. Y. Q., Kaur, M., Quek, F. Y. X., Chia, J. L., & Majeed, N. M. (2024) · Technology Mind and Behavior · doi:10.1037/tmb0000123 · Cited by 20
LaSota, R. R., Polanin, J. R., Perna, L. W., Rodgers, M. A., & Austin, M. J. (2024) · Review of Educational Research · doi:10.3102/00346543241239955 · Cited by 15
“To address this concern, we conducted the Egger Sandwich test (Rodgers & Pustejovsky, 2020), a variant of the classic Egger’s regression that accounts for dependent effect sizes using the robust variance estimation technique.”
Self-citationHolgado, D., Sanabria, D., Vadillo, M. A., & Román-Caballero, R. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105821 · Cited by 17
“The logic of PET-PEESE can be extended to multilevel models (Fern ´ andez-Castilla et al., 2021; Friese et al., 2017; Rodgers and Pustejovsky, 2020).”
MethodYan, S., Liu, Z., Peng, P., & Yan, N. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09955-1 · Cited by 19
“Furthermore, the three-level funnel plot, Egger’s regression test, and the trim-and-fill method are commonly used to identify and address publication bias in three-level meta-analysis (Rodgers & Pustejovsky, 2021).”
Cho, M., & Cho, Y. H. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168524 · Cited by 18
Cipriano, C., Strambler, M. J., Naples, L. H., Ha, C., Kirk, M., Wood, M., Sehgal, K., Zieher, A. K., Eveleigh, A., McCarthy, M. F., et al. (2023) · doi:10.31219/osf.io/mk35u · Cited by 15
Gender and Accuracy in Decoding Affect Cues: A Meta-Analysis
Hall, J. A., Gunnery, S. D., & Schlegel, K. (2025) · Journal of Intelligence · doi:10.3390/jintelligence13030038 · Cited by 17
“Given that selection models cannot currently account for nested data structures, this analysis was conducted on a dataset where effect sizes were aggregated within the 1011 studies, following the recommendations of Rodgers and Pustejovsky (2021).”
Do Pedagogical Agents Enhance Student Motivation? Unraveling the Evidence Through Meta-Analysis
Gladstone, J. R., Schroeder, N. L., Heidig, S., Zhang, S., Palaguachi, C., & Pitera, M. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10050-2 · Cited by 17
“Conventional approaches to quantifying publication bias do not function well in 3LMA models (Rodgers & Pustejovsky, 2021).”
P ‐hacking in meta‐analyses: A formalization and new meta‐analytic methods
Mathur, M. B. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1701 · Cited by 25
“There has been substantial recent progress toward considering SAS in the context of multiple estimates per study 9,39,65 ; can these conceptual and technical advances be parlayed into methods that additionally accommodate SWS?”
BackgroundCorporal punishment and violent behavior spectrum: a meta-analytic review
Pan, Q., Chen, S.-R., & Qu, Y. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1323784 · Cited by 13
“Therefore, we employed the latest MAML Egger’s test, which uses the standard error of the effect size as a function (Rodgers and Pustejovsky, 2020).”
MethodGenetic and environment influences on childhood victimization: a systematic review and meta-analysis
Dahoun, T., Peel, A. J., Baldwin, J. R., Coleman, O., Lewis, S. J., Wertz, J., Rijsdijk, F. V., & Danese, A. (2024) · Molecular Psychiatry · doi:10.1038/s41380-024-02868-z · Cited by 12
“We identi fi ed k = 21 studies [”
Spurious Precision in Meta-Analysis of Observational ResearchPreprint
Iršová, Z., Bom, P. R. D., Havránek, T., & Rachinger, H. J. (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/3qp2w · Cited by 11
Do socializers’ mindset beliefs matter for student mindset and achievement? A meta-analysis
Fong, C. J., Muenks, K. M., Fathi, Z., Adelugba, S. F., O'Grady, M., Lin, S., & Goldstein, M. G. (2025) · Learning and Individual Differences · doi:10.1016/j.lindif.2025.102709 · Cited by 16
No meta-analytical effect of economic inequality on well-being or mental health
Sommet, N., Fillon, A. A., Rudmann, O., Cunha, A. R. S., & Ehsan, A. (2025) · Nature · doi:10.1038/s41586-025-09797-z · Cited by 19
“To accommodate the multivariate data structure, we performed an Egger–MLMA regression 17 .”
Pak, N. S., Bailey, K. M., Ledford, J. R., & Kaiser, A. P. (2023) · American Journal of Speech-Language Pathology · doi:10.1044/2022_ajslp-22-00220 · Cited by 12
Mathur, M. B. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1667 · Cited by 13
“42,46,47 We adopt this model as well and explore its performance under model misspecification in an extensive simulation study (Section 5).”
MethodStephen, D. M., & Stahlschmidt, S. (2024) · Scientometrics · doi:10.1007/s11192-024-05006-2 · Cited by 12
“…we did not test for publication bias because the standard tests, such as Egger’s regression and Trim and Fill, have been found to have limited ability to detect publication bias in such cases and no suitable alternative is yet available (Assink & Wibbelink, 2016; Rodgers & Pustejovsky, 2021).”
Smokeless tobacco and periodontitis: A systematic review with meta‐analysis
Quadri, M. F. A., Kamel, A. M., Nayeem, M., John, T., Mohanan, A. T., Tartaglia, G. M., & Tadakamadla, S. K. (2024) · Journal of Periodontal Research · doi:10.1111/jre.13274 · Cited by 19
Metadata
- Authors
- Rodgers, M. A., & Pustejovsky, J. E.
- Affiliations
- The University of Texas at Austin (US)
- Venue
- Psychological Methods
- Published
- 2021-04
- Volume
- 26
- Issue
- 2
- Pages
- 141-160
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W3043790905
- Cited by
- 492
- References
- 0
- Citations by year
- 2026: 111 · 2025: 111 · 2024: 95 · 2023: 72 · 2022: 61 · 2021: 34
- FWCI
- 26.68
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- MetArXiv (OSF Preprints) (submittedVersion)
- MetArXiv (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- selective reporting, publication bias, small-study effects, dependent effect sizes, robust variance estimation, multilevel meta-analysis, type I error rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
