The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.
Open accessPublished version of a preprint
Abstract
Selective reporting of results based on their statistical significance threatens the validity of meta-analytic findings. A variety of techniques for detecting selective reporting, publication bias, or small-study effects are available and are routinely used in research syntheses. Most such techniques are univariate, in that they assume that each study contributes a single, independent effect size estimate to the meta-analysis. In practice, however, studies often contribute multiple, statistically dependent effect size estimates, such as for multiple measures of a common outcome construct. Many methods are available for meta-analyzing dependent effect sizes, but methods for investigating selective reporting while also handling effect size dependencies require further investigation. Using Monte Carlo simulations, we evaluate three available univariate tests for small-study effects or selective reporting, including the trim and fill test, Egger's regression test, and a likelihood ratio test from a three-parameter selection model (3PSM), when dependence is ignored or handled using ad hoc techniques. We also examine two variants of Egger's regression test that incorporate robust variance estimation (RVE) or multilevel meta-analysis (MLMA) to handle dependence. Simulation results demonstrate that ignoring dependence inflates Type I error rates for all univariate tests. Variants of Egger's regression maintain Type I error rates when dependent effect sizes are sampled or handled using RVE or MLMA. The 3PSM likelihood ratio test does not fully control Type I error rates. With the exception of the 3PSM, all methods have limited power to detect selection bias except under strong selection for statistically significant effects. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Publication bias was assessed both visually, using funnel plots, and statistically, using a multilevel Egger-type regression test, in which the standard error of the effect size was included as a predictor in a mixed-effects model (39).”
From Carbohydrate ingestion during prolonged exercise and net skeletal muscle glycogen utilization: A meta-analysis. (2026) · cites this paper as [39]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
Parry, D. A., Davidson, B. I., Sewall, C. J. R., Fisher, J. T., Mieczkowski, H., & Quintana, D. S. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01117-5 · Cited by 793
“Egger’s regression test (incorporating RVE per the Egger–sandwich test) 82 , indicated no evidence of small study bias in this sample ( β = 0.55, P = 0.136); see Fig.”
MethodInfluentialCipriano, C., Strambler, M. J., Naples, L. H., Ha, C., Kirk, M., Wood, M., Sehgal, K., Zieher, A. K., Eveleigh, A., McCarthy, M., et al. (2023) · Child Development · doi:10.1111/cdev.13968 · Cited by 636
“These practices of data treatment allowed us to build upon a comprehensive catalog of data for review, and reduced researcher bias while maintaining methodological transparency (Pigott & Polanin, 2020; Rodgers & Pustejovsky, 2021).”
Methods for testing publication bias in ecological and evolutionary meta‐analyses
Nakagawa, S., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Sánchez‐Tójar, A., Yang, Y., & O'Dea, R. E. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13724 · Cited by 399
Visual attentional orienting by eye gaze: A meta-analytic review of the gaze-cueing effect.
McKay, K. T., Grainger, S. A., Coundouris, S. P., Skorich, D. P., Phillips, L. H., & Henry, J. D. (2021) · Psychological Bulletin · doi:10.1037/bul0000353 · Cited by 453
Baldwin, J. R., Wang, B., Karwatowska, L., Schoeler, T., Tsaligopoulou, A., Munafò, M. R., & Pingault, J.-B. (2023) · American Journal of Psychiatry · doi:10.1176/appi.ajp.20220174 · Cited by 313
Palmer, C. A., Bower, J. L., Cho, K. W., Clementi, M. A., Lau, S., Oosterhoff, B. J., & Alfano, C. A. (2023) · Psychological Bulletin · doi:10.1037/bul0000410 · Cited by 251
Johnson, D., Policelli, J., Li, M., Dharamsi, A., Hu, Q., Sheridan, M. A., McLaughlin, K. A., & Wade, M. (2021) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2021.2511 · Cited by 209
Polanin, J. R., Espelage, D. L., Grotpeter, J. K., Ingram, K., Michaelson, L., Spinney, E., Valido, A., Sheikh, A. E., Torgal, C., & Robinson, L. (2022) · Prevention Science · doi:10.1007/s11121-021-01259-y · Cited by 171
Hall, C., Dahl‐Leonard, K., Cho, E., Solari, E. J., Capin, P., Conner, C. L., Henry, A. R., Cook, L., Hayes, L. L., Vargas, I., et al. (2022) · Reading Research Quarterly · doi:10.1002/rrq.477 · Cited by 258
Examining the relations between spatial skills and mathematical performance: A meta-analysis
Atit, K., Power, J., Pigott, T., Lee, J., Geer, E. A., Uttal, D. H., Ganley, C. M., & Sorby, S. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-021-02012-w · Cited by 202
“To examine possible publication bias in the data, we implemented a modified version of Egger ’ s test of funnel plot asymmetry (Egger et al., 1997), or Egger ’ s sandwich (Rodgers & Pustejovsky, 2020), that can be used when dependent effect sizes exist in the meta-analysis data.”
MethodSandbank, M. P., Bottema‐Beutel, K., Crowley, S., Feldman, J. I., Barrett, D. J., Caldwell, N., Dunham-Carr, K., Crank, J., Albarran, S., & Woynaroski, T. G. (2023) · BMJ · doi:10.1136/bmj-2023-076733 · Cited by 206
“Results from the Egger multilevel meta-analysis test 31 indicated that, across all outcomes analyzed in our models, there was evidence for funnel plot asymmetry (B=2.98, P 0.001).”
MethodInfluentialAn umbrella review of randomized control trials on the effects of physical exercise on cognition
Ciria, L. F., Román-Caballero, R., Vadillo, M. A., Holgado, D., Luque-Casado, A., Perakakis, P., & Sanabria, D. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01554-4 · Cited by 157
Influencing climate change attitudes in the United States: A systematic review and meta-analysis
Rode, J. B., Dent, A. L., Benedict, C. N., Brosnahan, D., Martinez, R. L., & Ditto, P. H. (2021) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2021.101623 · Cited by 164
Wang, L. A. L., Petrulla, V., Zampella, C., Waller, R. E., & Schultz, R. T. (2022) · Psychological Bulletin · doi:10.1037/bul0000358 · Cited by 148
Baldwin, J. R., Coleman, O., Francis, E., & Danese, A. (2024) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2024.0818 · Cited by 145
Román-Caballero, R., Vadillo, M. A., Trainor, L. J., & Lupiáñez, J. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100436 · Cited by 118
“A way to account for dependence is to combine all the effect sizes coming from the same sample generating an average estimate for each study, and conduct the classic methods on these aggregated estimates (Rodgers & Pustejovsky, 2020).”
BackgroundMethodInfluentialYuan, T., Hu, Y., Zhou, X., Yang, L., Wang, H., Li, L., Wang, J., Qian, H., Clifford, G. M., & Zou, H. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101613 · Cited by 126
Wang, B., Baldwin, J. R., Schoeler, T., Cheesman, R., Barkhuizen, W., Dudbridge, F., Bann, D., Morris, T. T., & Pingault, J.-B. (2021) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2021.07.010 · Cited by 103
Oser, J. L., Grinson, A., Boulianne, S., & Halperin, E. (2022) · Political Communication · doi:10.1080/10584609.2022.2086329 · Cited by 127
“The results provide evidence of modest publication bias for OffPP but no evidence of such bias for OnPP, although there is not yet consensus regarding thresholds for publication bias at the effect-level in multilevel meta-analysis (Fernández-Castilla et al., 2021; Rodgers & Pustejovsky, 2021).”
BackgroundThe role of teaching staff in fostering perceived employability of university students
Petruzziello, G., Mariani, M. G., Guglielmi, D., HEIJDEN, B. I. J. M. V. D., de Jong, J. P., & Chiesa, R. (2022) · Studies in Higher Education · doi:10.1080/03075079.2022.2105830 · Cited by 94
Kittel, A. F. D., Olderbak, S. G., & Wilhelm, O. (2021) · Assessment · doi:10.1177/1073191121996469 · Cited by 90
“We could not estimate trim and fill analyses for theory of mind, cognitive empathy, or affective empathy because the nonindependent in the effect sizes is ignored by trim and fill analyses and results could lead to Type I errors (Rodgers & Pustejovsky, 2020).”
BackgroundJoyal‐Desmarais, K., Scharmer, A., Madzelan, M. K., See, J. V., Rothman, A. J., & Snyder, M. (2022) · Psychological Bulletin · doi:10.1037/bul0000377 · Cited by 99
Schoenherr, J., Strohmaier, A., & Schukajlow, S. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100639 · Cited by 170
Effects of action video game play on cognitive skills: A meta-analysis.
Bediou, B., Rodgers, M. A., Tipton, E., Mayer, R. E., Green, C. S., & Bavelier, D. (2023) · Technology Mind and Behavior · doi:10.1037/tmb0000102 · Cited by 112
Self-citationReinebo, G., Alfonsson, S., Jansson‐Fröjmark, M., Rozental, A., & Lundgren, T. (2023) · Sports Medicine · doi:10.1007/s40279-023-01931-z · Cited by 135
“An adapted version of Egger’s regression was also conducted to test for funnel plot asymmetry [45, 46].”
MethodThe sleep benefit in episodic memory: An integrative review and a meta-analysis.
Berres, S., & Erdfelder, E. (2021) · Psychological Bulletin · doi:10.1037/bul0000350 · Cited by 74
Depression and episodic memory across the adult lifespan: A meta-analytic review.
James, T., Weiss-Cowie, S., Hopton, Z. W., Verhaeghen, P., Dotson, V. M., & Duarte, A. (2021) · Psychological Bulletin · doi:10.1037/bul0000344 · Cited by 96
“Regression analysis on the funnel plot (Figure 2) was conducted using an Egger test appropriate for multilevel metaanalysis (Rodgers & Pustejovsky, 2020).”
MethodNickow, A., Oreopoulos, P., & Quan, V. (2024) · American Educational Research Journal · doi:10.3102/00028312231208687 · Cited by 82
“Regressing ES on W i using the same RVE models as for our main estimates, we then observe the significance of the W i term and especially changes in the regression constant, which represents pooled ES (Rodgers & Pustejovsky, 2021).”
MethodPredictors of Political Violence Outcomes among Young People: A Systematic Review and Meta‐Analysis
Jahnke, S., Borger, K. A., & Beelmann, A. (2021) · Political Psychology · doi:10.1111/pops.12743 · Cited by 79
“Rodgers and Pustejovsky (2019) have recently introduced a new version of the traditional Egger test that is based on RVE and can be used for dependent samples.”
BackgroundBackhaus, S., Leijten, P. H. O., Jochim, J., Meléndez‐Torres, G. J., & Gardner, F. E. M. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102003 · Cited by 89
Sex drive: Theoretical conceptualization and meta-analytic review of gender differences.
Frankenbach, J., Weber, M., Loschelder, D. D., Kilger, H., & Friese, M. (2022) · Psychological Bulletin · doi:10.1037/bul0000366 · Cited by 77
Wollburg, C., Steinert, J., Reeves, A., & Nye, E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281283 · Cited by 70
“It should be noted that funnel plot asymmetry may occur for reasons other than publication bias and traditional methods may not adequately account for effect size dependency [65,67].”
BackgroundWang, F., Cheung, A. C. K., Neitzel, A. J., & Chai, C. S. (2025) · Review of Educational Research · doi:10.3102/00346543241255621 · Cited by 64
“Other methods to test for and assess the degree of publication bias exist, but few have been utilized with robust variance estimation and may rely on larger sample sizes (Rodgers & Pustejovsky, 2021).”
McWeeny, S., Choi, S. J., Choe, J., LaTourrette, A. S., Roberts, M. Y., & Norton, E. S. (2022) · Reading Research Quarterly · doi:10.1002/rrq.467 · Cited by 97
Emmer, C., Dorn, J., & Mata, J. (2024) · Psychological Bulletin · doi:10.1037/bul0000419 · Cited by 93
Salmerón, L., Altamura, L., Delgado, P., Karagiorgi, A., & Vargas, C. (2023) · Journal of Educational Psychology · doi:10.1037/edu0000830 · Cited by 96
Self-regulated learning strategies in continuing education: A systematic review and meta-analysis
Hemmler, Y. M., & Ifenthaler, D. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100629 · Cited by 110
“However, reliable methods for detecting publication bias in three-level meta-analyses are still lacking, and research has shown that the three-level version of the Egger ’ s regression test may suffer from a lack of statistical power (Rodgers & Pustejovsky, 2021).”
InfluentialStucke, N. J., & Doebel, S. (2024) · Psychological Bulletin · doi:10.1037/bul0000445 · Cited by 69
Myers, J. A., Witzel, B. S., Powell, S. R., Li, H., Pigott, T., Xin, Y. P., & Hughes, E. M. (2022) · Review of Educational Research · doi:10.3102/00346543211070049 · Cited by 96
de Roo, M., Veenstra, R., & Kretschmer, T. (2022) · Social Development · doi:10.1111/sode.12590 · Cited by 75
“To assess the likelihood of publication bias, we conducted Egger’s test for funnel plot asymmetry using sample size as amoderator to account for dependence among effect sizes (Rodgers & Pustejovsky, 2020).”
MethodWan, S., Lauermann, F., Bailey, D. H., & Eccles, J. S. (2021) · Psychological Bulletin · doi:10.1037/bul0000340 · Cited by 64
Roest, J. J., de Poll, M. J. W.-V., van der Helm, P. G., Stams, G. J. J. M., & Hoeve, M. (2022) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2022.2093210 · Cited by 61
“Moreover, for multilevel meta-analysis, specific methods to take into account dependency of effect sizes in the assessment of publication bias are still under development (Assink & Wibbelink, 2016; Fernández-Castilla et al., 2021; Rodgers & Pustejovsky, 2021).”
MethodSchmaußer, M., Hoffmann, S., Raab, M., & Laborde, S. (2022) · Journal of Neuroscience Research · doi:10.1002/jnr.25062 · Cited by 59
Bourke, M., Wang, H., McNaughton, S. A., Thomas, G., Firth, J., Trott, M., & Cairney, J. (2025) · Clinical Psychology Review · doi:10.1016/j.cpr.2025.102585 · Cited by 65
A meta-analysis of Libet-style experiments
Braun, M. N., Wessler, J., & Friese, M. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.06.018 · Cited by 63
Meta-Analytic Methods to Detect Publication Bias in Behavior Science Research
Dowdy, A., Hantula, D. A., Travers, J. C., & Tincani, M. (2021) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00303-0 · Cited by 61
Schwabe, A., Lind, F., Kosch, L., & Boomgaarden, H. G. (2022) · Media Psychology · doi:10.1080/15213269.2022.2070216 · Cited by 75
“However, the detected funnel plot asymmetry could have another cause than a publication bias (Rodgers & Pustejovsky, 2021).”
BackgroundMethodReal-time fMRI neurofeedback as a new treatment for psychiatric disorders: A meta-analysisPreprint
Pindi, P., Houenou, J., Piguet, C., & Favre, P. (2021) · PsyArXiv (OSF Preprints) · doi:10.1016/j.pnpbp.2022.110605 · Cited by 63
Dietrichson, J., Filges, T., Seerup, J. K., Klokker, R. H., Viinholt, B. C. A., Bøg, M., & Eiberg, M. (2021) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1152 · Cited by 54
Chacón-Candia, J. A., Román-Caballero, R., Aranda‐Martín, B., Casagrande, M., Lupiáñez, J., & Marotta, A. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104993 · Cited by 67
Winters, K. L., Jasso, J., Pustejovsky, J. E., & Byrd, C. T. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2022_jslhr-22-00017 · Cited by 74
Self-citationManipulating Belief in Free Will and Its Downstream Consequences: A Meta-Analysis
Genschow, O., Cracco, E., Schneider, J., Protzko, J., Wisniewski, D., Braß, M., & Schooler, J. W. (2022) · Personality and Social Psychology Review · doi:10.1177/10888683221087527 · Cited by 63
Patall, E. A., Yates, N., Lee, J., Chen, M., Bhat, B. H., Lee, K., Beretvas, S. N., Lin, S., Hooper, S. Y., Jacobson, N. G., et al. (2023) · Educational Psychologist · doi:10.1080/00461520.2023.2274104 · Cited by 85
“Finally, we examined the possibility of publication bias and funnel plot asymmetry by conducting a modified version of Egger’s regression (Egger et al., 1997) that accounted for the dependent effect sizes (Rodgers & Pustejovsky, 2020).”
MethodA Meta-Analysis on Teachers’ Growth Mindset
Bardach, L., Bostwick, K. C. P., Fütterer, T., Kopatz, M., Hobbi, D. M., Klassen, R. M., & Pietschnig, J. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09925-7 · Cited by 76
“Therefore, we used three methods that have been shown to be suitable for multilevel meta-analyses in targeted investigations (Rodgers & Pustejovsky, 2021).”
MethodInfluentialImproving Low-Performing Schools: A Meta-Analysis of Impact Evaluation Studies
Schueler, B. E., Asher, C. A., Larned, K. E., Mehrotra, S., & Pollard, C. (2021) · American Educational Research Journal · doi:10.3102/00028312211060855 · Cited by 59
“We implement RVE in Stata using the Robumeta package (Tanner-Smith & Tipton, 2014) parameterized for hierarchical effects.”
MethodInfluentialNeglect of publication bias compromises meta-analyses of educational research
Ropovik, I., Adamkovic, M., & Greger, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252415 · Cited by 55
“The meta-analyst then needs to be aware that if dependencies are not properly modeled (see [61]) or handled otherwise (e.”
Yang, Y., Macleod, M. R., Pan, J., Lagisz, M., & Nakagawa, S. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.105016 · Cited by 53
Girdwood, M., Culvenor, A. G., Rio, E., Patterson, B., Haberfield, M. J., Couch, J. L., Mentiplay, B. F., Hedger, M., & Crossley, K. M. (2024) · British Journal of Sports Medicine · doi:10.1136/bjsports-2023-107977 · Cited by 48
Dudek, E. M., & Dodell‐Feder, D. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.020 · Cited by 45
A systematic review and meta-analysis of strategies to promote vaccination uptake
Liu, S., Durantini, M. R., Calabrese, C., Sánchez, F., & Albarracín, D. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01940-6 · Cited by 48
“…effect sizes and involves one-sided t -tests to detect the linear relation between effect size magnitude and the associated precision 170,171 , together with the visual inspection of the funnel plot, whose asymmetry of data point distribution suggests missing studies due to reporting…”
MethodIdentifying Key Antecedents of Quiet Quitting Among Nurses: A Cross‐Profession Meta‐Analytic Review
Geng, R., Geng, X., & Geng, S. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.16934 · Cited by 63
Cross-cultural variation in cooperation: A meta-analysis.
Spadaro, G., Graf, C., Jin, S., Arai, S., Inoue, Y., Lieberman, E., Rinderu, M. I., Yuan, M., Lissa, C. J. V., & Balliet, D. P. (2022) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000389 · Cited by 56
Denton, C. A., Hall, C., Cho, E., Cannon, G., Scammacca, N. K., & Wanzek, J. (2021) · Learning and Individual Differences · doi:10.1016/j.lindif.2021.102062 · Cited by 92
Speech‐in‐noise perception in musicians and non‐musicians: A multi‐level meta-analysis
Hennessy, S., Mack, W. J., & Habibi, A. (2022) · Hearing Research · doi:10.1016/j.heares.2022.108442 · Cited by 50
Roberts, B. R. T., MacLeod, C. M., & Fernandes, M. A. (2022) · Psychological Bulletin · doi:10.1037/bul0000360 · Cited by 51
“To assess publication bias while maintaining robust variance estimates, we used the “ Egger sandwich ” analysis method (Rodgers & Pustejovsky, 2020).”
MethodHow universal is preference for visual curvature? A systematic review and meta‐analysis
Chuquichambi, E. G., Vartanian, O. A., Skov, M., Corradi, G. B., Nadal, M., Silvia, P. J., & Munar, E. (2022) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14919 · Cited by 51
Wallrich, L., Opara, V., Wesołowska, M., Barnoth, D., & Yousefi, S. (2024) · Journal of Business and Psychology · doi:10.1007/s10869-024-09977-0 · Cited by 62
A broadscale analysis of host‐symbiont cophylogeny reveals the drivers of phylogenetic congruence
Hayward, A., Poulin, R., & Nakagawa, S. (2021) · Ecology Letters · doi:10.1111/ele.13757 · Cited by 42
Vembye, M. H., Weiss, F., & Bhat, B. H. (2023) · Review of Educational Research · doi:10.3102/00346543231186588 · Cited by 82
“Moreover, selection models based on dependent effect sizes aggregated to the study level do not fully calibrate the nominal Type-I error rate (Rodgers & Pustejovsky, 2021), and the WAAP test used was based on just three adequately powered studies.”
MethodMcNally, X., Webb, T. L., Smith, C., Moss, A., & Miller, J. G. (2024) · Conservation Biology · doi:10.1111/cobi.14237 · Cited by 46
Zhou, L., Sommer, I. E., Yang, P., Sikirin, L., van Os, J., Bentall, R. P., Varese, F., & Begemann, M. (2025) · American Journal of Psychiatry · doi:10.1176/appi.ajp.20240456 · Cited by 51
Effects of the 5E Instructional Model: A Systematic Review and Meta-Analysis
Polanin, J. R., Austin, M. J., Taylor, J. A., Steingut, R. R., Rodgers, M. A., & Williams, R. T. (2024) · AERA Open · doi:10.1177/23328584241269866 · Cited by 78
Self-citationJoshi, M., Pustejovsky, J. E., & Beretvas, S. N. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1554 · Cited by 40
Self-citationEmotional intelligence and romantic relationship satisfaction: A systematic review and meta-analysis
Jardine, B. B., Vannier, S. A., & Voyer, D. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2022.111713 · Cited by 62
Vieira, A. P. A., Peng, P., Antoniuk, A., DeVries, J., Rothou, K. M., Parrila, R., & Georgiou, G. K. (2023) · Annals of Dyslexia · doi:10.1007/s11881-023-00294-4 · Cited by 48
Demangeon, A., Claudel-Valentin, S., Aubry, A., & Tazouti, Y. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102182 · Cited by 55
Ye, L., Li, Y., Deng, Q., Zhao, X., Zhong, L., & Yang, L. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301226 · Cited by 43
Kong, F., Meng, S., Deng, H., Wang, M., & Sun, X. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09746-0 · Cited by 51
Meta-analysis of second language research with complex research designs
Norouzian, R., & Bui, G. (2023) · Studies in Second Language Acquisition · doi:10.1017/s0272263123000311 · Cited by 38
January, S. A., & Klingbeil, D. A. (2020) · Journal of School Psychology · doi:10.1016/j.jsp.2020.08.007 · Cited by 45
Ivaniushina, V. A., & Викторовна, Т. В. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250169 · Cited by 39
“While no bias was detected, it should be noted that such tests do not have strong power, especially when the number of articles is small, as in case of our meta-analysis [34].”
BackgroundMethodThe Impasse on Gender Differences in Intelligence: a Meta-Analysis on WISC Batteries
Giofrè, D., Allen, K., Toffalini, E., & Caviola, S. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09705-1 · Cited by 51
“In addition, regression 1 3 tests (PET-PEESE is a modification of Egger’s test; Stanley & Doucouliagos, 2014) can be used within the multi-level meta-analytic framework (Rodgers & Pustejovsky, 2021).”
MethodDanielsen, P. A., Mentzoni, R. A., & Låg, T. (2023) · Addictive Behaviors · doi:10.1016/j.addbeh.2023.107887 · Cited by 39
Cui, X., Zhang, S., Yu, S., Ding, Q., & Li, X. (2024) · Behaviour Research and Therapy · doi:10.1016/j.brat.2024.104549 · Cited by 37
Effects of peer observation on risky decision-making in adolescence: A meta-analytic review.
Powers, K. E., Schaefer, L., Figner, B., & Somerville, L. H. (2022) · Psychological Bulletin · doi:10.1037/bul0000382 · Cited by 41
“In addition, we implemented Vevea and Hedges (1995) three-parameter model that provides greater power and can be interpreted as a more direct test of publication bias (Rodgers & Pustejovsky, 2020).”
MethodChang, D. F., Donald, J. N., Whitney, J. M., Miao, I. Y., & Sahdra, B. K. (2023) · Personality and Social Psychology Bulletin · doi:10.1177/01461672231178518 · Cited by 35
Anxiety and Depression in Gifted Individuals: A Systematic and Meta-Analytic Review
Duplenne, L., Bourdin, B., Fernandez, D., Blondelle, G., & Aubry, A. (2023) · Gifted Child Quarterly · doi:10.1177/00169862231208922 · Cited by 51
“This test could be adapted with effect size dependence (Fernández-Castilla et al., 2021; Rodgers & Pustejovsky, 2021).”
MethodSwaine, M., Bergna, A., Oyserman, B. O., Vasileiadis, S., Karas, P. A., Screpanti, C., & Karpouzas, D. G. (2025) · FEMS Microbiology Ecology · doi:10.1093/femsec/fiaf052 · Cited by 49
“Publication bias was assessed using Egger’s test for multilevel models (Rodgers and Pustejovsky 2021 ).”
Patten, R. V., Mulhauser, K. R. W., Austin, T. A., Bellone, J. A., Cotton, E., Chan, L. C. N., Twamley, E. W., Sawyer, K., & LaFrance, W. C., Jr. (2025) · Clinical Psychology Review · doi:10.1016/j.cpr.2025.102648 · Cited by 44
Webber, H. E., de Dios, C., Kessler, D. A., Schmitz, J. M., Lane, S. D., & Suchting, R. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104835 · Cited by 35
Donolato, E., Toffalini, E., Rogde, K., Nordahl‐Hansen, A., Lervåg, A., Norbury, C., & Melby‐Lervåg, M. (2023) · Campbell Systematic Reviews · doi:10.1002/cl2.1368 · Cited by 66
“Unfortunately, the p ‐ curve method may present the problem of not accounting for the presence of dependency structures between the effect sizes, and it is known to perform poorly when there is between ‐ study heterogeneity (Rodgers & Pustejovsky, 2021).”
Dietrichson, J., Filges, T., Klokker, R. H., Viinholt, B. C. A., Bøg, M., & Jensen, U. H. (2020) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1081 · Cited by 33
Miroshnik, K. G., Forthmann, B., Karwowski, M., & Benedek, M. (2023) · Intelligence · doi:10.1016/j.intell.2023.101739 · Cited by 33
The effect of notetaking method on academic performance: A systematic review and meta-analysis
Voyer, D., Ronis, S. T., & Byers, N. (2021) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2021.102025 · Cited by 47
Siepe, B. S., Bartos̆, F., Morris, T. P., Boulesteix, A., Heck, D. W., & Pawel, S. (2024) · Psychological Methods · doi:10.1037/met0000695 · Cited by 71
“Hypothesis testing, e.g., comparing different tests of publication bias (Rodgers & Pustejovsky, 2021).”
Hagger, M. S. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1966824 · Cited by 41
“For a more detailed explanation, see Rodgers and Pustejovsky (2021).”
BackgroundGolan, O. K., Totaram, R., Perry, E. W., Fortson, K., Rivera-Atilano, R., Entress, R. M., Golan, M., Andraka‐Christou, B., Whitaker, D. J., & Pigott, T. (2022) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2022.109445 · Cited by 34
Conservation interventions are required to improve bird breeding performance in artificial wetlands
Cheng, C., & Ma, Z. (2022) · Biological Conservation · doi:10.1016/j.biocon.2022.109872 · Cited by 31
Zhang, Y., Zhou, S., Wu, X., & Cheung, A. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100627 · Cited by 51
The effectiveness of refutation text in confronting scientific misconceptions: A meta-analysis
Danielson, R., Jacobson, N. G., Patall, E. A., Sinatra, G. M., Adesope, O., Kennedy, A. A. U., Bhat, B. H., Ramazan, O., Akinrotimi, B., Nketah, G., et al. (2024) · Educational Psychologist · doi:10.1080/00461520.2024.2365628 · Cited by 36
“…of publication bias and funnel plot asymmetry by creating funnel plots for IG and IGRM effects and each outcome separately and conducting a modified version of Egger’s regression (Egger et al., 1997) for each outcome that accounted for the dependent effect sizes (Rodgers & Pustejovsky, 2020).”
Metadata
- Authors
- Rodgers, M. A., & Pustejovsky, J. E.
- Affiliations
- The University of Texas at Austin (US)
- Venue
- Psychological Methods
- Published
- 2021-04
- Volume
- 26
- Issue
- 2
- Pages
- 141-160
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W3043790905
- Cited by
- 492
- References
- 0
- Citations by year
- 2026: 111 · 2025: 111 · 2024: 95 · 2023: 72 · 2022: 61 · 2021: 34
- FWCI
- 26.68
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- MetArXiv (OSF Preprints) (submittedVersion)
- MetArXiv (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- selective reporting, publication bias, small-study effects, dependent effect sizes, robust variance estimation, multilevel meta-analysis, type I error rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
