The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Open accessPublished version of a preprint
Abstract
Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/ .
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The absence of publication bias is highly unlikely unless a set of preregistered reports is examined (Carter et al. 2019).”
From The gamification of learning: a meta-analysis (2020) · cites this paper as Carter et al. 2019
In the citing paper
“The trim-and-fill method (Duval & Tweedie, 2000) tends to adjust too little when bias is strong (Carter et al., 2019), and Fail-Safe N (Rosenthal, 1979) neither tests nor adjusts for the presence of bias.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Bias adjustments can improve performance relative to a meta-analysis that does not adjust for publication bias (Carter et al., 2019; McShane et al., 2016; Moreno et al., 2009; Simonsohn et al., 2014).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“negative and statistically significant SPET estimates have been observed in both simulation studies (Carter et al., 2019) and meta-analysis (Hilgard et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“On the one hand, this may be an underestimate of average study power—in simulation studies, p-hacking adds large p values to the p-curve, canceling out the right skew and biasing the p-curve estimate downward (Carter et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“SPET and SPEESE have a downward bias in the presence of p-hacking (Carter et al., 2019)”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“There may also be downward bias inflicted by p-hacking (Carter et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Further, PET-PEESE can underestimate the effect size when there is publication bias and when p-hacking is present (Carter et al., 2019), which are two conditions likely affecting the literature.”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“However, this counterintuitive finding may be explained by the fact that PET is prone to producing imprecise and biased estimates under high heterogeneity, small sample sizes, and a large proportion of p-hacking (Stanley & Doucouliagos, 2014; Stanley 2017; Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Simulation studies by Carter et al. (2019) showed that WAAP-WLS can provide a reasonably unbiased estimate of the effect size with a high power to detect the effect given that a true underlying effect exists.”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al. (2019)
In the citing paper
“This could lead selection model to underestimate effects in the presence of publication bias and p-hacking, as shown in simulation studies (Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Lastly, PET-PEESE can underestimate the size of small effects in the presence of p-hacking and publication bias (Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“However, when effects are highly heterogeneous and when sample sizes are small, the PET-PEESE analysis is known to yield imprecise estimates (Stanley & Doucouliagos, 2014; Stanley, 2017; Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“there is a danger that uncritical application can lead to inappropriate conclusions being drawn 68,73,74”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [74]
In the citing paper
“the available evidence suggests that no single method outperforms others in all scenarios”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [74]
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Ego depletion and the strength model of self-control: a meta-analysis
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010) · Psychological Bulletin · doi:10.1037/a0019486 · Cited by 2,265
Research works printed but not linked: 1
- Augusteijn H. E. M., 2019, Q, 24, 116
Cited by
Sorted by citations
Items 101–200 of 578
Systematic Review and Meta-Analysis of Correlates of FFMQ Mindfulness Facets
Mattes, J. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02684 · Cited by 88
The benefits of preregistration and Registered Reports
Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024) · Evidence-Based Toxicology · doi:10.1080/2833373x.2024.2376046 · Cited by 93
The sleep benefit in episodic memory: An integrative review and a meta-analysis.
Berres, S., & Erdfelder, E. (2021) · Psychological Bulletin · doi:10.1037/bul0000350 · Cited by 74
Howard, M. C., & Zandt, E. C. V. (2020) · Journal of Research in Personality · doi:10.1016/j.jrp.2020.103982 · Cited by 85
Nguyen, L., Murphy, K. A., & Andrews, G. (2021) · Neuropsychology Review · doi:10.1007/s11065-021-09515-2 · Cited by 76
The effect of publication bias on the Q test and assessment of heterogeneity.
Augusteijn, H. E. M., van Aert, R. C. M., & van Assen, M. A. L. M. (2018) · Psychological Methods · doi:10.1037/met0000197 · Cited by 77
Hoogsteder, L. M., Thije, L. T., Schippers, E. E., & Stams, G. J. J. M. (2021) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x211010290 · Cited by 79
Lucca, K., Yuen, F. L., Wang, Y., Alessandroni, N., Allison, O., Alvarez, M., Axelsson, E. L., Baumer, J., Baumgartner, H. A., Bertels, J., et al. (2024) · Developmental Science · doi:10.1111/desc.13581 · Cited by 78
Self-Reported Trait Mindfulness and Couples’ Relationship Satisfaction: a Meta-Analysis
Quinn‐Nilas, C. (2020) · Mindfulness · doi:10.1007/s12671-020-01303-y · Cited by 78
The Cooperation Databank: Machine-Readable Science Accelerates Research Synthesis
Spadaro, G., Tiddi, I., Columbus, S., Jin, S., ten Teije, A., CoDa Team, & Balliet, D. P. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211053319 · Cited by 67
Meta-analysis in vocational behavior: A systematic review and recommendations for best practices
Rudolph, C. W., Chang, C. K., Rauvola, R. S., & Zacher, H. (2020) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2020.103397 · Cited by 67
Riet, J. V. ‘., & van Stekelenburg, A. (2021) · Human Communication Research · doi:10.1093/hcr/hqab022 · Cited by 74
Child physical abuse risk factors: A systematic review and a meta-analysis
Milner, J. S., Crouch, J. L., McCarthy, R. J., Ammar, J., Dominguez-Martinez, R., Thomas, C. L., & Jensen, A. P. (2022) · Aggression and Violent Behavior · doi:10.1016/j.avb.2022.101778 · Cited by 75
Sala, G., Aksayli, N. D., Tatlidil, K. S., Gondo, Y., & Gobet, F. (2019) · Intelligence · doi:10.1016/j.intell.2019.101386 · Cited by 72
The relationship between mind wandering and reading comprehension: A meta-analysis
Bonifacci, P., Viroli, C., Vassura, C., Colombini, E., & Desideri, L. (2022) · Psychonomic Bulletin & Review · doi:10.3758/s13423-022-02141-w · Cited by 71
The relation of mindfulness and prosocial behavior: What do we (not) know?
Schindler, S., & Friese, M. (2021) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2021.09.010 · Cited by 69
Emotion regulation and mental health across cultures: a systematic review and meta-analysis
Chen, S., Cai, Q., Omari, D. I., Sanghvi, D. E., Lyu, S., & Bonanno, G. A. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02168-8 · Cited by 61
Miller, R. E., & Kirschbaum, C. (2018) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2018.12.236 · Cited by 61
Manipulating Belief in Free Will and Its Downstream Consequences: A Meta-Analysis
Genschow, O., Cracco, E., Schneider, J., Protzko, J., Wisniewski, D., Braß, M., & Schooler, J. W. (2022) · Personality and Social Psychology Review · doi:10.1177/10888683221087527 · Cited by 63
Liu, C., Hua, M.-H., Lu, M., & Goh, K. K. (2023) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12455 · Cited by 60
A meta-analysis of Libet-style experiments
Braun, M. N., Wessler, J., & Friese, M. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.06.018 · Cited by 63
A meta-analysis of teacher language practices in early childhood classrooms
Hadley, E. B., Barnes, E. M., Wiernik, B. M., & Raghavan, M. (2022) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2021.12.002 · Cited by 59
Chacón-Candia, J. A., Román-Caballero, R., Aranda‐Martín, B., Casagrande, M., Lupiáñez, J., & Marotta, A. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104993 · Cited by 67
Mok, S. Y., Rupp, D., & Holzberger, D. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100552 · Cited by 59
Anomalous information reception by mediums: A meta-analysis of the scientific evidence
Sarraf, M. A., Menie, M. A. W. O., & Tressoldi, P. E. (2020) · EXPLORE · doi:10.1016/j.explore.2020.04.002 · Cited by 58
Siepe, B. S., Bartos̆, F., Morris, T. P., Boulesteix, A., Heck, D. W., & Pawel, S. (2024) · Psychological Methods · doi:10.1037/met0000695 · Cited by 71
Maier, M., VanderWeele, T. J., & Mathur, M. B. (2022) · Campbell Systematic Reviews · doi:10.1002/cl2.1256 · Cited by 60
How to Detect Publication Bias in Psychological Research
Renkewitz, F., & Keiner, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000386 · Cited by 62
Siegel, M., Eder, J. S. N., Wicherts, J. M., & Pietschnig, J. (2021) · Journal of Applied Psychology · doi:10.1037/apl0000991 · Cited by 65
Clayson, P. E., Carbine, K. A., & Larson, M. J. (2020) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2020.01.005 · Cited by 58
Digital mental health interventions for the treatment of depression: A multiverse meta-analysis
Plessen, C. Y., Panagiotopoulou, O. M., Tong, L., Cuijpers, P., & Karyotaki, E. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.10.018 · Cited by 53
West, G., Melby‐Lervåg, M., & Hulme, C. (2021) · Developmental Psychology · doi:10.1037/dev0001172 · Cited by 63
Neglect of publication bias compromises meta-analyses of educational research
Ropovik, I., Adamkovic, M., & Greger, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252415 · Cited by 55
Allen, M. S., Robson, D. A., Martin, L. J., & Laborde, S. (2019) · Personality and Social Psychology Bulletin · doi:10.1177/0146167219893995 · Cited by 51
Joye, N., Broc, L., Olive, T., & Dockrell, J. E. (2018) · Scientific Studies of Reading · doi:10.1080/10888438.2018.1491584 · Cited by 56
Saunders, B., & Inzlicht, M. (2020) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2020.05.008 · Cited by 50
Craig, R., Cox, A., Tourish, D., & Thorpe, A. (2020) · Research Policy · doi:10.1016/j.respol.2020.103930 · Cited by 53
Challenging the challenge hypothesis on testosterone in fathers: Limited meta-analytic support
Meijer, W. M., van IJzendoorn, M. H., & Bakermans - Kranenburg, M. J. (2019) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2019.104435 · Cited by 49
Collin, S., Rowse, G., Martinez, A. P., & Bentall, R. P. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2023.102303 · Cited by 49
Chernikova, O., Sommerhoff, D., Stadler, M. J., Holzberger, D., Nickl, M., Seidel, T., Kasneci, E., Küchemann, S., Kühn, J., Fischer, F., et al. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100662 · Cited by 54
Pietschnig, J., Siegel, M., Eder, J. S. N., & Gittler, G. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02874 · Cited by 52
Oberlader, V. A., Quinten, L. K. J. L., Banse, R., Volbert, R., Schmidt, A. F., & Schönbrodt, F. D. (2020) · Applied Cognitive Psychology · doi:10.1002/acp.3776 · Cited by 53
Self-citationHouben, S., & Bonnechère, B. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19137748 · Cited by 52
McKay, B., Bacelar, M. F. B., Parma, J. O., Miller, M. W., & Carter, M. J. (2023) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2023.2207255 · Cited by 56
Nordgård, R., & Låg, T. (2021) · Frontiers in Virtual Reality · doi:10.3389/frvir.2021.699383 · Cited by 53
McKay, B., Yantha, Z. D., Hussien, J., Carter, M. J., & Ste‐Marie, D. M. (2022) · Meta-Psychology · doi:10.15626/mp.2021.2803 · Cited by 52
Choowan, P., Daovisan, H., & Suwanwong, C. (2024) · International Journal of Financial Studies · doi:10.3390/ijfs13010001 · Cited by 47
Tsipa, A., O’Connor, D. B., Branley-Bell, D., Day, F., Hall, L. H., Sykes-Muskett, B., Wilding, S., Taylor, N., & Conner, M. (2021) · Health Psychology Review · doi:10.1080/17437199.2020.1760726 · Cited by 46
Meta-analyses in psychology often overestimate evidence for and size of effects
Bartos̆, F., Maier, M., Shanks, D. R., Stanley, T. D., Sladekova, M., & Wagenmakers, E. (2023) · Royal Society Open Science · doi:10.1098/rsos.230224 · Cited by 57
Does FDI promote entrepreneurial activities? A meta-analysis
Hong, S., Reed, W. R., Tian, B., Wu, T., & Chen, G. (2021) · World Development · doi:10.1016/j.worlddev.2021.105436 · Cited by 45
Mazzolari, R., Porcelli, S., Bishop, D. J., & Lakens, D. (2022) · Experimental Physiology · doi:10.1113/ep090171 · Cited by 44
Williams, R. T., Citkowicz, M., Miller, D. I., Lindsay, J., & Walters, K. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2021.2009072 · Cited by 50
Holgado, D., Sanabria, D., Perales, J. C., & Vadillo, M. A. (2020) · Journal of Cognition · doi:10.5334/joc.126 · Cited by 44
Dudek, E. M., & Dodell‐Feder, D. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.020 · Cited by 45
Cuijpers, P. J., Harrer, M., Miguel, C., Čihařová, M., & Karyotaki, E. (2023) · American Psychologist · doi:10.1037/amp0001250 · Cited by 47
Houben, S. T. L., Otgaar, H., Roelofs, J., Merckelbach, H. L. G. J., & Muris, P. (2020) · Journal of Experimental Psychopathology · doi:10.1177/2043808720907744 · Cited by 43
Assessing Publication Bias: a 7-Step User’s Guide with Best-Practice Recommendations
Kepes, S., Wang, W., & Cortina, J. M. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-022-09840-0 · Cited by 47
Identification of and correction for publication biasPreprint
Kasy, M., & Andrews, I. (2018) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/49yst · Cited by 42
Oldrati, V., Corti, C., Poggi, G., Borgatti, R., Urgesi, C., & Bardoni, A. (2020) · Neuropsychology Review · doi:10.1007/s11065-020-09429-5 · Cited by 41
Game-Based Learning for Students’ Computational Thinking: A Meta-Analysis
Ma, J., Zhang, Y., Zhu, Z., Zhao, S., & Wang, Q. (2023) · Journal of Educational Computing Research · doi:10.1177/07356331231178948 · Cited by 45
Effect of game-based learning on students’ mathematics high order thinking skills: A meta-analysis
Anggoro, B. S., Dewantara, A. H., Suherman, S., Muhammad, R. R., & Saraswati, S. (2024) · Revista de Psicodidáctica (English ed ) · doi:10.1016/j.psicoe.2024.500158 · Cited by 49
Resting Frontal Eeg Asymmetry and Personality Traits: A Meta–Analysis
Kuper, N., Käckenmester, W., & Wacker, J. (2019) · European Journal of Personality · doi:10.1002/per.2197 · Cited by 49
Berezowski, L., Ludwig, L., Martin, A., Löwe, B., & Mora, M. C. S. (2021) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001011 · Cited by 44
Has the evidence for moral licensing been inflated by publication bias?
Kuper, N., & Bott, A. (2019) · Meta-Psychology · doi:10.15626/mp.2018.878 · Cited by 30
Using Monte Carlo experiments to select meta‐analytic estimators
Hong, S., & Reed, W. R. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1467 · Cited by 43
Meta-analysis of the reduced relative feedback frequency effect on motor learning and performance
McKay, B., Hussien, J., Vinh, M.-A., Mir-Orefice, A., Brooks, H., & Ste‐Marie, D. M. (2022) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2022.102165 · Cited by 37
Creativity enhancement methods for adults: A meta-analysis.
Haase, J., Hanel, P. H. P., & Gronau, N. (2023) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/aca0000557 · Cited by 46
Diriba, C., Abessa, T. G., Lamba, D., & Bonnechère, B. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14061783 · Cited by 37
A Series of Meta-Analytic Tests of the Efficacy of Long-Term Psychoanalytic Psychotherapy
Woll-Weber, C. F. J., & Schönbrodt, F. D. (2019) · European Psychologist · doi:10.1027/1016-9040/a000385 · Cited by 37
Self-citationČihařová, M., Karyotaki, E., Miguel, C., Walsh, E., de Ponti, N., Amarnath, A., van Ballegooijen, W., Riper, H. M., Arroll, B., & Cuijpers, P. J. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.05.070 · Cited by 36
Röseler, L., Schütz, A., Blank, P. A., Dück, M., Fels, S., Kupfer, J., Scheelje, L., & Seida, C. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104066 · Cited by 36
Hagger, M. S. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1966824 · Cited by 41
An aberrant abundance of Cronbach’s alpha values at .70Preprint
Hussey, I., Alsalti, T., Bosco, F. A., Elson, M., & Arslan, R. C. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dm8xn · Cited by 39
Meta-Analysis in Organizational Research: A Guide to Methodological Options
Morris, S. B. (2022) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-031921-021922 · Cited by 37
Video games and attitude change: A meta-analysis
Kolek, L., Ropovik, I., Šisler, V., van Oostendorp, H., & Brom, C. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102225 · Cited by 43
Comparison of Student Health and Well-Being Profiles and Social Media Use
Wright, R. R., Schaeffer, C., Mullins, R., Evans, A. M., & Cast, L. (2020) · Psi Chi Journal of Psychological Research · doi:10.24839/2325-7342.jn25.1.14 · Cited by 35
Expert opinions on the smallest effect size of interest in false memory research
Riesthuis, P., Mangiulli, I., Broers, N. J., & Otgaar, H. (2021) · Applied Cognitive Psychology · doi:10.1002/acp.3911 · Cited by 45
Promoting learning from null or negative results in prevention science trials
Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022) · Prevention Science · doi:10.1007/s11121-020-01140-4 · Cited by 34
Schindler, S., Hilgard, J., Fritsche, I., Burke, B., & Pfattheicher, S. (2023) · Personality and Social Psychology Review · doi:10.1177/10888683221107267 · Cited by 43
“Naturally, both regression models require substantial extrapolation, but they often provide better estimates than unadjusted metaanalysis or trim-and-fill (Carter & McCullough, 2014; Carter et al., 2019; Moreno et al., 2009).”
BackgroundMethodResultInfluentialSelf-citationMeta-analyzing the multiverse: A peek under the hood of selective reporting.
Olsson-Collentine, A., van Aert, R. C. M., Bakker, M., & Wicherts, J. M. (2023) · Psychological Methods · doi:10.1037/met0000559 · Cited by 40
How robust is our cumulative knowledge on turnover?
Field, J. G., Bosco, F. A., & Kepes, S. (2020) · Journal of Business and Psychology · doi:10.1007/s10869-020-09687-3 · Cited by 38
Implications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 37
Beyond Random Effects: When Small-Study Findings Are More Heterogeneous
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221120427 · Cited by 36
Overestimation of Action-Game Training Effects: Publication Bias and Salami Slicing
Hilgard, J., Sala, G., Boot, W. R., & Simons, D. J. (2019) · Collabra: Psychology · doi:10.1525/collabra.231 · Cited by 34
Self-citationChang, D. F., Donald, J. N., Whitney, J. M., Miao, I. Y., & Sahdra, B. K. (2023) · Personality and Social Psychology Bulletin · doi:10.1177/01461672231178518 · Cited by 35
Sensitivity Analysis of Mathematical Models
Sysoev, A. S. (2023) · Computation · doi:10.3390/computation11080159 · Cited by 33
Viljoen, J. L., Goossens, I., Monjazeb, S., Cochrane, D. M., Vargen, L. M., Jonnson, M. R., Blanchard, A. J. E., Li, S. M. Y., & Jackson, J. R. (2024) · Behavioral Sciences & the Law · doi:10.1002/bsl.2698 · Cited by 38
Dawtry, R. J., Callan, M. J., Harvey, A. J., & Gheorghiu, A. I. (2020) · Personality and Social Psychology Review · doi:10.1177/1088868320914208 · Cited by 33
Aschwanden, D., Sutin, A. R., Luchetti, M., Allemand, M., Stéphan, Y., & Terracciano, A. (2020) · Social and Personality Psychology Compass · doi:10.1111/spc3.12565 · Cited by 38
Siegel, M., Legler, M., Neziraj, F., Goldberg, A. E., & Zemp, M. (2022) · Children · doi:10.3390/children9091364 · Cited by 35
Intuitions About the Reference of Proper Names: a Meta-Analysis
van Dongen, N. N. N., Colombo, M., Romero, F., & Sprenger, J. (2020) · Review of Philosophy and Psychology · doi:10.1007/s13164-020-00503-8 · Cited by 31
Schulz, S., Nelemans, S. A., Hadiwijaya, H., Klimstra, T. A., Crocetti, E., Branje, S., & Meeus, W. (2022) · Child Development · doi:10.1111/cdev.13849 · Cited by 39
Implicit and explicit influences of religious cognition on Dictator Game transfers
Billingsley, J., Gomes, C., & McCullough, M. E. (2018) · Royal Society Open Science · doi:10.1098/rsos.170238 · Cited by 34
Potentially harmful therapies: A meta-scientific review of evidential value.
Williams, A. J., Botanov, Y., Kilshaw, R. E., Wong, R. E., & Sakaluk, J. K. (2020) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12331 · Cited by 34
Miroshnik, K. G., Forthmann, B., Karwowski, M., & Benedek, M. (2023) · Intelligence · doi:10.1016/j.intell.2023.101739 · Cited by 33
Becker, L., Kaltenegger, H. C., Nowak, D., Rohleder, N., & Weigl, M. (2022) · Health Psychology Review · doi:10.1080/17437199.2022.2071323 · Cited by 30
Pietschnig, J., Gerdesmann, D., Zeiler, M. D., & Voracek, M. (2022) · Royal Society Open Science · doi:10.1098/rsos.211621 · Cited by 31
Yang, C., Zhao, W., Yuan, B., Luo, L., & Shanks, D. R. (2022) · Review of Educational Research · doi:10.3102/00346543221094083 · Cited by 34
Social physical anxiety and eating disorders: A systematic review and meta-analysis
Alcaraz‐Ibáñez, M., Paterna, A., & Griffiths, M. D. (2023) · Body Image · doi:10.1016/j.bodyim.2023.02.008 · Cited by 31
The impact of open educational resources on student achievement: A meta-analysis
Cho, K. W., & Permzadian, V. (2024) · International Journal of Educational Research · doi:10.1016/j.ijer.2024.102365 · Cited by 29
Metadata
- Authors
- Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J.
- Affiliations
- DEVCOM Army Research Laboratory (US)
- Ludwig-Maximilians-Universität München (DE)
- University of Kentucky (US)
- Illinois State University (US)
- Corresponding author
- Carter, Evan C.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2019-06
- Volume
- 2
- Issue
- 2
- Pages
- 115-144
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- OpenAlex
- W2781957993
- Cited by
- 588
- References
- 76
- Citations by year
- 2026: 43 · 2025: 60 · 2024: 69 · 2023: 86 · 2022: 83 · 2021: 87 · 2020: 79 · 2019: 54
- FWCI
- 50.69
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Advances in Methods and Practices in Psychological Science (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- publication bias, questionable research practices, effect size overestimation, meta-analytic methods, sensitivity analysis, heterogeneity, simulation study, pre-registered replication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
