The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Open accessPublished version of a preprint
Abstract
Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/ .
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The absence of publication bias is highly unlikely unless a set of preregistered reports is examined (Carter et al. 2019).”
From The gamification of learning: a meta-analysis (2020) · cites this paper as Carter et al. 2019
In the citing paper
“The trim-and-fill method (Duval & Tweedie, 2000) tends to adjust too little when bias is strong (Carter et al., 2019), and Fail-Safe N (Rosenthal, 1979) neither tests nor adjusts for the presence of bias.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Bias adjustments can improve performance relative to a meta-analysis that does not adjust for publication bias (Carter et al., 2019; McShane et al., 2016; Moreno et al., 2009; Simonsohn et al., 2014).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“negative and statistically significant SPET estimates have been observed in both simulation studies (Carter et al., 2019) and meta-analysis (Hilgard et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“On the one hand, this may be an underestimate of average study power—in simulation studies, p-hacking adds large p values to the p-curve, canceling out the right skew and biasing the p-curve estimate downward (Carter et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“SPET and SPEESE have a downward bias in the presence of p-hacking (Carter et al., 2019)”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“There may also be downward bias inflicted by p-hacking (Carter et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Further, PET-PEESE can underestimate the effect size when there is publication bias and when p-hacking is present (Carter et al., 2019), which are two conditions likely affecting the literature.”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“However, this counterintuitive finding may be explained by the fact that PET is prone to producing imprecise and biased estimates under high heterogeneity, small sample sizes, and a large proportion of p-hacking (Stanley & Doucouliagos, 2014; Stanley 2017; Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Simulation studies by Carter et al. (2019) showed that WAAP-WLS can provide a reasonably unbiased estimate of the effect size with a high power to detect the effect given that a true underlying effect exists.”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al. (2019)
In the citing paper
“This could lead selection model to underestimate effects in the presence of publication bias and p-hacking, as shown in simulation studies (Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“Lastly, PET-PEESE can underestimate the size of small effects in the presence of p-hacking and publication bias (Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“However, when effects are highly heterogeneous and when sample sizes are small, the PET-PEESE analysis is known to yield imprecise estimates (Stanley & Doucouliagos, 2014; Stanley, 2017; Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
In the citing paper
“there is a danger that uncritical application can lead to inappropriate conclusions being drawn 68,73,74”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [74]
In the citing paper
“the available evidence suggests that no single method outperforms others in all scenarios”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [74]
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Ego depletion and the strength model of self-control: a meta-analysis
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010) · Psychological Bulletin · doi:10.1037/a0019486 · Cited by 2,265
Research works printed but not linked: 1
- Augusteijn H. E. M., 2019, Q, 24, 116
Cited by
Sorted by citations
Items 201–300 of 578
Xu, G., Zheng, Z., Zhang, J., Sun, T., & Liu, G. (2025) · Systems · doi:10.3390/systems13060409 · Cited by 29
Rakes, C. R., Ronau, R. N., Bush, S. B., Driskell, S. O. S., Niess, M. L., & Pugalee, D. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100337 · Cited by 30
Meissner, C. A. (2021) · Applied Cognitive Psychology · doi:10.1002/acp.3808 · Cited by 34
Meta-analyzing enhanced expectancies on motor learning: positive effects but methodological concerns
Bacelar, M. F. B., Parma, J. O., Murrah, W. M., & Miller, M. W. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2022.2042839 · Cited by 31
Moving in the presence of others – a systematic review and meta-analysis on social facilitation
van Meurs, E., Greve, J., & Strauß, B. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2022.2111663 · Cited by 29
Cuijpers, P. J., Miguel, C., Čihařová, M., Quero, S., Plessen, C. Y., Ebert, D. D., Harrer, M., van Straten, A., & Karyotaki, E. (2023) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2023.2166578 · Cited by 33
Zickfeld, J., Karg, S. T., Engen, S. S., Gonzalez, A. S. R., Michael, J., & Mitkidis, P. (2024) · Psychological Bulletin · doi:10.1037/bul0000429 · Cited by 33
The validity of situation awareness for performance: a meta-analysis
Bakdash, J. Z., Marusich, L. R., Cox, K. R., Geuss, M. N., Zaroukian, E. G., & Morris, K. (2021) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2021.1921310 · Cited by 29
Retrospective median power, false positive meta‐analysis and large‐scale replication
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1529 · Cited by 31
Where salience goes beyond authenticity
Chernikova, O., Holzberger, D., Heitzmann, N., Stadler, M. J., Seidel, T., & Fischer, F. (2023) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000357 · Cited by 40
Assessing the Evidential Value of Mental Fatigue and Exercise Research
Holgado, D., Mesquida, C., & Román-Caballero, R. (2023) · Sports Medicine · doi:10.1007/s40279-023-01926-w · Cited by 31
Cuijpers, P. J., Harrer, M., Miguel, C., Keshen, A. R., Karyotaki, E., & Linardon, J. (2024) · Eating Disorders · doi:10.1080/10640266.2024.2421057 · Cited by 26
Bylund, E., Antfolk, J., Abrahamsson, N., Olstad, A. M. H., Norrman, G., & Lehtonen, M. (2022) · Psychonomic Bulletin & Review · doi:10.3758/s13423-022-02136-7 · Cited by 26
The efficacy of psychotherapy for social anxiety disorder, a systematic review and meta-analysis
de Ponti, N., Matbouriahi, M., Franco, P., Harrer, M., Miguel, C., Papola, D., Sicimoğlu, A., Cuijpers, P. J., & Karyotaki, E. (2024) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2024.102881 · Cited by 30
Spontaneous Trait Inferences From Behavior: A Systematic Meta-Analysis
Bott, A., Brockmann, L., Denneberg, I., Henken, E., Kuper, N., Kruse, F., & Degner, J. (2022) · Personality and Social Psychology Bulletin · doi:10.1177/01461672221100336 · Cited by 29
A meta‐analysis on the effects of just‐below versus round prices
Troll, E. S., Frankenbach, J., Friese, M., & Loschelder, D. D. (2023) · Journal of Consumer Psychology · doi:10.1002/jcpy.1353 · Cited by 26
Action Effects on Visual Perception of Distances: A Multilevel Bayesian Meta-Analysis
Molto, L., Nalborczyk, L., Palluel-Germain, R., & Morgado, N. (2020) · Psychological Science · doi:10.1177/0956797619900336 · Cited by 26
Maier, M., Bartos̆, F., Oh, M., Wagenmakers, E., Shanks, D. R., & Harris, A. J. L. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/r8nyu · Cited by 28
Assessing robustness to worst case publication bias using a simple subset meta-analysis
Mathur, M. B. (2024) · BMJ · doi:10.1136/bmj-2023-076851 · Cited by 25
Self‐Determination Theory and Workplace Outcomes: A Meta‐Analysis
Hagger, M. S., & McAnally, K. (2026) · Stress and Health · doi:10.1002/smi.70151 · Cited by 33
Examining Psychological Science Through Systematic Meta-Method Analysis: A Call for Research
Elson, M. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919863296 · Cited by 25
Walker, S., Double, K. S., Birney, D. P., & MacCann, C. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2022.111622 · Cited by 26
P ‐hacking in meta‐analyses: A formalization and new meta‐analytic methods
Mathur, M. B. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1701 · Cited by 25
Intervention effects on physical activity identity: a systematic review and meta-analysis
Rhodes, R. E., Wierts, C. M., Kullman, S. M., Magel, E., & Strachan, S. M. (2024) · Health Psychology Review · doi:10.1080/17437199.2024.2412996 · Cited by 23
Fuchs, A., Radkowitsch, A., & Sommerhoff, D. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100648 · Cited by 23
Sorting the File Drawer: A Typology for Describing Unpublished Studies
Lishner, D. A. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620979831 · Cited by 24
Bonnechère, B., Rintala, A., Spooren, A. I. F., Lamers, I., & Feys, P. (2021) · Brain Sciences · doi:10.3390/brainsci11091187 · Cited by 23
Heterogeneity estimates in a biased world
Hönekopp, J., & Linden, A. H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262809 · Cited by 25
Remote Viewing: A 1974-2022 Systematic Review and Meta-Analysis
Tressoldi, P. E., & Katz, D. L. (2023) · Journal of Scientific Exploration · doi:10.31275/20232931 · Cited by 23
Seven steps toward transparency and replicability in psychological science.
Lindsay, D. S. (2020) · Canadian Psychology/Psychologie canadienne · doi:10.1037/cap0000222 · Cited by 23
Lewis, M. L., Mathur, M. B., VanderWeele, T. J., & Frank, M. C. (2022) · Royal Society Open Science · doi:10.1098/rsos.211499 · Cited by 26
Emotion Regulation and Aggression: A Systematic Review and Meta‐Analysis
Smith, K., Jones, A., Daly, N., Widdrington, H., Garofalo, C., & Gillespie, S. M. (2025) · Aggressive Behavior · doi:10.1002/ab.70055 · Cited by 29
Covert attention leads to fast and accurate decision-making.
Perkovic, S., Schoemann, M., Lagerkvist, C. J., & Orquin, J. L. (2022) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000425 · Cited by 22
The Replication Crisis and Open Science in Psychology
Renkewitz, F., & Heene, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000389 · Cited by 25
Creativity in the West and the East: A Meta-Analysis of Cross-Cultural Differences
Barth, P., & Stadtmann, G. (2024) · Creativity Research Journal · doi:10.1080/10400419.2024.2369442 · Cited by 25
Narcissism and Wellbeing: A Cross-Cultural Meta-Analysis
Sedikides, C., Tang, Y., Liu, Y., de Boer, E., Assink, M., Thomaes, S. C. E., & Brummelman, E. (2024) · Personality and Social Psychology Bulletin · doi:10.1177/01461672241307531 · Cited by 27
Alcaraz‐Ibáñez, M., Paterna, A., Griffiths, M. D., Demetrovics, Z., & Sicilia, Á. (2022) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2022.102280 · Cited by 24
Kung, K. T. F., Louie, K., Spencer, D., & Hines, M. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105616 · Cited by 23
Schütt, M. (2021) · Environmental and Resource Economics · doi:10.1007/s10640-021-00536-2 · Cited by 21
Meta-Analysis of the “Ironic” Effects of Intergroup ContactPreprint
Reimer, N. K., & Sengupta, N. K. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/vrsqe · Cited by 22
Maier, M., Wong, Y. C., & Feldman, G. (2024) · Collabra Psychology · doi:10.1525/collabra.90203 · Cited by 23
Changing the logic of replication: A case from infant studies
Margoni, F., & Shepperd, M. (2020) · Infant Behavior and Development · doi:10.1016/j.infbeh.2020.101483 · Cited by 22
Schizotypy, neuroticism, and saccadic eye movements: New data and meta‐analysis
Thomas, E. H. X., Steffens, M., Harms, C., Rossell, S. L., Gurvich, C., & Ettinger, U. (2020) · Psychophysiology · doi:10.1111/psyp.13706 · Cited by 20
Duggan, M., & Tressoldi, P. E. (2018) · F1000Research · doi:10.12688/f1000research.14330.2 · Cited by 22
Has Stereotype Threat Dissipated Over Time? A Cross-Temporal Meta-AnalysisPreprint
Lewis, N. A., & Michalak, N. M. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/w4ta2 · Cited by 25
Non-directive supportive therapy for depression: A meta-analytic review
Cuijpers, P. J., Miguel, C., Čihařová, M., Harrer, M., & Karyotaki, E. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.01.073 · Cited by 20
Domenicucci, R., Ferrandes, F., Sarlo, M., Borella, E., & Belacchi, C. (2022) · Ageing Research Reviews · doi:10.1016/j.arr.2022.101781 · Cited by 20
Deng, F., Bueber, M. A., Cao, Y., Tang, J., Bai, X., Cho, Y. I., Lee, J., Lin, Z., Yang, Q., Keshavan, M. S., et al. (2024) · Psychological Medicine · doi:10.1017/s0033291723003501 · Cited by 19
The Replication Database: Documenting the Replicability of Psychological Science
Röseler, L., Kaiser, L., Doetsch, C., Klett, N., Seida, C., Schütz, A., Aczél, B., Adelina, N., Agostini, V., Alarie, S., et al. (2024) · Journal of Open Psychology Data · doi:10.5334/jopd.101 · Cited by 9
An integrative framework for the mechanisms underlying mindfulness-induced cognitive change
Cásedas, L., Schooler, J. W., Vadillo, M. A., & Lupiáñez, J. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00374-1 · Cited by 19
Action Video-Game Training and Its Effects on Perception and Attentional Control
Green, C. S., Gorman, T. E., & Bavelier, D. (2016) · Springer eBooks · doi:10.1007/978-3-319-42662-4_10 · Cited by 22
Physical activity and sense of coherence: a meta-analysis
Moyers, S. A., & Hagger, M. S. (2020) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2020.1846068 · Cited by 24
Reimer, N. K., Hughes, J., Blaylock, D., Donnelly, C., Wölfer, R., & Hewstone, M. (2021) · Developmental Psychology · doi:10.1037/dev0001274 · Cited by 19
Identification of and Correction for Publication Bias
Andrews, I., & Kasy, M. (2017) · National Bureau of Economic Research · doi:10.3386/w23298 · Cited by 21
Loneliness and the Big Five Personality Traits: A Meta-AnalysisPreprint
Buecker, S., Maes, M., Denissen, J. J. A., & Luhmann, M. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/fx5bq · Cited by 20
Not Published Is Not Perished: Addressing Publication Bias in Meta-Analytic Studies in Communication
Sun, Y., & Pan, Z. (2020) · Human Communication Research · doi:10.1093/hcr/hqz015 · Cited by 18
Body image and risk of exercise addiction in adults: A systematic review and meta-analysis
Guo, S., Kamionka, A., Xue, Q., Izydorczyk, B., Lipowska, M., & Lipowski, M. (2025) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00085 · Cited by 22
Yang, H., Hu, L., Wang, H., & Xin, Y. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70079 · Cited by 20
Ney, L. J., O'Donohue, M. P., Lowe, B. G., & Lipp, O. V. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104756 · Cited by 18
Publication bias in psychology: A closer look at the correlation between sample size and effect size
Linden, A. H., Pollet, T. V., & Hönekopp, J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0297075 · Cited by 20
Humans adaptively select different computational strategies in different learning environments.
Verbeke, P., & Verguts, T. (2024) · Psychological Review · doi:10.1037/rev0000474 · Cited by 18
Replicating and extending the effects of auditory religious cues on dishonest behavior
Nichols, A. D., Lang, M., Kavanagh, C. M., Kundt, R., Yamada, J., Dan Ariely, C., & Mitkidis, P. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237007 · Cited by 21
Navas, J. F., Verdejo‐García, A., & Vadillo, M. A. (2021) · Obesity Reviews · doi:10.1111/obr.13338 · Cited by 19
Guo, L. (2024) · Mindfulness · doi:10.1007/s12671-024-02395-6 · Cited by 21
Evaluating the Replicability of Social Priming Studies
Giolla, E. M., Karlsson, S., Neequaye, D. A., & Bergquist, M. (2024) · Meta-Psychology · doi:10.15626/mp.2022.3308 · Cited by 23
Worku, B. T., Abdulahi, M., Amenu, D., & Bonnechère, B. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07286-9 · Cited by 17
A meta-analytic review of the relationship between cyber aggression and substance use
Crane, C. A., Wiernik, B. M., Berbary, C. M., Crawford, M., Schlauch, R. C., & Easton, C. J. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.108510 · Cited by 18
Modelling publication bias and p ‐hacking
Moss, J., & Bin, R. D. (2021) · Biometrics · doi:10.1111/biom.13560 · Cited by 21
Detecting Evidential Value and p-Hacking With the p-Curve Tool
Erdfelder, E., & Heck, D. W. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000383 · Cited by 18
The replication crisis has led to positive structural, procedural, and community changesPreprint
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M. M., Breznau, N., Robertson, O. M., Kalandadze, T., et al. (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/r6cvx · Cited by 18
Pawel, S., Bartos̆, F., Siepe, B. S., & Lohmann, A. (2025) · The American Statistician · doi:10.1080/00031305.2025.2540002 · Cited by 16
No meta-analytical effect of economic inequality on well-being or mental health
Sommet, N., Fillon, A. A., Rudmann, O., Cunha, A. R. S., & Ehsan, A. (2025) · Nature · doi:10.1038/s41586-025-09797-z · Cited by 19
Cuijpers, P. J., Miguel, C., Čihařová, M., Harrer, M., Moir, F., Roskvist, R. P., van Straten, A., Karyotaki, E., & Arroll, B. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.07.011 · Cited by 17
The current phase of social sciences research: A thematic overview of the literature
Babalola, S. S., & Nwanzu, C. L. (2021) · Cogent Social Sciences · doi:10.1080/23311886.2021.1892263 · Cited by 16
Optimizing the scientific study of suicide with open and transparent research practices
Carpenter, T. P., & Law, K. C. (2021) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12665 · Cited by 20
Cuijpers, P. J., Harrer, M., Miguel, C., Karyotaki, E., & Papola, D. (2026) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.121106 · Cited by 15
A meta-analysis on air traffic controllers selection: cognitive and non-cognitive predictors
Mouratille, D., Amadieu, F., & Matton, N. (2022) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2022.103769 · Cited by 16
Swinton, P. A., Shim, J., Pavlova, A. V., Moss, R. A., MacLean, C., Brandie, D., Mitchell, L., Greig, L., Parkinson, E., Brown, V. T., et al. (2023) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2022-001389 · Cited by 17
Ribeiro, L. A., Donolato, E., Aguiar, C., Correia, N. E. F. G., & Zachrisson, H. D. (2023) · Journal of Cognition and Development · doi:10.1080/15248372.2023.2248259 · Cited by 17
Mitchaϊ, P. M., Mapinduzi, J., Verbrugghe, J., Michiels, S., Janssens, L., Kossi, O., Bonnechère, B., & Timmermans, A. A. A. (2023) · European Spine Journal · doi:10.1007/s00586-023-07964-2 · Cited by 15
How (not) to measure replication
Fletcher, S. C. (2021) · European Journal for Philosophy of Science · doi:10.1007/s13194-021-00377-2 · Cited by 16
Accounting for intangible assets – insights from meta-analysis of R&D research
Jeny, A., & Moldovan, R. (2021) · Journal of Accounting Literature · doi:10.1108/jal-11-2021-0004 · Cited by 17
Sladekova, M., Webb, L. E. A., & Field, A. P. (2022) · Psychological Methods · doi:10.1037/met0000470 · Cited by 18
Strengths and Weaknesses of Meta-Analyses
Corker, K. S. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780190938550.003.0006 · Cited by 16
Bossert, L., Arzberger, K., Dorok, F., Kern, J., Stickler, C., Wunderlich, M., & Tran, U. S. (2023) · Mindfulness · doi:10.1007/s12671-023-02075-x · Cited by 15
Stanley, T. D., Ioannidis, J. P. A., Maier, M., Doucouliagos, H., Otte, W. M., & Bartos̆, F. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.03.004 · Cited by 18
p-Hacking and Publication Bias Interact to Distort Meta-Analytic Effect Size EstimatesPreprint
Friese, M., & Frankenbach, J. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/bvm58 · Cited by 15
The winner's curse and related perils of low statistical power − spelled out and illustrated
Lindstromberg, S. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100059 · Cited by 16
Grilling the data: application of specification curve analysis to red meat and all-cause mortality
Wang, Y., Pitre, T., Wallach, J. D., de Souza, R. J., Jassal, T., Bier, D. M., Patel, C. J., & Zeraatkar, D. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111278 · Cited by 14
Low Prevalence of A Priori Power Analyses in Motor Behavior Research
McKay, B., Corson, A. E., Vinh, M.-A., Jeyarajan, G., Tandon, C., Brooks, H., Hubley, J., & Carter, M. J. (2022) · Journal of Motor Learning and Development · doi:10.1123/jmld.2022-0042 · Cited by 14
Chen, L., Benjamin, R., Lai, A., & Heine, S. (2022) · doi:10.31234/osf.io/kuhy6 · Cited by 14
Renkewitz, F., & Keiner, M. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/w94ep · Cited by 15
Chao, C., & Weatherly, K. I. C. H. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100710 · Cited by 16
Levine, T. R., & Weber, R. (2020) · Human Communication Research · doi:10.1093/hcr/hqz019 · Cited by 13
Khudair, M., Marcuzzi, A., Ng, K., Tempest, G. D., Bartos̆, F., Peric, R., Maier, M., Beccia, F., Boccia, S., Brandes, M., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059202 · Cited by 14
Category learning in autistic individuals: A meta-analysis
Wimmer, L., Steininger, T. M., Schmid, A., & Wittwer, J. (2023) · Psychonomic Bulletin & Review · doi:10.3758/s13423-023-02365-4 · Cited by 14
Mathur, M. B. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1667 · Cited by 13
Preregistration of Psychology Meta-Analyses: A Cross-Sectional Study of Prevalence and Practice
Sandoval‐Lentisco, A., Tortajada, M., López‐Nicolás, R., López-López, J. A., Wagenmakers, E., Sánchez‐Meca, J., & Hardwicke, T. E. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241300113 · Cited by 14
Huang, S., & Ma, Q. (2025) · Learning and Individual Differences · doi:10.1016/j.lindif.2025.102752 · Cited by 13
Tressoldi, P. E., & Storm, L. (2021) · F1000Research · doi:10.12688/f1000research.51746.1 · Cited by 13
Metadata
- Authors
- Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J.
- Affiliations
- DEVCOM Army Research Laboratory (US)
- Ludwig-Maximilians-Universität München (DE)
- University of Kentucky (US)
- Illinois State University (US)
- Corresponding author
- Carter, Evan C.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2019-06
- Volume
- 2
- Issue
- 2
- Pages
- 115-144
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- OpenAlex
- W2781957993
- Cited by
- 588
- References
- 76
- Citations by year
- 2026: 43 · 2025: 60 · 2024: 69 · 2023: 86 · 2022: 83 · 2021: 87 · 2020: 79 · 2019: 54
- FWCI
- 50.69
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Advances in Methods and Practices in Psychological Science (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- publication bias, questionable research practices, effect size overestimation, meta-analytic methods, sensitivity analysis, heterogeneity, simulation study, pre-registered replication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
