Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Open accessPublished version of a preprint
Abstract
Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/ .
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The absence of publication bias is highly unlikely unless a set of preregistered reports is examined.
“The absence of publication bias is highly unlikely unless a set of preregistered reports is examined (Carter et al. 2019).”
From The gamification of learning: a meta-analysis (2020) · cites this paper as Carter et al. 2019
The trim-and-fill method tends to adjust too little when bias is strong, and Fail-Safe N neither tests nor adjusts for the presence of bias.
“The trim-and-fill method (Duval & Tweedie, 2000) tends to adjust too little when bias is strong (Carter et al., 2019), and Fail-Safe N (Rosenthal, 1979) neither tests nor adjusts for the presence of bias.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
Bias adjustments can improve performance relative to a meta-analysis that does not adjust for publication bias.
“Bias adjustments can improve performance relative to a meta-analysis that does not adjust for publication bias (Carter et al., 2019; McShane et al., 2016; Moreno et al., 2009; Simonsohn et al., 2014).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
Negative and statistically significant SPET estimates have been observed in both simulation studies and meta-analysis.
“negative and statistically significant SPET estimates have been observed in both simulation studies (Carter et al., 2019) and meta-analysis (Hilgard et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
The p-curve estimate of average study power may be an underestimate because p-hacking adds large p values to the p-curve, canceling out the right skew and biasing the estimate downward.
“On the one hand, this may be an underestimate of average study power—in simulation studies, p-hacking adds large p values to the p-curve, canceling out the right skew and biasing the p-curve estimate downward (Carter et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
SPET and SPEESE have a downward bias in the presence of p-hacking.
“SPET and SPEESE have a downward bias in the presence of p-hacking (Carter et al., 2019)”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
There may also be downward bias inflicted by p-hacking.
“There may also be downward bias inflicted by p-hacking (Carter et al., 2019).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Carter et al., 2019
PET-PEESE can underestimate the effect size when publication bias and p-hacking are present.
“Further, PET-PEESE can underestimate the effect size when there is publication bias and when p-hacking is present (Carter et al., 2019), which are two conditions likely affecting the literature.”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
The counterintuitive PET finding may be explained by PET being prone to producing imprecise and biased estimates under high heterogeneity, small sample sizes, and a large proportion of p-hacking.
“However, this counterintuitive finding may be explained by the fact that PET is prone to producing imprecise and biased estimates under high heterogeneity, small sample sizes, and a large proportion of p-hacking (Stanley & Doucouliagos, 2014; Stanley 2017; Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
Simulation studies by Carter et al. (2019) showed that WAAP-WLS can provide a reasonably unbiased estimate of the effect size with a high power to detect the effect given that a true underlying effect exists.
“Simulation studies by Carter et al. (2019) showed that WAAP-WLS can provide a reasonably unbiased estimate of the effect size with a high power to detect the effect given that a true underlying effect exists.”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al. (2019)
Selection model may underestimate effects in the presence of publication bias and p-hacking.
“This could lead selection model to underestimate effects in the presence of publication bias and p-hacking, as shown in simulation studies (Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
PET-PEESE can underestimate the size of small effects in the presence of p-hacking and publication bias.
“Lastly, PET-PEESE can underestimate the size of small effects in the presence of p-hacking and publication bias (Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
When effects are highly heterogeneous and sample sizes are small, PET-PEESE analysis is known to yield imprecise estimates.
“However, when effects are highly heterogeneous and when sample sizes are small, the PET-PEESE analysis is known to yield imprecise estimates (Stanley & Doucouliagos, 2014; Stanley, 2017; Carter et al., 2019).”
From Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature (2023) · cites this paper as Carter et al., 2019
Uncritical application of these methods can lead to inappropriate conclusions.
“there is a danger that uncritical application can lead to inappropriate conclusions being drawn 68,73,74”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [74]
Available evidence indicates that no single method performs better than others across all scenarios.
“the available evidence suggests that no single method outperforms others in all scenarios”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [74]
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Conducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553 · see citing works
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · see citing works
Ego depletion and the strength model of self-control: a meta-analysis
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010) · Psychological Bulletin · doi:10.1037/a0019486 · Cited by 2,265 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
Self-control as a limited resource: Regulatory depletion patterns.
Muraven, M., Tice, D. M., & Baumeister, R. F. (1998) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.74.3.774 · Cited by 2,099 · see citing works
Self-control as a limited resource: Regulatory depletion patterns.
Muraven, M., Tice, D. M., & Baumeister, R. F. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.3.774 · Cited by 1,724 · see citing works
Methods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685 · see citing works
One Hundred Years of Social Psychology Quantitatively Described
Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003) · Review of General Psychology · doi:10.1037/1089-2680.7.4.331 · Cited by 1,558 · see citing works
Publication bias in meta-analysis: Prevention, assessment and adjustments
Viechtbauer, W. (2007) · Psychometrika · doi:10.1007/s11336-006-1450-y · Cited by 1,507 · see citing works
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · see citing works
A Multilab Preregistered Replication of the Ego-Depletion Effect
Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H. J. E. M., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brand, M., Brewer, G., Bruyneel, S. D., et al. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616652873 · Cited by 1,097 · see citing works
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 2× · see citing works
“For example, if researchers strongly believe that an effect is real and positive, statistically non-significant or negative estimates of that effect may never be submitted for publication or may be rejected by reviewers and editors (Greenwald, 1975; Sterling, Rosenbaum, & Weinkam, 1995; Rothstein et al., 2006; Ferguso…”
BackgroundMany Labs 2: Investigating variation in replicability across samples and settings
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918810225 · Cited by 944 · see citing works
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916 · see citing works
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809 · see citing works
Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills.
Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., Green, C. S., & Bavelier, D. (2018) · Psychological Bulletin · doi:10.1037/bul0000130 · Cited by 807 · see citing works
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800 · see citing works
Correlational effect size benchmarks
Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015) · Journal of Applied Psychology · doi:10.1037/a0038047 · Cited by 768 · see citing works
The design of simulation studies in medical statistics
Burton, A., Altman, D. G., Royston, P. J., & HOLDER, R. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2673 · Cited by 755 · see citing works
<i>p</i> -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717 · cited 3× · see citing works
“A p-curve is the distribution of all statistically significant p-values from the set of studies of interest (i.e., ps < 0.05; Simonsohn et al., 2014).”
BackgroundMethodInfluentialPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007) · Statistics in Medicine · doi:10.1002/sim.2889 · Cited by 716 · see citing works
Incorporating variations in the quality of individual randomized trials into meta-analysis
Detsky, A. S., Naylor, C. D., O'Rourke, K., McGeer, A. J., & L'Abbé, K. A. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90085-2 · Cited by 640 · see citing works
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634 · see citing works
Ferguson, C. J., & Heene, M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459059 · Cited by 603 · see citing works
Reporting standards for research in psychology: why do we need them? What might they be?
APA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008) · American Psychologist · doi:10.1037/0003-066x.63.9.839 · Cited by 571 · see citing works
Carter, E. C., Kofler, L., Forster, D. E., & McCullough, M. E. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000083 · Cited by 539 · see citing works
Self-citationMeta-analysis, funnel plots and sensitivity analysis
Copas, J. (2000) · Biostatistics · doi:10.1093/biostatistics/1.3.247 · Cited by 503 · see citing works
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461 · see citing works
Pigott, T. D. (2012) · doi:10.1007/978-1-4614-2278-5 · Cited by 453 · see citing works
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453 · see citing works
Kühberger, A., Fritz, A., & Scherndl, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105825 · Cited by 399 · see citing works
Selection Models and the File Drawer Problem
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013012 · Cited by 399 · see citing works
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616662243 · Cited by 398 · see citing works
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373 · see citing works
Meta-assessment of bias in science
Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618569114 · Cited by 364 · see citing works
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000104 · Cited by 343 · see citing works
What meta-analyses reveal about the replicability of psychological research.
Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018) · Psychological Bulletin · doi:10.1037/bul0000169 · Cited by 307 · see citing works
Self-citationOn the Assessment of Monte Carlo Error in Simulation-Based Statistical Analyses
Koehler, E., Brown, E. R., & Haneuse, S. (2009) · The American Statistician · doi:10.1198/tast.2009.0030 · Cited by 303 · see citing works
Continuously Cumulating Meta-Analysis and Replicability
Braver, S. L., Thoemmes, F. J., & Rosenthal, R. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614529796 · Cited by 298 · see citing works
A re‐evaluation of fixed effect(s) meta‐analysis
Rice, K., Higgins, J. P. T., & Lumley, T. (2018) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12275 · Cited by 287 · see citing works
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265 · see citing works
Neither fixed nor random: weighted least squares meta‐analysis
Stanley, T. D., & Doucouliagos, H. (2015) · Statistics in Medicine · doi:10.1002/sim.6481 · Cited by 249 · see citing works
Hedges, L. V. (1984) · Journal of Educational Statistics · doi:10.3102/10769986009001061 · Cited by 241 · see citing works
Limitations of PET-PEESE and Other Meta-Analysis Methods
Stanley, T. D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617693062 · Cited by 229 · see citing works
Hedges, L., & Vevea, J. L. (1996) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986021004299 · Cited by 223 · see citing works
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219 · see citing works
Stanley, D. J., & Spence, J. R. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614528518 · Cited by 211 · see citing works
Neither fixed nor random: weighted least squares meta‐regression
Stanley, T. D., & Doucouliagos, H. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1211 · Cited by 205 · see citing works
Specification Curve: Descriptive and Inferential Statistics on All Reasonable SpecificationsPreprint
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2694998 · Cited by 199 · see citing works
van Elk, M., Matzke, D., Gronau, Q. F., Guan, M., Vandekerckhove, J., & Wagenmakers, E. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01365 · Cited by 192 · see citing works
Conceptualizing and evaluating the replication of research results
Fabrigar, L. R., & Wegener, D. T. (2015) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.009 · Cited by 167 · see citing works
Sample size in psychological research over the past 30 years
Marszalek, J. M., Barber, C., Kohlhart, J., & Cooper, B. H. (2011) · Perceptual and Motor Skills · doi:10.2466/03.11.pms.112.2.331-348 · Cited by 167 · see citing works
Hilgard, J., Engelhardt, C. R., & Rouder, J. N. (2017) · Psychological Bulletin · doi:10.1037/bul0000074 · Cited by 162 · see citing works
Self-citationDetecting and adjusting for small‐study effects in meta‐analysis
Rücker, G., Carpenter, J. R., & Schwarzer, G. (2011) · Biometrical Journal · doi:10.1002/bimj.201000151 · Cited by 132 · see citing works
Finding the power to reduce publication bias
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2017) · Statistics in Medicine · doi:10.1002/sim.7228 · Cited by 131 · see citing works
Underreporting in Psychology Experiments
Franco, A., Malhotra, N. A., & Simonovits, G. (2015) · Social Psychological and Personality Science · doi:10.1177/1948550615598377 · Cited by 119 · see citing works
Correcting for Publication Bias in a Meta-Analysis with the P-uniform* MethodPreprint
van Aert, R. C. M., & van Assen, M. A. L. M. (2018) · doi:10.31222/osf.io/zqjr9 · Cited by 108 · see citing works
Distributions of p -values smaller than .05 in psychology: what is going on?
Hartgerink, C. H. J., van Aert, R. C. M., Nuijten, M. B., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · PeerJ · doi:10.7717/peerj.1935 · Cited by 90 · see citing works
Boulesteix, A., Wilson, R., & Hapfelmeier, A. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0417-2 · Cited by 84 · see citing works
A Bayesian approach to mitigation of publication bias
Guan, M., & Vandekerckhove, J. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0868-6 · Cited by 59 · see citing works
A Likelihood-Based Sensitivity Analysis for Publication Bias in Meta-Analysis
Copas, J. B. (2012) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/j.1467-9876.2012.01049.x · Cited by 41 · see citing works
Meta‐analysis inside and outside particle physics: two traditions that should converge?
Baker, R., & Jackson, D. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1065 · Cited by 39 · see citing works
A Monte Carlo Analysis of Alternative Meta-Analysis Estimators in the Presence of Publication Bias
Reed, W. R. (2015) · Economics · doi:10.5018/economics-ejournal.ja.2015-30 · Cited by 39 · see citing works
van Erp, S., Verhagen, J., Grasman, R. P. P. P., & Wagenmakers, E. (2017) · doi:10.31234/osf.io/myu9c · Cited by 25 · see citing works
A Bayesian “Fill-In” Method for Correcting for Publication Bias in Meta-Analysis
Du, H., Liu, F., & Wang, L.-J. (2017) · Psychological methods · doi:10.1037/met0000164 · Cited by 24 · see citing works
Jackson, D. (2007) · Biometrics · doi:10.1111/j.1541-0420.2006.00663.x · Cited by 23 · see citing works
Eyo (2012) · American Journal of Applied Sciences · doi:10.3844/ajassp.2012.1512.1517 · Cited by 18 · see citing works
A Simple, Principled Approach to Combining Evidence From Meta-Analysis and High-Quality Replications
Carter, E. C., & McCullough, M. E. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918756858 · Cited by 12 · see citing works
Self-citationHeterogeneity: multiplicative, additive or both?
Schmid, C. H. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1223 · Cited by 5 · see citing works
Dissertation R.C.M. van AertPreprint
van Aert, R. C. M. (2018) · doi:10.31222/osf.io/eqhjd · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962 · see citing works
The gamification of learning: a meta-analysis
Sailer, M., & Homner, L. (2020) · Educational Psychology Review · doi:10.1007/s10648-019-09498-w · Cited by 1,458 · see citing works
Simulation-Based Learning in Higher Education: A Meta-Analysis
Chernikova, O., Heitzmann, N., Stadler, M., Holzberger, D., Seidel, T., & Fischer, F. (2020) · Review of Educational Research · doi:10.3102/0034654320933544 · Cited by 1,072 · see citing works
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
Self-citationIs bilingualism associated with enhanced executive functioning in adults? A meta-analytic review.
Lehtonen, M., Soveri, A., Laine, A., Järvenpää, J., de Bruin, A., & Antfolk, J. (2018) · Psychological Bulletin · doi:10.1037/bul0000142 · Cited by 646 · see citing works
Rodgers, M. A., & Pustejovsky, J. E. (2021) · Psychological Methods · doi:10.1037/met0000300 · Cited by 492 · see citing works
A meta-analysis of the health action process approach.
Zhang, C.-Q., Zhang, R., Schwarzer, R., & Hagger, M. S. (2019) · Health Psychology · doi:10.1037/hea0000728 · Cited by 485 · see citing works
Methods for testing publication bias in ecological and evolutionary meta‐analyses
Nakagawa, S., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Sánchez‐Tójar, A., Yang, Y., & O’Dea, R. E. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13724 · Cited by 455 · see citing works
Music therapy for stress reduction: a systematic review and meta-analysis
de Witte, M., Pinho, A. D. S., Stams, G.-J., Moonen, X., Bos, A. E. R., & van Hooren, S. (2022) · Health Psychology Review · doi:10.1080/17437199.2020.1846580 · Cited by 444 · see citing works
Sexual orientation concealment and mental health: A conceptual and meta-analytic review.
Pachankis, J. E., Mahon, C. P., Jackson, S. D., Fetzner, B. K., & Bränström, R. (2020) · Psychological Bulletin · doi:10.1037/bul0000271 · Cited by 433 · see citing works
Evaluation of a decided sample size in machine learning applications
Rajput, D., Wang, W.-J., & Chen, C.-C. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05156-9 · Cited by 411 · see citing works
Is Ego Depletion Real? An Analysis of Arguments
Friese, M., Loschelder, D. D., Gieseler, K., Frankenbach, J., & Inzlicht, M. (2018) · Personality and Social Psychology Review · doi:10.1177/1088868318762183 · Cited by 404 · see citing works
Loneliness and the Big Five Personality Traits: A Meta–Analysis
Buecker, S., Maes, M., Denissen, J. J. A., & Luhmann, M. (2020) · European Journal of Personality · doi:10.1002/per.2229 · Cited by 385 · see citing works
The association between weight stigma and mental health: A meta‐analysis
Emmer, C., Bošnjak, M., & Mata, J. (2019) · Obesity Reviews · doi:10.1111/obr.12935 · Cited by 381 · see citing works
Donald, J. N., Sahdra, B. K., Van Zanden, B., Duineveld, J. J., Atkins, P. W. B., Marshall, S. L., & Ciarrochi, J. (2019) · British Journal of Psychology · doi:10.1111/bjop.12338 · Cited by 371 · see citing works
Macnamara, B. N., & Burgoyne, A. P. (2022) · Psychological Bulletin · doi:10.1037/bul0000352 · Cited by 358 · see citing works
Cuijpers, P., Miguel, C., Harrer, M., Plessen, C. Y., Ciharova, M., Ebert, D., & Karyotaki, E. (2023) · World Psychiatry · doi:10.1002/wps.21069 · Cited by 354 · see citing works
Testing for funnel plot asymmetry of standardized mean differences
Pustejovsky, J. E., & Rodgers, M. A. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1332 · Cited by 354 · see citing works
Association of Drug Cues and Craving With Drug Use and Relapse
Vafaie, N., & Hedy Kober, A. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.1240 · Cited by 353 · see citing works
Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215052 · Cited by 345 · see citing works
Testing (quizzing) boosts classroom learning: a systematic and meta-analytic review
Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021) · Psychological Bulletin · doi:10.1037/bul0000309 · Cited by 333 · see citing works
Joel, S., Eastwick, P. W., Allison, C. J., Arriaga, X. B., Baker, Z. G., Bar-Kalifa, E., Bergeron, S., Birnbaum, G. E., Brock, R. L., Brumbaugh, C. C., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1917036117 · Cited by 315 · see citing works
Is bilingualism related to a cognitive advantage in children? A systematic review and meta-analysis.
Gunnerud, H. L., ten Braak, D., Reikerås, E. K. L., Donolato, E., & Melby‐Lervåg, M. (2020) · Psychological Bulletin · doi:10.1037/bul0000301 · Cited by 303 · see citing works
Agostini, M., & van Zomeren, M. (2021) · Psychological Bulletin · doi:10.1037/bul0000256 · Cited by 301 · see citing works
Utesch, T., Bardid, F., Büsch, D., & Strauß, B. (2019) · Sports Medicine · doi:10.1007/s40279-019-01068-y · Cited by 301 · see citing works
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294 · see citing works
A tutorial on Bayes Factor Design Analysis using an informed prior
Stefan, A. M., Gronau, Q. F., Schönbrodt, F. D., & Wagenmakers, E.-J. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-01189-8 · Cited by 284 · see citing works
Self-citationComparing meta-analyses and preregistered multiple-laboratory replication projects
Kvarven, A., Strømland, E., & Johannesson, M. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0787-z · Cited by 271 · see citing works
Promoting memory consolidation during sleep: A meta-analysis of targeted memory reactivation
Hu, X., Cheng, L. Y., Chiu, M. H., & Paller, K. A. (2020) · Psychological Bulletin · doi:10.1037/bul0000223 · Cited by 232 · see citing works
Social Value Orientation, Expectations, and Cooperation in Social Dilemmas: A Meta–Analysis
Pletzer, J. L., Balliet, D. P., Joireman, J. A., Kuhlman, D. M., Voelpel, S. C., & Lange, P. A. M. V. (2018) · European Journal of Personality · doi:10.1002/per.2139 · Cited by 228 · see citing works
A century of research on conscientiousness at work
Wilmot, M. P., & Ones, D. S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1908430116 · Cited by 215 · see citing works
Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09595-9 · Cited by 211 · see citing works
Trait mindfulness and health behaviours: a meta-analysis
Sala, M., Rochefort, C., Lui, P. P., & Baldwin, A. S. (2020) · Health Psychology Review · doi:10.1080/17437199.2019.1650290 · Cited by 199 · see citing works
A meta-analysis of factors related to recycling
Geiger, J. L., Steg, L., van der Werff, E., & Ünal, A. B. (2019) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2019.05.004 · Cited by 199 · see citing works
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 190 · see citing works
Clayson, P. E., Carbine, K. A., Baldwin, S. A., & Larson, M. J. (2019) · Psychophysiology · doi:10.1111/psyp.13437 · Cited by 186 · see citing works
The Past, Present, and Future of Ego Depletion
Inzlicht, M., & Friese, M. (2019) · Social Psychology · doi:10.1027/1864-9335/a000398 · Cited by 180 · see citing works
Association of Testosterone Treatment With Alleviation of Depressive Symptoms in Men
Walther, A., Breidenstein, J., & Miller, R. E. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.2734 · Cited by 178 · see citing works
Galuschka, K., Görgen, R., Kalmar, J., Haberstroh, S., Schmalz, X., & Schulte‐Körne, G. (2020) · Educational Psychologist · doi:10.1080/00461520.2019.1659794 · Cited by 176 · see citing works
Morgenroth, T., Kirby, T. A., Ryan, M. K., & Sudkämper, A. (2020) · Psychological Bulletin · doi:10.1037/bul0000234 · Cited by 169 · see citing works
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165 · see citing works
An extended theory of planned behavior for parent-for-child health behaviors: A meta-analysis.
Hamilton, K., van Dongen, A., & Hagger, M. S. (2020) · Health Psychology · doi:10.1037/hea0000940 · Cited by 164 · see citing works
Lessons From Pinocchio: Cues to Deception May Be Highly Exaggerated
Luke, T. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691619838258 · Cited by 135 · see citing works
An Aberrant Abundance of Cronbach’s Alpha Values at .70
Hussey, I., Alsalti, T., Bosco, F. A., Elson, M., & Arslan, R. C. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241287123 · Cited by 158 · see citing works
An umbrella review of randomized control trials on the effects of physical exercise on cognition
Ciria, L. F., Román-Caballero, R., Vadillo, M. A., Holgado, D., Luque-Casado, A., Perakakis, P., & Sanabria, D. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01554-4 · Cited by 157 · see citing works
Internalization of body shape ideals and body dissatisfaction: A systematic review and meta‐analysis
Paterna, A., Alcaraz‐Ibáñez, M., Fuller‐Tyszkiewicz, M., & Sicilia, Á. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23568 · Cited by 157 · see citing works
A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration
Coles, N. A., March, D. S., Marmolejo‐Ramos, F., Larsen, J. T., Arinze, N. C., Ndukaihe, I. L. G., Willis, M. L., Foroni, F., Reggev, N., Mokady, A., et al. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01458-9 · Cited by 152 · see citing works
Täschner, J., Dicke, T., Reinhold, S., & Holzberger, D. (2024) · Review of Educational Research · doi:10.3102/00346543231221499 · Cited by 154 · see citing works
Wartenberg, G., Aldrup, K., Grund, S., & Klusmann, U. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09831-4 · Cited by 151 · see citing works
Big little lies: a compendium and simulation of p -hacking strategies
Stefan, A. M., & Schönbrodt, F. D. (2023) · Royal Society Open Science · doi:10.1098/rsos.220346 · Cited by 149 · see citing works
Self-citationRobust Bayesian meta-analysis: Addressing publication bias with model-averaging.
Maier, M., Bartos̆, F., & Wagenmakers, E. (2022) · Psychological Methods · doi:10.1037/met0000405 · Cited by 144 · see citing works
Longitudinal tests of the theory of planned behaviour: A meta-analysis
Hagger, M. S., & Hamilton, K. (2023) · European Review of Social Psychology · doi:10.1080/10463283.2023.2225897 · Cited by 143 · see citing works
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 138 · see citing works
The Replication Crisis Has Led to Positive Structural, Procedural, and Community Changes
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M., Breznau, N., Robertson, O., Kalandadze, T., et al. (2023) · Communications Psychology · doi:10.1038/s44271-023-00003-2 · Cited by 137 · see citing works
Long-term Persuasive Effects in Narrative Communication Research: A Meta-Analysis
Oschatz, C., & Marker, C. (2020) · Journal of Communication · doi:10.1093/joc/jqaa017 · Cited by 120 · see citing works
Agreeableness and Its Consequences: A Quantitative Review of Meta-Analytic Findings
Wilmot, M. P., & Ones, D. S. (2022) · Personality and Social Psychology Review · doi:10.1177/10888683211073007 · Cited by 130 · see citing works
Ironic Effects of Thought Suppression: A Meta-Analysis
Wang, D., Hagger, M. S., & Chatzisarantis, N. L. D. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691619898795 · Cited by 128 · see citing works
Do ‘watching eyes’ influence antisocial behavior? A systematic review & meta-analysis
Dear, K., Dutton, K., & Fox, E. (2019) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2019.01.006 · Cited by 127 · see citing works
Almaatouq, A., Griffiths, T. L., Suchow, J. W., Whiting, M. E., Evans, J., & Watts, D. J. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x22002874 · Cited by 126 · see citing works
P-hacking and publication bias interact to distort meta-analytic effect size estimates
Friese, M., & Frankenbach, J. (2020) · Psychological Methods · doi:10.1037/met0000246 · Cited by 122 · see citing works
Current practices in meta‐regression in psychology, education, and medicine
Tipton, E., Pustejovsky, J. E., & Ahmadi, H. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1339 · Cited by 122 · see citing works
Replication is important for educational psychology: Recent developments and key issues
Plucker, J. A., & Makel, M. C. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1895796 · Cited by 122 · see citing works
Siegel, R., König, C. J., & Lazar, V. (2022) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2022.100227 · Cited by 121 · see citing works
The Relationships of Intergroup Ideologies to Ethnic Prejudice: A Meta-Analysis
Whitley, B. E., & Webster, G. D. (2018) · Personality and Social Psychology Review · doi:10.1177/1088868318761423 · Cited by 121 · see citing works
The use of commercial computerised cognitive games in older adults: a meta-analysis
Bonnechère, B., Langley, C., & Sahakian, B. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-72281-3 · Cited by 120 · see citing works
Is personality associated with dementia risk? A meta-analytic investigation
Aschwanden, D., Strickhouser, J. E., Luchetti, M., Stéphan, Y., Sutin, A. R., & Terracciano, A. (2021) · Ageing Research Reviews · doi:10.1016/j.arr.2021.101269 · Cited by 120 · see citing works
Román-Caballero, R., Vadillo, M. A., Trainor, L. J., & Lupiáñez, J. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100436 · Cited by 118 · see citing works
Aging and recognition memory: A meta-analysis.
Fraundorf, S. H., Hourihan, K. L., Peters, R. A., & Benjamin, A. S. (2019) · Psychological Bulletin · doi:10.1037/bul0000185 · Cited by 118 · see citing works
Spaniol, M., & Danielsson, H. (2021) · Journal of Intellectual Disability Research · doi:10.1111/jir.12878 · Cited by 118 · see citing works
Intuitive Honesty Versus Dishonesty: Meta-Analytic Evidence
Köbis, N., Verschuère, B., Bereby‐Meyer, Y., Rand, D. G., & Shalvi, S. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619851778 · Cited by 117 · see citing works
Pair-bonding, fatherhood, and the role of testosterone: a meta-analytic review
Grebe, N. M., Sarafin, R. E., Strenth, C. R., & Zilioli, S. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.01.010 · Cited by 116 · see citing works
Oser, J. L., Grinson, A., Boulianne, S., & Halperin, E. (2022) · Political Communication · doi:10.1080/10584609.2022.2086329 · Cited by 116 · see citing works
Bartos̆, F., Maier, M., Quintana, D. S., & Wagenmakers, E. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221109259 · Cited by 115 · see citing works
Aulbach, M. B., Knittle, K. P., & Haukkala, A. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1571933 · Cited by 114 · see citing works
Rhodes, R. E., Gray, S. M., & Husband, C. (2018) · Health Psychology Review · doi:10.1080/17437199.2018.1530067 · Cited by 113 · see citing works
Lin, H., Saunders, B., Friese, M., Evans, N. J., & Inzlicht, M. (2020) · Psychological Science · doi:10.1177/0956797620904990 · Cited by 111 · see citing works
McBreen, M., & Savage, R. S. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09584-4 · Cited by 110 · see citing works
Correcting for publication bias in a meta-analysis with the p-uniform* method
van Aert, R. C. M., & van Assen, M. A. L. M. (2026) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02812-4 · Cited by 108 · see citing works
Nudging Social Media toward Accuracy
Pennycook, G., & Rand, D. G. (2022) · The ANNALS of the American Academy of Political and Social Science · doi:10.1177/00027162221092342 · Cited by 106 · see citing works
Meta-Analysis of Cognition in Parkinson’s Disease Mild Cognitive Impairment and Dementia Progression
Wallace, E. R., Segerstrom, S. C., van Horne, C. G., Schmitt, F. A., & Koehl, L. (2021) · Neuropsychology Review · doi:10.1007/s11065-021-09502-7 · Cited by 98 · see citing works
Effects of habit and intention on behavior: Meta-analysis and test of key moderators.
Hagger, M. S., Hamilton, K., Phipps, D. J., Protogerou, C., Zhang, C., Girelli, L., Mallia, L., & Lucidi, F. (2023) · Motivation Science · doi:10.1037/mot0000294 · Cited by 97 · see citing works
Effectiveness of conceptual change strategies in science education: A meta‐analysis
Paçacı, Ç., Üstün, U., & Özdemir, Ö. F. (2023) · Journal of Research in Science Teaching · doi:10.1002/tea.21887 · Cited by 96 · see citing works
Meta-Analysis of Present-Bias Estimation using Convex Time Budgets
Imai, T., Rutter, T. A., & Camerer, C. F. (2020) · The Economic Journal · doi:10.1093/ej/ueaa115 · Cited by 94 · see citing works
Hagger, M. S., & Hamilton, K. (2020) · European Journal of Personality · doi:10.1177/0890207020962330 · Cited by 91 · see citing works
When is narcissism associated with low empathy? A meta-analytic review
Urbonaviciute, G., & Hepper, E. G. (2020) · Journal of Research in Personality · doi:10.1016/j.jrp.2020.104036 · Cited by 91 · see citing works
Corruption Information and Vote Share: A Meta-Analysis and Lessons for Experimental Design
Incerti, T. (2020) · American Political Science Review · doi:10.1017/s000305542000012x · Cited by 89 · see citing works
Handling effect size dependency in meta-analysis
Gucciardi, D. F., Lines, R. L. J., & Ntoumanis, N. (2021) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1946835 · Cited by 89 · see citing works
McAuliffe, W. H. B., Carter, E. C., Berhane, J., Snihur, A., & McCullough, M. E. (2019) · Personality and Social Psychology Review · doi:10.1177/1088868319887599 · Cited by 87 · see citing works
Self-citationHow Strongly Is Personality Associated With Burnout Among Teachers? A Meta-analysis
Roloff, J., Kirstges, J., Grund, S., & Klusmann, U. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09672-7 · Cited by 86 · see citing works
Putting the Self in Self-Correction: Findings From the Loss-of-Confidence Project
Rohrer, J. M., Tierney, W., Uhlmann, E. L., DeBruine, L. M., Heyman, T., Jones, B. C., Schmukle, S. C., Silberzahn, R., Willén, R. M., Carlsson, R., et al. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620964106 · Cited by 85 · see citing works
Unemployment and health: A meta‐analysis
Picchio, M., & Ubaldi, M. (2023) · Journal of Economic Surveys · doi:10.1111/joes.12588 · Cited by 85 · see citing works
Problematic smartphone use and academic achievement: A systematic review and meta-analysis
Paterna, A., Alcaraz-Ibáñez, M., Aguilar-Parra, J. M., Salavera, C., Demetrovics, Z., & Griffiths, M. D. (2024) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00014 · Cited by 83 · see citing works
A comprehensive meta-analysis of money priming.
Lodder, P., Ong, H. H., Grasman, R. P. P. P., & Wicherts, J. M. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000570 · Cited by 83 · see citing works
Physical activity and cortisol regulation: A meta-analysis
Moyers, S. A., & Hagger, M. S. (2023) · Biological Psychology · doi:10.1016/j.biopsycho.2023.108548 · Cited by 83 · see citing works
A systematic review and meta-analysis of the evidence for unaware fear conditioning
Mertens, G., & Engelhard, I. M. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.11.012 · Cited by 81 · see citing works
Gieseler, K., Loschelder, D. D., & Friese, M. (2019) · Springer eBooks · doi:10.1007/978-3-030-13788-5_1 · Cited by 79 · see citing works
Detecting publication selection bias through excess statistical significance
Stanley, T. D., Doucouliagos, H., Ioannidis, J. P. A., & Carter, E. C. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1512 · Cited by 79 · see citing works
Self-citationHeterogeneity in direct replications in psychology and its association with effect size.
Olsson-Collentine, A., Wicherts, J. M., & van Assen, M. A. L. M. (2020) · Psychological Bulletin · doi:10.1037/bul0000294 · Cited by 78 · see citing works
Systematic Review and Meta-Analysis of Correlates of FFMQ Mindfulness Facets
Mattes, J. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02684 · Cited by 77 · see citing works
- Authors
- Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J.
- Affiliations
- DEVCOM Army Research Laboratory (US)
- Ludwig-Maximilians-Universität München (DE)
- University of Kentucky (US)
- Illinois State University (US)
- Corresponding author
- Carter, Evan C.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2019-06
- Volume
- 2
- Issue
- 2
- Pages
- 115-144
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- OpenAlex
- W2781957993
- Cited by
- 588
- References
- 76
- Citations by year
- 2026: 43 · 2025: 60 · 2024: 69 · 2023: 86 · 2022: 83 · 2021: 87 · 2020: 79 · 2019: 54 · 2018: 23 · 2017: 3 · 2016: 1
- FWCI
- 50.69
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Advances in Methods and Practices in Psychological Science (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- publication bias, questionable research practices, effect size overestimation, meta-analytic methods, sensitivity analysis, heterogeneity, simulation study, pre-registered replication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
