The DOI matches its Crossref record, with a small difference in title or year; 49 of the 49 references it cites were found in a registry or an index.
Adjusting for Publication Bias in Meta-Analysis: An Evaluation of Selection Methods and Some Cautionary Notes
Open accessAbstract
We review and evaluate selection methods, a prominent class of techniques first proposed by Hedges (1984) that assess and adjust for publication bias in meta-analysis, via an extensive simulation study. Our simulation covers both restrictive settings as well as more realistic settings and proceeds across multiple metrics that assess different aspects of model performance. This evaluation is timely in light of two recently proposed approaches, the so-called p-curve and p-uniform approaches, that can be viewed as alternative implementations of the original Hedges selection method approach. We find that the p-curve and p-uniform approaches perform reasonably well but not as well as the original Hedges approach in the restrictive setting for which all three were designed. We also find they perform poorly in more realistic settings, whereas variants of the Hedges approach perform well. We conclude by urging caution in the application of selection methods: Given the idealistic model assumptions underlying selection methods and the sensitivity of population average effect size estimates to them, we advocate that selection methods should be used less for obtaining a single estimate that purports to adjust for publication bias ex post and more for sensitivity analysis—that is, exploring the range of estimates that result from assuming different forms of and severity of publication bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Consistent with those findings, the relative probabilities estimated by selection method analyses (e.g., McShane et al., 2016) also suggest a general lack of publication bias.”
From Examining domains of psychological flexibility and inflexibility as treatment mechanisms in acceptance and commitment therapy: A comprehensive systematic and meta-analytic review (2024) · cites this paper as McShane et al., 2016
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the other hand, this could also be an overestimate of average study power, as heterogeneity in effect sizes can cause the p-curve to overestimate mean power (Brunner & Schimmack, 2020; McShane et al., 2016).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as McShane et al., 2016
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we ran selection method analyses for each meta-analytic effect using R functions developed and validated by McShane et al. (2016).”
From Examining domains of psychological flexibility and inflexibility as treatment mechanisms in acceptance and commitment therapy: A comprehensive systematic and meta-analytic review (2024) · cites this paper as McShane et al., 2016
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, the credibility of such results is contingent on a lack of selective reporting or on the ability to effectively correct for missing evidence (McShane et al., 2016; Stanley and Doucouliagos, 2014).”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as McShane et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“In general, cumulative or meta-analytic evidence obtained from multiple independently conducted experiments provides a better basis for assessing the reliability of findings than evidence from a single experiment.”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as McShane et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“p-uniform does not model heterogeneity, and this can lead it to overestimate the average effect size of all studies (McShane et al., 2016).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as McShane et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“Bias adjustments can improve performance relative to a meta-analysis that does not adjust for publication bias (Carter et al., 2019; McShane et al., 2016; Moreno et al., 2009; Simonsohn et al., 2014).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as McShane et al., 2016
In the citing paper
“Statisticians have proposed several statistical approaches to assess how robust meta-analyses are to various assumptions about the extent and nature of missing results, including trim-andfill 67 , selection models 68 , and regression-based adjustment methods 69 .”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [69]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 2×
“…& Mazumdar, 1994; Egger, Smith, Schneider, & Minder, 1997; Sterne & Egger, 2005), Rosenthal’s fail-safe N method and its many variants (Becker, 2005; Rosenthal, 1979), the trim-and-fill method (Duval, 2005; Duval & Tweedie, 2000a, 2000b), and the Precision-Effect Test– Precision-Effect Estimate…”
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Research works printed but not linked: 4
- Boyle R., 1965, The sceptical chymist
- Ferriar J., 1792, Medical histories and reflection
- Fisher Ronald A, 1938, Sankhya, 4, 14
- Sterling T. D., 1959, Journal of the American Statistical Association, 54, 30
Cited by
Sorted by citations
Salk, R. H., Hyde, J. S., & Abramson, L. Y. (2017) · Psychological Bulletin · doi:10.1037/bul0000102 · Cited by 2,622
Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation
Chan, M.-P. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017) · Psychological Science · doi:10.1177/0956797617714579 · Cited by 898
A social comparison theory meta-analysis 60+ years on.
Gerber, J. P., Wheeler, L., & Suls, J. (2018) · Psychological Bulletin · doi:10.1037/bul0000127 · Cited by 808
Self-regulation in childhood as a predictor of future outcomes: A meta-analytic review.
Robson, D. A., Allen, M. S., & Howard, S. J. (2020) · Psychological Bulletin · doi:10.1037/bul0000227 · Cited by 759
Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis
Shrout, P. E., & Rodgers, J. L. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011845 · Cited by 830
O’Dea, R. E., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Gurevitch, J., Page, M. J., Stewart, G. B., Moher, D., et al. (2021) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12721 · Cited by 825
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588
Siegel, E. H., Sands, M. K., Van den Noortgate, W., Condon, P., Chang, Y., Dy, J., Quigley, K. S., & Barrett, L. F. (2018) · Psychological Bulletin · doi:10.1037/bul0000128 · Cited by 515
Wrzus, C., & Neubauer, A. B. (2023) · Assessment · doi:10.1177/10731911211067538 · Cited by 500
Single-Paper Meta-Analysis: Benefits for Study Summary, Theory Testing, and Replicability
McShane, B. B., & Böckenholt, U. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucw085 · Cited by 474
Self-citationMethods for testing publication bias in ecological and evolutionary meta‐analyses
Nakagawa, S., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Sánchez‐Tójar, A., Yang, Y., & O'Dea, R. E. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13724 · Cited by 399
Zell, E., Strickhouser, J. E., Sedikides, C., & Alicke, M. D. (2019) · Psychological Bulletin · doi:10.1037/bul0000218 · Cited by 439
Is Ego Depletion Real? An Analysis of Arguments
Friese, M., Loschelder, D. D., Gieseler, K., Frankenbach, J., & Inzlicht, M. (2018) · Personality and Social Psychology Review · doi:10.1177/1088868318762183 · Cited by 438
A meta-analysis of the problematic social media use and mental health
Huang, C. (2022) · International Journal of Social Psychiatry · doi:10.1177/0020764020978434 · Cited by 399
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
“However, the credibility of such results is contingent on a lack of selective reporting or on the ability to effectively correct for missing evidence (McShane et al., 2016; Stanley and Doucouliagos, 2014).”
BackgroundDonald, J. N., Sahdra, B. K., Van Zanden, B., Duineveld, J. J., Atkins, P. W. B., Marshall, S. L., & Ciarrochi, J. (2019) · British Journal of Psychology · doi:10.1111/bjop.12338 · Cited by 371
Cuijpers, P., Miguel, C., Harrer, M., Plessen, C. Y., Ciharova, M., Ebert, D., & Karyotaki, E. (2023) · World Psychiatry · doi:10.1002/wps.21069 · Cited by 354
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354
Testing for funnel plot asymmetry of standardized mean differences
Pustejovsky, J. E., & Rodgers, M. A. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1332 · Cited by 354
Assessing risk of bias due to missing results in a synthesis
Page, M. J., Higgins, J. P., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch13 · Cited by 402
Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215052 · Cited by 365
Testing (quizzing) boosts classroom learning: a systematic and meta-analytic review
Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021) · Psychological Bulletin · doi:10.1037/bul0000309 · Cited by 333
Risk factors for child sexual abuse victimization: A meta-analytic review.
Assink, M., van der Put, C. E., Meeuwsen, M. W. C. M., de Jong, N. M., Oort, F. J., Stams, G. J. J. M., & Hoeve, M. (2019) · Psychological bulletin · doi:10.1037/bul0000188 · Cited by 322
Cognitive Training Does Not Enhance General Cognition
Sala, G., & Gobet, F. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.10.004 · Cited by 303
The impact of shared book reading on children's language skills: A meta-analysis
Noble, C. H., Sala, G., Peter, M., Lingwood, J., Rowland, C. F., Gobet, F. R., & Pine, J. M. (2019) · Educational Research Review · doi:10.1016/j.edurev.2019.100290 · Cited by 326
Agostini, M., & van Zomeren, M. (2021) · Psychological Bulletin · doi:10.1037/bul0000256 · Cited by 301
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294
Persistence and Fade-Out of Educational-Intervention Effects: Mechanisms and Potential Solutions
Bailey, D. H., Duncan, G. J., Cunha, F., Foorman, B. R., & Yeager, D. S. (2020) · Gothic.net · doi:10.1177/1529100620915848 · Cited by 336
Comparing meta-analyses and preregistered multiple-laboratory replication projects
Kvarven, A., Strømland, E., & Johannesson, M. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0787-z · Cited by 271
The development of gender differences in spatial reasoning: A meta-analytic review.
Lauer, J. E., Yhang, E., & Lourenco, S. F. (2019) · Psychological Bulletin · doi:10.1037/bul0000191 · Cited by 239
Promoting memory consolidation during sleep: A meta-analysis of targeted memory reactivation
Hu, X., Cheng, L. Y., Chiu, M. H., & Paller, K. A. (2020) · Psychological Bulletin · doi:10.1037/bul0000223 · Cited by 232
What meta-analyses reveal about the replicability of psychological research.
Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018) · Psychological Bulletin · doi:10.1037/bul0000169 · Cited by 307
Limitations of PET-PEESE and Other Meta-Analysis Methods
Stanley, T. D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617693062 · Cited by 229
Power to Detect What? Considerations for Planning and Evaluating Sample Size
Giner‐Sorolla, R., Montoya, A. K., Reifman, A. S., Carpenter, T., Lewis, N. A., Aberson, C. L., Bostyn, D. H., Conrique, B. G., Ng, B., Schoemann, A. M., et al. (2024) · Personality and Social Psychology Review · doi:10.1177/10888683241228328 · Cited by 324
Tourism and Economic Growth: A Meta-regression Analysis
Nunkoo, R., Seetanah, B., Jaffur, Z. K., Moraghen, P. G. W., & Sannassee, R. V. (2019) · Journal of Travel Research · doi:10.1177/0047287519844833 · Cited by 245
Palmer, C. A., Bower, J. L., Cho, K. W., Clementi, M. A., Lau, S., Oosterhoff, B. J., & Alfano, C. A. (2023) · Psychological Bulletin · doi:10.1037/bul0000410 · Cited by 251
Debray, T. P. A., Moons, K. G. M., & Riley, R. D. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1266 · Cited by 212
Country differences in the relationship between leadership and employee engagement: A meta-analysis
Li, P., Sun, J., Taris, T. W., Xing, L., & Peeters, M. C. W. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101458 · Cited by 235
Galuschka, K., Görgen, R., Kalmar, J., Haberstroh, S., Schmalz, X., & Schulte‐Körne, G. (2020) · Educational Psychologist · doi:10.1080/00461520.2019.1659794 · Cited by 198
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165
Privacy concerns in e‐commerce: A multilevel meta‐analysis
Maseeh, H. I., Jebarajakirthy, C., Pentecost, R. D., Arli, D. I., Weaven, S. K., & Ashaduzzaman, M. (2021) · Psychology and Marketing · doi:10.1002/mar.21493 · Cited by 162
Morgenroth, T., Kirby, T. A., Ryan, M. K., & Sudkämper, A. (2020) · Psychological Bulletin · doi:10.1037/bul0000234 · Cited by 169
Blake, K. R., & Gangestad, S. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167220913363 · Cited by 159
Kenny, D. A., & Judd, C. M. (2019) · Psychological Methods · doi:10.1037/met0000209 · Cited by 167
Gencer, B., Djousse, L., Al-Ramady, O. T., Cook, N. R., Manson, J. E., & Albert, C. M. (2021) · Circulation · doi:10.1161/circulationaha.121.055654 · Cited by 150
Linden, A. H., & Hönekopp, J. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620964193 · Cited by 159
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 156
Effects of Mindfulness-Based Therapies for Female Sexual Dysfunction: A Meta-Analytic Review
Stephenson, K. R., & Kerth, J. (2017) · The Journal of Sex Research · doi:10.1080/00224499.2017.1331199 · Cited by 150
Radkowitsch, A., Vogel, F., & Fischer, F. (2020) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-020-09316-4 · Cited by 152
Robust Bayesian meta-analysis: Addressing publication bias with model-averaging.
Maier, M., Bartos̆, F., & Wagenmakers, E. (2022) · Psychological Methods · doi:10.1037/met0000405 · Cited by 153
ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis
Chiocchia, V., Nikolakopoulou, A., Higgins, J. P. T., Page, M. J., Papakonstantinou, T., Cipriani, A., Furukawa, T. A., Siontis, G. C. M., Egger, M., & Salanti, G. (2021) · BMC Medicine · doi:10.1186/s12916-021-02166-3 · Cited by 129
Conceptualizing and evaluating replication across domains of behavioral research
Tackett, J. L., & McShane, B. B. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000882 · Cited by 127
Self-citationCurrent practices in meta‐regression in psychology, education, and medicine
Tipton, E., Pustejovsky, J. E., & Ahmadi, H. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1339 · Cited by 133
Longitudinal tests of the theory of planned behaviour: A meta-analysis
Hagger, M. S., & Hamilton, K. (2023) · European Review of Social Psychology · doi:10.1080/10463283.2023.2225897 · Cited by 165
Aging and recognition memory: A meta-analysis.
Fraundorf, S. H., Hourihan, K. L., Peters, R. A., & Benjamin, A. S. (2019) · Psychological Bulletin · doi:10.1037/bul0000185 · Cited by 118
Pair-bonding, fatherhood, and the role of testosterone: a meta-analytic review
Grebe, N. M., Sarafin, R. E., Strenth, C. R., & Zilioli, S. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.01.010 · Cited by 116
Brain volume and intelligence: The moderating role of intelligence measurement quality
Gignac, G. E., & Bates, T. C. (2017) · Intelligence · doi:10.1016/j.intell.2017.06.004 · Cited by 141
Infant Motor Development in Autism Spectrum Disorder: A Synthesis and Meta-analysis
West, K. L. (2018) · Child Development · doi:10.1111/cdev.13086 · Cited by 128
Bartos̆, F., Maier, M., Quintana, D. S., & Wagenmakers, E. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221109259 · Cited by 122
Correcting for Publication Bias in a Meta-Analysis with the P-uniform* MethodPreprint
van Aert, R. C. M., & van Assen, M. A. L. M. (2018) · doi:10.31222/osf.io/zqjr9 · Cited by 108
Large-Scale Replication Projects in Contemporary Psychological Research
McShane, B. B., Tackett, J. L., Böckenholt, U., & Gelman, A. (2019) · The American Statistician · doi:10.1080/00031305.2018.1505655 · Cited by 117
Self-citationA systematic review and Bayesian meta-analysis of the acoustic features of infant-directed speech
Cox, C. M. M., Bergmann, C., Fowler, E., Keren‐Portnoy, T., Roepstorff, A., Bryant, G. A., & Fusaroli, R. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01452-1 · Cited by 123
Risk and protective factors for (internet) gaming disorder: A meta-analysis of pre-COVID studies
Ropovik, I., Martončik, M., Babinčák, P., Baník, G., Vargová, L., & Adamkovic, M. (2022) · Addictive Behaviors · doi:10.1016/j.addbeh.2022.107590 · Cited by 111
Daks, J. S., & Rogge, R. D. (2020) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2020.09.010 · Cited by 101
Effects of habit and intention on behavior: Meta-analysis and test of key moderators.
Hagger, M. S., Hamilton, K., Phipps, D. J., Protogerou, C., Zhang, C., Girelli, L., Mallia, L., & Lucidi, F. (2023) · Motivation Science · doi:10.1037/mot0000294 · Cited by 102
Bresin, K., Mekawi, Y., & Verona, E. (2018) · Psychology of Addictive Behaviors · doi:10.1037/adb0000383 · Cited by 97
Grgić, J., McIlvenna, L. C., Fyfe, J. J., Sabol, F., Bishop, D. J., Schoenfeld, B. J., & Pedišić, Ž. (2018) · Sports Medicine · doi:10.1007/s40279-018-1008-z · Cited by 89
Hemispheric asymmetries in the amygdala: A comparative primer
Ocklenburg, S., Peterburs, J., & Mundorf, A. (2022) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2022.102283 · Cited by 86
Social network site use and Big Five personality traits: A meta-analysis
Huang, C. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.03.009 · Cited by 91
McAuliffe, W. H. B., Carter, E. C., Berhane, J., Snihur, A., & McCullough, M. E. (2019) · Personality and Social Psychology Review · doi:10.1177/1088868319887599 · Cited by 92
Macri, J. A., & Rogge, R. D. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102432 · Cited by 89
Detecting publication selection bias through excess statistical significance
Stanley, T. D., Doucouliagos, H., Ioannidis, J. P. A., & Carter, E. C. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1512 · Cited by 83
Effects of action video game play on cognitive skills: A meta-analysis.
Bediou, B., Rodgers, M. A., Tipton, E., Mayer, R. E., Green, C. S., & Bavelier, D. (2023) · Technology Mind and Behavior · doi:10.1037/tmb0000102 · Cited by 112
A comprehensive meta-analysis of money priming.
Lodder, P., Ong, H. H., Grasman, R. P. P. P., & Wicherts, J. M. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000570 · Cited by 86
Preregistration Is Neither Sufficient nor Necessary for Good Science
Pham, M. T., & Oh, T. T. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1209 · Cited by 91
“Statistical Significance” and Statistical Reporting: Moving Beyond Binary
McShane, B. B., Bradlow, E. T., Lynch, J. G., & Meyer, R. J. (2023) · Journal of Marketing · doi:10.1177/00222429231216910 · Cited by 79
Self-citationHeterogeneity in direct replications in psychology and its association with effect size.
Olsson-Collentine, A., Wicherts, J. M., & van Assen, M. A. L. M. (2020) · Psychological Bulletin · doi:10.1037/bul0000294 · Cited by 81
Frost, A. D. S., & Ledgerwood, A. (2020) · Journal of Pacific Rim Psychology · doi:10.1017/prp.2020.7 · Cited by 75
Systematic Review and Meta-Analysis of Correlates of FFMQ Mindfulness Facets
Mattes, J. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02684 · Cited by 88
Estimating Population Mean Power Under Conditions of Heterogeneity and Selection for Significance
Brunner, J., & Schimmack, U. (2020) · Meta-Psychology · doi:10.15626/mp.2018.874 · Cited by 76
Psychological Effects of Meditation for Healthy Practitioners: An Update
Sedlmeier, P., Loße, C., & Quasten, L. C. (2018) · Mindfulness · doi:10.1007/s12671-017-0780-4 · Cited by 74
The sleep benefit in episodic memory: An integrative review and a meta-analysis.
Berres, S., & Erdfelder, E. (2021) · Psychological Bulletin · doi:10.1037/bul0000350 · Cited by 74
Self-Reported Trait Mindfulness and Couples’ Relationship Satisfaction: a Meta-Analysis
Quinn‐Nilas, C. (2020) · Mindfulness · doi:10.1007/s12671-020-01303-y · Cited by 78
Self- and Response Efficacy Information in Fear Appeals: A Meta-Analysis
Bigsby, E., & Albarracín, D. (2022) · Journal of Communication · doi:10.1093/joc/jqab048 · Cited by 72
Examining the Efficacy of Time Scarcity Marketing Promotions in Online Retail
Hmurovic, J., Lamberton, C. P., & Goldsmith, K. (2022) · Journal of Marketing Research · doi:10.1177/00222437221118856 · Cited by 95
Cognitive Behavior Therapy for Mental Disorders in Adults
Cuijpers, P. J., Harrer, M., Miguel, C., Čihařová, M., Papola, D., Basic, D., Botella, C., Cristea, I. A., de Ponti, N., Donker, T., et al. (2025) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2025.0482 · Cited by 91
Pavlacic, J. M., Buchanan, E. M., Maxwell, N. P., Hopke, T. G., & Schulenberg, S. E. (2019) · Review of General Psychology · doi:10.1177/1089268019831645 · Cited by 81
Maseeh, H. I., Sangroya, D., Jebarajakirthy, C., Adil, M., Kaur, J., Yadav, M. P., & Saha, R. (2022) · Psychology and Marketing · doi:10.1002/mar.21748 · Cited by 77
Child physical abuse risk factors: A systematic review and a meta-analysis
Milner, J. S., Crouch, J. L., McCarthy, R. J., Ammar, J., Dominguez-Martinez, R., Thomas, C. L., & Jensen, A. P. (2022) · Aggression and Violent Behavior · doi:10.1016/j.avb.2022.101778 · Cited by 75
Meta‐analysis of social science research: A practitioner's guide
Iršová, Z., Doucouliagos, H., Havránek, T., & Stanley, T. D. (2023) · Journal of Economic Surveys · doi:10.1111/joes.12595 · Cited by 76
Does Interpersonal Discussion Increase Political Knowledge? A Meta-Analysis
Amsalem, E., & Nir, L. (2019) · Communication Research · doi:10.1177/0093650219866357 · Cited by 68
More Than One Replication Study Is Needed for Unambiguous Tests of Replication
Hedges, L. V., & Schauer, J. M. (2019) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998619852953 · Cited by 66
Scientific-Consensus Communication About Contested Science: A Preregistered Meta-Analysis
van Stekelenburg, A., Schaap, G., Veling, H., Riet, J. V. ‘., & Buijzen, M. (2022) · Psychological Science · doi:10.1177/09567976221083219 · Cited by 69
Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003)
Colling, L. J., Szűcs, D., Marco, D. D., Cipora, K., Ulrich, R., Nuerk, H., Soltanlou, M., Bryce, D., Chen, S.-C., Schroeder, P. A., et al. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920903079 · Cited by 64
Self-citationBurnay, J., Kepes, S., & Bushman, B. J. (2021) · Aggressive Behavior · doi:10.1002/ab.21998 · Cited by 68
Menhas, R., Yang, L., Saqib, Z. A., Younas, M. S., & Saeed, M. M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1228271 · Cited by 70
Bayesian evaluation of effect size after replicating an original study
van Aert, R. C. M., & van Assen, M. A. L. M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0175302 · Cited by 57
Power to Detect What? Considerations for Planning and Evaluating Sample SizePreprint
Giner‐Sorolla, R., Montoya, A. K., Aberson, C. L., Carpenter, T., Lewis, N. A., Bostyn, D. H., Conrique, B. G., Ng, B., Reifman, A. S., Schoemann, A. M., et al. (2023) · doi:10.31234/osf.io/rv3kw · Cited by 62
Wang, Y. A., Sparks, J., Gonzales, J. E., Hess, Y. D., & Ledgerwood, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.04.011 · Cited by 55
Metadata
- Authors
- McShane, B. B., Böckenholt, U., & Hansen, K. T.
- Affiliations
- Northwestern University (US)
- University of California San Diego (US)
- Corresponding author
- McShane, Blakeley B.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2016-09
- Volume
- 11
- Issue
- 5
- Pages
- 730-749
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2524737933
- Cited by
- 398
- References
- 49
- Citations by year
- 2026: 30 · 2025: 43 · 2024: 43 · 2023: 46 · 2022: 50 · 2021: 46 · 2020: 44 · 2019: 50 · 2018: 23 · 2017: 21 · 2016: 2
- FWCI
- 29.71
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, selection methods, p-curve, meta-analysis, sensitivity analysis, effect size estimation, simulation study
- MeSH terms
- Computer Simulation, Data Interpretation, Statistical, Humans, Meta-Analysis as Topic, Models, Statistical, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
