A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null

Ferguson, Christopher J.; Heene, Moritz

doi:10.1177/1745691612459059

Cited by 603Meta-analysis and systematic reviewsPsychology Research and BibliometricsAcademic Publishing and Open Access

A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null

Ferguson, Christopher J., Heene, Moritz·2012·Perspectives on Psychological Science

DOI 10.1177/1745691612459059

Open this paper in CoChat — free account

Abstract

Publication bias remains a controversial issue in psychological science. The tendency of psychological science to avoid publishing null results produces a situation that limits the replicability assumption of science, as replication cannot be meaningful without the potential acknowledgment of failed replications. We argue that the field often constructs arguments to block the publication and interpretation of null results and that null results may be further extinguished through questionable researcher practices. Given that science is dependent on the process of falsification, we argue that these problems reduce psychological science's capability to have a proper mechanism for theory falsification, thus resulting in the promulgation of numerous "undead" theories that are ideologically popular but have little basis in fact.

Integrity signals

Registry

Crossref · 10.1177/1745691612459059

Cites

65 of 65 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

In psychological research, arguments are often made to block publication and interpretation of null results, and questionable practices may extinguish null results.
“the field often constructs arguments to block the publication and interpretation of null results and that null results may be extinguished through questionable researcher practices”
From Intranasal Oxytocin: Myths and Delusions (2016) · cites this paper as [99]

Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · see citing works
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Cochrane Database of Systematic Reviews
(2008) · Springer eBooks · doi:10.1007/978-1-4020-5614-7_436 · Cited by 4,552 · see citing works
Chronicle of Higher Education
Cohen, D. (1998) · The Women s Review of Books · doi:10.2307/4022859 · Cited by 4,323 · see citing works
The earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003 · see citing works
An effect size primer: A guide for clinicians and researchers.
Ferguson, C. J. (2009) · Professional Psychology: Research and Practice · doi:10.1037/a0015808 · Cited by 3,467 · see citing works
Self-citation
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · see citing works
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791 · see citing works
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · see citing works
Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: A meta-analytic review
Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein, H. R., & Saleem, M. (2010) · Psychological Bulletin · doi:10.1037/a0018251 · Cited by 2,048 · see citing works
Psychological testing and psychological assessment: A review of evidence and issues.
Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2001) · American Psychologist · doi:10.1037//0003-066x.56.2.128 · Cited by 1,469 · see citing works
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291 · see citing works
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · see citing works
Interpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,143 · see citing works
A comprehensive meta-analysis of the relationship between Emotional Intelligence and health
Martins, A., Ramalho, N., & Morin, E. (2010) · Personality and Individual Differences · doi:10.1016/j.paid.2010.05.029 · Cited by 1,070 · see citing works
The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey
Ioannidis, J. P. A., & Trikalinos, T. A. (2007) · Canadian Medical Association Journal · doi:10.1503/cmaj.060410 · Cited by 1,052 · see citing works
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 1× · see citing works
“In addition to publication bias at the journal level, publication bias also likely occurs at the individual scholar level, insofar as scholars decline to submit null results for publication either because such results conflict with beloved theoretical models in which they are invested or simply because they believe nu…”
Background
Full publication of results initially presented in abstracts.
Scherer, R. W., Langenberg, P., & von Elm, E. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub3 · Cited by 997 · see citing works
A meta-analytic examination of assumed properties of child sexual abuse using college samples.
Rind, B., Tromovitch, P., & Bauserman, R. L. (1998) · Psychological Bulletin · doi:10.1037/0033-2909.124.1.22 · Cited by 982 · see citing works
Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977 · see citing works
Egos inflating over time: A cross-temporal meta-analysis of the Narcissistic Personality Inventory
Twenge, J. M., Konrath, S., Foster, J. D., Keith Campbell, W., & Bushman, B. J. (2008) · Journal of Personality · doi:10.1111/j.1467-6494.2008.00507.x · Cited by 972 · see citing works
Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect.
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893 · see citing works
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805 · see citing works
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734 · see citing works
The relations between parents’ Big Five personality factors and parenting: A meta-analytic review.
Prinzie, P., Stams, G. J. J. M., Deković, M., Reijntjes, A. H. A., & Belsky, J. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015823 · Cited by 549 · see citing works
Mass Media Effects Research : Advances Through Meta-Analysis
Roskos‐Ewoldsen, D. R., Klinger, M. R., & Roskos‐Ewoldsen, B. (2006) · Routledge eBooks · Cited by 507 · see citing works
On Being Rejected: A Meta-Analysis of Experimental Research on Rejection
Gerber, J., & Wheeler, L. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01158.x · Cited by 498 · see citing works
Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses.
Ferguson, C. J., & Brannick, M. Τ. (2011) · Psychological Methods · doi:10.1037/a0024445 · Cited by 493 · see citing works
Self-citation
Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453 · see citing works
Replication and p Intervals: p Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better
Cumming, G. (2008) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2008.00079.x · Cited by 451 · see citing works
Publication Bias: The Problem That Won't Go Away
DICKERSIN, K., & MIN, Y. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26343.x · Cited by 427 · see citing works
Effect sizes for experimenting psychologists.
Rosnow, R. L., & Rosenthal, R. (2003) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087427 · Cited by 416 · see citing works
Selection Models and the File Drawer Problem
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013012 · Cited by 399 · see citing works
The Public Health Risks of Media Violence: A Meta-Analytic Review
Ferguson, C. J., & Kilburn, J. C. (2009) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2008.11.033 · Cited by 330 · see citing works
Self-citation
Publication Bias (The "File-Drawer Problem") in Scientific Inference
Scargle, J. D. (1999) · arXiv · doi:10.48550/arxiv.physics/9909033 · Cited by 313 · see citing works
Much ado about nothing: the misestimation and overinterpretation of violent video game effects in eastern and western nations: comment on Anderson et al. (2010)
Ferguson, C. J., & Kilburn, J. (2010) · Psychological Bulletin · doi:10.1037/a0018566 · Cited by 311 · see citing works
Self-citation
There Is Nothing So Theoretical as a Good Method
Greenwald, A. G. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691611434210 · Cited by 282 · see citing works
The Theory of Reasoned Action
Trafimow, D. (2009) · Theory & Psychology · doi:10.1177/0959354309336319 · Cited by 187 · see citing works
Time limits and gender differences on paper-and-pencil tests of mental rotation: a meta-analysis
Voyer, D. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-010-0042-0 · Cited by 186 · see citing works
Do Today's Young People Really Think They Are So Extraordinary?: An Examination of Secular Trends in Narcissism and Self-Enhancement
Trzesniewski, K. H., Donnellan, M. B., & Robins, R. W. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02065.x · Cited by 183 · see citing works
Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa
Sterling, T. D., Rosenbaum, W. L., & Jj, W. (1995) · The American Statistician · doi:10.2307/2684823 · Cited by 168 · see citing works
Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias.
Coursol, A., & Wagner, E. E. (1986) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.17.2.136 · Cited by 165 · see citing works
Sample Sizes and Effect Sizes are Negatively Correlated in Meta-Analyses: Evidence and Implications of a Publication Bias Against NonSignificant Findings
Levine, T. R., Asada, K. J. K., & Carpenter, C. (2009) · Communication Monographs · doi:10.1080/03637750903074685 · Cited by 159 · see citing works
Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice
LeBel, E. P., & Peters, K. R. (2011) · Review of General Psychology · doi:10.1037/a0025172 · Cited by 152 · see citing works
A longitudinal test of video game violence influences on dating and aggression: A 3-year longitudinal study of adolescents
Ferguson, C. J., Miguel, C. S., Garza, A., & Jerabeck, J. M. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2011.10.014 · Cited by 141 · see citing works
Self-citation
Confidence intervals and replication: Where will the next mean fall?
Cumming, G., & Maillardet, R. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.3.217 · Cited by 122 · see citing works
Statistical significance, reviewer evaluations, and the scientific process: Is there a (statistically) significant relationship?
Atkinson, D. R., Furlong, M., & Wampold, B. (1982) · Journal of Counseling Psychology · doi:10.1037/0022-0167.29.2.189 · Cited by 120 · see citing works
A Communication Researchers’ Guide to Null Hypothesis Significance Testing and Alternatives
Levine, T. R., Weber, R., Park, H. S., & Hullett, C. R. (2008) · Human Communication Research · doi:10.1111/j.1468-2958.2008.00318.x · Cited by 97 · see citing works
The effects of child sexual abuse: Comment on Rind, Tromovitch, and Bauserman (1998).
Dallam, S. J., Gleaves, D. H., Antonio Cepeda-benito, A., Silberg, J. L., KRAEMER, H. C., & Spiegel, D. (2001) · Psychological Bulletin · doi:10.1037/0033-2909.127.6.715 · Cited by 96 · see citing works
When worlds collide: Social science, politics, and the Rind et al. (1998) child sexual abuse meta-analysis.
Lilienfeld, S. O. (2002) · American Psychologist · doi:10.1037/0003-066x.57.3.176 · Cited by 94 · see citing works
The effects of child sexual abuse
MacIntyre, D., & Carr, A. (1999) · Lenus, The Irish Health Repository (Dr Steevens Hospital Library) · Cited by 69 · see citing works
Publication bias in psychological science: Comment on Ferguson and Brannick (2012).
Rothstein, H. R., & Bushman, B. J. (2012) · Psychological Methods · doi:10.1037/a0027128 · Cited by 68 · see citing works
Mirror, Mirror on the Wall: Peer Competition, Television Influences, and Body Image Dissatisfaction
Ferguson, C. J., Muñoz, M. E., Contreras, S., & Velasquez, K. (2011) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2011.30.5.458 · Cited by 64 · see citing works
Self-citation
Social Rejection, Control, Numbness, and Emotion: How Not to be Fooled by Gerber and Wheeler (2009)
Baumeister, R., DeWall, C., & Vohs, K. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01159.x · Cited by 61 · see citing works
Openness to experience and all‐cause mortality: A meta‐analysis and r equivalent from risk ratios and odds ratios
Ferguson, E., & Bibby, P. A. (2012) · British Journal of Health Psychology · doi:10.1111/j.2044-8287.2011.02055.x · Cited by 53 · see citing works
Editorial: A Call for Replications of Research in Pediatric Psychology and Guidance for Authors
Drotar, D. (2010) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsq049 · Cited by 34 · see citing works
Fail-Safe Ns for One- Versus Two-Tailed Tests Lead to Different Conclusions About Publication Bias
Hsu, L. (2002) · Understanding Statistics · doi:10.1207/s15328031us0102_02 · Cited by 24 · see citing works
[Selection Models and the File Drawer Problem]: Rejoinder
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013019 · Cited by 22 · see citing works
A Generalized Publication Bias ModelPreprint
Schönemann, P., & Scargle, J. (2008) · arXiv (Cornell University) · doi:10.6129/cjp.2008.5001.02 · Cited by 14 · see citing works
A brief history of the Fail Safe Number in Applied ResearchPreprint
Heene, M. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.2326 · Cited by 10 · see citing works
Self-citation
Some Theories are Unfalsifiable
Wallach, L., & Wallach, M. A. (2010) · Theory & Psychology · doi:10.1177/0959354310373676 · Cited by 9 · see citing works
Box scores are for baseball
ELASHOFF, J. D. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075579 · Cited by 7 · see citing works
WITHDRAWNPreprint
Schönemann, P. H., Heene, M., Schonemann, P. H., et al. (2017) · doi:10.31234/osf.io/pcq6a · Cited by 3 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

The Triple P-Positive Parenting Program: a systematic review and meta-analysis of a multi-level system of parenting support
Sanders, M. R., Kirby, J. N., Tellegen, C. L., & Day, J. J. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.04.003 · Cited by 1,077 · see citing works
Effect Size Guidelines, Sample Size Calculations, and Statistical Power in Gerontology
Brydges, C. R. (2019) · Innovation in Aging · doi:10.1093/geroni/igz036 · Cited by 826 · see citing works
Intranasal Oxytocin: Myths and Delusions
Leng, G., & Ludwig, M. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.05.003 · Cited by 615 · see citing works
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588 · see citing works
How Methodological Features Affect Effect Sizes in Education
Cheung, A. C. K., & Slavin, R. E. (2016) · Educational Researcher · doi:10.3102/0013189x16656615 · Cited by 529 · see citing works
Theory of mind and prosocial behavior in childhood: A meta-analytic review.
Imuta, K., Henry, J. D., Slaughter, V., Selcuk, B., & Ruffman, T. (2016) · Developmental Psychology · doi:10.1037/dev0000140 · Cited by 474 · see citing works
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436 · see citing works
Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size
Kühberger, A., Fritz, A., & Scherndl, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105825 · Cited by 399 · see citing works
PERSONALITY AND DEPRESSIVE SYMPTOMS: INDIVIDUAL PARTICIPANT META-ANALYSIS OF 10 COHORT STUDIES: Research Article: Personality and Depression
Hakulinen, C., Elovainio, M., Pulkki-Råback, L., Virtanen, M., Kivimäki, M., & Jokela, M. (2015) · Depression and Anxiety · doi:10.1002/da.22376 · Cited by 418 · see citing works
Contextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398 · see citing works
Effects of video-game play on information processing: a meta-analytic investigation
Powers, K. L., Brooks, P. J., Aldrich, N. J., Palladino, M. A., & Alfieri, L. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0418-z · Cited by 348 · see citing works
Testing for funnel plot asymmetry of standardized mean differences
Pustejovsky, J. E., & Rodgers, M. A. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1332 · Cited by 354 · see citing works
Theory of planned behavior and adherence in chronic illness: a meta-analysis
Rich, A., Brandes, K., Mullan, B., & Hagger, M. S. (2015) · Journal of Behavioral Medicine · doi:10.1007/s10865-015-9644-3 · Cited by 272 · see citing works
A Short (Personal) Future History of Revolution 2.0
Spellman, B. A. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615609918 · Cited by 213 · see citing works
Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016) · Review of Educational Research · doi:10.3102/0034654315582067 · Cited by 234 · see citing works
Retrieval practice consistently benefits student learning: A systematic review of applied research in schools and classrooms
Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09595-9 · Cited by 211 · see citing works
A systematic literature review of sex differences in childhood language and brain development
Etchell, A., Adhikari, A., Weinberg, L. S., Choo, A. L., Garnett, E. O., Chow, H. M., & Chang, S.-E. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.04.011 · Cited by 193 · see citing works
A survey of publication practices of single-case design researchers when treatments have small or large effects.
Shadish, W. R., Zelinsky, N. A. M., Vevea, J. L., & Kratochwill, T. R. (2016) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.308 · Cited by 130 · see citing works
P-hacking and publication bias interact to distort meta-analytic effect size estimates
Friese, M., & Frankenbach, J. (2020) · Psychological Methods · doi:10.1037/met0000246 · Cited by 122 · see citing works
A meta-analysis of Geogebra software decade of assisted mathematics learning: what to learn and where to go?
Juandi, D., Kusumah, Y. S., Tamur, M., Perbowo, K. S., & Wijaya, T. T. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e06953 · Cited by 119 · see citing works
The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices
Giofrè, D., Cumming, G., Fresc, L., Boedker, I., & Tressoldi, P. (2017) · PLOS ONE · doi:10.1371/journal.pone.0175583 · Cited by 98 · see citing works
Failure to Demonstrate That Playing Violent Video Games Diminishes Prosocial Behavior
Tear, M. J., & Nielsen, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068382 · Cited by 90 · see citing works
The registration continuum in clinical science: A guide toward transparent practices
Benning, S. D., Bachrach, R. L., Smith, E. A., Freeman, A. J., & Wright, A. G. C. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000451 · Cited by 77 · see citing works
Do published studies yield larger effect sizes than unpublished studies in education and special education? A meta-review
Chow, J. C., & Ekholm, E. (2018) · Educational Psychology Review · doi:10.1007/s10648-018-9437-7 · Cited by 80 · see citing works
A surge of p -values between 0.041 and 0.049 in recent decades (but negative results are increasing rapidly too)
de Winter, J. C., & Dodou, D. (2015) · PeerJ · doi:10.7717/peerj.733 · Cited by 78 · see citing works
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39 · see citing works
Correlations of online social network size with well-being and distress: a meta-analysis
Huang, C. (2021) · Cyberpsychology: Journal of Psychosocial Research on Cyberspace · doi:10.5817/cp2021-2-3 · Cited by 33 · see citing works
A retrospective analysis of dissemination biases in the brief alcohol intervention literature.
Tanner-Smith, E. E., & Polanin, J. R. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000014 · Cited by 26 · see citing works
“It is little surprise, then, that publication bias has been call the “800 lb gorilla in psychology’s living room” (Ferguson & Heene, 2012, p. 556).”
Background
Addressing selective reporting of experiments through predefined exclusion criteria
Neves, K., & Amaral, O. B. (2020) · eLife · doi:10.7554/elife.56626 · Cited by 15 · see citing works
The replicability of the negative testing effect: differences across participant populations
Mulligan, N. W., Rawson, K. A., Peterson, D. J., & Wissman, K. T. (2018) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000490 · Cited by 10 · see citing works
Mortality salience in an offline and online setting
Chew, P. K. H., & Yap, B. L. M. (2022) · Death Studies · doi:10.1080/07481187.2021.1888826 · Cited by 8 · see citing works
Scientific integrity and the market for lemons
Cottrell, R. C. (2014) · Research Ethics · doi:10.1177/1747016113494651 · Cited by 6 · see citing works
When Can AI Models Explain Learning? Validity Criteria for AI as Cognitive Models in Education
Wang, P., Viberg, O., Law, E. L.-C., & Greiff, S. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10202-y · Cited by 3 · see citing works

Authors: Ferguson, C. J., & Heene, M.
Affiliations: Texas A&M International University (US)
Ludwig-Maximilians-Universität München (DE)
ORCID: Ferguson, Christopher J.
Heene, Moritz
Venue: Perspectives on Psychological Science
Publisher: SAGE Publications
Published: 2012-11
Volume: 7
Issue: 6
Pages: 555-561
ISSN: 1745-6916 (ISSN-L)
1745-6924
DOI: 10.1177/1745691612459059
OpenAlex: W2158647686
Cited by: 603
References: 65
Citations by year: 2026: 19 · 2025: 28 · 2024: 17 · 2023: 42 · 2022: 37 · 2021: 87 · 2020: 47 · 2019: 56 · 2018: 51 · 2017: 70 · 2016: 37 · 2015: 46 · 2014: 37 · 2013: 29
FWCI: 40.91
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Perspectives on Psychological Science (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
Keywords: publication bias, negative results, replication, questionable research practices, replicability, replication failure, falsification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks