A peculiar prevalence of p values just below .05
Abstract
In null hypothesis significance testing (NHST), p values are judged relative to an arbitrary threshold for significance (.05). The present work examined whether that standard influences the distribution of p values reported in the psychology literature. We examined a large subset of papers from three highly regarded journals. Distributions of p were found to be similar across the different journals. Moreover, p values were much more common immediately below .05 than would be expected based on the number of p values occurring in other ranges. This prevalence of p values just below the arbitrary criterion for significance was observed in all three journals. We discuss potential sources of this pattern, including publication bias and researcher degrees of freedom.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many published studies only barely achieve nominal statistical significance.
“Many published studies only barely achieve nominal statistical significance 15 .”
From Power failure: why small sample size undermines the reliability of neuroscience (2013) · cites this paper as [15]
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289 · see citing works
The earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003 · see citing works
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · see citing works
Sackett, D. L. (1979) · Journal of Chronic Diseases · doi:10.1016/0021-9681(79)90012-2 · Cited by 2,075 · see citing works
The review of educational research.
(1931) · Journal of Educational Psychology · doi:10.1037/h0066448 · Cited by 1,562 · see citing works
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090 · see citing works
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034 · see citing works
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,031 · see citing works
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977 · see citing works
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 1× · see citing works
“Authors may assume that they must obtain statistical significance in their studies in order to publish (Sterling, 1959).”
BackgroundThe case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931 · see citing works
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 856 · see citing works
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 738 · see citing works
Cumming, G. (2008) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2008.00079.x · Cited by 451 · see citing works
An Alternative to Null-Hypothesis Significance Tests
Killeen, P. R. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01538.x · Cited by 443 · see citing works
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011) · PLoS ONE · doi:10.1371/journal.pone.0026828 · Cited by 424 · see citing works
Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing
Hubbard, R., & Lindsay, R. (2008) · Theory & Psychology · doi:10.1177/0959354307086923 · Cited by 244 · see citing works
Becoming a behavioral science researcher: a guide to producing research that matters
Kline, R. B. (2009) · Choice Reviews Online · doi:10.5860/choice.46-3545 · Cited by 240 · see citing works
The Interpretation of Levels of Significance by Psychological Researchers
Rosenthal, R., & Gaito, J. M. (1963) · The Journal of Psychology · doi:10.1080/00223980.1963.9916596 · Cited by 183 · see citing works
Trafimow, D. (2003) · Psychological Review · doi:10.1037/0033-295x.110.3.526 · Cited by 158 · see citing works
Statistical Reform in Psychology: Is Anything Changing?
Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., Lo, J., McMenamin, N., & Wilson, S. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01881.x · Cited by 122 · see citing works
When are ‘positive’ clinical trials in oncology truly positive?
Ocana, A., & Tannock, I. F. (2011) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djq463 · Cited by 110 · see citing works
Post Hoc Pair-Wise Comparisons for the Chi-Square Test of Homogeneity of Proportions
Cox, M. K., & Key, C. H. (1993) · Educational and Psychological Measurement · doi:10.1177/0013164493053004008 · Cited by 64 · see citing works
Cicchetti, D. V., Koenig, K., Klin, A. J., Volkmar, F. R., Paul, R., & Sparrow, S. S. (2010) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-010-1035-6 · Cited by 49 · see citing works
Poitevineau, J., & Lecoutre, B. (2001) · Psychonomic Bulletin & Review · doi:10.3758/bf03196227 · Cited by 41 · see citing works
Hypothesis Testing in Relation to Statistical Methodology
Clark, C. A. (1963) · Review of Educational Research · doi:10.2307/1169648 · Cited by 35 · see citing works
p rep misestimates the probability of replication
Iverson, G., Lee, M., & Wagenmakers, E. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.424 · Cited by 27 · see citing works
Chapter I: Hypothesis Testing in Relation to Statistical Methodology
Clark, C. A. (1963) · Review of Educational Research · doi:10.3102/00346543033005455 · Cited by 15 · see citing works
How often is prep close to the true replication probability?
Trafimow, D., MacDonald, J. A., Rice, S. C., & Clason, D. L. (2010) · Psychological Methods · doi:10.1037/a0018533 · Cited by 6 · see citing works
Flagged first, then by citations
1 of the 32 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · see citing works
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · see citing works
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388 · see citing works
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · see citing works
Registered Reports: A new publishing initiative at Cortex
Chambers, C. (2012) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 574 · see citing works
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471 · see citing works
Kühberger, A., Fritz, A., & Scherndl, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105825 · Cited by 399 · see citing works
How Bayes factors change scientific practice
Dienes, Z. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.10.003 · Cited by 364 · see citing works
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354 · see citing works
What’s in a p? Reassessing best practices for conducting and reporting hypothesis-testing research
Meyer, K., Witteloostuijn, A., & Beugelsdijk, S. (2017) · Journal of International Business Studies · doi:10.1057/s41267-017-0078-8 · Cited by 337 · see citing works
The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles
O’Boyle, E. H., Banks, G. C., & Gonzalez-Mulé, E. (2017) · Journal of Management · doi:10.1177/0149206314527133 · Cited by 246 · see citing works
Improving the Dependability of Research in Personality and Social Psychology
Funder, D. C., Levine, J. M., Mackie, D. M., Morf, C. C., Sansone, C., Vazire, S., & West, S. G. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313507536 · Cited by 254 · see citing works
A tutorial on testing hypotheses using the Bayes factor.
Hoijtink, H. J. A., Mulder, J., Lissa, C. J. V., & Gu, X. (2019) · Psychological Methods · doi:10.1037/met0000201 · Cited by 237 · see citing works
Low statistical power in biomedical science: a review of three human research domains
Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F., & Munafò, M. R. (2017) · Royal Society Open Science · doi:10.1098/rsos.160254 · Cited by 208 · see citing works
Star Wars: The Empirics Strike Back
Brodeur, A., Lé, M., Sangnier, M., & Zylberberg, Y. (2016) · American Economic Journal: Applied Economics · doi:10.1257/app.20150044 · Cited by 430 · see citing works
Safeguard Power as a Protection Against Imprecise Power Estimates
Perugini, M., Gallucci, M., & Costantini, G. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614528519 · Cited by 203 · see citing works
Editorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly
Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016) · Journal of Business and Psychology · doi:10.1007/s10869-016-9456-7 · Cited by 176 · see citing works
Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614535216 · Cited by 191 · see citing works
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165 · see citing works
Teixeira da Silva, J. A., & Dobránszki, J. (2015) · Accountability in Research · doi:10.1080/08989621.2014.899909 · Cited by 158 · see citing works
Elson, M., Mohseni, M. R., Breuer, J., Scharkow, M., & Quandt, T. (2014) · Psychological Assessment · doi:10.1037/a0035569 · Cited by 159 · see citing works
The influence of deliberate practice on musical achievement: a meta-analysis.
Platz, F., Kopiez, R., Lehmann, A. C., & Wolf, A. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00646 · Cited by 152 · see citing works
Replicability and Reproducibility in Comparative Psychology
Stevens, J. R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00862 · Cited by 155 · see citing works
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 138 · see citing works
Robust Bayesian meta-analysis: Addressing publication bias with model-averaging.
Maier, M., Bartos̆, F., & Wagenmakers, E. (2022) · Psychological Methods · doi:10.1037/met0000405 · Cited by 144 · see citing works
Fields, E. C., & Kuperberg, G. R. (2019) · Psychophysiology · doi:10.1111/psyp.13468 · Cited by 143 · see citing works
Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, P. G. S., & Barnett, A. G. (2023) · BMC Medicine · doi:10.1186/s12916-023-03048-6 · Cited by 149 · see citing works
Potential Reporting Bias in Neuroimaging Studies of Sex Differences
David, S., Naudet, F., Laude, J. R., Raduà, J., Fusar-Poli, P., Chu, I., Stefanick, M., & Ioannidis, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-23976-1 · Cited by 123 · see citing works
P-hacking and publication bias interact to distort meta-analytic effect size estimates
Friese, M., & Frankenbach, J. (2020) · Psychological Methods · doi:10.1037/met0000246 · Cited by 122 · see citing works
Big little lies: a compendium and simulation of p -hacking strategies
Stefan, A. M., & Schönbrodt, F. D. (2023) · Royal Society Open Science · doi:10.1098/rsos.220346 · Cited by 149 · see citing works
A kinked meta‐regression model for publication bias correction
Bom, P. R. D., & Rachinger, H. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1352 · Cited by 133 · see citing works
Yom, S. L. (2014) · Comparative Political Studies · doi:10.1177/0010414014554685 · Cited by 114 · see citing works
Does psychotherapy work? An umbrella review of meta‐analyses of randomized controlled trials
Dragioti, E., Karathanos, V. T., Gerdle, B. U. C., & Εvangelou, Ε. (2017) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12713 · Cited by 112 · see citing works
Predicting replication outcomes in the Many Labs 2 study
Forsell, E., Viganola, D., Pfeiffer, T., Almenberg, J., Wilson, B., Chen, Y.-L., Nosek, B. A., Johannesson, M., & Dreber, A. (2018) · Journal of Economic Psychology · doi:10.1016/j.joep.2018.10.009 · Cited by 97 · see citing works
A Call for Examining Replication and Bias in Special Education Research
Cook, B. G. (2014) · Remedial and Special Education · doi:10.1177/0741932514528995 · Cited by 97 · see citing works
Research Ethics for Students in the Social Sciences
Bos, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-48415-6 · Cited by 110 · see citing works
An Aberrant Abundance of Cronbach’s Alpha Values at .70
Hussey, I., Alsalti, T., Bosco, F. A., Elson, M., & Arslan, R. C. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241287123 · Cited by 158 · see citing works
Gildersleeve, K. A., Haselton, M. G., & Fales, M. R. (2014) · Psychological Bulletin · doi:10.1037/a0037714 · Cited by 91 · see citing works
Cybulski, Ł., Mayo‐Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 91 · see citing works
Distributions of p -values smaller than .05 in psychology: what is going on?
Hartgerink, C. H. J., van Aert, R. C. M., Nuijten, M. B., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · PeerJ · doi:10.7717/peerj.1935 · Cited by 90 · see citing works
Bradley, S. H., DeVito, N., Lloyd, K. E., Richards, G. C., Rombey, T., Wayant, C., & Gill, P. J. (2020) · Journal of the Royal Society of Medicine · doi:10.1177/0141076820956799 · Cited by 90 · see citing works
Article Commentary: On the Persistence of Low Power in Psychological Science
Vankov, I., Bowers, J., & Munafò, M. R. (2014) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2014.885986 · Cited by 82 · see citing works
Mathur, M. B., & VanderWeele, T. J. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1464 · Cited by 88 · see citing works
Reproducing Statistical Results
Stodden, V. (2015) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-010814-020127 · Cited by 77 · see citing works
Bishop, D. V. M., & Thompson, P. A. (2016) · PeerJ · doi:10.7717/peerj.1715 · Cited by 90 · see citing works
Sijtsma, K. (2015) · Psychometrika · doi:10.1007/s11336-015-9446-0 · Cited by 79 · see citing works
“Statistical Significance” and Statistical Reporting: Moving Beyond Binary
McShane, B. B., Bradlow, E. T., Lynch, J. G., & Meyer, R. J. (2023) · Journal of Marketing · doi:10.1177/00222429231216910 · Cited by 79 · see citing works
Barch, D. M., & Yarkoni, T. (2013) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-013-0201-7 · Cited by 67 · see citing works
de Winter, J. C., & Dodou, D. (2015) · PeerJ · doi:10.7717/peerj.733 · Cited by 78 · see citing works
The quest for an optimal alpha
Miller, J., & Ulrich, R. (2019) · PLoS ONE · doi:10.1371/journal.pone.0208631 · Cited by 63 · see citing works
Excess Success for Psychology Articles in the Journal Science
Francis, G., Tanzman, J. S., & Matthews, W. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114255 · Cited by 68 · see citing works
Neuroadaptive Bayesian Optimization and Hypothesis Testing
Lorenz, R., Hampshire, A., & Leech, R. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.01.006 · Cited by 67 · see citing works
Matthes, J., Marquart, F., Naderer, B., Arendt, F., Schmuck, D., & Adam, K. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096334 · Cited by 66 · see citing works
The Life of <i>p</i> : “Just Significant” Results are on the Rise
Leggett, N. C., Thomas, N. A., Loetscher, T., & Nicholls, M. E. R. (2013) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2013.863371 · Cited by 64 · see citing works
Elliott, G., Kudrin, N., & Wüthrich, K. (2022) · Econometrica · doi:10.3982/ecta18583 · Cited by 61 · see citing works
A Bayesian approach to mitigation of publication bias
Guan, M., & Vandekerckhove, J. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0868-6 · Cited by 59 · see citing works
Good scientific practice in EEG and MEG research: Progress and perspectives
Niso, G., Krol, L. R., Combrisson, E., Dubarry, A., Elliott, M., Clément, F., Héjja‐Brichard, Y., Herbst, S. K., Jerbi, K., Ković, V., et al. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119056 · Cited by 64 · see citing works
Comment: What p -hacking really looks like: A comment on Masicampo and LaLande (2012)
Lakens, D. (2014) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2014.982664 · Cited by 56 · see citing works
Full Disclosure: Doing behavioural science necessitates sharingPreprint
Peters, G.-J., Abraham, C., & Crutzen, R. (2018) · doi:10.31234/osf.io/n7p5m · Cited by 55 · see citing works
Cristea, I. A., & Ioannidis, J. P. A. (2018) · PLoS ONE · doi:10.1371/journal.pone.0197440 · Cited by 55 · see citing works
Krawczyk, M. W. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127872 · Cited by 52 · see citing works
Incorrect results in software engineering experiments: How to improve research practices
Jørgensen, M., Dybå, T., Liestøl, K., & Sjøberg, D. I. K. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.03.065 · Cited by 51 · see citing works
Maier, M., VanderWeele, T. J., & Mathur, M. B. (2022) · Campbell Systematic Reviews · doi:10.1002/cl2.1256 · Cited by 56 · see citing works
Brekelmans, G., Lavan, N., Saito, H., Clayards, M. A., & Wonnacott, E. (2022) · Journal of Memory and Language · doi:10.1016/j.jml.2022.104352 · Cited by 56 · see citing works
Perneger, T., & Combescure, C. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.04.003 · Cited by 41 · see citing works
Liu, I., Liu, F., Xiao, Y., Huang, Y., Wu, S., & Ni, S. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2300015 · Cited by 44 · see citing works
Replication crisis or an opportunity to improve scientific production?
Navarro, M. D. F., Pascual‐Llobell, J., Pascual‐Soler, M., Perezgonzalez, J. D., & Berríos-Riquelme, J. (2020) · European Journal of Education · doi:10.1111/ejed.12417 · Cited by 46 · see citing works
On the challenges of drawing conclusions from p -values just below 0.05
Lakens, D. (2015) · PeerJ · doi:10.7717/peerj.1142 · Cited by 44 · see citing works
Increasing the Credibility of Political Science Research: A Proposal for Journal Reforms
Nyhan, B. J. (2015) · PS Political Science & Politics · doi:10.1017/s1049096515000463 · Cited by 43 · see citing works
The Place of Experimental Design and Statistics in the 3Rs
Parker, R. M., & Browne, W. J. (2014) · ILAR Journal · doi:10.1093/ilar/ilu044 · Cited by 46 · see citing works
Vermeulen, I., Beukeboom, C. J., Batenburg, A., Avrămiea, A.-E., Stoyanov, D. S., van de Velde, B., & Oegema, D. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096333 · Cited by 41 · see citing works
Anderson, S. F. (2019) · Psychological Methods · doi:10.1037/met0000248 · Cited by 41 · see citing works
Statistical Inference and the Replication Crisis
Colling, L. J., & Szűcs, D. (2019) · PhilPapers (PhilPapers Foundation) · doi:10.17863/cam.35452 · Cited by 41 · see citing works
A 20-Year Review of Outcome Reporting Bias in Moderated Multiple Regression
O’Boyle, E. H., Banks, G. C., Carter, K., Walter, S. L., & Yuan, Z. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9539-8 · Cited by 40 · see citing works
Some properties of p-curves, with an application to gradual publication bias.
Ulrich, R., & Miller, J. (2018) · Psychological methods · doi:10.1037/met0000125 · Cited by 28 · see citing works
Toward Open and Reproducible Epidemiology
Mathur, M. B., & Fox, M. P. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad007 · Cited by 46 · see citing works
Statistical Inference and the Replication Crisis
Colling, L. J., & Szűcs, D. (2018) · Review of Philosophy and Psychology · doi:10.1007/s13164-018-0421-4 · Cited by 47 · see citing works
How can preregistration contribute to research in our field?
Jonas, K. J., & Cesario, J. (2015) · Comprehensive Results in Social Psychology · doi:10.1080/23743603.2015.1070611 · Cited by 36 · see citing works
Chang, X., Gao, H., & Li, W. (2024) · Journal of Accounting Research · doi:10.1111/1475-679x.12579 · Cited by 35 · see citing works
Hyatt, C. S., Chester, D. S., Zeichner, A., & Miller, J. D. (2019) · Aggressive Behavior · doi:10.1002/ab.21830 · Cited by 35 · see citing works
Lind, S. E., Williams, D. M., Nicholson, T., Grainger, C., & Carruthers, P. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000467 · Cited by 35 · see citing works
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 36 · see citing works
Panteleimon Ekkekakis, A., Swinton, P. A., & Tiller, N. B. (2023) · Sports Medicine · doi:10.1007/s40279-023-01880-7 · Cited by 35 · see citing works
Big Little Lies: A Compendium and Simulation of p-Hacking StrategiesPreprint
Stefan, A. M., & Schönbrodt, F. D. (2022) · doi:10.31234/osf.io/xy2dk · Cited by 35 · see citing works
The Heuristic Value of p in Inductive Statistical Inference
Krueger, J. I., & Heck, P. R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00908 · Cited by 32 · see citing works
Type M Error Might Explain Weisburd’s Paradox
Gelman, A., Skarðhamar, T., & Aaltonen, M. (2017) · Journal of Quantitative Criminology · doi:10.1007/s10940-017-9374-5 · Cited by 33 · see citing works
Invalidation of Models and Fitness-for-Purpose: A Rejectionist Approach
Beven, K. J., & Lane, S. N. (2019) · Simulation foundations, methods and applications · doi:10.1007/978-3-319-70766-2_6 · Cited by 33 · see citing works
Inhibition of HBsAg secretion from Alexander cell line by Phyllanthus amarus.
Jayaram, S., & Thyagarajan, S. P. (1996) · PubMed · Cited by 33 · see citing works
Full disclosure: Doing behavioural science necessitates sharing
Peters, G.-J., Abraham, C., & Crutzen, R. (2012) · PsycEXTRA Dataset · doi:10.1037/e544792013-002 · Cited by 36 · see citing works
Best (but oft forgotten) practices: sample size planning for powerful studies
Anderson, S. F. (2019) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqz058 · Cited by 34 · see citing works
Stone, D. N. (2017) · Accounting Horizons · doi:10.2308/acch-51949 · Cited by 29 · see citing works
An aberrant abundance of Cronbach’s alpha values at .70Preprint
Hussey, I., Alsalti, T., Bosco, F. A., Elson, M., & Arslan, R. C. (2023) · doi:10.31234/osf.io/dm8xn · Cited by 36 · see citing works
Kallitsounaki, A., & Williams, D. (2020) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-020-04478-4 · Cited by 26 · see citing works
Reactions to threatening health messages
ten Hoor, G. A., Peters, G.-J., Kalagi, J., de Groot, L., Grootjans, K., Huschens, A., Köhninger, C., Kölgen, L., Pelssers, I., Schütt, T., et al. (2012) · BMC Public Health · doi:10.1186/1471-2458-12-1011 · Cited by 26 · see citing works
Reanalyzing Head et al. (2015): investigating the robustness of widespread p -hacking
Hartgerink, C. H. J. (2017) · PeerJ · doi:10.7717/peerj.3068 · Cited by 25 · see citing works
Means to valuable exploration: I. The blending of confirmation and exploration and how to resolve it
Höfler, M., Scherbaum, S., Kanske, P., McDonald, B., & Miller, R. (2022) · Meta-Psychology · doi:10.15626/mp.2021.2837 · Cited by 27 · see citing works
Publication Bias in Recent Empirical Accounting ResearchPreprint
Basu, S., & Park, H.-U. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2379889 · Cited by 12 · see citing works
Kwaśnicka, D., ten Hoor, G. A., van Dongen, A., Gruszczyńska, E., Hagger, M. S., Hamilton, K., Hankonen, N. E., Heino, M. T. J., Kotzur, M., Noone, C., et al. (2020) · Health Psychology Review · doi:10.1080/17437199.2020.1844037 · Cited by 24 · see citing works
Assessing robustness to worst case publication bias using a simple subset meta-analysis
Mathur, M. B. (2024) · BMJ · doi:10.1136/bmj-2023-076851 · Cited by 24 · see citing works
- Authors
- Masicampo, E. J., & Lalande, D. R.
- Affiliations
- Wake Forest University (US)
- Université du Québec à Chicoutimi (CA)
- Corresponding author
- Masicampo, E.J.
- Lalande, Daniel R.
- Venue
- Quarterly Journal of Experimental Psychology
- Publisher
- SAGE Publications
- Published
- 2012-11
- Volume
- 65
- Issue
- 11
- Pages
- 2271-2279
- ISSN
- 0033-555X (ISSN-L)
- 1747-0218
- 1747-0226
- OpenAlex
- W2112039868
- Cited by
- 266
- References
- 32
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 17 · 2023: 18 · 2022: 23 · 2021: 24 · 2020: 24 · 2019: 28 · 2018: 26 · 2017: 23 · 2016: 15 · 2015: 21 · 2014: 22 · 2013: 11 · 2012: 4
- FWCI
- 21.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Quarterly Journal of Experimental Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- p-values, null hypothesis significance testing, significance threshold, p-value distribution, publication bias, researcher degrees of freedom, statistical reporting, psychology journals
- MeSH terms
- Humans, Probability, Psychology, Statistics as Topic, Prevalence, Databases, Bibliographic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
