The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
A peculiar prevalence of p values just below .05
Abstract
In null hypothesis significance testing (NHST), p values are judged relative to an arbitrary threshold for significance (.05). The present work examined whether that standard influences the distribution of p values reported in the psychology literature. We examined a large subset of papers from three highly regarded journals. Distributions of p were found to be similar across the different journals. Moreover, p values were much more common immediately below .05 than would be expected based on the number of p values occurring in other ranges. This prevalence of p values just below the arbitrary criterion for significance was observed in all three journals. We discuss potential sources of this pattern, including publication bias and researcher degrees of freedom.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many published studies only barely achieve nominal statistical significance 15 .”
From Power failure: why small sample size undermines the reliability of neuroscience (2013) · cites this paper as [15]
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Sackett, D. L. (1979) · Journal of Chronic Diseases · doi:10.1016/0021-9681(79)90012-2 · Cited by 2,075
The review of educational research.
(1931) · Journal of Educational Psychology · doi:10.1037/h0066448 · Cited by 1,575
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,044
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034
Cited by
Sorted by citations
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
Metadata
- Authors
- Masicampo, E. J., & Lalande, D. R.
- Affiliations
- Wake Forest University (US)
- Université du Québec à Chicoutimi (CA)
- Corresponding author
- Masicampo, E.J.
- Lalande, Daniel R.
- Venue
- Quarterly Journal of Experimental Psychology
- Publisher
- SAGE Publications
- Published
- 2012-11
- Volume
- 65
- Issue
- 11
- Pages
- 2271-2279
- ISSN
- 0033-555X (ISSN-L)
- 1747-0218
- 1747-0226
- OpenAlex
- W2112039868
- Cited by
- 266
- References
- 32
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 17 · 2023: 18 · 2022: 23 · 2021: 24 · 2020: 24 · 2019: 28 · 2018: 26 · 2017: 23 · 2016: 15 · 2015: 21 · 2014: 22 · 2013: 11 · 2012: 4
- FWCI
- 21.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Quarterly Journal of Experimental Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- p-values, null hypothesis significance testing, significance threshold, p-value distribution, publication bias, researcher degrees of freedom, statistical reporting, psychology journals
- MeSH terms
- Humans, Probability, Psychology, Statistics as Topic, Prevalence, Databases, Bibliographic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
