The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
The earth is round (p < .05).
Abstract
After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a sacred .05 criterion) still persists. This article reviews the problems with this practice, including near universal misinterpretation of p as the probability that H₀ is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test. Exploratory data analysis and the use of graphic methods, a steady improvement in and a movement toward standardization in measurement, an emphasis on estimating effect sizes using confidence intervals, and the informed use of available statistical methods are suggested. For generalization, psychologists must finally rely, as has been done in all the older sciences, on replication. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As in any statistical application, accepting the null hypothesis (in this case, a null hypothesis of no interaction) should be understood as being distinct from a failure to reject it (Cohen, 1994).”
From Enrichment Effects on Adult Cognitive Development: Can the Functional Capacity of Older Adults Be Preserved and Enhanced? (2008) · cites this paper as Cohen, 1994
In the citing paper
“Therefore, NHST only tells us about the probability of obtaining data which are equally or more discrepant than those obtained in the event that H0 is true (Cohen, 1994; Nickerson, 2000; Balluerka et al., 2009; Kline, 2013).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Cohen, 1994
In the citing paper
“The most common misconceptions about p-value are “inverse probability fallacy,” “replication fallacy,” “effect size fallacy,” and “clinical or practical significance fallacy” (Carver, 1978; Cohen, 1994; Kirk, 1996; Nickerson, 2000; Kline, 2013).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Cohen, 1994
In the citing paper
“This is might be because of two reasons: the programmer could have found a problem while coding, who at first will try to solve it by reading the code, switching out from the IDE, navigating the call stack and debugging, before going to ask another teammate, according to LaToza et al. [35]; this set of actions end up with an interruption and reduce the observed activity within the IDE.”
From Work fragmentation in developer interaction data (2017) · cites this paper as [35]
In the citing paper
“These categories of events agree with our previous hypothesis that positive interruptions are related to information seeking [9,35], while negative interruptions are related to actual task switches where context needs to be rebuilt afterwards.”
From Work fragmentation in developer interaction data (2017) · cites this paper as [35]
In the citing paper
“Our results and theoretical foundations are based on observational studies performed on information workers' development environments [5,6,35,10].”
From Work fragmentation in developer interaction data (2017) · cites this paper as [35]
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Self-citationStatistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Self-citationDavid, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,707
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The Logic of Scientific Discovery
Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765
The visual display of quantitative information
(1985) · General Pharmacology The Vascular System · doi:10.1016/0306-3623(85)90230-7 · Cited by 5,090
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Kemp, C. D., & Tufte, E. R. (1992) · Biometrics · doi:10.2307/2532323 · Cited by 2,728
Meehl, P. E. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.4.806 · Cited by 2,061
Cited by
Sorted by citations
Items 1,001–1,029 of 1,029
Inconsistencies between reported test statistics and p‐values in two psychiatry journals
Berle, D., & Starcevic, V. (2007) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.225 · Cited by 38
Polat, N., Mahalingappa, L., Hughes, E., & Karayigit, C. (2019) · International Multilingual Research Journal · doi:10.1080/19313152.2019.1597607 · Cited by 39
Corkin, M. T., Henderson, A. M. E., Peterson, E. R., Kennedy- Costantini, S., Sharplin, H. S., & Morrison, S. (2021) · Infant Behavior and Development · doi:10.1016/j.infbeh.2021.101611 · Cited by 39
Misconceptions of the p-value among Chilean and Italian Academic Psychologists
Badenes-Ribera, L., Frias-Navarro, D., Iotti, B., Bonilla-Campos, A., & Longobardi, C. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01247 · Cited by 35
“Therefore, NHST only tells us about the probability of obtaining data which are equally or more discrepant than those obtained in the event that H0 is true (Cohen, 1994; Nickerson, 2000; Balluerka et al., 2009; Kline, 2013).”
BackgroundResultInfluentialAschieri, F., & Smith, J. D. (2012) · Journal of Personality Assessment · doi:10.1080/00223891.2011.627964 · Cited by 38
Do hotter temperatures increase the incidence of self-harm hospitalisations?
Williams, M. N., Hill, S. R., & Spicer, J. (2016) · Psychology, Health & Medicine · doi:10.1080/13548506.2015.1028945 · Cited by 36
Fowler, C. A., Ballistrea, L. M., Mazzone, K. E., Martin, A. M., Kaplan, H., Kip, K. E., Ralston, K., Murphy, J. L., & Winkler, S. L. (2019) · JMIR Formative Research · doi:10.2196/11266 · Cited by 36
Worden, R. E., Bonner, H. S., & McLean, S. J. (2018) · Police Quarterly · doi:10.1177/1098611117739812 · Cited by 31
Mahalingappa, L., Hughes, E. M., & Polat, N. (2018) · Language and Education · doi:10.1080/09500782.2017.1417996 · Cited by 30
Ortega, A., & Navarrete, G. (2017) · Bayesian Inference · doi:10.5772/intechopen.70230 · Cited by 31
Zhang, S., Heck, P. R., Meyer, M. N., Chabris, C. F., Goldstein, D. G., & Hofman, J. M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2302491120 · Cited by 30
Uppercase Premium Effect: The Role of Brand Letter Case in Brand Premiumness
Yu, Y., Zhou, X., Wang, L., & Wang, Q. (2022) · Journal of Retailing · doi:10.1016/j.jretai.2021.03.002 · Cited by 29
Eminoğlu, A., & Çelikkanat, Ş. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105386 · Cited by 30
Consistency errors in p-values reported in Spanish psychology journals
Caperos, J., & Pardo, A. (2013) · Psicothema · doi:10.7334/psicothema2012.207 · Cited by 23
Dominique-Ferreira, S. (2017) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2016.09.013 · Cited by 25
Heteronormative relationship education for same-gender couples
Hatch, S. G., Rothman, K., Roddy, M. K., Dominguez, R. M., Le, Y., & Doss, B. D. (2021) · Family Process · doi:10.1111/famp.12548 · Cited by 23
Rabovsky, K., Trombini, M., Allemann, D., & Stoppe, G. (2012) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-012-0291-1 · Cited by 20
Shi, D., Song, H., DiStefano, C., Maydeu-Olivares, A., McDaniel, H. L., & Jiang, Z. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1514484 · Cited by 19
Finch, S., Cumming, G., & Thomason, N. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401612001 · Cited by 52
Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2014) · Quality of Life Research · doi:10.1007/s11136-013-0437-2 · Cited by 18
Theobald, D. L. (2011) · Biology Direct · doi:10.1186/1745-6150-6-60 · Cited by 16
“Frequentist hypothesis tests (of both Fisher and Neyman-Pearson flavors) have been highly criticized and disparaged for many decades by professional statisticians [44-71].”
BackgroundWilliams, Z. J., Suzman, E., & Woynaroski, T. G. (2021) · American Journal of Audiology · doi:10.1044/2021_aja-20-00209 · Cited by 16
Monterde-i-Bort, H., Frías-Navarro, D., & Pascual-Llobell, J. (2010) · European Journal of Psychology of Education · doi:10.1007/s10212-010-0021-x · Cited by 14
Follow‐up on the U.S. Central Intelligence Agency's (CIA) remote viewing experiments☆
Escolà‐Gascón, Á., Houran, J., Dagnall, N., Drinkwater, K., & Denovan, A. (2023) · Brain and Behavior · doi:10.1002/brb3.3026 · Cited by 6
Work fragmentation in developer interaction data
Cruz, L. C., Sanchez, H., González, V. M., & Robbes, R. (2017) · Journal of Software: Evolution and Process · doi:10.1002/smr.1839 · Cited by 6
Sadeghi-Bahmani, D., Rigotti, V., Stanga, Z., Lang, U. E., Blais, R. K., Kelley, M. L., & Brand, S. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1390636 · Cited by 6
“For t-tests, Cohen’s d effect sizes are reported with the following cut-off values: Trivial (ds: 0–0.19), small (ds: 0.20–0.49), medium (ds: 0.50–0.79) or large (ds: 0.80 and greater) (146–148).”
MethodInfluentialManolov, R., & Onghena, P. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02230-1 · Cited by 5
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
Evans (2000) · doi:10.1002/(sici)1520-6807(200003)37:2
Metadata
- Authors
- Cohen, J.
- Affiliations
- New York University (US)
- Corresponding author
- Cohen, Jacob
- Venue
- American Psychologist
- Published
- 1994-12
- Volume
- 49
- Issue
- 12
- Pages
- 997-1003
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2030360178
- Cited by
- 4,003
- References
- 43
- Citations by year
- 2026: 94 · 2025: 122 · 2024: 87 · 2023: 116 · 2022: 128 · 2021: 161 · 2020: 255 · 2019: 178 · 2018: 184 · 2017: 177 · 2016: 187 · 2015: 182 · 2014: 183 · 2013: 167 · 2012: 172
- FWCI
- 87.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Psychologist (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- null hypothesis significance testing, misinterpretation of p-values, replication, effect size estimation, confidence intervals, exploratory data analysis, measurement standardization, psychology research practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
