The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
The earth is round (p < .05).
Abstract
After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a sacred .05 criterion) still persists. This article reviews the problems with this practice, including near universal misinterpretation of p as the probability that H₀ is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test. Exploratory data analysis and the use of graphic methods, a steady improvement in and a movement toward standardization in measurement, an emphasis on estimating effect sizes using confidence intervals, and the informed use of available statistical methods are suggested. For generalization, psychologists must finally rely, as has been done in all the older sciences, on replication. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As in any statistical application, accepting the null hypothesis (in this case, a null hypothesis of no interaction) should be understood as being distinct from a failure to reject it (Cohen, 1994).”
From Enrichment Effects on Adult Cognitive Development: Can the Functional Capacity of Older Adults Be Preserved and Enhanced? (2008) · cites this paper as Cohen, 1994
In the citing paper
“Therefore, NHST only tells us about the probability of obtaining data which are equally or more discrepant than those obtained in the event that H0 is true (Cohen, 1994; Nickerson, 2000; Balluerka et al., 2009; Kline, 2013).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Cohen, 1994
In the citing paper
“The most common misconceptions about p-value are “inverse probability fallacy,” “replication fallacy,” “effect size fallacy,” and “clinical or practical significance fallacy” (Carver, 1978; Cohen, 1994; Kirk, 1996; Nickerson, 2000; Kline, 2013).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Cohen, 1994
In the citing paper
“This is might be because of two reasons: the programmer could have found a problem while coding, who at first will try to solve it by reading the code, switching out from the IDE, navigating the call stack and debugging, before going to ask another teammate, according to LaToza et al. [35]; this set of actions end up with an interruption and reduce the observed activity within the IDE.”
From Work fragmentation in developer interaction data (2017) · cites this paper as [35]
In the citing paper
“These categories of events agree with our previous hypothesis that positive interruptions are related to information seeking [9,35], while negative interruptions are related to actual task switches where context needs to be rebuilt afterwards.”
From Work fragmentation in developer interaction data (2017) · cites this paper as [35]
In the citing paper
“Our results and theoretical foundations are based on observational studies performed on information workers' development environments [5,6,35,10].”
From Work fragmentation in developer interaction data (2017) · cites this paper as [35]
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Self-citationStatistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Self-citationDavid, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,707
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The Logic of Scientific Discovery
Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765
The visual display of quantitative information
(1985) · General Pharmacology The Vascular System · doi:10.1016/0306-3623(85)90230-7 · Cited by 5,090
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Kemp, C. D., & Tufte, E. R. (1992) · Biometrics · doi:10.2307/2532323 · Cited by 2,728
Meehl, P. E. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.4.806 · Cited by 2,061
Tukey, J. W. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704711 · Cited by 1,934
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Self-citationCleveland, W. S. (1993) · Cited by 1,698
Statistical significance in psychological research.
Lykken, D. T. (1968) · Psychological bulletin · doi:10.1037/h0026141 · Cited by 1,363
Rational choice in an uncertain world
Dawes, R. M., & Kagan, J. (1988) · Medical Entomology and Zoology · Cited by 1,291
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 2×
“Meehl (1990a) attributed to the distinguished philosopher Morris Raphael Cohen the saying "All logic texts are divided into two parts.”
BackgroundMethodTheory-Testing in Psychology and Physics: A Methodological Paradox
Meehl, P. E. (1967) · Philosophy of Science · doi:10.1086/288135 · Cited by 962
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 957
Statistical Methods for Research Workers
Tukey, J. W., & Fisher, R. A. (1952) · Econometrica · doi:10.2307/1907425 · Cited by 950
WHY SUMMARIES OF RESEARCH ON PSYCHOLOGICAL THEORIES ARE OFTEN UNINTERPRETABLE
Meehl, P. E. (1990) · Psychological Reports · doi:10.2466/pr0.1990.66.1.195 · Cited by 934
The Philosophy of Multiple Comparisons
Tukey, J. W. (1991) · Statistical Science · doi:10.1214/ss/1177011945 · Cited by 773
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 740
Meehl, P. E. (1990) · Psychological Inquiry · doi:10.1207/s15327965pli0102_1 · Cited by 673
What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology.
Schmidt, F. L. (1992) · American Psychologist · doi:10.1037/0003-066x.47.10.1173 · Cited by 564 · cited 2×
“Forty-two years ago, Frank Yates, a close colleague and friend of R. A. Fisher, wrote about Fisher's "Statistical Methods for Research Workers" (1925/1951), It has caused scientific research workers to pay undue attention to the results of the tests of significance they perform on their data. . . and too little to the…”
BackgroundThings I have learned (so far).
Cohen, J. (1992) · American Psychological Association eBooks · doi:10.1037/10109-028 · Cited by 553
Self-citationA Handbook for data analysis in the behavioral sciences: methodological issues
(1993) · Choice Reviews Online · doi:10.5860/choice.31-0602 · Cited by 544
The Significance Test Controversy.
Carter, L. F., Morrison, D. E., & Henkel, R. E. (1971) · Social Forces · doi:10.2307/2576949 · Cited by 443
Statistical Inference: A Commentary for the Social and Behavioural Sciences
Oakes, M. W. (1986) · Medical Entomology and Zoology · Cited by 388
Analyzing data: Sanctification or detective work?
Tukey, J. W. (1969) · American Psychologist · doi:10.1037/h0027108 · Cited by 321
Some Difficulties of Interpretation Encountered in the Application of the Chi-Square Test
Berkson, J. (1938) · Journal of the American Statistical Association · doi:10.1080/01621459.1938.10502329 · Cited by 317
Significance Tests Die Hard: The Amazing Persistence of a Probabilistic Misconception
Falk, R., & Greenbaum, C. W. (1995) · Theory & Psychology · doi:10.1177/0959354395051004 · Cited by 279
Dynamic Graphics for Statistics
Buckingham, J. (1990) · Technometrics · doi:10.1080/00401706.1990.10484731 · Cited by 279
Some Difficulties of Interpretation Encountered in the Application of the Chi-Square Test
Berkson, J. (1938) · Journal of the American Statistical Association · doi:10.2307/2279690 · Cited by 259
The Superego, the Ego, and the Id in Statistical Reasoning
Gigerenzer, G. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.003.0013 · Cited by 207
Wainer, H., & Thissen, D. (1981) · Annual Review of Psychology · doi:10.1146/annurev.ps.32.020181.001203 · Cited by 159
On the probability of making Type I errors.
Pollard, P., & Richardson, J. T. E. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.102.1.159 · Cited by 155
Yates, F. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500764 · Cited by 118
Nachmias, D. (1987) · American Political Science Review · doi:10.2307/1962690 · Cited by 114
Two and One‐Half Decades of Leadership in Measurement and Evaluation
THOMPSON, B. (1992) · Journal of Counseling & Development · doi:10.1002/j.1556-6676.1992.tb01631.x · Cited by 97
Confidence Curves: An Omnibus Technique for Estimation and Testing Statistical Hypotheses
Birnbaum, A. (1961) · Journal of the American Statistical Association · doi:10.1080/01621459.1961.10482107 · Cited by 80
Rational appraisal of psychological research and the good-enough principle.
Serlin, R. C., & Lapsley, D. K. (1993) · Cited by 78
A Handbook for data analysis in the behavioral sciences: statistical issues
(1993) · Choice Reviews Online · doi:10.5860/choice.31-0603 · Cited by 78
Some Distributions of Sample Means
Brown, G. W., & Tukey, J. W. (1946) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177731017 · Cited by 64
Cited by
Sorted by citations
Mediation in experimental and nonexperimental studies: New procedures and recommendations.
Shrout, P. E., & Bolger, N. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.4.422 · Cited by 11,335
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 12,051
Corporate Social and Financial Performance: A Meta-Analysis
Orlitzky, M., Schmidt, F. L., & Rynes, S. L. (2003) · Organization Studies · doi:10.1177/0170840603024003910 · Cited by 7,641
Using pls path modeling in new technology research: updated guidelines
Henseler, J., Hubona, G., & Ray, P. A. (2016) · Industrial Management & Data Systems · doi:10.1108/imds-09-2015-0382 · Cited by 7,173
Assessing measurement model quality in PLS-SEM using confirmatory composite analysis
Hair, J. F., Howard, M. C., & Nitzl, C. (2020) · Journal of Business Research · doi:10.1016/j.jbusres.2019.11.069 · Cited by 5,446
Metadata
- Authors
- Cohen, J.
- Affiliations
- New York University (US)
- Corresponding author
- Cohen, Jacob
- Venue
- American Psychologist
- Published
- 1994-12
- Volume
- 49
- Issue
- 12
- Pages
- 997-1003
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2030360178
- Cited by
- 4,003
- References
- 43
- Citations by year
- 2026: 94 · 2025: 122 · 2024: 87 · 2023: 116 · 2022: 128 · 2021: 161 · 2020: 255 · 2019: 178 · 2018: 184 · 2017: 177 · 2016: 187 · 2015: 182 · 2014: 183 · 2013: 167 · 2012: 172
- FWCI
- 87.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Psychologist (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- null hypothesis significance testing, misinterpretation of p-values, replication, effect size estimation, confidence intervals, exploratory data analysis, measurement standardization, psychology research practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
