CoChat
Sign Up
Cited by 4,003Meta-analysis and systematic reviewsStatistical Methods and Bayesian InferencePsychology Research and Bibliometrics

The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.

The earth is round (p < .05).

Cohen, Jacob·1994·American Psychologist
DOI 10.1037/0003-066x.49.12.997
Open this paper in CoChat — free account

Abstract

After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a sacred .05 criterion) still persists. This article reviews the problems with this practice, including near universal misinterpretation of p as the probability that H₀ is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test. Exploratory data analysis and the use of graphic methods, a steady improvement in and a movement toward standardization in measurement, an emphasis on estimating effect sizes using confidence intervals, and the informed use of available statistical methods are suggested. For generalization, psychologists must finally rely, as has been done in all the older sciences, on replication. (PsycINFO Database Record (c) 2012 APA, all rights reserved)

Overview

Integrity signals

Registry

Crossref · 10.1037/0003-066x.49.12.997

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Accepting the null hypothesis should be understood as distinct from failing to reject it.

    “As in any statistical application, accepting the null hypothesis (in this case, a null hypothesis of no interaction) should be understood as being distinct from a failure to reject it (Cohen, 1994).”

    From Enrichment Effects on Adult Cognitive Development: Can the Functional Capacity of Older Adults Be Preserved and Enhanced? (2008) · cites this paper as Cohen, 1994

  • NHST only tells us about the probability of obtaining data which are equally or more discrepant than those obtained in the event that H0 is true.

    “Therefore, NHST only tells us about the probability of obtaining data which are equally or more discrepant than those obtained in the event that H0 is true (Cohen, 1994; Nickerson, 2000; Balluerka et al., 2009; Kline, 2013).”

    From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Cohen, 1994

  • The most common misconceptions about p-values are the inverse probability fallacy, the replication fallacy, the effect size fallacy, and the clinical or practical significance fallacy.

    “The most common misconceptions about p-value are “inverse probability fallacy,” “replication fallacy,” “effect size fallacy,” and “clinical or practical significance fallacy” (Carver, 1978; Cohen, 1994; Kirk, 1996; Nickerson, 2000; Kline, 2013).”

    From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Cohen, 1994

  • The drop in activity before an interruption may be because the programmer found a problem and engaged in debugging or asking teammates, which reduces IDE activity.

    “This is might be because of two reasons: the programmer could have found a problem while coding, who at first will try to solve it by reading the code, switching out from the IDE, navigating the call stack and debugging, before going to ask another teammate, according to LaToza et al. [35]; this set of actions end up with an interruption and reduce the observed activity within the IDE.”

    From Work fragmentation in developer interaction data (2017) · cites this paper as [35]

  • These categories of events agree with the previous hypothesis that positive interruptions are related to information seeking, while negative interruptions are related to actual task switches where context needs to be rebuilt afterwards.

    “These categories of events agree with our previous hypothesis that positive interruptions are related to information seeking [9,35], while negative interruptions are related to actual task switches where context needs to be rebuilt afterwards.”

    From Work fragmentation in developer interaction data (2017) · cites this paper as [35]

Citations

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Metadata

Authors
Cohen, J.
Affiliations
New York University (US)
Corresponding author
Cohen, Jacob
Venue
American Psychologist
Publisher
American Psychological Association (APA)
Published
1994-12
Volume
49
Issue
12
Pages
997-1003
ISSN
  • 0003-066X (ISSN-L)
  • 1935-990X
DOI
10.1037/0003-066x.49.12.997
OpenAlex
W2030360178
Cited by
4,003
References
43
Citations by year
2026: 94 · 2025: 122 · 2024: 87 · 2023: 116 · 2022: 128 · 2021: 161 · 2020: 255 · 2019: 178 · 2018: 184 · 2017: 177 · 2016: 187 · 2015: 182 · 2014: 183 · 2013: 167 · 2012: 172
FWCI
87.17
Type
journal article
Language
en
Open access
closed
Locations
American Psychologist (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
null hypothesis significance testing, misinterpretation of p-values, replication, effect size estimation, confidence intervals, exploratory data analysis, measurement standardization, psychology research practice

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review