Null hypothesis significance testing: On the survival of a flawed method.
Abstract
Null hypothesis significance testing (NHST) is the researcher's workhorse for making inductive inferences. This method has often been challenged, has occasionally been defended, and has persistently been used through most of the history of scientific psychology. This article reviews both the criticisms of NHST and the arguments brought to its defense. The review shows that the criticisms address the logical validity of inferences arising from NHST, whereas the defenses stress the pragmatic value of these inferences. The author suggests that both critics and apologists implicitly rely on Bayesian assumptions. When these assumptions are made explicit, the primary challenge for NHST--and any system of induction--can be confronted. The challenge is to find a solution to the question of replicability.
Cites (49)
- Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty.Tversky, Amos, Kahneman, Daniel · 1974 · Science
- Judgment under Uncertainty: Heuristics and BiasesTversky, Amos, Kahneman, Daniel · 1975 · Utility, Probability, and Human Decision Making
- A treatise of human nature (1739-40)Hume, David · 1996 · The Sublime
- Distribution Theory for Glass's Estimator of Effect Size and Related EstimatorsHedges, Larry V. · 1981 · Journal of Educational Statistics
- Design of ExperimentsFisher, R. A. · 1936 · BMJ
- The earth is round (p < .05).Cohen, Jacob · 1994 · American Psychologist
- Belief in the law of small numbers.Tversky, Amos, Kahneman, Daniel · 1971 · Psychological Bulletin
- Statistical methods in psychology journals: Guidelines and explanations.Wilkinson, Leland, Task Force on Statistical Inference · 1999 · American Psychologist
- Introduction to the Practice of Statistics2004 · Technometrics
- Statistics for the social sciencesWilliam L. Hays · 1973 · Medical Entomology and Zoology
Cited by (24)
- A tutorial on a practical Bayesian alternative to null-hypothesis significance testingMasson, Michael E. J. · 2011 · Behavior Research Methods
- Using confidence intervals for graphically based data interpretation.Masson, Michael E. J., Loftus, Geoffrey R. · 2003 · Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale
- Cautionary note on reporting eta-squared values from multifactor ANOVA designs.Pierce, Charles A., Block, Richard A., Aguinis, Herman · 2004 · Educational and Psychological Measurement
- The prevalence of statistical reporting errors in psychology (1985–2013)Nuijten, Michèle B., Hartgerink, Chris H. J., van Assen, Marcel A. L. M., Epskamp, Sacha, Wicherts, Jelte M. · 2016 · Behavior Research Methods
- Social Projection to Ingroups and Outgroups: A Review and Meta-AnalysisRobbins, Jordan M., Krueger, Joachim I. · 2005 · Personality and Social Psychology Review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Null hypothesis significance testing: On the survival of a flawed method. https://app.cochat.ai/papers/10.1037//0003-066x.56.1.16
- The paper
- Krueger, J. (2001). Null hypothesis significance testing: On the survival of a flawed method. American Psychologist. https://doi.org/10.1037//0003-066x.56.1.16
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.