CoChat

Malignant side effects of null-hypothesis significance testing

Branch, Marc · 2014 · Theory & Psychology

The DOI matches its Crossref record; 88 of the 89 references it cites were found in a registry or an index.

Abstract

Six decades-worth of published information has shown irrefutably that null-hypothesis significance tests (NHSTs) provide no information about the reliability of research outcomes. Nevertheless, they are still the core of editorial decision-making in Psychology. Two reasons appear to contribute to the continuing practice. One, survey information suggests that a majority of psychological researchers incorrectly believe that p values provide information about reliability of results. Two, a position sometimes taken is that using them to make decisions has been essentially benign. The mistaken belief has been pointed out many times, so it is briefly covered because of the apparent persistence of the misunderstanding. The idea that NHSTs have been benign is challenged by seven “side-effects” that continue to retard effective development of psychological science. The article concludes with both a few suggestions about possible alternatives and a challenge to psychological researchers to develop new methods that actually assess the reliability of research findings.

Overview

Integrity signals

Registry

Crossref · 10.1177/0959354314525282

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 88 references

Research works printed but not linked: 12

  1. Beaudette C. G., 2002, Excess heat: Why cold fusion research prevailed
  2. Berger J. O., 1988, American Scientist, 76, 159
  3. 10.1080/01621459.1942.10501760 · doi:10.1080/01621459.1942.10501760 · Paper page
  4. Cohen J., 1988, Statistical power analysis for the behavior sciences
  5. Committee on CMMP 2010, Solid State Sciences Committee, National Research Council, 2007, Condensed-matter and materials physics: The science of the world around us
  6. Ebbinghaus H., 1885, Über das Gedächtnis
  7. 10.1201/9780429258411 · doi:10.1201/9780429258411 · Paper page
  8. Haller S., 2002, Methods of Psychological Research, 7, 1
  9. Hays W. L., 1981, Statistics
  10. 10.1037/0003-066X.56.2.119 · doi:10.1037/0003-066x.56.2.119 · Paper page
  11. Oakes M., 1986, Statistical inference: A commentary for the social and behavioral sciences
  12. Piaget J., 1928, The child’s conception of the world

Cited by

Metadata

Authors
Branch, M.
Affiliations
University of Florida (US)
Corresponding author
Branch, Marc
Venue
Theory & Psychology
Published
2014-04
Volume
24
Issue
2
Pages
256-277
ISSN
  • 0959-3543 (ISSN-L)
  • 1461-7447
OpenAlex
W2335213174
Cited by
81
References
89
Citations by year
2026: 3 · 2025: 3 · 2024: 2 · 2023: 9 · 2022: 7 · 2021: 7 · 2020: 5 · 2019: 7 · 2018: 12 · 2017: 12 · 2016: 9 · 2015: 5
FWCI
12.05
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Theory & Psychology (publishedVersion)
Funders
  • U.S. Public Health Service
  • National Institute on Drug Abuse
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
null hypothesis significance testing, p-values, editorial decision-making, research methodology critique

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.