Significance tests and the duplicity of binary decisions.

Folger, Robert·1989·Psychological Bulletin

Abstract

Presents a logical justification for the following statements and discusses their implications: It is duplicitous (misleading) to use significance tests for making binary (either/or) decisions regarding the validity of a theory; the binary choice between calling results significant or not significant should not govern the confidence placed i n a theory, because such confidence cannot be gained in the either/ or fashion characterizing deductive certainty. The implications include grounds for describing ways that effect size estimates become useful in makingjudgments about the value of theories. Charles Sanders Peirce, who some say was America's greatest philosopher of science, wrote that "science can only rest on experience"; he then added that "experience can never result in absolute certainty, exactitude, or universality " (Buchler, 1955, p. 47). Hypothesis-testing procedures make explicit science's lack of certainty. In particular, inferential statistics provide estimates of whether a finding reflects merely random variation, an outcome owing to "chance " rather than to causal mechanisms hypothesized by theories. I contend that such procedures can

Cites (3)

  1. Falsification and the Methodology of Scientific Research Programmes
    Lakatos, I. · 1970 · Criticism and the Growth of Knowledge
  2. The Aim and Structure of Physical Theory
    Duhem, Pierre Maurice Marie · 1954
  3. The effects of severity of initiation on liking for a group: A replication
    Gerard, Harold B, Mathewson, Grover C · 1966 · Journal of Experimental Social Psychology

Show all 3 references

Cited by (6)

  1. Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies
    Voorhees, Clay M., Brady, Michael K., Calantone, Roger, Ramirez, Edward · 2016 · Journal of the Academy of Marketing Science
  2. Null hypothesis significance testing: a review of an old and continuing controversy
    Nickerson, Raymond S. · 2000 · Psychological Methods
  3. Replication, falsification, and the crisis of confidence in social psychology
    Earp, Brian D., Trafimow, David · 2015 · Frontiers in Psychology
  4. Effect Size, Practical Importance, and Social Policy for Children
    McCartney, Kathleen, Rosenthal, Robert · 2000 · Child Development
  5. Significance Tests Die Hard: The Amazing Persistence of a Probabilistic Misconception
    Falk, Ruma, Greenbaum, Charles W. · 1995 · Theory & Psychology

See all 45

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Significance tests and the duplicity of binary decisions. https://app.cochat.ai/papers/10.1037//0033-2909.106.1.155
The paper
Folger, R. (1989). Significance tests and the duplicity of binary decisions. Psychological Bulletin. https://doi.org/10.1037//0033-2909.106.1.155

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.