On the probability of making Type I errors.

Paul Pollard, John T. E. Richardson·1987·Psychological Bulletin

Abstract

A statistical test leads to a Type I error whenever it leads to the rejection of a null hypothesis that is in fact true. The probability of making a Type I error can be characterized in the following three ways: the conditional prior probability (the probability of making a Type I error whenever a true null hypothesis is tested), the overall prior probability (the probability of making a Type I error across all experiments), and the conditional posterior probability (the probability of having made a Type I error in situations in which the null hypothesis is rejected). In this article, we show (a) that the alpha level can be equated with the first of these and (b) that it provides an upper bound for the second but (c) that it does not provide an estimate of the third, although it is commonly assumed to do so. We trace the source of this erroneous assumption first to statistical texts used by psychologists, which are generally ambiguous about which of the three interpretations is intended at any point in their discussions of Type I errors and which typically confound the conditional prior and posterior probabilities. Underlying this, however, is a more general fallacy in reasoning about probabilities, and we suggest that this may be the result of erroneous inferences about probabilistic conditional statements. Finally, we consider the possibility of estimating the (posterior) probability of a Type I error in situations in which the null hypothesis is rejected and, hence, the proportion of statistically significant results that may be Type I errors. © 1987 American Psychological Association.

Cites (5)

  1. On the psychology of prediction.
    Kahneman, Daniel, Tversky, Amos · 1973 · Psychological Review
  2. Belief in the law of small numbers.
    Tversky, Amos, Kahneman, Daniel · 1971 · Psychological Bulletin
  3. Consequences of prejudice against the null hypothesis.
    Greenwald, Anthony G. · 1975 · Psychological Bulletin
  4. The case against statistical significance testing
    Carver, Ronald · 1978 · Harvard Educational Review
  5. Quantitative methods for literature reviews
    Green, Bert F., Hall, Judith A. · 1984 · Annual Review of Psychology

Show all 5 references

Cited by (16)

  1. GPOWER: A general power analysis program
    Erdfelder, Edgar, Faul, Franz, Buchner, Axel · 1996 · Behavior Research Methods, Instruments, & Computers
  2. The earth is round (p < .05).
    Cohen, Jacob · 1994 · American Psychologist
  3. Null hypothesis significance testing: a review of an old and continuing controversy
    Nickerson, Raymond S. · 2000 · Psychological Methods
  4. Mindless statistics.
    Gigerenzer, Gerd · 2004 · The Journal of Socio-Economics
  5. Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature
    Szucs, Denes, Ioannidis, John P. A. · 2017 · PLOS Biology

See all 148

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: On the probability of making Type I errors. https://app.cochat.ai/papers/10.1037//0033-2909.102.1.159
The paper
Pollard, P., & Richardson, J. T. E. (1987). On the probability of making Type I errors. Psychological Bulletin. https://doi.org/10.1037//0033-2909.102.1.159

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.