On the probability of making Type I errors.
Abstract
A statistical test leads to a Type I error whenever it leads to the rejection of a null hypothesis that is in fact true. The probability of making a Type I error can be characterized in the following three ways: the conditional prior probability (the probability of making a Type I error whenever a true null hypothesis is tested), the overall prior probability (the probability of making a Type I error across all experiments), and the conditional posterior probability (the probability of having made a Type I error in situations in which the null hypothesis is rejected). In this article, we show (a) that the alpha level can be equated with the first of these and (b) that it provides an upper bound for the second but (c) that it does not provide an estimate of the third, although it is commonly assumed to do so. We trace the source of this erroneous assumption first to statistical texts used by psychologists, which are generally ambiguous about which of the three interpretations is intended at any point in their discussions of Type I errors and which typically confound the conditional prior and posterior probabilities. Underlying this, however, is a more general fallacy in reasoning about probabilities, and we suggest that this may be the result of erroneous inferences about probabilistic conditional statements. Finally, we consider the possibility of estimating the (posterior) probability of a Type I error in situations in which the null hypothesis is rejected and, hence, the proportion of statistically significant results that may be Type I errors. © 1987 American Psychological Association.
Cites (5)
- On the psychology of prediction.Kahneman, Daniel, Tversky, Amos · 1973 · Psychological Review
- Belief in the law of small numbers.Tversky, Amos, Kahneman, Daniel · 1971 · Psychological Bulletin
- Consequences of prejudice against the null hypothesis.Greenwald, Anthony G. · 1975 · Psychological Bulletin
- The case against statistical significance testingCarver, Ronald · 1978 · Harvard Educational Review
- Quantitative methods for literature reviewsGreen, Bert F., Hall, Judith A. · 1984 · Annual Review of Psychology
Cited by (16)
- GPOWER: A general power analysis programErdfelder, Edgar, Faul, Franz, Buchner, Axel · 1996 · Behavior Research Methods, Instruments, & Computers
- The earth is round (p < .05).Cohen, Jacob · 1994 · American Psychologist
- Null hypothesis significance testing: a review of an old and continuing controversyNickerson, Raymond S. · 2000 · Psychological Methods
- Mindless statistics.Gigerenzer, Gerd · 2004 · The Journal of Socio-Economics
- Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literatureSzucs, Denes, Ioannidis, John P. A. · 2017 · PLOS Biology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the probability of making Type I errors. https://app.cochat.ai/papers/10.1037//0033-2909.102.1.159
- The paper
- Pollard, P., & Richardson, J. T. E. (1987). On the probability of making Type I errors. Psychological Bulletin. https://doi.org/10.1037//0033-2909.102.1.159
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.