The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Confusion Over Measures of Evidence (p's) Versus Errors (α's) in Classical Statistical Testing
Abstract
Confusion surrounding the reporting and interpretation of results of classical statistical tests is widespread among applied researchers, most of whom erroneously believe that such tests are prescribed by a single coherent theory of statistical inference. This is not the case: Classical statistical testing is an anonymous hybrid of the competing and frequently contradictory approaches formulated by R. A. Fisher on the one hand, and Jerzy Neyman and Egon Pearson on the other. In particular, there is a widespread failure to appreciate the incompatibility of Fisher's evidential p value with the Type I error rate, α, of Neyman-Pearson statistical orthodoxy. The distinction between evidence (p's) and error (α's) is not trivial. Instead, it reflects the fundamental differences between Fisher's ideas on significance testing and inductive inference, and Neyman-Pearson's views on hypothesis testing and inductive behavior. The emphasis of the article is to expose this incompatibility, but we also briefly note a possible reconciliation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Philosophical transactions of the Royal Society of London. B
Press, H. (1886) · Cited by 10,080
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Mathematical Statistics and Data Analysis
Rabinowitz, L. (1989) · Technometrics · doi:10.1080/00401706.1989.10488573 · Cited by 3,116
IX. On the problem of the most efficient tests of statistical hypotheses
Neyman, J., & Pearson, E. S. (1933) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1933.0009 · Cited by 2,341
A First Course in Probability.
Sherman, M. J., & Ross, S. (1984) · Journal of the American Statistical Association · doi:10.2307/2288753 · Cited by 1,344
Statistical Methods and Scientific Inference.
Bartlett, M. S., & Fisher, R. A. (1957) · Biometrika · doi:10.2307/2333276 · Cited by 1,278
The Logic of Inductive Inference
Fisher, R. A. (1935) · Journal of the Royal Statistical Society · doi:10.2307/2342435 · Cited by 1,265
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,185
ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE
NEYMAN, J., & PEARSON, E. S. (1928) · Biometrika · doi:10.1093/biomet/20a.3-4.263 · Cited by 1,070
The Arrangement of Field Experiments
Fisher, R. A. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_8 · Cited by 964
124: The Logic of Inductive Inference.
Fisher, R. A. (1935) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 881
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 854
Self-citationTesting a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478397 · Cited by 775
Berger, J. O., & Delampady, M. (1987) · Statistical Science · doi:10.1214/ss/1177013238 · Cited by 664
Statistical Methods and Scientific Induction
Fisher, R. (1955) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1955.tb00180.x · Cited by 596
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.2307/2289131 · Cited by 548
Reconciling Bayesian and Frequentist Evidence in the One-Sided Testing Problem
Casella, G., & Berger, R. L. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478396 · Cited by 453
The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two?
Lehmann, E. L. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476404 · Cited by 382
Objective Bayesian Methods for Model Selection: Introduction and Comparison
Berger, J. O., & Pericchi, L. R. (2001) · Institute of Mathematical Statistics Lecture Notes - Monograph Series · doi:10.1214/lnms/1215540968 · Cited by 355
Could Fisher, Jeffreys and Neyman Have Agreed on Testing?
Berger, J. O. (2003) · Statistical Science · doi:10.1214/ss/1056397485 · Cited by 346
P Values for Composite Null Models
Bayarri, M. J., & Berger, J. O. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474309 · Cited by 335
Self-citationA comment on replication,P‐values and evidence
Goodman, S. N. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110705 · Cited by 309
Rothman, J. (1978) · New England Journal of Medicine · doi:10.1056/nejm197812142992410 · Cited by 303
Goodman, S. N. (1993) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116700 · Cited by 303
The Behavior of the P-Value When the Alternative Hypothesis is True
Hung, H. M. J., O'Neill, R. T., Bauer, P., & Kohne, K. (1997) · Biometrics · doi:10.2307/2533093 · Cited by 281
P Values: What They are and What They are Not
Schervish, M. J. (1996) · The American Statistician · doi:10.1080/00031305.1996.10474380 · Cited by 244
Frequentist probability and frequentist statistics
Neyman, J. (1977) · Synthese · doi:10.1007/bf00485695 · Cited by 230
On Probability as a Basis for Action
Deming, W. E. (1975) · The American Statistician · doi:10.1080/00031305.1975.10477402 · Cited by 202
P Values: What They Are and What They Are Not
Schervish, M. J. (1996) · The American Statistician · doi:10.2307/2684655 · Cited by 198
P Values for Composite Null Models
Bayarri, M. J., & Berger, J. O. (2000) · Journal of the American Statistical Association · doi:10.2307/2669749 · Cited by 186
Self-citationBayesian Analysis: A Look at Today and Thoughts of Tomorrow
Berger, J. O. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474328 · Cited by 155
P-Values: Interpretation and Methodology
Gibbons, J. D., & Pratt, J. W. (1975) · The American Statistician · doi:10.2307/2683674 · Cited by 121
Some Thoughts on Statistical Inference
Pearson, E. S. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704566 · Cited by 103
(1999) · Choice Reviews Online · doi:10.5860/choice.37-1497 · Cited by 92
R. A. Fisher: The Founder of Modern Statistics
Rao, C. R. (1992) · Statistical Science · doi:10.1214/ss/1177011442 · Cited by 72
The problem of inductive inference
Neyman, J. (1955) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.3160080103 · Cited by 71
Tests of Significance in Theory and Practice
Johnstone, D. J., Barnard, G. A., & Lindley, D. V. (1986) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2987965 · Cited by 63
Philosophical Problems of Statistical Inference. Learning from R. A. Fisher.
Marriott, F. H. C., & Seidenfeld, T. (1980) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981990 · Cited by 61
The Logic of Tests of Significance
Spielman, S. (1974) · Philosophy of Science · doi:10.1086/288590 · Cited by 54
Tests of Significance Following R. A. Fisher1
Johnstone, D. J. (1987) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/38.4.481 · Cited by 51
A Note on Information Seldom Reported via the P Value
Donahue, R. M. J. (1999) · The American Statistician · doi:10.1080/00031305.1999.10474481 · Cited by 27
R. A. Fisher (1890—1962): An Appreciation
Neyman, J. (1967) · Science · doi:10.1126/science.156.3781.1456 · Cited by 25
On the Interpretation of Hypothesis Tests following Neyman and Pearson
Johnstone, D. J. (1987) · Springer eBooks · doi:10.1007/978-1-4613-1885-9_28 · Cited by 15
HINKLEY, D. V. (1987) · Journal of the American Statistical Association · doi:10.2307/2289134 · Cited by 5
Cited by
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Ellison, A. M. (2004) · Ecology Letters · doi:10.1111/j.1461-0248.2004.00603.x · Cited by 786
Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method
Wagenmakers, E.-J., Lodewyckx, T., Kuriyal, H., & Grasman, R. (2010) · Cognitive Psychology · doi:10.1016/j.cogpsych.2009.12.001 · Cited by 740
Triballeau, N., Acher, F., Brabet, I., Pin, J.-P., & Bertrand, H.-O. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm049092j · Cited by 631
The epistemology of mathematical and statistical modeling: A quiet methodological revolution.
Rodgers, J. L. (2010) · American Psychologist · doi:10.1037/a0018326 · Cited by 402
Metadata
- Authors
- Hubbard, R., & Bayarri, M. J.
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2003-08
- Volume
- 57
- Issue
- 3
- Pages
- 171-178
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W2104162921
- Cited by
- 287
- References
- 45
- FWCI
- 5.54
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.