The DOI matches its Crossref record; 274 of the 274 references it cites were found in a registry or an index.
Final Collapse of the Neyman-Pearson Decision Theoretic Framework and Rise of the neoFisherian
Abstract
This essay grew out of an examination of one-tailed significance testing. One-tailed tests were little advocated by the founders of modern statistics but are widely used and recommended nowadays in the biological, behavioral and social sciences. The high frequency of their use in ecology and animal behavior and their logical indefensibility have been documented in a companion review paper. In the present one, we trace the roots of this problem and counter some attacks on significance testing in general. Roots include: the early but irrational dichotomization of the P scale and adoption of the ‘significant/non-significant’ terminology; the mistaken notion that a high P value is evidence favoring the null hypothesis over the alternative hypothesis; and confusion over the distinction between statistical and research hypotheses. Resultant widespread misuse and misinterpretation of significance tests have also led to other problems, such as unjustifiable demands that reporting of P values be disallowed or greatly reduced and that reporting of confidence intervals and standardized effect sizes be required in their place. Our analysis of these matters thus leads us to a recommendation that for standard types of significance assessment the paleoFisherian and Neyman-Pearsonian paradigms be replaced by a neoFisherian one. The essence of the latter is that a critical α (probability of type I error) is not specified, the terms ‘significant’ and ‘non-significant’ are abandoned, that high P values lead only to suspended judgments, and that the so-called “three-valued logic” of Cox, Kaiser, Tukey, Tryon and Harris is adopted explicitly. Confidence intervals and bands, power analyses, and severity curves remain useful adjuncts in particular situations. Analyses conducted under this paradigm we term neoFisherian significance assessments (NFSA). Their role is assessment of the existence, sign and magnitude of statistical effects. The common label of null hypothesis significance tests (NHST) is retained for paleoFisherian and Neyman-Pearsonian approaches and their hybrids. The original Neyman-Pearson framework has no utility outside quality control type applications. Some advocates of Bayesian, likelihood and information-theoretic approaches to model selection have argued that P values and NFSAs are of little or no value, but those arguments do not withstand critical review. Champions of Bayesian methods in particular continue to overstate their value and relevance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–274 of 274
Statistical Significance and Effect Size Reporting: Portrait of a Possible Future.
Thompson, B. (1998) · Cited by 89
Note on an Article by Sir Ronald Fisher
Neyman, J. (1956) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1956.tb00236.x · Cited by 88
Statistics and the Scientific Method
Freedman, D. A. (1985) · Springer eBooks · doi:10.1007/978-1-4613-8536-3_11 · Cited by 85
Some Principles of the Theory of Testing Hypotheses
Lehmann, E. L. (2011) · Springer eBooks · doi:10.1007/978-1-4614-1412-4_14 · Cited by 84
Yates, F. (1951) · Journal of the American Statistical Association · doi:10.2307/2280090 · Cited by 84
Confidence Intervals: Better Answers to Better Questions
Cumming, G., & Fidler, F. (2009) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.217.1.15 · Cited by 83
Huberty, C. J. (1993) · The Journal of Experimental Education · doi:10.1080/00220973.1993.10806593 · Cited by 83
Hypothesis Testing: From p Values to Bayes Factors
Marden, J. I. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474339 · Cited by 81
The Controversy over Null Hypothesis Significance Testing Revisited
Balluerka, N., Gómez, J., & Hidalgo, D. (2005) · Methodology · doi:10.1027/1614-1881.1.2.55 · Cited by 79
Reid, C. (1982) · Cited by 79
Roberts, L. J. (1990) · Science · doi:10.1126/science.2294585 · Cited by 79
Sound and fury: McCloskey and significance testing in economics
Hoover, K. D., & Siegler, M. V. (2008) · Journal of Economic Methodology · doi:10.1080/13501780801913298 · Cited by 75
Behavioristic, Evidentialist, and Learning Models of Statistical Testing
Mayo, D. G. (1985) · Philosophy of Science · doi:10.1086/289272 · Cited by 72
Jaccard, J. J., & Guilamo‐Ramos, V. (2002) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3102_13 · Cited by 72
Significance Tests Have Their Place
Harris, R. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00535.x · Cited by 71
Evaluating the medical literature part II: Statistical analysis
Elenbaas, R. M., Elenbaas, J. K., & Cuddy, P. G. (1983) · Annals of Emergency Medicine · doi:10.1016/s0196-0644(83)80205-4 · Cited by 71
Fuson, W. M., & Hagood, M. J. (1942) · American Sociological Review · doi:10.2307/2085378 · Cited by 69
Toward improved statistical reporting in the journal of consulting and clinical psychology.
Fidler, F., Cumming, G., Thomason, N., Pannuzzo, D., Smith, J., Fyffe, P., Edmonds, H., Harrington, C., & Schmitt, R. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.1.136 · Cited by 67
FISHER, K. A. (1935) · Nature · doi:10.1038/136474b0 · Cited by 64
Karl Pearson and R. A. Fisher on Statistical Tests: A 1935 Exchange from Nature
Pearson, K., Fisher, R. A., & Inman, H. F. (1994) · The American Statistician · doi:10.1080/00031305.1994.10476010 · Cited by 63
Why We Don't Really Know What Statistical Significance Means: Implications for Educators
Hubbard, R., & Armstrong, J. S. (2006) · Journal of Marketing Education · doi:10.1177/0273475306288399 · Cited by 63
Are significance thresholds appropriate for the study of animal behaviour?
Stoehr, A. M. (1999) · Animal Behaviour · doi:10.1006/anbe.1998.1016 · Cited by 62
What If There Were No More Bickering About Statistical Significance Tests
Levin, J. R. (1998) · Research in the schools · Cited by 61
Compared to What? Assessing Brazil’s Political Institutions
Armijo, L. E., Faucher, P., & Dembińska, M. (2006) · Comparative Political Studies · doi:10.1177/0010414006287895 · Cited by 61
THE DIFFERENCE BETWEEN STATISTICAL HYPOTHESES AND SCIENTIFIC HYPOTHESES
Bolles, R. C. (1962) · Psychological Reports · doi:10.2466/pr0.1962.11.3.639 · Cited by 60
Statistical Significance: Rationale, Validity and Utility
Frick, R. W., & Chow, S. L. (1998) · Journal of the American Statistical Association · doi:10.2307/2669649 · Cited by 60
Methods of Reporting Statistical Results from Medical Research Studies
Walter, S. D. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117356 · Cited by 57
Lecoutre, B., Lecoutre, M., & Poitevineau, J. (2001) · International Statistical Review · doi:10.1111/j.1751-5823.2001.tb00466.x · Cited by 56
Statistics in Psychology: An Historical Perspective
Cowles, M. P. (1989) · Cited by 55
Statistical Practices: The Seven Deadly Sins
Millis, S. R. (2003) · Child Neuropsychology · doi:10.1076/chin.9.3.221.16455 · Cited by 54
The concept of statistical significance and the controversy about one-tailed tests.
Eysenck, H. J. (1960) · Psychological Review · doi:10.1037/h0048412 · Cited by 54
One- Versus Two-Tailed Hypothesis Tests in Contemporary Educational Research
Pillemer, D. B. (1991) · Educational Researcher · doi:10.3102/0013189x020009013 · Cited by 54
A brief note on one-tailed tests.
Burke, C. J. (1953) · Psychological Bulletin · doi:10.1037/h0059627 · Cited by 46
Precursors of Coprosterol and the Bile Acids in the Animal Organism
Rosenheim, O., & Webster, T. A. (1935) · Nature · doi:10.1038/136474a0 · Cited by 44
Statistics in psychology: an historical perspective
(1990) · Choice Reviews Online · doi:10.5860/choice.27-4803 · Cited by 44
Mayo, D. G. (1979) · Cited by 43
Edwards, A. W. F. (1990) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-20865-4_10 · Cited by 42
Error and the Growth of Experimental Knowledge
Lehmann, E. L., & Mayo, D. G. (1997) · Journal of the American Statistical Association · doi:10.2307/2965729 · Cited by 41
Significance Testing in Psychological Research: Some Persisting Issues
Estes, W. K. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00538.x · Cited by 40
One-sided or two-sided ρ values: which most appropriately address the question of drug efficacy?
Peace, K. E. (1991) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543409108835010 · Cited by 39
[Testing Precise Hypotheses]: Comment
Casella, G., & Berger, R. L. (1987) · Statistical Science · doi:10.1214/ss/1177013243 · Cited by 37
A note on metastatistics or ?an essay toward stating a problem in the doctrine of chances?
LeCam, L. M. (1977) · Synthese · doi:10.1007/bf00485696 · Cited by 37
Design and Analysis: Uncertain Intent, Uncertain Result
Hurlbert, S. H., & Lombardi, C. M. (2003) · Ecology · doi:10.1890/0012-9658(2003)084[0810:daauiu]2.0.co;2 · Cited by 36
Self-citationHypothesis Testing in Relation to Statistical Methodology
Clark, C. A. (1963) · Review of Educational Research · doi:10.2307/1169648 · Cited by 35
Sense and Nonsense of Statistical Inference
DasGupta, A., & Wang, C. (1993) · Technometrics · doi:10.1080/00401706.1993.10485376 · Cited by 35
Kalbfleisch, J. G., & Sprott, D. A. (1976) · Springer eBooks · doi:10.1007/978-94-010-1436-6_7 · Cited by 34
One-tailed tests and "unexpected" results.
Goldfried, M. R. (1959) · Psychological Review · doi:10.1037/h0038521 · Cited by 33
Bayesians in clinical trials: Asleep at the switch
Moyé, L. A. (2007) · Statistics in Medicine · doi:10.1002/sim.2928 · Cited by 32
Sense and Nonsense of Statistical Inference.
Smith, J. T., & Wang, C. (1994) · Biometrics · doi:10.2307/2533463 · Cited by 32
Hypotheses, errors, and statistical assumptions
Simberloff, D. (1990) · Herpetologica · Cited by 31
Camilli, G. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.1.135 · Cited by 30
Kendall, P. C. (1997) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0092570 · Cited by 28
Statistics and the Scientific Method in Ecology
Dennis, B. (2004) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226789583.003.0011 · Cited by 27
The Use of Restricted Significance Tests in Clinical Trials
Salsburg, D. S. (1992) · Springer eBooks · doi:10.1007/978-1-4612-4414-1 · Cited by 26
Science, Ecological Validity and Experimentation
Chow, S. L. (1987) · Journal for the Theory of Social Behaviour · doi:10.1111/j.1468-5914.1987.tb00094.x · Cited by 26
Sockman, K. W. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001785 · Cited by 25
The alternative hypothesis: One-sided or two-sided?
Peace, K. E. (1989) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(89)90137-6 · Cited by 22
Commentary: The P-value, devalued
Goodman, S. N. (2003) · International Journal of Epidemiology · doi:10.1093/ije/dyg294 · Cited by 22
A further look at wrong reasons to abandon statistical testing.
Hagen, R. L. (1998) · American Psychologist · doi:10.1037//0003-066x.53.7.801 · Cited by 20
Tutorial: Using Confidence Curves in Medical Research
Bender, R., Berg‐Beckhoff, G., & Zeeb, H. (2005) · Biometrical Journal · doi:10.1002/bimj.200410104 · Cited by 20
Schweder, T. (2003) · Biometrics · doi:10.1111/j.0006-341x.2003.00112.x · Cited by 18
Fleiss, J. L. (1987) · Controlled Clinical Trials · doi:10.1016/0197-2456(87)90159-0 · Cited by 17
Book Reviews: To Test or Not to Test Ho?
Levin, J. R. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058002015 · Cited by 17
The null hypothesis is not called that for nothing: statistical tests in randomized trials
Boruch, R. F. (2007) · Journal of Experimental Criminology · doi:10.1007/s11292-007-9026-0 · Cited by 15
Moreau, M. F., Surico-Bennett, J., Vicario-Fisher, M., Gerads, R., Gersberg, R. M., & Hurlbert, S. H. (2007) · Lake and Reservoir Management · doi:10.1080/07438140709354037 · Cited by 15
Self-citationChapter I: Hypothesis Testing in Relation to Statistical Methodology
Clark, C. A. (1963) · Review of Educational Research · doi:10.3102/00346543033005455 · Cited by 15
Still Much to Learn About Confidence Intervals: Reply to Rouder and Morey (2005)
Fidler, F. M., Thomason, N. R., Cumming, G., Finch, S. J., & Leeman, J. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01562.x · Cited by 14
Ten rules for reading clinical research reports
Yancey, J. M. (1996) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/s0889-5406(96)70143-9 · Cited by 13
Wolterbeek, R. (1994) · BMJ · doi:10.1136/bmj.309.6958.873a · Cited by 12
Thompson, B. (2006) · International Statistical Review · doi:10.1111/j.1751-5823.2006.tb00157.x · Cited by 9
The New Stats: Attitudes for the Twenty-First Century
Fidler, F. M., & Cumming, G. (2008) · Cited by 5
Comments, Conjectures and Conclusions
Good, I. J. (2006) · Journal of Statistical Computation and Simulation · doi:10.1080/10629360600565426 · Cited by 4
Classical Statistical Inference: Practice Versus Presentation
Harris, R. (2005) · Encyclopedia of Statistics in Behavioral Science · doi:10.1002/0470013192.bsa092 · Cited by 2
Salsburg, D. S. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90019-6 · Cited by 2
Cited by
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Microbial formation of stable soil carbon is more efficient from belowground than aboveground input
Sokol, N. W., & Bradford, M. A. (2019) · Nature Geoscience · doi:10.1038/s41561-018-0258-6 · Cited by 891
Abandon Statistical Significance
McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L. (2019) · The American Statistician · doi:10.1080/00031305.2018.1527253 · Cited by 791
Sokol, N. W., Kuebbing, S. E., Karlsen‐Ayala, E., & Bradford, M. A. (2019) · New Phytologist · doi:10.1111/nph.15361 · Cited by 631
Amrhein, V., Trafimow, D., & Greenland, S. (2019) · The American Statistician · doi:10.1080/00031305.2018.1543137 · Cited by 425
Metadata
- Authors
- Hurlbert, S. H., & Lombardi, C. M.
- Venue
- Annales Zoologici Fennici
- Published
- 2009-10
- Volume
- 46
- Issue
- 5
- Pages
- 311-349
- ISSN
- 0003-455X (ISSN-L)
- 1797-2450
- OpenAlex
- W2120312320
- Cited by
- 301
- References
- 274
- FWCI
- 8.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.