The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
The Insignificance of Statistical Significance Testing
Abstract
Despite their wide use in scientific journals such as The Journal of Wildlife Management, statistical hypothesis tests add very little value to the products of research. Indeed, they frequently confuse the interpretation of data. This paper describes how statistical hypothesis tests are often viewed, and then contrasts that interpretation with the correct one. I discuss the arbitrariness of P-values, conclusions that the null hypothesis is true, power analysis, and distinctions between statistical and biological significance. Statistical hypothesis testing, in which the null hypothesis about the properties of a population is almost always known a priori to be false, is contrasted with scientific hypothesis testing, which examines a credible null hypothesis about phenomena in nature. More meaningful alternatives are briefly outlined, including estimation and confidence intervals for determining the importance of factors, decision theory for guiding actions in the face of uncertainty, and Bayesian approaches to hypothesis testing and other statistical practices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
The Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,499
Applied Nonparametric Statistics
Wozniak, P. J., & Daniel, W. W. (1991) · Technometrics · doi:10.2307/1268795 · Cited by 4,987
Model Selection and Inference: A Practical Information-Theoretic Approach
Strawderman, R. L., Burnham, K. P., & Anderson, D. R. (2000) · Journal of the American Statistical Association · doi:10.2307/2669574 · Cited by 4,194
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Bayesian Inference in Statistical Analysis.
Kadane, J. B., Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.2307/2285412 · Cited by 3,824
Experiments in ecology their logical design and interpretation using analysis of variance
Underwood, A. J. (1997) · Cited by 3,799
Optimal Statistical Decisions.
Siegmund, D., & DeGroot, M. H. (1972) · Journal of the American Statistical Association · doi:10.2307/2284675 · Cited by 3,386
Adaptive Management of Renewable Resources.
Rutherford, A. A., & Walters, C. (1987) · Biometrics · doi:10.2307/2531565 · Cited by 3,308
Adaptive Environmental Assessment and Management
(1980) · Forest Science · doi:10.1093/forestscience/26.3.435 · Cited by 3,222
AN ESSAY TOWARDS SOLVING A PROBLEM IN THE DOCTRINE OF CHANCES
BAYES (1958) · Biometrika · doi:10.1093/biomet/45.3-4.296 · Cited by 2,911
Stastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,905
Hatfield, J. L. (1998) · Journal of Environmental Quality · doi:10.2134/jeq1998.00472425002700010038x · Cited by 2,871
Model Selection and Inference: A Practical Information-Theoretic Approach
Beier, P., Burnham, K. P., & Anderson, D. R. (2001) · The Journal of Wildlife Management · doi:10.2307/3803117 · Cited by 2,440
Bayes and Empirical Bayes Methods for Data Analysis
(2001) · Technometrics · doi:10.1198/tech.2001.s608 · Cited by 2,098
Bayesian Inference in Statistical Analysis
Hill, B. M. (1974) · Technometrics · doi:10.1080/00401706.1974.10489222 · Cited by 1,833
Model Selection: An Integral Part of Inference
Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997) · Biometrics · doi:10.2307/2533961 · Cited by 1,717
Adaptive environmental assessment and management
Jørgensen, S. E. (1980) · Ecological Modelling · doi:10.1016/0304-3800(80)90047-2 · Cited by 1,615
Randolph, P. H. (1970) · Technometrics · doi:10.1080/00401706.1970.10488715 · Cited by 986
The case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931
Sampling and Bayes' Inference in Scientific Modelling and Robustness
Box, G. E. P. (1980) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2982063 · Cited by 924
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 911
Statistical Power Analysis can Improve Fisheries Research and Management
Peterman, R. M. (1990) · Canadian Journal of Fisheries and Aquatic Sciences · doi:10.1139/f90-001 · Cited by 796
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478397 · Cited by 775
Introduction to Statistical Decision Theory.
Schervish, M. J., Pratt, J. W., Raiffa, H., & Schlaifer, R. (1996) · Journal of the American Statistical Association · doi:10.2307/2291759 · Cited by 705
Berger, J. O., & Delampady, M. (1987) · Statistical Science · doi:10.1214/ss/1177013238 · Cited by 664
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.2307/2289131 · Cited by 548
An introduction to Bayesian inference for ecological research and environmental decision-making
Ellison, A. M. (1996) · Ecological Applications · doi:10.2307/2269588 · Cited by 482
Bayes and Empirical Bayes Methods for Data Analysis
Rukhin, A. L. (1997) · Technometrics · doi:10.1080/00401706.1997.10485131 · Cited by 462
Statistical Power Analysis in Wildlife Research
Steidl, R. J., Hayes, J. P., & Schauber, E. (1997) · The Journal of Wildlife Management · doi:10.2307/3802582 · Cited by 420
Some Difficulties of Interpretation Encountered in the Application of the Chi-Square Test
Berkson, J. (1938) · Journal of the American Statistical Association · doi:10.1080/01621459.1938.10502329 · Cited by 411
Statistical analysis and the illusion of objectivity
(1988) · Infectious Diseases Newsletter · doi:10.1016/0278-2316(88)90057-6 · Cited by 408
On Hypothesis Testing in Ecology and Evolution
Quinn, J. F., & Dunham, A. E. (1983) · The American Naturalist · doi:10.1086/284161 · Cited by 338
Analyzing data: Sanctification or detective work?
Tukey, J. W. (1969) · American Psychologist · doi:10.1037/h0027108 · Cited by 313
Savage, I. R. (1957) · Journal of the American Statistical Association · doi:10.1080/01621459.1957.10501392 · Cited by 303
Detecting Community-Wide Patterns: Estimating Power Strengthens Statistical Inference
Toft, C. A., & Shea, P. J. (1983) · The American Naturalist · doi:10.1086/284162 · Cited by 269
Reversal of the Burden of Proof in Fisheries Management
Dayton, P. K. (1998) · Science · doi:10.1126/science.279.5352.821 · Cited by 207
On Probability as a Basis for Action
Deming, W. E. (1975) · The American Statistician · doi:10.1080/00031305.1975.10477402 · Cited by 202
What Statistical Significance Testing Is, and What It Is Not
Shaver, J. P. (1993) · The Journal of Experimental Education · doi:10.1080/00220973.1993.10806592 · Cited by 196
The Place of Statistics in Psychology
Nunnally, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000401 · Cited by 189
Limits of Retrospective Power Analysis
Gerard, P. D., Smith, D. R., & Weerakkody, G. (1998) · The Journal of Wildlife Management · doi:10.2307/3802357 · Cited by 163
On the Tyranny of Hypothesis Testing in the Social Sciences
Loftus, G. R. (1991) · Contemporary Psychology · doi:10.1037/029395 · Cited by 136
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (2013) · doi:10.4324/9781315629049-15 · Cited by 134
An Applied Statistician's Creed
Nester, M. R. (1996) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986064 · Cited by 132
Sir Ronald Fisher and the Design of Experiments
Yates, F. (1964) · Biometrics · doi:10.2307/2528399 · Cited by 121
Howson, C., & Urbach, P. (1991) · Nature · doi:10.1038/350371a0 · Cited by 119
The illogic of statistical inference for cumulative science
Guttman, L. (1985) · Applied Stochastic Models and Data Analysis · doi:10.1002/asm.3150010103 · Cited by 118
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (2016) · doi:10.4324/9781315629049-29 · Cited by 112
Statistical Sirens: The Allure of Nonparametrics
Johnson, D. H. (1995) · Ecology · doi:10.2307/1940733 · Cited by 110
Self-citationThe Role of Statistical Significance Testing In Educational Research
McLean, J. E., & Ernest, J. M. (1998) · Cited by 105
Valuation of Experimental Management Options for Ecological Systems
Walters, C. J., & Green, R. (1997) · Journal of Wildlife Management · doi:10.2307/3802096 · Cited by 99
Huberty, C. J. (1993) · The Journal of Experimental Education · doi:10.1080/00220973.1993.10806593 · Cited by 83
Bayesian Environmental Policy Decisions: Two Case Studies
Wolfson, L. J., Kadane, J. B., & Small, M. J. (1996) · Ecological Applications · doi:10.2307/2269590 · Cited by 62
R. A. Fisher and Experimental Design: A Review
Preece, D. A. (1990) · Biometrics · doi:10.2307/2532438 · Cited by 59
Hypothesis Testing in Relation to Statistical Methodology
Clark, C. A. (1963) · Review of Educational Research · doi:10.2307/1169648 · Cited by 35
Hypotheses, errors, and statistical assumptions
Simberloff, D. (1990) · Herpetologica · Cited by 31
There Is a Time and a Place for Significance Testing
Mulaik, S. A., Raju, N. S., & Harshman, R. A. (2016) · doi:10.4324/9781315629049-16 · Cited by 30
Testing 'Small', not Null, Hypotheses: Classical and Bayesian Approaches
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (2013) · doi:10.4324/9781315629049-26 · Cited by 27
Edwards, A. W. F. (1991) · Nature · doi:10.1038/352386b0 · Cited by 8
Cited by
CLASSIFICATION AND REGRESSION TREES: A POWERFUL YET SIMPLE TECHNIQUE FOR ECOLOGICAL DATA ANALYSIS
De'ath, G., & Fabricius, K. E. (2000) · Ecology · doi:10.1890/0012-9658(2000)081[3178:cartap]2.0.co;2 · Cited by 3,167
Evaluating resource selection functions
Boyce, M. S., Vernier, P. R., Nielsen, S. E., & Schmiegelow, F. K. A. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00200-4 · Cited by 2,566
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
STRENGTH OF EVIDENCE FOR DENSITY DEPENDENCE IN ABUNDANCE TIME SERIES OF 1198 SPECIES
Brook, B. W., & Bradshaw, C. J. A. (2006) · Ecology · doi:10.1890/0012-9658(2006)87[1445:soefdd]2.0.co;2 · Cited by 2,082
Null Hypothesis Testing: Problems, Prevalence, and an Alternative
Anderson, D. R., Burnham, K. P., & Thompson, W. L. (2000) · The Journal of Wildlife Management · doi:10.2307/3803199 · Cited by 1,930
Metadata
- Authors
- Johnson, D. H.
- Venue
- The Journal of Wildlife Management
- Publisher
- JSTOR
- Published
- 1999-07
- Volume
- 63
- Issue
- 3
- Pages
- 763
- ISSN
- 0022-541X (ISSN-L)
- OpenAlex
- W2157002521
- Cited by
- 1,104
- References
- 62
- FWCI
- 50.57
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.