The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Null Hypothesis Testing: Problems, Prevalence, and an Alternative
Abstract
This paper presents a review and critique of statistical null hypothesis testing in ecological studies in general, and wildlife studies in particular, and describes an alternative. Our review of Ecology and the Journal of Wildlife Management found the use of null hypothesis testing to be pervasive. The estimated number of P-values appearing within articles of Ecology exceeded 8,000 in 1991 and has exceeded 3,000 in each year since 1984, whereas the estimated number of P-values in the Journal of Wildlife Management exceeded 8,000 in 1997 and has exceeded 3,000 in each year since 1994. We estimated that 47% (SE = 3.9%) of the P-values in the Journal of Wildlife Management lacked estimates of means or effect sizes or even the sign of the difference in means or other parameters. We find that null hypothesis testing is uninformative when no estimates of means or effect size and their precision are given. Contrary to common dogma, tests of statistical null hypotheses have relatively little utility in science and are not a fundamental aspect of the scientific method. We recommend their use be reduced in favor of more informative approaches. Towards this objective, we describe a relatively new paradigm of data analysis based on Kullback-Leibler information. This paradigm is an extension of likelihood theory and, when used correctly, avoids many of the fundamental limitations and common misuses of null hypothesis testing. Information-theoretic methods focus on providing a strength of evidence for an a priori set of alternative hypotheses, rather than a statistical test of a null hypothesis. This paradigm allows the following types of evidence for the alternative hypotheses: the rank of each hypothesis, expressed as a model; an estimate of the formal likelihood of each model, given the data; a measure of precision that incorporates model selection uncertainty; and simple methods to allow the use of the set of alternative models in making, formal inference. We provide an example of the information-theoretic approach using data on the effect of lead on survival in spectacled eider ducks (Somateria fischeri). Regardless of the analysis paradigm used, we strongly recommend inferences based on a priori considerations be clearly separated from those resulting from some form of data dredging.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Program MARK: survival estimation from populations of marked animals
White, G. C., & Burnham, K. P. (1999) · Bird Study · doi:10.1080/00063659909477239 · Cited by 7,499
Self-citationApplied Nonparametric Statistics
Wozniak, P. J., & Daniel, W. W. (1991) · Technometrics · doi:10.2307/1268795 · Cited by 4,987
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Information theory as an extension of the maximum likelihood principle
Akaike, H. (1973) · Medical Entomology and Zoology · Cited by 2,357
Model Selection: An Integral Part of Inference
Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997) · Biometrics · doi:10.2307/2533961 · Cited by 1,717
Self-citationON THE PROBLEM OF THE MOST EFFICIENT TESTS OF STATISTICAL HYPOTHESES
(1967) · Joint Statistical Papers · doi:10.1525/9780520339897-008 · Cited by 1,504
Design and Analysis Methods for Fish Survival Experiments Based on Release-Recapture.
Lakhani, K. H., Burnham, K. P., Anderson, D. R., White, G. C., Brownie, C., & Pollock, K. H. (1988) · The Journal of Applied Ecology · doi:10.2307/2403770 · Cited by 1,210
Self-citationThe Insignificance of Statistical Significance Testing
Johnson, D. H. (1999) · The Journal of Wildlife Management · doi:10.2307/3802789 · Cited by 1,104
ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE
NEYMAN, J., & PEARSON, E. S. (1928) · Biometrika · doi:10.1093/biomet/20a.3-4.263 · Cited by 1,070
Chamberlin, T. C. (1965) · Science · doi:10.1126/science.148.3671.754 · Cited by 1,069
The case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931
Statistical Evidence: A Likelihood Paradigm
Reeves, J. H., & Royall, R. M. (1998) · Journal of the American Statistical Association · doi:10.2307/2669869 · Cited by 814
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037//1082-989x.1.2.115 · Cited by 783
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478397 · Cited by 775
Regression and Time Series Model Selection
McQuarrie, A. D. R., & Tsai, C. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789812385451 · Cited by 688
Goodman, S. N., & Berlin, J. A. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-3-199408010-00008 · Cited by 654
The Empire of chance: how probability changed science and everyday life
(1989) · Choice Reviews Online · doi:10.5860/choice.27-2084 · Cited by 597
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.2307/2289131 · Cited by 548
Some Problems Connected with Statistical Inference
Cox, D. R. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706618 · Cited by 536
An introduction to Bayesian inference for ecological research and environmental decision-making
Ellison, A. M. (1996) · Ecological Applications · doi:10.2307/2269588 · Cited by 482
Some Difficulties of Interpretation Encountered in the Application of the Chi-Square Test
Berkson, J. (1938) · Journal of the American Statistical Association · doi:10.1080/01621459.1938.10502329 · Cited by 411
Statistical analysis and the illusion of objectivity
(1988) · Infectious Diseases Newsletter · doi:10.1016/0278-2316(88)90057-6 · Cited by 408
Selection Models and the File Drawer Problem
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013012 · Cited by 384
Wildlife Science: Gaining Reliable Knowledge
Romesburg, H. C. (1981) · The Journal of Wildlife Management · doi:10.2307/3807913 · Cited by 380
Goodman, S. N. (1993) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116700 · Cited by 303
Edwards, A. W. F. (1993) · Biometrics · doi:10.2307/2532282 · Cited by 272
Discussion: Should Ecologists Become Bayesians?
Dennis, B. (1996) · Ecological Applications · doi:10.2307/2269594 · Cited by 244
Evidence and scientific research.
Goodman, S. N., & Royall, R. (1988) · American Journal of Public Health · doi:10.2105/ajph.78.12.1568 · Cited by 227
Stuart, A., & Edwards, A. W. F. (1973) · Applied Statistics · doi:10.2307/2346929 · Cited by 199
The Significance Test Controversy: A Reader
Henkel, R. E. (2006) · CERN Document Server (European Organization for Nuclear Research) · Cited by 190
Tests of Significance Considered as Evidence
Berkson, J. (1942) · Journal of the American Statistical Association · doi:10.2307/2279000 · Cited by 174
Selected Papers of Hirotugu Akaike
(1999) · Technometrics · doi:10.1080/00401706.1999.10485965 · Cited by 169
Limits of Retrospective Power Analysis
Gerard, P. D., Smith, D. R., & Weerakkody, G. (1998) · The Journal of Wildlife Management · doi:10.2307/3802357 · Cited by 163
Yates, F. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500764 · Cited by 154
Regression Analysis: Concepts and Applications
Barrett, B. E., Graybill, F. A., & Iyer, H. K. (1995) · Technometrics · doi:10.1080/00401706.1995.10484308 · Cited by 150
Statistical Sirens: The Allure of Nonparametrics
Johnson, D. H. (1995) · Ecology · doi:10.2307/1940733 · Cited by 110
Henderson, A. R. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.6.929 · Cited by 43
Cited by
Rebuilding community ecology from functional traits
MCGILL, B., ENQUIST, B., WEIHER, E., & WESTOBY, M. (2006) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2006.02.002 · Cited by 4,631
Burnham, K. P., Anderson, D. R., & Huyvaert, K. P. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1029-6 · Cited by 4,104
Self-citationPosada, D., & Buckley, T. R. (2004) · Systematic Biology · doi:10.1080/10635150490522304 · Cited by 3,927
Model selection in ecology and evolution
Johnson, J. B., & Omland, K. S. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2003.10.013 · Cited by 3,838
Evaluating resource selection functions
Boyce, M. S., Vernier, P. R., Nielsen, S. E., & Schmiegelow, F. K. A. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00200-4 · Cited by 2,566
Metadata
- Authors
- Anderson, D. R., Burnham, K. P., & Thompson, W. L.
- Venue
- The Journal of Wildlife Management
- Publisher
- JSTOR
- Published
- 2000-10
- Volume
- 64
- Issue
- 4
- Pages
- 912
- ISSN
- 0022-541X (ISSN-L)
- OpenAlex
- W1991445071
- Cited by
- 1,930
- References
- 42
- FWCI
- 58.53
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.