The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Information theory and hypothesis testing: a call for pluralism
Open accessAbstract
Summary: A major paradigm shift is occurring in the approach of ecologists to statistical analysis. The use of the traditional approach of null‐hypothesis testing has been questioned and an alternative, model selection by information–theoretic methods, has been strongly promoted and is now widely used. For certain types of analysis, information–theoretic approaches offer powerful and compelling advantages over null‐hypothesis testing. The benefits of information–theoretic methods are often framed as criticisms of null‐hypothesis testing. We argue that many of these criticisms are neither irremediable nor always fair. Many are criticisms of the paradigm's application, rather than of its formulation. Information–theoretic methods are equally vulnerable to many such misuses. Care must be taken in the use of either approach but users of null‐hypothesis tests, in particular, must greatly improve standards of reporting and interpretation. Recent critiques have suggested that the distinction between experimental and observational studies defines the limits of the utility of null‐hypothesis testing (with the paradigm being applicable to the former but not the latter). However, we believe that there are many situations in which observational data are collected that lend themselves to analysis under the null‐hypothesis testing paradigm. We suggest that the applicability of the two analytical paradigms is more accurately defined by studies that assess univariate causality (when null‐hypothesis testing is adequate) and those that assess multivariate patterns of causality (when information–theoretic methods are more suitable). Synthesis and applications. Many ecologists are confused about the circumstances under which different inferential paradigms might apply. We address some of the major criticisms of the null‐hypothesis testing paradigm, assess those criticisms in relation to the information–theoretic paradigm, propose methods for improving the use of null‐hypothesis testing, and discuss situations in which the use of null‐hypothesis testing would be appropriate. We urge instructors and practitioners of statistical methods to heighten awareness of the limitations of null‐hypothesis testing and to use information–theoretic methods whenever prior evidence suggests that multiple research hypotheses are plausible. We contend, however, that by marginalizing the use of null‐hypothesis testing, ecologists risk rejecting a powerful, informative and well‐established analytical tool.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Model selection in ecology and evolution
Johnson, J. B., & Omland, K. S. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2003.10.013 · Cited by 3,838
Evaluating resource selection functions
Boyce, M. S., Vernier, P. R., Nielsen, S. E., & Schmiegelow, F. K. A. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00200-4 · Cited by 2,566
Directions in Conservation Biology
Caughley, G. (1994) · The Journal of Animal Ecology · doi:10.2307/5542 · Cited by 2,405
Evaluating the predictive performance of habitat models developed using logistic regression
Pearce, J., & Ferrier, S. (2000) · Ecological Modelling · doi:10.1016/s0304-3800(00)00322-7 · Cited by 2,002
Null Hypothesis Testing: Problems, Prevalence, and an Alternative
Anderson, D. R., Burnham, K. P., & Thompson, W. L. (2000) · The Journal of Wildlife Management · doi:10.2307/3803199 · Cited by 1,930
Understanding robust and exploratory data analysis
Hoaglin, D. C., Mosteller, F. C., & Tukey, J. W. (1983) · Wiley eBooks · Cited by 1,906
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778 · cited 3×
“gical significance is of greater importance than statistical significance (Yoccoz 1991; Kirk 1996); and (iii) the null is framed more imaginatively than the standard ‘there is no difference between a and b ’.”
BackgroundMathematical Statistics With Applications
Meek, G. E. (1987) · Technometrics · doi:10.1080/00401706.1987.10488256 · Cited by 1,425
Fitting curves to data using nonlinear regression: a practical and nonmathematical review
Motulsky, H. J., & Ransnas, L. A. (1987) · The FASEB Journal · doi:10.1096/fasebj.1.5.3315805 · Cited by 1,327 · cited 2×
“Importantly, there is no reason why the sampling distribution of the test statistic cannot be inferred from a post-hoc analysis of, for example, residuals from a regression line (e.g. Motulsky & Ransnas 1987, p. 370) or distribution of data collected.”
BackgroundAvoiding Pitfalls When Using Information-Theoretic Methods
Anderson, D. R., & Burnham, K. P. (2002) · The Journal of Wildlife Management · doi:10.2307/3803155 · Cited by 1,324
Statistical data analysis in the computer age
Efron, B., & Tibshirani, R. (1991) · Science · doi:10.1126/science.253.5018.390 · Cited by 1,192 · cited 4×
“In particular, non-parametric bootstrapping approaches make no distributional assumptions beyond those indicated by the observed data, but have been shown to be very powerful ( Efron & Tibshirani 1991, 1993 ).”
BackgroundMethodInfluentialThe Insignificance of Statistical Significance Testing
Johnson, D. H. (1999) · The Journal of Wildlife Management · doi:10.2307/3802789 · Cited by 1,104
Model uncertainty, data mining and statistical inference
Chatfield, C. (1995) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983440 · Cited by 1,103 · cited 2×
“Some authors have suggested that concentrating on the information criteria of models may be less important than more pragmatic concerns of accuracy (Chatfield 1995) and it appears that accepting a ‘best’ model or set of models on the basis of AIC may not always result in the selection of the most useful model (C.”
BackgroundThe case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931
Kullback-Leibler information as a basis for strong inference in ecological studies
Burnham, K. P., & Anderson, D. R. (2001) · Wildlife Research · doi:10.1071/wr99107 · Cited by 926
Peters, R. H. (1991) · Cited by 864
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 830
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037//1082-989x.1.2.115 · Cited by 783
Derksen, S., & Keselman, H. J. (1992) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1992.tb00992.x · Cited by 771
Goodman, S. N., & Berlin, J. A. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-3-199408010-00008 · Cited by 654
Hayes, T., Haston, K., Tsui, M., Hoang, A., Haeffele, C., & Vonk, A. (2003) · Environmental Health Perspectives · doi:10.1289/ehp.5932 · Cited by 606
New paradigms for modelling species distributions?
Rushton, S. P., Ormerod, S. J., & Kerby, G. (2004) · Journal of Applied Ecology · doi:10.1111/j.0021-8901.2004.00903.x · Cited by 585
Density Dependence in Time Series Observations of Natural Populations: Estimation and Testing
Dennis, B., & Taper, M. L. (1994) · Ecological Monographs · doi:10.2307/2937041 · Cited by 583
Suggestions for Presenting the Results of Data Analyses
Anderson, D. R., Link, W. A., Johnson, D. H., & Burnham, K. P. (2001) · The Journal of Wildlife Management · doi:10.2307/3803088 · Cited by 465
Removing GPS collar bias in habitat selection studies
Frair, J. L., Nielsen, S. E., Merrill, E. H., Lele, S. R., Boyce, M. S., Munro, R. H. M., Stenhouse, G. B., & Beyer, H. L. (2004) · Journal of Applied Ecology · doi:10.1111/j.0021-8901.2004.00902.x · Cited by 358
Gibson, N., & Whittaker, J. (1996) · Journal of Management in Engineering · doi:10.1061/(asce)0742-597x(1996)12:6(34 · Cited by 294
Use, Overuse, and Misuse of Significance Tests in Evolutionary Biology and Ecology
(1991) · The Bulletin of the Ecological Society of America · doi:10.2307/20167258 · Cited by 281
Understanding Robust and Exploratory Data Analysis
(2001) · Technometrics · doi:10.1198/tech.2001.s648 · Cited by 261
Spatial prediction of rufous bristlebird habitat in a coastal heathland: a GIS‐based approach
Gibson, L. A., Wilson, B. A., Cahill, D. M., & Hill, J. (2004) · Journal of Applied Ecology · doi:10.1111/j.0021-8901.2004.00896.x · Cited by 222
In praise of the null hypothesis statistical test.
Hagen, R. L. (1997) · American Psychologist · doi:10.1037//0003-066x.52.1.15 · Cited by 193
Rules of thumb for judging ecological theories
Ginzburg, L. R., & Jensen, C. X. J. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2003.11.004 · Cited by 167
Joint Effects of Density Dependence and Rainfall on Abundance of San Joaquin Kit Fox
Dennis, B., & Otten, M. R. M. (2000) · The Journal of Wildlife Management · doi:10.2307/3803237 · Cited by 160
Mathematical statistics with applications
(2005) · Choice Reviews Online · doi:10.5860/choice.43-1620 · Cited by 135
An Applied Statistician's Creed
Nester, M. R. (1996) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986064 · Cited by 132
Statistical Sirens: The Allure of Nonparametrics
Johnson, D. H. (1995) · Ecology · doi:10.2307/1940733 · Cited by 110
What Should We Do about Hypothesis Testing?
Eberhardt, L. L. (2003) · Journal of Wildlife Management · doi:10.2307/3802765 · Cited by 96
On the Past and Future of Null Hypothesis Significance Testing
Robinson, D. H., & Wainer, H. (2002) · Journal of Wildlife Management · doi:10.2307/3803158 · Cited by 90
The Role of Hypothesis Testing in Wildlife Science
Johnson, D. H. (2002) · Journal of Wildlife Management · doi:10.2307/3803159 · Cited by 64
Cited by
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910
A protocol for data exploration to avoid common statistical problems: Data exploration
Zuur, A. F., Ieno, E. N., & Elphick, C. S. (2010) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2009.00001.x · Cited by 8,228
Effect size, confidence interval and statistical significance: a practical guide for biologists
Nakagawa, S., & Cuthill, I. C. (2007) · Biological Reviews · doi:10.1111/j.1469-185x.2007.00027.x · Cited by 3,835
Symonds, M. R. E., & Moussalli, A. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1037-6 · Cited by 2,543
Multimodel inference in ecology and evolution: challenges and solutions
GRUEBER, C. E., NAKAGAWA, S., LAWS, R. J., & JAMIESON, I. G. (2011) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2010.02210.x · Cited by 2,263
Metadata
- Authors
- STEPHENS, P. A., BUSKIRK, S. W., HAYWARD, G. D., & MARTÍNEZ DEL RIO, C.
- Venue
- Journal of Applied Ecology
- Publisher
- Wiley
- Published
- 2005-02
- Volume
- 42
- Issue
- 1
- Pages
- 4-12
- ISSN
- 0021-8901 (ISSN-L)
- 1365-2664
- OpenAlex
- W2169605107
- Cited by
- 360
- References
- 43
- FWCI
- 27.78
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.