Information theory and hypothesis testing: a call for pluralism

Abstract

Summary: A major paradigm shift is occurring in the approach of ecologists to statistical analysis. The use of the traditional approach of null‐hypothesis testing has been questioned and an alternative, model selection by information–theoretic methods, has been strongly promoted and is now widely used. For certain types of analysis, information–theoretic approaches offer powerful and compelling advantages over null‐hypothesis testing. The benefits of information–theoretic methods are often framed as criticisms of null‐hypothesis testing. We argue that many of these criticisms are neither irremediable nor always fair. Many are criticisms of the paradigm's application, rather than of its formulation. Information–theoretic methods are equally vulnerable to many such misuses. Care must be taken in the use of either approach but users of null‐hypothesis tests, in particular, must greatly improve standards of reporting and interpretation. Recent critiques have suggested that the distinction between experimental and observational studies defines the limits of the utility of null‐hypothesis testing (with the paradigm being applicable to the former but not the latter). However, we believe that there are many situations in which observational data are collected that lend themselves to analysis under the null‐hypothesis testing paradigm. We suggest that the applicability of the two analytical paradigms is more accurately defined by studies that assess univariate causality (when null‐hypothesis testing is adequate) and those that assess multivariate patterns of causality (when information–theoretic methods are more suitable). Synthesis and applications. Many ecologists are confused about the circumstances under which different inferential paradigms might apply. We address some of the major criticisms of the null‐hypothesis testing paradigm, assess those criticisms in relation to the information–theoretic paradigm, propose methods for improving the use of null‐hypothesis testing, and discuss situations in which the use of null‐hypothesis testing would be appropriate. We urge instructors and practitioners of statistical methods to heighten awareness of the limitations of null‐hypothesis testing and to use information–theoretic methods whenever prior evidence suggests that multiple research hypotheses are plausible. We contend, however, that by marginalizing the use of null‐hypothesis testing, ecologists risk rejecting a powerful, informative and well‐established analytical tool.