Concerns regarding a call for pluralism of information theory and hypothesis testing

Abstract

Summary: Stephens et al. (2005) argue for ‘pluralism’ in statistical analysis, combining null hypothesis testing and information‐theoretic (I‐T) methods. We show that I‐T methods are more informative even in single variable problems and we provide an ecological example. I‐T methods allow inferences to be made from multiple models simultaneously. We believe multimodel inference is the future of data analysis, which cannot be achieved with null hypothesis‐testing approaches. We argue for a stronger emphasis on critical thinking in science in general and less reliance on exploratory data analysis and data dredging. Deriving alternative hypotheses is central to science; deriving a single interesting science hypothesis and then comparing it to a default null hypothesis (e.g. ‘no difference’) is not an efficient strategy for gaining knowledge. We think this single‐hypothesis strategy has been relied upon too often in the past. We clarify misconceptions presented by Stephens et al. (2005). We think inference should be made about models, directly linked to scientific hypotheses, and their parameters conditioned on data, Prob(Hj | data). I‐T methods provide a basis for this inference. Null hypothesis testing merely provides a probability statement about the data conditioned on a null model, Prob(data | H0). Synthesis and applications. I‐T methods provide a more informative approach to inference. I‐T methods provide a direct measure of evidence for or against hypotheses and a means to consider simultaneously multiple hypotheses as a basis for rigorous inference. Progress in our science can be accelerated if modern methods can be used intelligently; this includes various I‐T and Bayesian methods.