An analysis of the controversy over classical one-sided tests

Abstract

Background When applying classical tests of the null hypothesis in clinical trials, there has been considerable controversy over the choice between a one-sided versus a two-sided test. The choice between a one-sided and two-sided test still impacts on sample size calculations, assessment of study results by regulatory authorities, and publication of study results in academic journals. Purpose To analyze the main elements in the controversy, and examine the procedures from both a Bayesian and classical viewpoint. Methods and results Using a Bayesian decision framework, it is shown that there is no reason to double the p-value when moving from a one-sided to a two-sided test. Within the classical framework, it is shown that the doubling of the p-value results from a discontinuity due to testing a point null hypothesis. A three-decision rule, credited to Neyman or Wald, is presented that does not require the doubling of the p-value, and is consistent with a Bayesian approach. Conclusions For most comparative clinical trials the three-decision rule is appropriate, and its use would abolish the controversy over one-sided tests.