The Maximum Familywise Error Rate of Fisher's Least Significant Difference Test

Abstract

In this article an investigation is made of the maximum familywise error rate (MFWER) of Fisher's least significant difference (LSD) test for testing the equality of k population means in a one-way layout. An exact expression for the MFWER is derived (see Theorem 1) for all balanced models and for an unbalanced model with k = 3 populations (Type I models). A close upper bound for the MFWER is derived for all unbalanced models with four or more populations (Type II models). These expressions are used to illustrate that the MFWER may greatly exceed the nominal size α of the LSD test. In addition, a simple modification of the LSD test is proposed to control the MFWER. This modified procedure has MFWER equal to the nominal level α for Type I models and no greater than α for Type II models (Theorem 2) and is, therefore, recommended as an improvement over the LSD test. The key to the analysis is two theorems concerning the ranges of independent normal random variables, which are contained in the Appendix. The first of these theorems (Theorem A.1) is concerned with the conservative nature of unbalanced models as compared with balanced models and was published in Hayter (1984), although in a different context. The second theorem (Theorem A.2) presents a new chain of inequalities concerning the ranges of independent normal random variables partitioned into groups of differing sizes. It is believed that this new latter theorem may be of independent interest.