Use and Interpretation of Common Statistical Tests in Method Comparison Studies

Abstract

Feature Article: Westgard JO, Hunt MR. Use and interpretation of common statistical tests in method comparison studies. Clin Chem 1973;19:49–57.1 Marian Hunt and I published this paper to improve the understanding and application of commonly used statistics in method validation studies. We specifically wanted to clarify the misuse of the correlation coefficient and t-test statistics, and to point out important factors that affected the reliability of least squares statistics. This paper also provided some direction in making decisions on the acceptability of a method by using statistics to estimate the sizes of analytical errors. A subsequent paper (1) provided criteria for comparing those estimates of errors with quality standards that defined the amount of allowable error. This paper was written at a time when I was very involved in the evaluation of new automated analytic systems. At that time little guidance was available as to how to analyze and interpret the data from method evaluation experiments. Decisions on acceptability of new methods seldom provided any rational explanation as to why a method was judged acceptable or not. The correlation coefficient was the statistic most often being used to justify decisions on acceptability, followed by the t-test, in which the decision was based on whether or not the t-value indicated a statistically significant difference (i.e., calculated t-value greater than the critical t-value) between methods. Neither of these approaches took into account the actual quality required for the application of laboratory tests.