On the misuse of residuals in ecology: testing regression residuals vs. the analysis of covariance

Abstract

Summary: An analysis of variance (anova) or other linear models of the residuals of a simple linear regression is being increasingly used in ecology to compare two or more groups. Such a procedure (hereafter, ‘residual index’) was used in 8% and 2% of the papers published during 1999 in the Journal of Animal Ecology and in Ecology, respectively, and has been recently recommended for studying condition. Although the residual index is similar to an analysis of covariance (ancova), it is not identical and is incorrect for at least four reasons: the regression coefficient used by the residual index differs from the one used in ancova and is not the least‐squares estimator of the model. in contrast to the ancova, the error d.f. in the residual index are overestimated because of the estimation of the regression coefficient. the residual index also assumes the homogeneity of regression coefficients (parallelism assumption), which should be tested with a special ancova design. even if the assumptions of the linear model hold for the original variables, they will not hold for the residuals. More importantly, the residual index is an ad hoc sequential procedure with no statistical justification, unlike the well‐known ancova. For these reasons, I suggest that a t‐test or an anova of the residuals should never be used in place of an ancova to study condition or any other variable.