Residuals in Generalized Linear Models

Abstract

Generalized linear models are regression-type models for data not normally distributed, appropriately fitted by maximum likelihood rather than least squares. Typical examples are models for binomial or Poisson data, with a linear regression model for a given, ordinarily nonlinear, function of the expected values of the observations. Use of such models has become very common in recent years, and there is a clear need to study the issue of appropriate residuals to be used for diagnostic purposes. Several definitions of residuals are possible for generalized linear models. The statistical package GLIM (Baker and Nelder 1978) routinely prints out residuals , where V(μ) is the function relating the variance to the mean of y and is the maximum likelihood estimate of the ith mean as fitted to the regression model. These residuals are the signed square roots of the contributions to the Pearson goodness-of-fit statistic. Another choice of residual is the signed square root of the contribution to the deviance (likelihood ratio) goodness-of-fit statistic: where 1(μ i , yi ,) is the log-likelihood function for yi . Other definitions are considered in the article, but primary interest will center on the deviance-based residuals. For many purposes these appear to be a very good choice. Not only are they very nearly normally distributed, after appropriate allowance for discreteness, but in addition they constitute a natural choice of residual for likelihood-based methods. Some uses of generalized residuals include (a) examining them to identify individual poorly fitting observations, (b) plotting them to examine effects of potential new covariates or nonlinear effects of those already in the fitted model, (c) combining them into overall goodness-of-fit tests, and (d) using them as building blocks in the sense of Pregibon (1982) for case-influence diagnostics. The primary objectives in this article are to discuss the remarkable appropriateness of deviance-based residuals for use (a) and to provide some resulting insight into the contrast of the Pearson chi-squared and residual deviance statistics for use (c). Some brief discussion of point (b) is also given, but no consideration is given to item (d). The deviance residuals, which have been advocated by others as well, appear to be very nearly the same as those based on the best possible normalizing transformation for specific models, such as the Wilson-Hilferty transformation for gamma response variables, and yet have the advantages of generality of definition and ease of computation. Some theoretical aspects of this excellent behavior are discussed, including the connection to the approximate distribution of likelihood ratios and to recent developments in second-order saddlepoint approximations to the distribution of maximum likelihood estimators. The excellent performance of the deviance-based residuals raises the question of why the Pearson goodness-of-fit statistic often has more nearly a chi-squared distribution than does the residual deviance. Some explanation and numerical results for this comparison are provided, including the suggestion that the residual deviance should provide a better basis for goodness-of-fit tests than the Pearson statistic, in spite of common assertions to the contrary.