Modeling Certainty with Clustered Data: A Comparison of Methods

Abstract

Political scientists often analyze data in which the observational units are clustered into politically or socially meaningful groups with an interest in estimating the effects that group-level factors have on individual-level behavior. Even in the presence of low levels of intracluster correlation, it is well known among statisticians that ignoring the clustered nature of such data overstates the precision estimates for group-level effects. Although a number of methods that account for clustering are available, their precision estimates are poorly understood, making it difficult for researchers to choose among approaches. In this paper, we explicate and compare commonly used methods (clustered robust standard errors (SEs), random effects, hierarchical linear model, and aggregated ordinary least squares) of estimating the SEs for group-level effects. We demonstrate analytically and with the help of empirical examples that under ideal conditions there is no meaningful difference in the SEs generated by these methods. We conclude with advice on the ways in which analysts can increase the efficiency of clustered designs.