The Accuracy of Multilevel Structural Equation Modeling With Pseudobalanced Groups and Small Samples

Abstract

Hierarchical structured data cause problems in analysis, because the usual assumptions of independently and identically distributed variables are violated. Muthén (1989) described an estimation method for multilevel factor and path analysis with hierarchical data. This article assesses the robustness of the method with unequal groups, small sample sizes at both the individual and the group level, in the presence of a low or a high intraclass correlation (ICC). The within-groups part of the model poses no problems. The most important problem in the between-groups part of the model is the occurrence of inadmissible estimates, especially when group level sample size is small (50) while the intracluster correlation is low. This is partly compensated by using large group sizes. When an admissible solution is reached, the factor loadings are generally accurate. However, the residual variances are underestimated, and the standard errors are generally too small. Having more or larger groups or a higher ICC does not effectively compensate for this. Therefore, although the nominal alpha level is 5%, the operating alpha level is about 8% in all simulated conditions with unbalanced groups. The strongest factor is an inadequate sample size at the group level. Imbalance is only a problem for the overall fit test. For balanced data, the chi-square fit test is accurate. The size of the biases is comparable to the effect of moderate nonnormality in ordinary modeling, and in our view, the approximate solution remains a useful analysis tool, provided the group level sample size is at least 100.