Assessing Between-Group Differences in Information Systems Research: A Comparison of Covariance- and component-Based SEM1

Abstract

Multigroup or between-group analyses are common in the information systems literature. The ability to detect the presence or absence of between-group differences and accurately estimate the strength of moderating effects is important in studies that attempt to show contingent effects. In the past, IS scholars have used a variety of approaches to examine these effects, with the partial least squares (PLS) pooled significance test for multigroup becoming the most common (e.g., Ahuja and Thatcher 2005; Enns et al. 2003; Zhu et al. 2006). In other areas of social sciences (Epitropaki and Martin 2005) and management (Mayer and Gavin 2005; Song et al. 2005) research, however, there is greater emphasis on the use of covariance-based structural equation modeling multigroup analysis. This paper compares these two methods through Monte Carlo simulation. Our findings demonstrate the conditions under which covariance-based multigroup analysis is more appropriate as well as those under which there either is no difference or the component-based approach is preferable. In particular, we find that when data are normally distributed, with a small sample size and correlated exogenous variables, the component-based approach is more likely to detect differences between-group than is the covariance-based approach. Both approaches will consistently detect differences under conditions of normality with large sample sizes. With non-normally distributed data, neither technique could consistently detect differences across the groups in two of the paths, suggesting that both techniques struggle with the prediction of a highly skewed and kurtotic dependent variable. Both techniques detected the differences in the other paths consistently under conditions of non-normality, with the component-based approach preferable at moderate effect sizes, particularly for smaller samples.