On the Misuse of Manifest Variables in the Detection of Measurement Bias

Abstract

Measurement invariance (lack of bias) of a manifest variable Y with respect to a latent variable W is defined as invariance of the conditional distribution of Y given W over selected subpopulations. Invariance is commonly assessed by studying subpopulation differences in the conditional distribution of Y given a manifest variable Z, chosen to substitute for W. A unified treatment of conditions that may allow the detection of measurement bias using statistical procedures involving only observed or manifest variables is presented. Theorems are provided that give conditions for measurement invariance, and for invariance of the conditional distribution of Y given Z. Additional theorems and examples explore the Bayes sufficiency of Z, stochastic ordering in W, local independence of Y and Z, exponential families, and the reliability of Z. It is shown that when Bayes sufficiency of Z fails, the two forms of invariance will often not be equivalent in practice. Bayes sufficiency holds under Rasch model assumptions, and in long tests under certain conditions. It is concluded that bias detection procedures that rely strictly on observed variables are not in general diagnostic of measurement bias, or the lack of bias.