The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index.
Testing measurement invariance in longitudinal data with ordered-categorical measures.
Open accessAbstract
A goal of developmental research is to examine individual changes in constructs over time. The accuracy of the models answering such research questions hinges on the assumption of longitudinal measurement invariance: The repeatedly measured variables need to represent the same construct in the same metric over time. Measurement invariance can be studied through factor models examining the relations between the observed indicators and the latent constructs. In longitudinal research, ordered-categorical indicators such as self- or observer-report Likert scales are commonly used, and these measures often do not approximate continuous normal distributions. The present didactic article extends previous work on measurement invariance to the longitudinal case for ordered-categorical indicators. We address a number of problems that commonly arise in testing measurement invariance with longitudinal data, including model identification and interpretation, sparse data, missing data, and estimation issues. We also develop a procedure and associated R program for gauging the practical significance of the violations of invariance. We illustrate these issues with an empirical example using a subscale from the Mexican American Cultural Values scale. Finally, we provide comparisons of the current capabilities of 3 major latent variable programs (lavaan, Mplus, OpenMx) and computer scripts for addressing longitudinal measurement invariance. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“To do so, we tested for longitudinal measurement invariance by comparing a series of nested factor models, adding specific restrictions to the models65,66.”
From Adolescent leisure reading and its longitudinal association with prosocial behavior and social adjustment (2023) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Comparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.588 · Cited by 20,015
Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance
Cheung, G. W., & Rensvold, R. B. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_5 · Cited by 16,280
Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance
Chen, F. F. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301834 · Cited by 12,217
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
A Reliability Coefficient for Maximum Likelihood Factor Analysis
Tucker, L. R., & Lewis, C. (1973) · Psychometrika · doi:10.1007/bf02291170 · Cited by 7,330 · cited 1×
“…measurement invariance used by standard statistical packages to calculate the CFI (and other relative fit indices like the TuckerLewis index; Tucker & Lewis, 1973) places no constraint on the mean structure or the unique factor structure, and becomes an inappropriate comparison model (cf.…”
Missing data analysis: making it work in the real world
Graham, J. W. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085530 · Cited by 6,163
A scaled difference chi-square test statistic for moment structure analysis
Satorra, A., & Bentler, P. M. (2001) · Psychometrika · doi:10.1007/bf02296192 · Cited by 5,282
Principles and practice in reporting structural equation analyses.
McDonald, R. P., & Ho, M.-H. R. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.64 · Cited by 4,734 · cited 1×
“Thus, it is important to also examine local fit indices (residuals; modification indices; cf. McDonald & Ho, 2002).”
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536
Flora, D. B., & Curran, P. J. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.466 · Cited by 3,105
Ensuring Positiveness of the Scaled Difference Chi-square Test Statistic
Satorra, A., & Bentler, P. M. (2009) · Psychometrika · doi:10.1007/s11336-009-9135-y · Cited by 2,880
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012) · Psychological Methods · doi:10.1037/a0029315 · Cited by 2,816
Muthén, B. (1984) · Psychometrika · doi:10.1007/bf02294210 · Cited by 2,029
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778 · cited 2×
“…tests of measurement invariance fail to be supported for one or more of the four models, it is useful to have a method with which to gauge the practical significance of the differences in measurement model parameters over time (i.e., whether the differences have practical implications; Kirk, 1996).”
Beauducel, A., & Herzberg, P. Y. (2006) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1302_2 · Cited by 1,390
Maximum Likelihood Estimation of the Polychoric Correlation Coefficient
Olsson, U. (1979) · Psychometrika · doi:10.1007/bf02296207 · Cited by 1,127
Widaman, K. F., Ferrer, E. E., & Conger, R. D. (2010) · Child Development Perspectives · doi:10.1111/j.1750-8606.2009.00110.x · Cited by 961
Item factor analysis: Current approaches and future directions.
Wirth, R. J., & Edwards, M. C. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.58 · Cited by 958
Assessing Factorial Invariance in Ordered-Categorical Measures
Millsap, R. E., & Tein, J. Y. (2004) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3903_4 · Cited by 864
Self-citationOn the Relationship between iTem Response Theory and Factor Analysis of Discretized Variables
Takane, Y., & de Leeuw, J. (1987) · Psychometrika · doi:10.1007/bf02294363 · Cited by 699
Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation
Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510903203573 · Cited by 685
Adding Missing-Data-Relevant Variables to FIML-Based Structural Equation Models
Graham, J. A. (2003) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1001_4 · Cited by 621
The Mexican American Cultural Values Scale for Adolescents and Adults
Knight, G. P., Gonzales, N. A., Saenz, D. S., Bonds, D. D., Germán, M., Deardorff, J., Roosav, M. W., & Updegraff, K. A. (2009) · The Journal of Early Adolescence · doi:10.1177/0272431609338178 · Cited by 554
Sass, D. A., Schmitt, T. A., & Marsh, H. W. (2014) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.882658 · Cited by 456
On specifying the null model for incremental fit indices in structural equation modeling.
Widaman, K. F., & Thompson, J. S. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.1.16 · Cited by 401
Factor Analysis of Ordinal Variables: A Comparison of Three Approaches
Jöreskog, K. G., & Moustaki, I. (2001) · Multivariate Behavioral Research · doi:10.1207/s15327906347-387 · Cited by 375
Performing likelihood ratio tests with multiply-imputed data sets
Meng, X., & Rubin, D. B. (1992) · Biometrika · doi:10.1093/biomet/79.1.103 · Cited by 357
Limited- and Full-Information Estimation and Goodness-of-Fit Testing in 2 n Contingency Tables
Maydeu‐Olivares, A., & Joe, H. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000002069 · Cited by 289
Dolan, C. V. (1994) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1994.tb01039.x · Cited by 281
The Impact of Categorization With Confirmatory Factor Analysis
DiStefano, C. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0903_2 · Cited by 276
Li, K. H., Raghunathan, T. E., & Rubin, D. B. (1991) · Journal of the American Statistical Association · doi:10.1080/01621459.1991.10475152 · Cited by 246
Wu, W., West, S. G., & Taylor, A. B. (2009) · Psychological Methods · doi:10.1037/a0015858 · Cited by 236
Self-citationEstimation of IRT graded response models: Limited versus full information methods.
Forero, C. G., & Maydeu‐Olivares, A. (2009) · Psychological Methods · doi:10.1037/a0015825 · Cited by 234
Yoon, M., & Millsap, R. E. (2007) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705510701301677 · Cited by 220
Self-citationMuthén, B., & Satorra, A. (1995) · Psychometrika · doi:10.1007/bf02294325 · Cited by 183
The performance of robust test statistics with categorical data
Savalei, V., & Rhemtulla, M. (2012) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.2012.02049.x · Cited by 165
Roosa, M. W., Liu, F. F., Torres, M., Gonzales, N. A., Knight, G. P., & Saenz, D. S. (2008) · Journal of Family Psychology · doi:10.1037/0893-3200.22.2.293 · Cited by 149
On the Mean and Variance of the Tetrachoric Correlation Coefficient
Brown, M. B., & Benedetti, J. (1977) · Psychometrika · doi:10.1007/bf02293655 · Cited by 73
A Comparison of Latent Growth Models for Constructs Measured by Multiple Items
Leite, W. L. (2007) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705510701575438 · Cited by 57
A General Family of Limited Information Goodness-of-Fit Statistics for Multinomial Data
Joe, H., & Maydeu‐Olivares, A. (2010) · Psychometrika · doi:10.1007/s11336-010-9165-5 · Cited by 52
Measurement Invariance Across Groups in Latent Growth Modeling
Kim, E., & Willson, V. L. (2014) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.915374 · Cited by 32
Kim, E., & Willson, V. L. (2014) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.919821 · Cited by 29
Research works printed but not linked: 9
- Bollen KA 1989 Structural equations with latent variables New York Wiley
- Browne MW Arminger G 1995 Specification and estimation of mean-and covariance-structure models Arminger G Clogg CC Sobel ME Handbook of statistical modeling for the social and behavioral sciences 185 249 New York Springer
- Grimm KJ Ram N Estabrook R in press Growth modeling: An illustrative approach New York Guilford Press
- Millsap RE 2011 Statistical approaches to measurement invariance New York Routledge
- Mislevy RJ in press Missing responses in item response theory van der Linden WJ Hambleton RK Handbook of modern item response theory 2 2 Boca Raton, FL Chapman & Hall/CRC Press
- Rosseel Y 2012 lavaan: An R package for structural equation modeling Journal of Statistical Software 48 2 1 36 http://www.jstatsoft.org/v48/i02/
- Rubin DB 1987 Multiple imputation for nonresponse in surveys New York J. Wiley & Sons
- Samejima F 1969 Estimation of latent ability using a response pattern of graded scores Psychometric Monograph No. 17 Richmond, VA Psychometric Society Retrieved from http://www.psychometrika.org/journal/online/MN17.pdf
- Wirth RJ 2008 The effects of measurement non-invariance on parameter estimation in latent growth models Doctoral dissertation Retrieved from ProQuest dissertation and theses database. (UMI No. 3331053)
Cited by
Sorted by citations
semTools: Useful Tools for Structural Equation Modeling
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2012) · doi:10.32614/cran.package.semtools · Cited by 664
Granziera, H., & Perera, H. N. (2019) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2019.02.003 · Cited by 281
Cognitive appraisals, achievement emotions, and students’ math achievement: A longitudinal analysis.
Forsblom, L., Pekrun, R. H., Stockinger, K., & Peixoto, F. (2021) · Journal of Educational Psychology · doi:10.1037/edu0000671 · Cited by 287
Are robots becoming unpopular? Changes in attitudes towards autonomous robotic systems in Europe
Gnambs, T., & Appel, M. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.11.045 · Cited by 256
Evaluation of Pressing Issues in Ecological Momentary Assessment
Stone, A. A., Schneider, S., & Smyth, J. M. (2022) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-080921-083128 · Cited by 247
Metadata
- Authors
- Liu, Y., Millsap, R. E., West, S. G., Tein, J.-Y., Tanaka, R., & Grimm, K. J.
- Affiliations
- Arizona State University (US)
- University Health Network (CA)
- Venue
- Psychological Methods
- Published
- 2017-09
- Volume
- 22
- Issue
- 3
- Pages
- 486-506
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- PMC
- PMC5121102
- OpenAlex
- W2395889112
- Cited by
- 351
- References
- 15
- Citations by year
- 2026: 45 · 2025: 32 · 2024: 45 · 2023: 49 · 2022: 46 · 2021: 53 · 2020: 36 · 2019: 20 · 2018: 15 · 2017: 6
- FWCI
- 16.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Psychological Methods (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01 MH68920)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- measurement invariance, longitudinal data, factor models, system identification, data sparsity, missing data, developmental research
- MeSH terms
- Data Interpretation, Statistical, Factor Analysis, Statistical, Humans, Individuality, Longitudinal Studies, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
