The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
A Primer to (Cross-Cultural) Multi-Group Invariance Testing Possibilities in R
Open accessAbstract
Psychology has become less WEIRD in recent years, marking progress toward becoming a truly global psychology. However, this increase in cultural diversity is not matched by greater attention to cultural biases in research. A significant challenge in culture-comparative research in psychology is that any comparisons are open to possible item bias and non-invariance. Unfortunately, many psychologists are not aware of problems and their implications, and do not know how to best test for invariance in their data. We provide a general introduction to invariance testing and a tutorial of three major classes of techniques that can be easily implemented in the free software and statistical language R. Specifically, we describe (1) confirmatory and multi-group confirmatory factor analysis, with extension to exploratory structural equation modeling, and multi-group alignment; (2) iterative hybrid logistic regression as well as (3) exploratory factor analysis and principal component analysis with Procrustes rotation. We pay specific attention to effect size measures of item biases and differential item function. Code in R is provided in the main text and online (see https://osf.io/agr5e/), and more extended code and a general introduction to R are available in the Supplementary Materials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 80,031
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955
Alternative Ways of Assessing Model Fit
Browne, M. W., & Cudeck, R. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021002005 · Cited by 27,200 · cited 2×
“The root mean square error of approximation (RMSEA; Browne and Cudeck, 1992) uses a similar logic, but also takes into account model complexity and”
BackgroundMethodComparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.3.588 · Cited by 19,023
Structural Equations with Latent Variables
Bollen, K. A. (1989) · Wiley series in probability and statistics · doi:10.1002/9781118619179 · Cited by 17,277
Hu, L.-T., & Bentler, P. M. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.424 · Cited by 12,298
Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance
Chen, F. F. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301834 · Cited by 12,217
(2007) · Cited by 10,092
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,714
Vandenberg, R. J., & Lance, C. E. (2000) · Organizational Research Methods · doi:10.1177/109442810031002 · Cited by 7,997
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1103_2 · Cited by 6,691
Validation of the five-factor model of personality across instruments and observers.
McCrae, R. R., & Costa, P. T. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.1.81 · Cited by 6,404
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536
Putnick, D. L., & Bornstein, M. H. (2016) · Developmental Review · doi:10.1016/j.dr.2016.06.004 · Cited by 4,369
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.105.3.456 · Cited by 4,115
Li, C.-H. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0619-7 · Cited by 3,031
Flora, D. B., & Curran, P. J. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.466 · Cited by 3,019
Exploratory Structural Equation Modeling
Asparouhov, T., & Muthén, B. (2009) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510903008204 · Cited by 2,665
Applications of Structural Equation Modeling in Psychological Research
MacCallum, R. C., & Austin, J. T. (2000) · Annual Review of Psychology · doi:10.1146/annurev.psych.51.1.201 · Cited by 2,623
Methods and Data Analysis for Cross-Cultural Research
van de Vijver, F. J. R., & Leung, K. (2021) · Cambridge University Press eBooks · doi:10.1017/9781107415188 · Cited by 2,612
Structural equation modelling: Adjudging model fit
Barrett, P. (2007) · Personality and Individual Differences · doi:10.1016/j.paid.2006.09.018 · Cited by 2,611
Discovering statistics using R
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2114 · Cited by 2,534
Testing measurement invariance across groups: applications in cross-cultural research.
L. Milfont, T., & Fischer, R. (2010) · International Journal of Psychological Research · doi:10.21500/20112084.857 · Cited by 1,565
Self-citationLong, J. (1983) · Sage eBooks · doi:10.4135/9781412983778 · Cited by 1,511
Little, T. D. (1997) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3201_3 · Cited by 1,393
Holgado–Tello, F. P., Chacón–Moscoso, S., Barbero–García, I., & Vila–Abad, E. (2010) · Quality & Quantity · doi:10.1007/s11135-008-9190-y · Cited by 1,200
Tucker's Congruence Coefficient as a Meaningful Index of Factor Similarity
Lorenzo-Seva, U., & ten Berge, J. M. F. (2006) · Methodology · doi:10.1027/1614-2241.2.2.57 · Cited by 1,115
Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510903008220 · Cited by 1,093
Chen, F. F. (2008) · Journal of Personality and Social Psychology · doi:10.1037/a0013193 · Cited by 962
Multiple-Group Factor Analysis Alignment
Asparouhov, T., & Muthén, B. (2014) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2014.919210 · Cited by 836
Rutkowski, L., & Svetina, D. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164413498257 · Cited by 767
Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v039.i08 · Cited by 755
Hambleton, R. K., & Jones, R. W. (1993) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.1993.tb00543.x · Cited by 731
metaSEM: an R package for meta-analysis using structural equation modeling
Cheung, M. W.-L. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01521 · Cited by 685
Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis
Bernaards, C. A., & Jennrich, R. I. (2005) · Educational and Psychological Measurement · doi:10.1177/0013164404272507 · Cited by 599
Cheung, G. W., & Rensvold, R. B. (2000) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022100031002003 · Cited by 577
The Attack of the Psychometricians
Borsboom, D. (2006) · Psychometrika · doi:10.1007/s11336-006-1447-6 · Cited by 547
Jodoin, M. G., & Gierl, M. J. (2001) · Applied Measurement in Education · doi:10.1207/s15324818ame1404_2 · Cited by 515
Covariance Structure Models: An Introduction to LISREL
Long, J. S. (1983) · Medical Entomology and Zoology · Cited by 408
The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models
McNeish, D., An, J., & Hancock, G. R. (2018) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1281286 · Cited by 398
Steinmetz, H., Schmidt, P., Tina-Booh, A., Wieczorek, S., & Schwartz, S. H. (2009) · Quality & Quantity · doi:10.1007/s11135-007-9143-x · Cited by 373
An index of goodness-of-fit based on noncentrality
McDonald, R. P. (1989) · Journal of Classification · doi:10.1007/bf01908590 · Cited by 306
Meade, A. W., & Lautenschlager, G. J. (2004) · Organizational Research Methods · doi:10.1177/1094428104268027 · Cited by 301
Boer, D., Hanke, K., & He, J. (2018) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022117749042 · Cited by 297
A Direct Comparison Approach for Testing Measurement Invariance
Cheung, G. W., & Lau, R. S. (2012) · Organizational Research Methods · doi:10.1177/1094428111421987 · Cited by 218
Nye, C. D., & Drasgow, F. (2011) · Journal of Applied Psychology · doi:10.1037/a0022955 · Cited by 211
Rotation to Perfect Congruence and the Cross Validation of Component Weights Across Populations
ten Berge, J. M. F. (1986) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2101_3 · Cited by 189
A Monte-Carlo Study of Confirmatory Factor Analytic Tests of Measurement Equivalence/Invariance
Meade, A. W., & Lautenschlager, G. J. (2004) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1101_5 · Cited by 188
Test bias in a cognitive test: differential item functioning in the CASI
Crane, P. K., Belle, G. V., & Larson, E. B. (2004) · Statistics in Medicine · doi:10.1002/sim.1713 · Cited by 170
Factor comparison: An examination of three methods
Barrett, P. T. (1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90008-5 · Cited by 134
Mai, Y., Zhang, Z. J., & Wen, Z. (2018) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2018.1444993 · Cited by 122
Kang, Y., McNeish, D., & Hancock, G. R. (2015) · Educational and Psychological Measurement · doi:10.1177/0013164415603764 · Cited by 118
Methods for Investigating Structural Equivalence
Fischer, R. D., & Fontaine, J. R. J. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511779381.010 · Cited by 113
Self-citationsirt: Supplementary Item Response Theory Models
Robitzsch, A. (2013) · doi:10.32614/cran.package.sirt · Cited by 102
Representing contextual effects in multiple-group MACS models.
Little, T. D., Card, N. A., Slegers, D. W., & Ledford, E. C. (2007) · Cited by 95
Addressing Methodological Challenges in Culture-Comparative Research
Fischer, R. D., & Poortinga, Y. H. (2018) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022117738086 · Cited by 93
Self-citationOn Chance and Factor Congruence Following Orthogonal Procrustes Rotation
Paunonen, S. V. (1997) · Educational and Psychological Measurement · doi:10.1177/0013164497057001003 · Cited by 74
Fischer, R. D., Ferreira, M. C., van Meurs, N., Gok, K., Jiang, D., Fontaine, J. R. J., Harb, C., Cieciuch, J., Achoui, M. M., Mendoza, M. S. D., et al. (2017) · Journal of International Business Studies · doi:10.1057/s41267-017-0132-6 · Cited by 59
Self-citationOn Cross-Cultural Comparative Studies with the Eysenck Personality Questionnaire
Bijnen, E. J., Net, T. Z. J. V. D., & Poortinga, Y. H. (1986) · Journal of Cross-Cultural Psychology · doi:10.1177/0022002186017001001 · Cited by 58
The Big Five and adolescent adjustment: An empirical test across six cultures
Vazsonyi, A. T., Ksinan, A. J., Mikuška, J., & Jiskrova, G. K. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.03.049 · Cited by 47
Fischer, R. D., Ferreira, M. C., Jiang, D., Cheng, B., Achoui, M. M., Wong, C. C., Barış, G., Mendoza, S., van Meurs, N., Achmadi, D., et al. (2011) · Social Justice Research · doi:10.1007/s11211-011-0142-7 · Cited by 41
Self-citationThe Questionable Value of Cross-Cultural Comparisons with the Eysenck Personality Questionnaire
Bijnen, E. J., & Poortinga, Y. H. (1988) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022188192005 · Cited by 29
Verbeke, A. C. (2019) · Journal of International Business Studies · doi:10.1057/s41267-018-0205-1 · Cited by 29
Using SEM Library in R software to Analyze Exploratory Structural Equation Models
Guàrdia‐Olmos, J., Peró, M., Bénitez-Borrego, S., & Fox, J. (2013) · Cited by 5
Cited by
The cross-cultural validity of the five-facet mindfulness questionnaire across 16 countries
Karl, J. A., Prado, S. M. M., Gračanin, A., Verhaeghen, P., Ramos, A., Mandal, S. P., Michalak, J., Zhang, C.-Q., Schmidt, C., Tran, U. S., et al. (2020) · Mindfulness · doi:10.1007/s12671-020-01333-6 · Cited by 85
Self-citationSong, H., Chan, J. S., & Ryan, C. (2024) · Journal of Cross-Cultural Psychology · doi:10.1177/00220221241285006 · Cited by 62
“…should be aware that the prerequisite of observing true cultural differences is to confirm the instrument has the same measurement properties across cultural samples, for instance, conducting CFA and/or EFA to test cultural measurement invariance (Boer et al., 2018; Fischer & Karl, 2019).”
Metadata
- Authors
- Fischer, R., & Karl, J. A.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2019-07-18
- Volume
- 10
- Pages
- 1507-1507
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2964208165
- Cited by
- 383
- References
- 68
- FWCI
- 25.58
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.