Still Comparing Apples With Oranges?: Some Thoughts on the Principles and Practices of Measurement Invariance Testing
Open accessAbstract
Measurement invariance testing has become an integral part of the methodological repertoire in psychological assessment.Whenever researchers attempt to compare scales or test scores across groups or over time, it needs to be ensured that they do not compare "apples with oranges."Measurement invariance testing is the road that leads to the information about whether the comparisons of means or variances are valid.In other words, invariance testing examines the between-group or between-time points variation in factor means and variances by ruling out differences in measurement (Bauer, 2017).This ensures the fairness of scales or tests and therefore contributes to crafting a validity argument (Kane, 2013).In this editorial, we briefly review some of the principles and practices of measurement invariance testing in psychological assessment contexts and encourage EJPA authors to critically review the limitations of the commonly used approach to invariance testingmulti-group confirmatory factor analysis.We suggest considering alternatives to this approach, for instance, when submitting to EJPA and we discuss the potential of these alternative approaches.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The evidence of scalar invariance permits comparing VQ scores across Spaniard and Colombian samples.
“The evidence of scalar invariance is relevant because it permits comparing scores across Spaniard and Colombian samples (Greiff & Scherer, 2018).”
From Psychometric properties of the Valuing Questionnaire in a Spaniard sample and factorial equivalence with a Colombian sample (2022) · cites this paper as Greiff & Scherer, 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536 · see citing works
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110 · see citing works
Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032813-153700 · Cited by 1,862 · see citing works
Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510903008220 · Cited by 1,093 · see citing works
Rutkowski, L., & Svetina, D. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164413498257 · Cited by 767 · see citing works
A more general model for testing measurement invariance and differential item functioning
Bauer, D. J. (2017) · Psychological Methods · doi:10.1037/met0000077 · Cited by 363 · see citing works
Marsh, H. W., Guo, J., Parker, P. D., Nagengast, B., Asparouhov, T., Muthén, B., & Dicke, T. (2017) · Psychological Methods · doi:10.1037/met0000113 · Cited by 277 · see citing works
Recent Methods for the Study of Measurement Invariance With Many Groups
Muthén, B., & Asparouhov, T. (2017) · Sociological Methods & Research · doi:10.1177/0049124117701488 · Cited by 271 · see citing works
Measurement Invariance Testing with Many Groups: A Comparison of Five Approaches
Kim, E. S., Cao, C.-H., Wang, Y., & Nguyen, D. T. (2017) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2017.1304822 · Cited by 232 · see citing works
Pendergast, L. L., von der Embse, N., Kilgus, S. P., & Eklund, K. R. (2017) · Journal of School Psychology · doi:10.1016/j.jsp.2016.11.002 · Cited by 229 · see citing works
van de Schoot, R., Kluytmans, A., Tummers, L. G., Lugtig, P. J., Hox, J. J., & Muthén, B. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00770 · Cited by 227 · see citing works
Evaluation of MIMIC-Model Methods for DIF Testing With Comparison to Two-Group Analysis
Woods, C. M. (2009) · Multivariate Behavioral Research · doi:10.1080/00273170802620121 · Cited by 196 · see citing works
Structural equation model trees.
Brandmaier, A. M., von Oertzen, T., McArdle, J. J., & Lindenberger, U. (2012) · Psychological Methods · doi:10.1037/a0030001 · Cited by 186 · see citing works
Why Psychological Assessment Needs to Start Worrying About Model Fit
Greiff, S., & Heene, M. (2017) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000450 · Cited by 126 · see citing works
Self-citationExploring Factor Model Parameters across Continuous Variables with Local Structural Equation Models
Hildebrandt, A., Lüdtke, O., Robitzsch, A., Sommer, C., & Wilhelm, O. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2016.1142856 · Cited by 116 · see citing works
Khojasteh, J., & Lo, W. (2015) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.937791 · Cited by 50 · see citing works
Hatlevik, O. E., Scherer, R., & Christophersen, K.-A. A. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.06.003 · Cited by 45 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Robitzsch, A., & Lüdtke, O. (2023) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2023.2191292 · Cited by 205 · see citing works
Zhou, Y., Xu, J., & Rief, W. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02859-8 · Cited by 122 · see citing works
Galiana, L., Oliver, A., Arena, F., Simone, G. G. D., Tomás, J. M., Vidal‐Blanco, G., Muñoz-Martínez, I., & Sansó, N. (2020) · Health and Quality of Life Outcomes · doi:10.1186/s12955-020-01618-3 · Cited by 86 · see citing works
Teig, N., Scherer, R., & Nilsen, T. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01697 · Cited by 88 · see citing works
Self-citationLara‐Cabrera, M. L., Betancort, M., Muñoz-Rubilar, C. A., Novo, N. R., Bjerkeset, O. A., & Cuevas, C. D. L. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191610106 · Cited by 64 · see citing works
Jovanović, V., Rudnev, M., Arslan, G., Buzea, C., Dimitrova, R., Góngora, V. C., Guse, T., Ho, R., Iqbal, N., Jámbori, S., et al. (2022) · Applied Research in Quality of Life · doi:10.1007/s11482-021-10024-w · Cited by 51 · see citing works
Martínez‐Molina, A., & Arias, V. B. (2018) · PeerJ · doi:10.7717/peerj.5542 · Cited by 39 · see citing works
Bravo, D., Suárez-Falcón, J. C., Bianchi, J. M., Segura‐Vargas, M. A., & Ruiz, F. J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18105118 · Cited by 42 · see citing works
Roover, K. D. (2021) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2020.1866577 · Cited by 40 · see citing works
Mixture multigroup factor analysis for unraveling factor loading noninvariance across many groups.
Roover, K. D., Vermunt, J. K., & Ceulemans, E. (2020) · Psychological Methods · doi:10.1037/met0000355 · Cited by 36 · see citing works
Psychometric properties of the Burnout Assessment Tool across four countries
Redelinghuys, K., & Morgan, B. (2023) · BMC Public Health · doi:10.1186/s12889-023-15604-z · Cited by 38 · see citing works
Primi, C., Fioravanti, G., Casale, S., & Donati, M. A. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18062979 · Cited by 28 · see citing works
Pauli, R., Gaum, P. M., Cropley, M., & Lang, J. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1013744 · Cited by 23 · see citing works
Wurster, S. (2022) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2022.101143 · Cited by 27 · see citing works
Ruiz, F. J., Suárez‐Falcón, J. C., Segura‐Vargas, M. A., & Gil‐Luciano, B. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23212 · Cited by 22 · see citing works
Donati, M. A., Montiel, C. B., Bernini, O., Gori, C., & Primi, C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0246434 · Cited by 20 · see citing works
Psychometric properties of the Avoidance and Fusion Questionnaire – Youth in Colombia
Salazar, D. M., Ruiz, F. J., Suárez-Falcón, J. C., Barreto-Zambrano, M. L., Gómez-Barreto, M. P., & Flórez, C. L. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2018.11.008 · Cited by 18 · see citing works
Large-Scale Cognitive Assessment
Maehler, D. B., & Rammstedt, B. (2020) · Methodology of educational measurement and assessment · doi:10.1007/978-3-030-47515-4 · Cited by 25 · see citing works
McDermott, R. C., Borgogna, N. C., Hammer, J. H., Berry, A. T., & Levant, R. F. (2019) · Psychology of Men & Masculinity · doi:10.1037/men0000251 · Cited by 16 · see citing works
Franzen, P., Arens, A. K., Greiff, S., van der Westhuizen, L., Fischbach, A., Wollschläger, R., & Niepel, C. (2021) · Journal of Personality Assessment · doi:10.1080/00223891.2021.1998083 · Cited by 16 · see citing works
Self-citationGrevenstein, D. (2020) · Measurement Instruments for the Social Sciences · doi:10.1186/s42409-020-00015-2 · Cited by 13 · see citing works
Ruiz, F. J., Salazar, D. M., Suárez-Falcón, J. C., Peña-Vargas, A., Ehring, T. W., Barreto-Zambrano, M. L., & Gómez-Barreto, M. P. (2019) · Assessment · doi:10.1177/1073191119843580 · Cited by 11 · see citing works
Validation of a German and English Version of the Revised Art-of-Living Inventory
Schmitz, B., Schumacher, B., Schwarz, M., & Feldmann, F. (2021) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000650 · Cited by 11 · see citing works
Donati, M., Borace, E., Franchi, E., & Primi, C. (2019) · Assessment (Odessa, Fla.) · doi:10.1177/1073191119864655 · Cited by 10 · see citing works
Well-being of pre-service teachers: A construct validation study across three countries
Haldimann, M., Hascher, T., & Holtsch, D. (2024) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2024.100346 · Cited by 10 · see citing works
Charles, N. E., Tennity, C. L., Anestis, J. C., Kim, H., & Barry, C. T. (2022) · Journal of Personality Assessment · doi:10.1080/00223891.2022.2076237 · Cited by 10 · see citing works
Analysing PIAAC Data with Structural Equation Modelling in Mplus
Scherer, R. (2020) · Methodology of educational measurement and assessment · doi:10.1007/978-3-030-47515-4_8 · Cited by 11 · see citing works
Self-citationRuiz, F. J., Odriozola-González, P., Suárez-Falcón, J. C., & Segura-Vargas, M. A. (2022) · PeerJ · doi:10.7717/peerj.12670 · Cited by 9 · see citing works
Generational differences in psychological ownership
Olckers, C., & Booysen, C. (2021) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v47i0.1844 · Cited by 9 · see citing works
Tomas, J., Seršić, D. M., Mikac, U., Rebernjak, B., Buško, V., & Witte, H. D. (2023) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12447 · Cited by 9 · see citing works
Fujiki, M., Brinton, B., Hart, C. H., Olsen, J. A., & Coombs, M. (2019) · Language Speech and Hearing Services in Schools · doi:10.1044/2018_lshss-18-0071 · Cited by 9 · see citing works
Yang, W., Laakkonen, E., & Silvén, M. (2021) · Journal of Psychoeducational Assessment · doi:10.1177/07342829211019150 · Cited by 7 · see citing works
Liou, P., & Lin, J. J. H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.674902 · Cited by 7 · see citing works
Büyüköksüz, E. (2024) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2024.100864 · Cited by 6 · see citing works
Measurement Invariance of Test Anxiety Across Four School Subjects
Schneider, R. M., Sparfeldt, J. R., Niepel, C., Buch, S. R., & Rost, D. H. (2021) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000676 · Cited by 6 · see citing works
Pauli, R., & Wilhelmy, S. (2021) · PeerJ · doi:10.7717/peerj.12604 · Cited by 6 · see citing works
Development and Validation of the Trait Sexual Motivation Scale (TSMS)
Weber, M., Reis, D., & Friese, M. (2023) · Journal of Personality Assessment · doi:10.1080/00223891.2023.2206896 · Cited by 6 · see citing works
Scholaske, L., Rodriguez, N., Sari, N. E., Spallek, J., Ziegler, M. D., & Entringer, S. (2020) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000567 · Cited by 7 · see citing works
Sources of measurement error in pediatric intelligence testing
van Grinsven, V. T. (2022) · Methodological Innovations · doi:10.1177/20597991221077919 · Cited by 5 · see citing works
Degrees of demand: a task-based analysis of the British graduate labour market
Henseke, G., Felstead, A., Gallie, D., & Green, F. (2024) · Oxford Economic Papers · doi:10.1093/oep/gpae020 · Cited by 5 · see citing works
Cross-Cultural Comparability of Latent Constructs in ILSAs
He, J., Buchholz, J., & Fischer, J. (2022) · Springer international handbooks of education · doi:10.1007/978-3-030-88178-8_58 · Cited by 5 · see citing works
Donati, M. A., Santisi, S., Romano, L., Panno, A., & Primi, C. (2025) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2025.102562 · Cited by 4 · see citing works
New Ways of Dealing with Lacking Measurement Invariance
Sauerwein, M., & Theis, D. (2021) · Accountability and educational improvement · doi:10.1007/978-3-030-69345-9_5 · Cited by 6 · see citing works
Omar, R. H., Fortin, J., Rivest‐Beauregard, M., Lonergan, M. H., & Brunet, A. (2022) · Stress and Health · doi:10.1002/smi.3172 · Cited by 3 · see citing works
Smith, W., Hermida, J., & Güss, C. D. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.965623 · Cited by 3 · see citing works
Bratt, C. (2025) · Sociological Methods & Research · doi:10.1177/00491241251405869 · Cited by 2 · see citing works
Fang, G., Teo, T. P., & Chan, P. W. K. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04870-4 · Cited by 2 · see citing works
Fradley, K., Sirois, F. M., Bentall, R. P., Ray, J., Bishop, R., Wadsley, J., & Danson, S. J. (2025) · British Journal of Health Psychology · doi:10.1111/bjhp.12810 · Cited by 2 · see citing works
Edelsbrunner, P. A., Ruggeri, K., Damnjanović, K., Greiff, S., Lemoine, J. E., & Ziegler, M. D. (2022) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000743 · Cited by 2 · see citing works
Self-citationTulowitzki, P., Pietsch, M., Grigoleit, E., & Sposato, G. (2024) · Swiss Journal of Educational Research · doi:10.24452/sjer.46.1.3 · Cited by 1 · see citing works
Rosales-Sarabia, R. M., Moore, C., Montoya, C. E., Peña, B., Ortiz, S., Santamaría, C., Odriozola-González, P., & Ruiz, F. J. (2025) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2025.100954 · Cited by 1 · see citing works
Dynamic measurement invariance cutoffs for longitudinal and dyadic data
McNeish, D. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-02964-8 · Cited by 1 · see citing works
Baghaei, P., Strietholt, R., & Christiansen, A. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1823761 · Cited by 1 · see citing works
Assessing Children in Developmental Research
Weber, A. M., Greiff, S., & Iliescu, D. (2023) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000808 · Cited by 1 · see citing works
Self-citationCross-Cultural Comparability of Latent Constructs in ILSAs
He, J., Buchholz, J., & Fischer, J. (2021) · Springer international handbooks of education · doi:10.1007/978-3-030-38298-8_58-1 · Cited by 1 · see citing works
Roover, K. D. (2020) · doi:10.31234/osf.io/82brm · Cited by 2 · see citing works
Psychometric Properties of the Generalized Pliance Questionnaire in Spain
Ruiz, F. J., Odriozola-González, P., & Suárez-Falcón, J. C. (2020) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000614
Schoeman, P. G. (2020) · Unisa Institutional Repository (University of South Africa)
Klopp, E., & Klößner, S. (2019) · doi:10.31234/osf.io/4vmku
Accountability Challenges for Distributed Implementation of Quality AssessmentsPreprint
Pages, R., Knopf, H. T., Sherlock, P., & Chapman, R. S. (2022) · doi:10.31234/osf.io/9jm2x
Dynamic Measurement Invariance Cutoffs for Two-Group Fit Index DifferencesPreprint
McNeish, D. (2025) · doi:10.31234/osf.io/d6rwx_v2 · Cited by 1 · see citing works
Dynamic Measurement Invariance Cutoffs for Two-Group Fit Index DifferencesPreprint
McNeish, D. (2024) · doi:10.31234/osf.io/d6rwx_v1
(2025) · IGI Global eBooks · doi:10.4018/979-8-3373-2377-0.ch010
Santisi, S., Ciancaleoni, M., Borghetto, M., Ferri, R., Landi, M. T., & Lang, M. (2025) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000905
Núñez, D., Trigueros, R., Yañez-Lizana, J., Aguilar-Parra, J. M., Salazar, J. M., Martínez, V., Gaete, J., & Langer, A. I. (2025) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2025.100933
Additional file 1 of Psychometric properties of the Burnout Assessment Tool across four countries
Redelinghuys, K., & Morgan, B. (2024) · Figshare · doi:10.6084/m9.figshare.26587053
Examining measurement invariance across gender in the BFAS-40
Samo, A., Stevenor, B. A., & McAbee, S. T. (2026) · Personality and Individual Differences · doi:10.1016/j.paid.2026.113695
Tafreshi, P. F., & Gravelines, Ž. (2026) · Journal of Marketing Analytics · doi:10.1057/s41270-026-00495-x
Lacomba‐Arnau, E., Martínez‐Molina, A., Perez-Diaz, O., Picó‐Pérez, M., & Barrós‐Loscertales, A. (2026) · NeuroImage · doi:10.1016/j.neuroimage.2026.122247
- Authors
- Greiff, S., & Scherer, R.
- Affiliations
- University of Luxembourg (LU)
- University of Oslo (NO)
- Venue
- European Journal of Psychological Assessment
- Publisher
- Hogrefe Publishing Group
- Published
- 2018-05
- Volume
- 34
- Issue
- 3
- Pages
- 141-144
- ISSN
- 1015-5759 (ISSN-L)
- 2151-2426
- OpenAlex
- W2805365218
- Cited by
- 76
- References
- 16
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 6 · 2023: 6 · 2022: 11 · 2021: 14 · 2020: 13 · 2019: 6 · 2018: 6
- FWCI
- 7.78
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Journal of Psychological Assessment (publishedVersion)
- Open Repository and Bibliography (University of Luxembourg) (submittedVersion, cc-by-nc-sa)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- measurement invariance, multi-group confirmatory factor analysis, cross-group comparison, test fairness, scale validity, psychological assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
