Power and sensitivity of alternative fit indices in tests of measurement invariance.

Meade, Adam W.; Johnson, Emily C.; Braddy, Phillip W.

doi:10.1037/0021-9010.93.3.568

Cited by 1,668Psychometric Methodologies and TestingEducational Assessment and PedagogyStatistical Methods and Bayesian Inference

Power and sensitivity of alternative fit indices in tests of measurement invariance.

Meade, Adam W., Johnson, Emily C., Braddy, Phillip W.·2008·Journal of Applied Psychology

DOI 10.1037/0021-9010.93.3.568

Open this paper in CoChat — free account

Abstract

Confirmatory factor analytic tests of measurement invariance (MI) based on the chi-square statistic are known to be highly sensitive to sample size. For this reason, G. W. Cheung and R. B. Rensvold (2002) recommended using alternative fit indices (AFIs) in MI investigations. In this article, the authors investigated the performance of AFIs with simulated data known to not be invariant. The results indicate that AFIs are much less sensitive to sample size and are more sensitive to a lack of invariance than chi-square-based tests of MI. The authors suggest reporting differences in comparative fit index (CFI) and R. P. McDonald's (1989) noncentrality index (NCI) to evaluate whether MI exists. Although a general value of change in CFI (.002) seemed to perform well in the analyses, condition specific change in McDonald's NCI values exhibited better performance than a single change in McDonald's NCI value. Tables of these values are provided as are recommendations for best practices in MI testing.

Integrity signals

Registry

Crossref · 10.1037/0021-9010.93.3.568

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The chi-square difference test is heavily influenced by sample size.
“Because the chi-square difference test is heavily influenced by sample size [83]”
From Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS) (2022) · cites this paper as [83]

Evaluating Structural Equation Models with Unobservable Variables and Measurement Error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.2307/3151312 · Cited by 109,314 · see citing works
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190 · see citing works
Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · see citing works
Alternative Ways of Assessing Model Fit
Browne, M. W., & Cudeck, R. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021002005 · Cited by 27,200 · see citing works
Comparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266 · see citing works
Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance
Cheung, G. W., & Rensvold, R. B. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_5 · Cited by 16,280 · see citing works
Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification
Hu, L.-T., & Bentler, P. M. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.424 · Cited by 12,298 · see citing works
A Reliability Coefficient for Maximum Likelihood Factor Analysis
Tucker, L. R., & Lewis, C. (1973) · Psychometrika · doi:10.1007/bf02291170 · Cited by 7,330 · see citing works
Factor Analysis and AIC
Akaike, H. (1987) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_29 · Cited by 4,027 · see citing works
The Analysis of Covariance Structures: Goodness-of-Fit Indices
HOELTER, J. W. (1983) · Sociological Methods & Research · doi:10.1177/0049124183011003003 · Cited by 1,624 · see citing works
Structural equation modelling in perspective
Kelloway, E. K. (1995) · Journal of Organizational Behavior · doi:10.1002/job.4030160304 · Cited by 495 · see citing works

Flagged first, then by citations

Sorted by citations

Measurement invariance conventions and reporting: The state of the art and future directions for psychological research
Putnick, D. L., & Bornstein, M. H. (2016) · Developmental Review · doi:10.1016/j.dr.2016.06.004 · Cited by 4,369 · see citing works
“Meade et al. (2008) recommended a ΔCFI < .002 based on the results of their Monte Carlo simulation, but Little (2013) suggested that the simulation parameters used by Meade and colleagues were too Dev Rev. Author manuscript; available in PMC 2017 September 01.”
BackgroundMethodInfluential
An Updated Guideline for Assessing Discriminant Validity
Rönkkö, M., & Cho, E. (2022) · Organizational Research Methods · doi:10.1177/1094428120968614 · Cited by 1,309 · see citing works
Assessing the Hypothesis of Measurement Invariance in the Context of Large-Scale International Surveys
Rutkowski, L., & Svetina, D. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164413498257 · Cited by 767 · see citing works
Translation and Validation of Body Image Instruments: Challenges, Good Practice Guidelines, and Reporting Recommendations for Test Adaptation
Swami, V., & Barron, D. (2019) · Body Image · doi:10.1016/j.bodyim.2018.08.014 · Cited by 454 · see citing works
Tackling the Problem of Construct Proliferation: A Guide to Assessing the Discriminant Validity of Conceptually Related Constructs
Shaffer, J. A., DeGeest, D., & Li, A. (2016) · Organizational Research Methods · doi:10.1177/1094428115598239 · Cited by 393 · see citing works
An examination of the latent structure of the difficulties in emotion regulation scale.
Bardeen, J. R., Fergus, T. A., & Orcutt, H. K. (2012) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-012-9280-y · Cited by 299 · see citing works
Stability and change in the Big Five personality domains: Evidence from a longitudinal study of Australians.
Wortman, J., Lucas, R. E., & Donnellan, M. B. (2012) · Psychology and Aging · doi:10.1037/a0029322 · Cited by 298 · see citing works
Measurement Invariance in International Surveys: Categorical Indicators and Fit Measure Performance
Rutkowski, L., & Svetina, D. (2017) · Applied Measurement in Education · doi:10.1080/08957347.2016.1243540 · Cited by 225 · see citing works
Race/ethnicity and measurement equivalence of the Everyday Discrimination Scale
Kim, G., Sellbom, M., & Ford, K.-L. (2014) · Psychological Assessment · doi:10.1037/a0036431 · Cited by 196 · see citing works
The Relationship Between Use of Social Network Sites, Online Social Support, and Well-Being: Results From a Six-Wave Longitudinal Study
Utz, S., & Breuer, J. (2017) · Journal of Media Psychology · doi:10.1027/1864-1105/a000222 · Cited by 188 · see citing works
25 years of higher-order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations: Higher-Order Factor Analysis
Credé, M., & Harms, P. D. (2015) · Journal of Organizational Behavior · doi:10.1002/job.2008 · Cited by 181 · see citing works
The Cyberchondria Severity Scale (CSS): An examination of structure and relations with health anxiety in a community sample
Fergus, T. A. (2014) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2014.05.006 · Cited by 159 · see citing works
Stress, self-compassion, and well-being during the transition to college
Kroshus, E., Hawrilenko, M., & Browning, A. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113514 · Cited by 142 · see citing works
Bifactor analysis and construct validity of the five facet mindfulness questionnaire (FFMQ) in non-clinical Spanish samples
Aguado, J., Luciano, J. V., Cebolla, A., Serrano-Blanco, A., Soler, J., & García-Campayo, J. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00404 · Cited by 137 · see citing works
Measuring burnout among university students: factorial validity, invariance, and latent profiles of the Italian version of the Maslach Burnout Inventory Student Survey (MBI-SS)
Portoghese, I., Leiter, M. P., Maslach, C., Galletta, M., Porru, F., D’Aloja, E., Finco, G., & Campagna, M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02105 · Cited by 124 · see citing works
Evidence for the Psychometric Validity, Internal Consistency and Measurement Invariance of Warwick Edinburgh Mental Well-being Scale Scores in Scottish and Irish Adolescents
McKay, M. T., & Andretta, J. R. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.06.071 · Cited by 122 · see citing works
Addressing Psychometric Limitations of the Difficulties in Emotion Regulation Scale Through Item Modification
Bardeen, J. R., Fergus, T. A., Hannan, S. M., & Orcutt, H. K. (2016) · Journal of Personality Assessment · doi:10.1080/00223891.2015.1091774 · Cited by 110 · see citing works
Testing a Hierarchical Model of Distress Tolerance
Bardeen, J. R., Fergus, T. A., & Orcutt, H. K. (2013) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-013-9359-0 · Cited by 72 · see citing works
Couples becoming parents: Trajectories for psychological distress and buffering effects of social support
Hughes, C., T. Devine, R., Foley, S., D. Ribner, A., Mesman, J., & Blair, C. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.01.133 · Cited by 67 · see citing works
<p>Examining psychometric properties and measurement invariance of a Chinese version of the Self-Compassion Scale – Short Form (SCS-SF) in nursing students and medical workers</p>
Meng, R., Yu, Y., Chai, S., Luo, X., Gong, B., Liu, B., Hu, Y., Luo, Y., & Yu, C. (2019) · Psychology Research and Behavior Management · doi:10.2147/prbm.s216411 · Cited by 72 · see citing works
Analyzing the developmental relation of academic self-concept and achievement in elementary school children: Alternative models point to different results.
Ehm, J.-H., Hasselhorn, M., & Schmiedek, F. (2019) · Developmental Psychology · doi:10.1037/dev0000796 · Cited by 64 · see citing works
Perfectionistic Strivings and Perfectionistic Concerns Interact to Predict Negative Emotionality: Support for the Tripartite Model of Perfectionism in Canadian and Chinese University Students
Smith, M. M., Saklofske, D. H., Yan, G., & Sherry, S. B. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2014.09.006 · Cited by 57 · see citing works
Structure of the Wechsler Intelligence Scale for Children – Fourth Edition in a Group of Children with ADHD
Gomez, R., Vance, A., & Watson, S. D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00737 · Cited by 55 · see citing works
Factorial invariance of pediatric patient self-reported fatigue across age and gender: a multigroup confirmatory factor analysis approach utilizing the PedsQL™ Multidimensional Fatigue Scale
Varni, J. W., Beaujean, A. A., & Limbers, C. A. (2013) · Quality of Life Research · doi:10.1007/s11136-013-0370-4 · Cited by 56 · see citing works
Are parenting practices associated with the development of narcissism? Findings from a longitudinal study of Mexican-origin youth
Wetzel, E., & Robins, R. W. (2016) · Journal of Research in Personality · doi:10.1016/j.jrp.2016.05.005 · Cited by 53 · see citing works
Selfitis Behavior: Assessing the Italian Version of the Selfitis Behavior Scale and Its Mediating Role in the Relationship of Dark Traits with Social Media Addiction
Monacis, L., Griffiths, M. D., Limone, P., Sinatra, M., & Servidio, R. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17165738 · Cited by 50 · see citing works
Relation between cognitive and behavioral strategies and future change in common mental health problems across 18 years.
Zainal, N. H., & Newman, M. G. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000428 · Cited by 45 · see citing works
Using the De Jong Gierveld Loneliness Scale With Early Adolescents: Factor Structure, Reliability, Stability, and External Validity
Grygiel, P., Humenny, G., & Rębisz, S. (2019) · Assessment · doi:10.1177/1073191116682298 · Cited by 44 · see citing works
Measurement invariance of cognitive abilities across ethnicity, gender, and time among older Americans
Blankson, A. N., & McArdle, J. J. (2015) · The Journals of Gerontology Series B: Psychological Sciences and Social Sciences · doi:10.1093/geronb/gbt106 · Cited by 43 · see citing works
The relationship between distress tolerance and symptoms of depression: validation of the Distress Tolerance Scale (DTS) and short-form (DTS-SF)
Brown, R. J., Burton, A. L., & Abbott, M. J. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23370 · Cited by 39 · see citing works
Exploring internal structure of a performance-based critical thinking assessment for new students in higher education
Kleemola, K., Hyytinen, H., & Toom, A. (2022) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2021.1946482 · Cited by 38 · see citing works
Using estimated factor scores from a bifactor analysis to examine the unique effects of the latent variables measured by the WAIS-IV on academic achievement.
Kranzler, J. H., Benson, N., & Floyd, R. G. (2015) · Psychological Assessment · doi:10.1037/pas0000119 · Cited by 39 · see citing works
Testing Spearman's hypotheses using a bi-factor model with WAIS-IV/WMS-IV standardization data
Frisby, C. L., & Beaujean, A. A. (2015) · Intelligence · doi:10.1016/j.intell.2015.04.007 · Cited by 39 · see citing works
Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS)
Lyon, A. R., Corbin, C. M., Brown, E. C., Ehrhart, M. G., Locke, J., Davis, C., Picozzi, E., Aarons, G. A., & Cook, C. R. (2022) · Implementation Science · doi:10.1186/s13012-022-01222-7 · Cited by 34 · see citing works
Evidence of support for dual language learners in a study of bilingual staffing patterns using the Classroom Assessment of Supports for Emergent Bilingual Acquisition (CASEBA)
Figueras-Daniel, A., & Li, Z. (2021) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2020.09.011 · Cited by 33 · see citing works
Measurement equivalence of the Social Interaction Anxiety Scale (SIAS) and Social Phobia Scale (SPS) across individuals with social anxiety disorder from Japanese and Australian sociocultural contexts
Wong, Q. J. J., Chen, J., Gregory, B., Baillie, A. J., Nagata, T., Furukawa, T. A., Kaiya, H., Peters, L., & Rapee, R. M. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.09.028 · Cited by 32 · see citing works
A multicultural examination of experiential avoidance: AAQ – II measurement comparisons across Asian American, Black, Latinx, Middle Eastern, and White college students
Borgogna, N. C., McDermott, R. C., Berry, A., Lathan, E. C., & Gonzales, J. (2020) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2020.01.011 · Cited by 29 · see citing works
The Psychometric Costs of Applicants' Faking: Examining Measurement Invariance and Retest Correlations Across Response Conditions
Krammer, G., Sommer, M., & Arendasy, M. E. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1285781 · Cited by 25 · see citing works
“Studies suggest the SRMR is influenced by the sample size, and advise against using DSRMR for measurement invariance analysis (e.g., Meade et al., 2008).”
Method
Cross-Cultural and Gender Invariance of Transdiagnostic Processes in the United States and Singapore
Zainal, N. H., Newman, M. G., & Hong, R. Y. (2021) · Assessment · doi:10.1177/1073191119869832 · Cited by 25 · see citing works
The relationships between WAIS-IV factor index scores and educational level: A bifactor model approach.
Abad, F. J., Sorrel, M. A., Román, F. J., & Colom, R. (2016) · Psychological Assessment · doi:10.1037/pas0000228 · Cited by 24 · see citing works
Measuring Psychological Inflexibility in Children and Adolescents: Evaluating the Avoidance and Fusion Questionnaire for Youth
Szemenyei, E., Reinhardt, M., Szabó, E., Szabó, K.-G., Urbán, R., Harvey, S. T., Morgan, A., Demetrovics, Z., & Kökönyei, G. (2020) · Assessment · doi:10.1177/1073191118796558 · Cited by 22 · see citing works
An explanatory model of dating violence risk factors in Spanish adolescents
Aizpitarte, A., Alonso‐Arbiol, I., & Van de Vijver, F. J. R. (2017) · Journal of Research on Adolescence · doi:10.1111/jora.12315 · Cited by 21 · see citing works
Developmental Pathways to BPD-Related Features in Adolescence: Infancy to Age 15
Brumariu, L. E., Owen, M. T., Dyer, N., & Lyons-Ruth, K. (2020) · Journal of Personality Disorders · doi:10.1521/pedi_2020_34_480 · Cited by 21 · see citing works
Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling
Shi, D., Song, H., DiStefano, C., Maydeu-Olivares, A., McDaniel, H. L., & Jiang, Z. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1514484 · Cited by 19 · see citing works
The road to addiction (might be) paved with good intentions: Motives for social media use and psychological distress among early adolescents
Maftei, A., & Diaconu-Gherasim, L. R. (2023) · Journal of Children and Media · doi:10.1080/17482798.2023.2255304 · Cited by 15 · see citing works
Does Cognitive Fusion show up similarly across two behavioral health samples? Psychometric properties and invariance of the Greek–Cognitive Fusion Questionnaire (G-CFQ)
Zacharia, M., Ioannou, M., Theofanous, A., Vasiliou, V. S., & Karekla, M. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.01.003 · Cited by 11 · see citing works
Narcissism and friendship quality: A longitudinal approach to long-term friendships
Wehner, C., & Ziegler, M. (2023) · Journal of Social and Personal Relationships · doi:10.1177/02654075221122023 · Cited by 13 · see citing works
“These cutoffs where chosen above those suggested by Meade et al. (2008), as the latter have been proposed to be too conservative (Little, 2013).”
Background
Age and Gender Invariance in the Taiwan Wechsler Intelligence Scale for Children, Fifth Edition: Higher Order Five-Factor Model
Chen, H., Zhu, J., Liao, Y.-K., & Keith, T. Z. (2020) · Journal of Psychoeducational Assessment · doi:10.1177/0734282920930542 · Cited by 8 · see citing works

Authors: Meade, A. W., Johnson, E. C., & Braddy, P. W.
Affiliations: North Carolina State University (US)
Center for Creative Leadership (US)
ORCID: Meade, Adam W.
Johnson, Emily C.
Braddy, Phillip W.
Venue: Journal of Applied Psychology
Publisher: American Psychological Association (APA)
Published: 2008-05
Volume: 93
Issue: 3
Pages: 568-592
ISSN: 0021-9010 (ISSN-L)
1939-1854
DOI: 10.1037/0021-9010.93.3.568
OpenAlex: W2091094156
Cited by: 1,668
References: 11
Citations by year: 2026: 84 · 2025: 128 · 2024: 142 · 2023: 132 · 2022: 141 · 2021: 172 · 2020: 165 · 2019: 127 · 2018: 125 · 2017: 102 · 2016: 97 · 2015: 65 · 2014: 48 · 2013: 66 · 2012: 33
FWCI: 20.51
Type: journal article
Language: en
Open access: closed
Locations: Journal of Applied Psychology (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Education › Educational Assessment and Pedagogy
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords: measurement invariance, confirmatory factor analysis, sample size sensitivity, chi-square test, comparative fit index, simulation study, best practices
MeSH terms: Humans, Power, Psychological, Psychological Tests, Sensitivity and Specificity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Power and sensitivity of alternative fit indices in tests of measurement invariance.

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Record

About these checks