Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices.
Abstract
Fit indices are widely used in order to test the model fit for structural equation models. In a highly influential study, Hu and Bentler (1999) showed that certain cutoff values for these indices could be derived, which, over time, has led to the reification of these suggested thresholds as "golden rules" for establishing the fit or other aspects of structural equation models. The current study shows how differences in unique variances influence the value of the global chi-square model test and the most commonly used fit indices: Root-mean-square error of approximation, standardized root-mean-square residual, and the comparative fit index. Using data simulation, the authors illustrate how the value of the chi-square test, the root-mean-square error of approximation, and the standardized root-mean-square residual are decreased when unique variances are increased although model misspecification is present. For a broader understanding of the phenomenon, the authors used different sample sizes, number of observed variables per factor, and types of misspecification. A theoretical explanation is provided, and implications for the application of structural equation modeling are discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The RMSEA can result in artificially lower estimates when there are weak factor loadings, as was the case in this model.
“However, the RMSEA will result in artificially lower estimates when there are weak factor loadings (Heene et al., 2011; Savalei, 2012), as is the case in this model, which can lead to an acceptance of a larger saturated model (Rigdon, 1996).”
From A psychometric analysis of the reading the mind in the eyes test: toward a brief form for research and applied settings (2015) · cites this paper as Heene et al., 2011
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · see citing works
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1103_2 · Cited by 6,691 · see citing works
Validation of the five-factor model of personality across instruments and observers.
McCrae, R. R., & Costa, P. T. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.1.81 · Cited by 6,404 · see citing works
Principles and practice in reporting structural equation analyses.
McDonald, R. P., & Ho, M.-H. R. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.64 · Cited by 4,734 · see citing works
How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power
Muthén, L. K., & Muthén, B. O. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0904_8 · Cited by 2,362 · see citing works
Effect of the number of variables on measures of fit in structural equation modeling
Kenny, D. A., & McCoach, D. B. (2003) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1003_1 · Cited by 1,231 · see citing works
Curran, P. J., Bollen, K. A., Paxton, P., Kirby, J., & Chen, F. (2002) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3701_01 · Cited by 137 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Achievement Emotions and Academic Performance: Longitudinal Models of Reciprocal Effects
Pekrun, R., Lichtenfeld, S., Marsh, H. W., Murayama, K., & Goetz, T. (2017) · Child Development · doi:10.1111/cdev.12704 · Cited by 1,091 · see citing works
What is working memory capacity, and how can we measure it?
Wilhelm, O., Hildebrandt, A., & Oberauer, K. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00433 · Cited by 468 · see citing works
Swami, V., & Barron, D. (2019) · Body Image · doi:10.1016/j.bodyim.2018.08.014 · Cited by 454 · see citing works
Examination of the Weighted Root Mean Square Residual: Evidence for Trustworthiness?
DiStefano, C., Liu, J., Jiang, N., & Shi, D. (2018) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2017.1390394 · Cited by 372 · see citing works
Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317 · see citing works
Reciprocal relations between students’ academic enjoyment, boredom, and achievement over time
Putwain, D. W., Becker, S., Symes, W., & Pekrun, R. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2017.08.004 · Cited by 228 · see citing works
Olderbak, S., Wilhelm, O., Olaru, G., Geiger, M., Brenneman, M. W., & Roberts, R. D. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01503 · Cited by 228 · see citing works
Yin, H., Huang, S., & Wang, W. (2016) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph13090907 · Cited by 214 · see citing works
Are fit indices used to test psychopathology structure biased? A simulation study
Greene, A. L., Eaton, N. R., Li, K., Forbes, M. K., Krueger, R. F., Markon, K. E., Waldman, I. D., Cicero, D. C., Conway, C. C., Docherty, A. R., et al. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000434 · Cited by 148 · see citing works
Obbarius, N., Fischer, F., Liegl, G., Obbarius, A., & Rose, M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.584333 · Cited by 138 · see citing works
Schroeders, U., Kubera, F., & Gnambs, T. (2022) · Assessment · doi:10.1177/10731911211033894 · Cited by 76 · see citing works
Behnke, A., Conrad, D., Kolassa, I.-T., & Rojas, R. (2019) · European Journal of Psychotraumatology · doi:10.1080/20008198.2019.1606628 · Cited by 26 · see citing works
Raimondi, G., Balsamo, M., Ebisch, S. J. H., Continisio, M., Lester, D., Saggino, A., & Innamorati, M. (2023) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-023-10098-w · Cited by 18 · see citing works
- Authors
- Heene, M., Hilbert, S., Draxler, C., Ziegler, M., & Bühner, M.
- Affiliations
- University of Graz (AT)
- Ludwig-Maximilians-Universität München (DE)
- Humboldt-Universität zu Berlin (DE)
- Venue
- Psychological Methods
- Published
- 2011-09
- Volume
- 16
- Issue
- 3
- Pages
- 319-336
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2152225530
- Cited by
- 406
- References
- 7
- Citations by year
- 2026: 19 · 2025: 28 · 2024: 35 · 2023: 52 · 2022: 48 · 2021: 36 · 2020: 30 · 2019: 33 · 2018: 31 · 2017: 21 · 2016: 25 · 2015: 19 · 2014: 15 · 2013: 8 · 2012: 5
- FWCI
- 13.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- model misfit, unique variance, model fit indices, cut-off values, chi-square test, RMSEA, comparative fit index, structural equation modeling, model misspecification
- MeSH terms
- Factor Analysis, Statistical, Humans, Models, Psychological, Psychology, Models, Statistical, Chi-Square Distribution, Likelihood Functions, Behavioral Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
