Model modifications in covariance structure analysis: The problem of capitalization on chance.

Abstract

In applications of covariance structure modeling in which an initial model does not fit sample data well, it has become common practice to modify that model to improve its fit. Because this process is data driven, it is inherently susceptible to capitalization on chance characteristics of the data, thus raising the question of whether model modifications generalize to other samples or to the population. This issue is discussed in detail and is explored empirically through sampling studies using 2 large sets of data. Results demonstrate that over repeated samples, model modifications may be very inconsistent and cross-validation results may behave erratically. These findings lead to skepticism about generalizability of models resulting from data-driven modifications of an initial model. The use of alternative a priori models is recommended as a preferred strategy.

Cites (1)

1. Structural model evaluation and modification: An interval estimation approach
   Steiger, James H. · 1990 · Multivariate Behavioral Research

Show all 1 references

Cited by (59)

1. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories.
   Lovibond, P.F., Lovibond, S.H. · 1995 · Behaviour Research and Therapy
2. The measurement of engagement and burnout: a two sample confirmatory factor analytic approach.
   Schaufeli, Wilmar B., Salanova, Marisa, González-romá, Vicente, Bakker, Arnold B. · 2002 · Journal of Happiness Studies
3. Job demands, job resources, and their relationship with burnout and engagement: a multi‐sample study
   Schaufeli, Wilmar B., Bakker, Arnold B. · 2004 · Journal of Organizational Behavior
4. Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures
   Schermelleh-Engel, Karin, Moosbrugger, Helfried, Müller, Hans · 2003 · Department of Psychology - University of Koblenz-Landau
5. Trust and TAM in Online Shopping: An Integrated Model1
   Gefen, David, Karahanna, Elena, Straub, Detmar W. · 2003 · MIS Quarterly

See all 1,632