Regularized Structural Equation Modeling
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
A new method is proposed that extends the use of regularization in both lasso and ridge regression to structural equation models. The method is termed regularized structural equation modeling (RegSEM). RegSEM penalizes specific parameters in structural equation models, with the goal of creating easier to understand and simpler models. Although regularization has gained wide adoption in regression, very little has transferred to models with latent variables. By adding penalties to specific parameters in a structural equation model, researchers have a high level of flexibility in reducing model complexity, overcoming poor fitting models, and the creation of models that are more likely to generalize to new samples. The proposed method was evaluated through a simulation study, two illustrative examples involving a measurement model, and one empirical example involving the structural part of the model to demonstrate RegSEM's utility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · cited 6×
“The two most common procedures for regularization in regression are the ridge (Hoerl & Kennard, 1970) and the least absolute shrinkage and selection operator (lasso; Tibshirani, 1996); however, there are various alternative forms that can be seen as subsets or generalizations of these two procedures.”
BackgroundMethodInfluentialEstimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033 · cited 1×
“Two different methods dominate: using a fit index that takes into account complexity, such as the Akaike information criteria (AIC; Akaike, 1973) or Bayesian information criteria (BIC; Schwarz, 1978), or using cross-validation (CV).”
Methodlavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955 · cited 2×
“The simulateData() function from the lavaan package (Rosseel, 2012) in R was used to simulate both training and test data sets that conformed to the factor structure and at the specified sample size.”
MethodThe Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Handbook of Structural Equation Modeling
Howard, A. L. (2013) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2013.769397 · Cited by 3,440
Sparse Principal Component Analysis
Zou, H., Hastie, T., & Tibshirani, R. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x113430 · Cited by 3,152 · cited 7×
“First proposed in the context of PCA (Jolliffe, Trendafilov, & Uddin, 2003; Zou et al., 2006), sparse estimation procedures for dimension reduction have flourished with a number of different methods proposed for both PCA and EFA (Choi et al., 2010; Hirose & Yamamoto, 2014b; Jung & Takane, 2008;…”
MethodInfluentialPark, T., & Casella, G. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000337 · Cited by 2,934 · cited 1×
“Although the equivalence or similarity between regularization in a frequentist and Bayesian framework has been established in the context of regression (Park & Casella, 2008; Tibshirani, 1996), this has not been the case in the context of SEM.”
Method(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n300 · Cited by 2,578
Statistical Learning with Sparsity: The Lasso and Generalizations
Hastie, T., Tibshirani, R., & Wainwright, M. (2015) · Chapman & Hall/CRC eBooks · doi:10.1201/b18401 · Cited by 2,520
A General Approach to Confirmatory Maximum Likelihood Factor Analysis
Jöreskog, K. G. (1969) · Psychometrika · doi:10.1007/bf02289343 · Cited by 2,489 · cited 1×
“…calculated, this can then be inserted into a loss functionFIGURE 1 Example one factor confirmatory factor analysis model. such as maximum likelihood (Jöreskog, 1969; ML; Lawley, 1940), FML ¼ logð j jÞ þ tr C 1 log Cj jð Þ p (2) where C is the sample covariance matrix and | · | is the determinant.”
MethodModel modifications in covariance structure analysis: The problem of capitalization on chance.
MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.111.3.490 · Cited by 1,632
Bayesian structural equation modeling: A more flexible representation of substantive theory
Muthén, B., & Asparouhov, T. (2012) · Psychological Methods · doi:10.1037/a0026802 · Cited by 1,562 · cited 1×
“Overconstraining models in an attempt to make interpretation easier, can lead to unacceptable levels of misfit and biased parameter estimates (Hsu, Troncoso Skidmore, Li, & Thompson, 2014; Muthén & Asparouhov, 2012).”
BackgroundHorst, P. (1948) · Psychological Bulletin · doi:10.1037/h0052785 · Cited by 1,545
Meredith, W., & Tisak, J. (1990) · Psychometrika · doi:10.1007/bf02294746 · Cited by 1,463 · cited 1×
“This might be understood easiest with latent-growth curve models (McArdle & Epstein, 1987; Meredith & Tisak, 1990), where models can be tested with anywhere from a linear model with fixed factor loadings from the slope to the latent basis growth model, which can be seen as an exploratory approach…”
An Overview of the Health and Retirement Study
Juster, F. T., & Suzman, R. (1995) · The Journal of Human Resources · doi:10.2307/146277 · Cited by 1,435 · cited 1×
“STUDY 3: EMPIRICAL EXAMPLE To demonstrate the utility of RegSEM at the structural part of a multiple indicators multiple causes (MIMIC) SEM, we analyzed data from the 2006 Health and Retirement Study (HRS; Juster & Suzman, 1995).”
MethodBrowne, M. W. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1279 · Cited by 1,348 · cited 1×
“The simplest form of CV, and the type that has been used widely in SEM applications (Browne, 2000), is splitting the sample into two: a training and test data set.”
New Developments and Techniques in Structural Equation Modeling
(2001) · Psychology Press eBooks · doi:10.4324/9781410601858 · Cited by 1,338
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
An Overview of Analytic Rotation in Exploratory Factor Analysis
Browne, M. W. (2001) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3601_05 · Cited by 1,140 · cited 4×
“However, the desire for simplicity in model structure comes by many names, including the aforementioned simple structure (Thurstone, 1935), variable complexity (Browne, 2001), parsimony (Marsh & Hau, 1996; Raykov & Marcoulides, 1999), “sparse loadings” in the context of principal components analysis (PCA; Zou, Hastie,…”
BackgroundMethodInfluentialOn the “degrees of freedom” of the lasso
Zou, H., Hastie, T., & Tibshirani, R. (2007) · The Annals of Statistics · doi:10.1214/009053607000000127 · Cited by 947 · cited 1×
“This concurs with work done on degrees of freedom for lasso regression, as Zou, Hastie, and Tibshirani (2007) proved that the number of nonzero coefficients is an unbiased estimate of the degrees of freedom for regression.”
MethodLatent Growth Curves within Developmental Structural Equation Models
McArdle, J. J., & EPSTEIN, D. A. (1987) · Child Development · doi:10.1111/j.1467-8624.1987.tb03494.x · Cited by 932
Self-citationSpecification searches in covariance structure modeling.
MacCallum, R. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.100.1.107 · Cited by 854
A Modified Principal Component Technique Based on the LASSO
Jolliffe, I. T., Trendafilov, N. T., & Uddin, M. (2003) · Journal of Computational and Graphical Statistics · doi:10.1198/1061860032148 · Cited by 809
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
Thurstone, L. L. (1978) · American psychology in historical perspective. · doi:10.1037/10049-016 · Cited by 748
Assessing Goodness of Fit: Is Parsimony Always Desirable?
Marsh, H. W., & Hau, K.-T. (1996) · The Journal of Experimental Education · doi:10.1080/00220973.1996.10806604 · Cited by 699
Computing and evaluating factor scores.
Grice, J. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.430 · Cited by 643
Structural Equation Modeling: A Bayesian Approach
Lee, S. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/9780470024737 · Cited by 595 · cited 1×
“Although not marketed as a method for performing regularization or more broadly for achieving simple structure, Bayesian SEM (BSEM; Kaplan & Depaoli, 2012; Lee, 2007; Levy, 2011; Muthén & Asparouhov, 2011) allows the researcher a high level of flexibility in specifying model structure.”
MethodMeinshausen, N. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.12.019 · Cited by 492 · cited 1×
“This is known as the relaxed lasso (Meinshausen, 2007).”
BackgroundStructural equation modeling: a Bayesian approach
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1510 · Cited by 441
McNeish, D. M. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2015.1036965 · Cited by 432 · cited 2×
“A recent overview of using the lasso for regression (McNeish, 2015), along with applications in differential item functioning (Magis, Tuerlinckx, & Boeck, 2014; Tutz & Schauberger, 2015), highlight this movement.”
BackgroundMethodSparsistency and rates of convergence in large covariance matrix estimation
Lam, C., & Fan, J. (2009) · The Annals of Statistics · doi:10.1214/09-aos720 · Cited by 418
Chou, C.-P., & Bentler, P. M. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2501_13 · Cited by 410 · cited 1×
“Conducting specification search with the use of modification indices is fraught with problems, including capitalizing on chance (MacCallum, Roznowski, & Necowitz, 1992), as well as a low probability of finding the best global solution (Chou & Bentler, 1990).”
VI.—The Estimation of Factor Loadings by the Method of Maximum Likelihood
Lawley, D. N. (1940) · Proceedings of the Royal Society of Edinburgh · doi:10.1017/s037016460002006x · Cited by 381 · cited 2×
“…calculated, this can then be inserted into a loss functionFIGURE 1 Example one factor confirmatory factor analysis model. such as maximum likelihood (Jöreskog, 1969; ML; Lawley, 1940), FML ¼ logð j jÞ þ tr C 1 log Cj jð Þ p (2) where C is the sample covariance matrix and | · | is the determinant.”
MethodSome algebraic properties of the Reticular Action Model for moment structures
McArdle, J. J., & McDonald, R. P. (1984) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1984.tb00802.x · Cited by 338 · cited 3×
“To provide the background on general SEM and make clear what matrices are available for regularization, we briefly detail Reticular Action Model (McArdle, 2005; RAM; McArdle & McDonald, 1984) notation as it provides a 1:1 correspondence between the graphical and matrix specifications (Boker, Neale, & Rausch, 2004), as…”
MethodSelf-citationA study in factor analysis : the stability of a bi-factor solution
Holzinger, K. J., & Swineford, F. (1939) · Medical Entomology and Zoology · Cited by 254
Watson, J. B. (2013) · Psychology: From the standpoint of a behaviorist (2nd ed.). · doi:10.1037/14262-002 · Cited by 172
Kaplan, D. (1988) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2301_4 · Cited by 163 · cited 1×
“Specification search (e.g., Kaplan, 1988; MacCallum, 1986) refers to the modification of a model to improve parsimony or fit.”
BackgroundLeite, W. L., Huang, I., & Marcoulides, G. A. (2008) · Multivariate Behavioral Research · doi:10.1080/00273170802285743 · Cited by 143
A Penalty Approach to Differential Item Functioning in Rasch Models
Tutz, G., & Schauberger, G. (2013) · Psychometrika · doi:10.1007/s11336-013-9377-6 · Cited by 135 · cited 1×
“A recent overview of using the lasso for regression (McNeish, 2015), along with applications in differential item functioning (Magis, Tuerlinckx, & Boeck, 2014; Tutz & Schauberger, 2015), highlight this movement.”
Method(2013) · Statistics and Computing · doi:10.1007/11222.1573-1375 · Cited by 118
On desirability of parsimony in structural equation model selection
Raykov, T., & Marcoulides, G. A. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540135 · Cited by 104 · cited 2×
“In addition to the increased interpretability of simpler models, parameter estimates often have smaller standard errors, allowing for more precise substantive conclusions (Raykov & Marcoulides, 1999).”
BackgroundMethodRidge structural equation modelling with correlation matrices for ordinal and continuous data
Yuan, K., Wu, R., & Bentler, P. M. (2011) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x497442 · Cited by 101
de Vries, A. L. M., Mellenbergh, G. J., van Heerden, J., & Vorst, H. C. M. (2008) · Cited by 101
Detection of Differential Item Functioning Using the Lasso Approach
Magis, D., Tuerlinckx, F., & de Boeck, P. A. L. (2015) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998614559747 · Cited by 99
Sparse estimation via nonconcave penalized likelihood in factor analysis model
Hirose, K., & Yamamoto, M. (2014) · Statistics and Computing · doi:10.1007/s11222-014-9458-0 · Cited by 95 · cited 2×
“…context of PCA (Jolliffe, Trendafilov, & Uddin, 2003; Zou et al., 2006), sparse estimation procedures for dimension reduction have flourished with a number of different methods proposed for both PCA and EFA (Choi et al., 2010; Hirose & Yamamoto, 2014b; Jung & Takane, 2008; Ning & Georgiou, 2011).”
MethodBayesian Data-Model Fit Assessment for Structural Equation Modeling
Levy, R. (2011) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2011.607723 · Cited by 93 · cited 1×
“Although not marketed as a method for performing regularization or more broadly for achieving simple structure, Bayesian SEM (BSEM; Kaplan & Depaoli, 2012; Lee, 2007; Levy, 2011; Muthén & Asparouhov, 2011) allows the researcher a high level of flexibility in specifying model structure.”
MethodLatent Differential Equation Modeling with Multivariate Multi-Occasion Indicators
Boker, S. M., Neale, M. C., & Rausch, J. R. (2004) · Mathematical modelling: theory and applications · doi:10.1007/978-1-4020-1958-6_9 · Cited by 92
Exploratory Latent Growth Models in the Structural Equation Modeling Framework
Grimm, K. J., Steele, J. S., Ram, N., & Nesselroade, J. R. (2013) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2013.824775 · Cited by 85
Self-citationModel Specification Searches Using Ant Colony Optimization Algorithms
Marcoulides, G. A., & Drezner, Z. (2003) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1001_8 · Cited by 79 · cited 1×
“This has led to the proposal of heuristic search algorithms such as ant colony optimization (Leite, Huang, & Marcoulides, 2008; Marcoulides & Drezner, 2003), genetic algorithm (Marcoulides & Drezner, 2001), and tabu search (Marcoulides, Drezner, & Schumacker, 1998; for overview, see Marcoulides &…”
MethodStructural equation modeling with near singular covariance matrices
Yuan, K., & Chan, W. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2008.03.030 · Cited by 76
Estimation of an oblique structure via penalized likelihood factor analysis
Hirose, K., & Yamamoto, M. (2014) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2014.05.011 · Cited by 58
A penalized maximum likelihood approach to sparse factor analysis
Choi, J., Oehlert, G. W., & Zou, H. (2010) · Statistics and Its Interface · doi:10.4310/sii.2010.v3.n4.a1 · Cited by 53 · cited 4×
“, 2006), sparse estimation procedures for dimension reduction have flourished with a number of different methods proposed for both PCA and EFA (Choi et al., 2010; Hirose & Yamamoto, 2014b; Jung & Takane, 2008; Ning & Georgiou, 2011).”
MethodInfluentialHsu, H.-Y. M., Skidmore, S. T., Li, Y., & Thompson, B. (2014) · Methodology · doi:10.1027/1614-2241/a000084 · Cited by 40
Sparse Versus Simple Structure Loadings
Trendafilov, N. T., & Adachi, K. (2014) · Psychometrika · doi:10.1007/s11336-014-9416-y · Cited by 33 · cited 2×
“In the context of latent variable models, sparse estimation techniques were first applied to PCA through the use of penalization to create sparse loadings (Zou et al., 2006) along with alternative methods of rotation (Trendafilov & Adachi, 2014).”
BackgroundMethodModel modification in structural equation modeling.
Chou, C., & Huh, J. (2012) · Cited by 31
TEACHER'S CORNER: Examining Identification Issues in Factor Analysis
Hayashi, K., & Marcoulides, G. A. (2006) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1304_7 · Cited by 29 · cited 1×
“One could imagine starting with the largest number of factors possible (given identification constraints; e.g., Hayashi & Marcoulides, 2006), and through increasing the penalties to each factor loading, eliminate latent factors from the model when either the number of non-zero loadings or sum of…”
BackgroundThe Development of the RAM Rules for Latent Variable Structural Equation Modeling
Maydeu‐Olivares, A., & McArdle, J. J. (2005) · doi:10.4324/9781410612977-18 · Cited by 25
Self-citationGrimm, K. J., & McArdle, J. J. (2005) · Research in multi-level issues · doi:10.1016/s1475-9144(05)04015-4 · Cited by 10 · cited 1×
“Because none of the models tested in the remainder of this article include mean structures, details regarding their inclusion in matrices and the resultant expectations are omitted (see Grimm & McArdle, 2005; McArdle, 2005).”
BackgroundSelf-citation
Cited by
Botha, M., & Frost, D. M. (2020) · Society and Mental Health · doi:10.1177/2156869318804297 · Cited by 651
Developmental cognitive neuroscience using latent change score models: A tutorial and applications
Kievit, R. A., Brandmaier, A. M., Ziegler, G., van Harmelen, A.-L., de Mooij, S. M. M., Moutoussis, M., Goodyer, I. M., Bullmore, E., Jones, P. B., Fonagy, P., et al. (2018) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.11.007 · Cited by 516
Generalized Network Psychometrics: Combining Network and Latent Variable Models
Epskamp, S., Rhemtulla, M., & Borsboom, D. (2017) · Psychometrika · doi:10.1007/s11336-017-9557-x · Cited by 471
Psychometric Network Models from Time-Series and Panel Data
Epskamp, S. (2020) · Psychometrika · doi:10.1007/s11336-020-09697-3 · Cited by 345
Mayer, A., Zimmermann, J., Hoyer, J., Salzer, S., Wiltink, J., Leibing, E., & Leichsenring, F. (2020) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2019.1671196 · Cited by 21
Metadata
- Authors
- Jacobucci, R., Grimm, K. J., & McArdle, J. J.
- Venue
- Structural Equation Modeling: A Multidisciplinary Journal
- Publisher
- Informa UK Limited
- Published
- 2016-04-12
- Volume
- 23
- Issue
- 4
- Pages
- 555-566
- ISSN
- 1070-5511 (ISSN-L)
- 1532-8007
- OpenAlex
- W2336766371
- Cited by
- 207
- References
- 62
- FWCI
- 14.96
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.