How to model and interpret cross-lagged effects in psychotherapy mechanisms of change research: A comparison of multilevel and structural equation models.
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Modeling cross-lagged effects in psychotherapy mechanisms of change studies is complex and requires careful attention to model selection and interpretation. However, there is a lack of field-specific guidelines. We aimed to (a) describe the estimation and interpretation of cross lagged effects using multilevel models (MLM) and random-intercept cross lagged panel model (RI-CLPM); (b) compare these models' performance and risk of bias using simulations and an applied research example to formulate recommendations for practice. METHOD: Part 1 is a tutorial focused on introducing/describing dynamic effects in the form of autoregression and bidirectionality. In Part 2, we compare the estimation of cross-lagged effects in RI-CLPM, which takes dynamic effects into account, with three commonly used MLMs that cannot accommodate dynamics. In Part 3, we describe a Monte Carlo simulation study testing model performance of RI-CLPM and MLM under realistic conditions for psychotherapy mechanisms of change studies. RESULTS: Our findings suggested that all three MLMs resulted in severely biased estimates of cross-lagged effects when dynamic effects were present in the data, with some experimental conditions generating statistically significant estimates in the wrong direction. MLMs performed comparably well only in conditions which are conceptually unrealistic for psychotherapy mechanisms of change research (i.e., no inertia in variables and no bidirectional effects). DISCUSSION: Based on conceptual fit and our simulation results, we strongly recommend using fully dynamic structural equation modeling models, such as the RI-CLPM, rather than static, unidirectional regression models (e.g., MLM) to study cross-lagged effects in mechanisms of change research. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A critique of the cross-lagged panel model
Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. P. P. (2015) · Psychological Methods · doi:10.1037/a0038889 · Cited by 4,005
Cole, D. A., & Maxwell, S. E. (2003) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.112.4.558 · Cited by 3,605
The disaggregation of within-person and between-person effects in longitudinal models of change
Curran, P. J., & Bauer, D. J. (2011) · Annual Review of Psychology · doi:10.1146/annurev.psych.093008.100356 · Cited by 2,056
Flückiger, C., Rubel, J., Del Re, A. C., Horvath, A. O., Wampold, B. E., Crits-Christoph, P., Atzil-Slonim, D., Compare, A., Falkenström, F., Ekeblad, A., et al. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000594 · Cited by 180
Self-citationCarvalho, J. P., & Hopko, D. R. (2011) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2010.10.001 · Cited by 176
Conklin, L. R., & Strunk, D. R. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.011 · Cited by 59
Cited by
Why the cross-lagged panel model is almost never the right choice
Lucas, R. E. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231158378 · Cited by 361
Falkenström, F. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102435 · Cited by 39
Self-citationSemcho, S. A., Southward, M. W., Stumpp, N. E., Smith, M. M., Fruhbauerova, M., & Sauer-Zavala, S. (2024) · Psychotherapy Research · doi:10.1080/10503307.2023.2254467 · Cited by 2
“We used hierarchical linear modeling (HLM) to test our first hypothesis and RI-CLPM in to test our third hypothesis given the specific goals of each hypothesis and to contribute to a more comprehensive understanding of the strengths and limitations of each approach (Falkenström et al., 2022).”
MethodTemporal Relations Among Changes in Activation, Avoidance, and Anxiety in the Unified Protocol.
Terrill, D. R., Southward, M. W., Crowley, L., Feddock, R., Kushner, M. L., & Sauer-Zavala, S. (2024) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-023-10099-9 · Cited by 1
Metadata
- Authors
- Falkenström, F., Solomonov, N., & Rubel, J. A.
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2022-05
- Volume
- 90
- Issue
- 5
- Pages
- 446-458
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W4281381742
- Cited by
- 79
- References
- 6
- FWCI
- 11.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.