Multilevel modeling of longitudinal data for psychotherapy researchers: II. The complexities
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
The authors previously reviewed the basic elements and steps to building multilevel models (MLMs) for longitudinal data typically found in psychotherapy research. The objective of this article is to focus on complexities associated with the MLM for longitudinal data analysis in psychotherapy research, which may result in proper use or misuse of the modeling structure. To do so, the authors illustrate complex scenarios and discuss issues in the implementation and interpretation of the MLM: (a) impact of missing data in the MLM, (b) determination of the complexity of the covariance structure and its implication on model interpretation, (c) issues with centering, (d) model diagnostics for MLM, (e) model formation, including implementation dependent on the treatment of time and distribution of outcome, and (f) model estimation. The authors also present data from psychotherapy research settings as examples of these complex situations. Finally, they offer some caveats and advice for recognizing these complexities and proper procession to ensure accurate implementation of the MLM and interpretation of the results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781119513469 · Cited by 4,639 · cited 1×
“Fitzmaurice, Laird, and Ware (2004) provide tables for the correct critical values for the mixture of chi-square distribution for set levels of statistical significance (e.g., a .01, a .05, a .10).”
BackgroundCentering predictor variables in cross-sectional multilevel models: A new look at an old issue.
Enders, C. K., & Tofighi, D. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.2.121 · Cited by 4,462
Linear Mixed Models for Longitudinal Data
Verbeke, G. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2294-1_3 · Cited by 3,506
Introducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., Gallop, R., McGlinchey, J. B., Markley, D. K., Gollan, J. K., et al. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.4.658 · Cited by 1,607 · cited 6×
“Figure 1 illustrates the slope estimates for dropouts and completers for the treatment conditions of Hedeker and Gibbons (1997) and Dimidjian et al. (2006), respectively.”
BackgroundMethodResultInfluentialSelf-citationModeling the Drop-Out Mechanism in Repeated-Measures Studies
Little, R. J. A. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476615 · Cited by 1,522
Keller, M. B., McCullough, J. P., Klein, D. N., Arnow, B., Dunner, D. L., Gelenberg, A. J., Markowitz, J. C., Nemeroff, C. B., Russell, J. M., Thase, M. E., et al. (2000) · New England Journal of Medicine · doi:10.1056/nejm200005183422001 · Cited by 1,331 · cited 3×
“( 2000 ) similarly saw various phases of change for which they modeled change as a piecewise model with two distinct phases (baseline to Week 4, then Week 4 to Week 12).”
BackgroundInfluentialUsing SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and Individual Growth Models
Singer, J. D. (1998) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165280 · Cited by 1,092
McCullagh, P., & Nelder, J. A. (2019) · doi:10.1201/9780203753736 · Cited by 1,013
Application of random-effects pattern-mixture models for missing data in longitudinal studies.
Hedeker, D., & Gibbons, R. D. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.1.64 · Cited by 907
Sudden gains and critical sessions in cognitive-behavioral therapy for depression.
Tang, T. Z., & DeRubeis, R. J. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.6.894 · Cited by 684 · cited 2×
“Cognitive therapy, for example, is often characterized by periods of sudden improvement (Tang & DeRubeis, 1999 ), as well as an early rapid period of improvement (Ilardi & Craighead, 1994 ; Wilson, 1999 ).”
BackgroundArbitrary metrics in psychology
Blanton, H., & Jaccard, J. (2006) · American Psychologist · doi:10.1037/0003-066x.61.1.27 · Cited by 681 · cited 2×
“Centering, whether applied to ordinary least squares (OLS) regression models or MLM, is used to provide an interpretable or meaningful zero point (Blanton & Jaccard, 2006 ; Enders & Tofighi, 2007 ).”
MethodThe role of nonspecific factors in cognitive-behavior therapy for depression.
Ilardi, S. S., & Craighead, W. E. (1994) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.1994.tb00016.x · Cited by 501
Raudenbush, S. W., & Liu, X.-F. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.387 · Cited by 455 · cited 2×
“Our attention turns toward considering the effect size associated with the γ 13 parameter and the statistical significance of γ 12 parameter (Raudenbush & Liu, 2001 ; Robert & Monaco, 2004 ).”
BackgroundDeterminants of change in cognitive therapy for depression
DeRubeis, R. J., & Feeley, M. (1990) · Cognitive Therapy and Research · doi:10.1007/bf01172968 · Cited by 401 · cited 2×
“Results from the few existing studies that have examined the relation between adherence/competence scores and outcome measures have been mixed, some positive (DeRubeis & Feeley, 1990; Feeley, DeRubeis, & Gelfand, 1999; Barber, CritsChristoph, & Luborsky, 1996) and some negative (Elkin, 1988).”
ResultUsing Multilevel Models to Analyze Couple and Family Treatment Data: Basic and Advanced Issues.
Atkins, D. C. (2005) · Journal of Family Psychology · doi:10.1037/0893-3200.19.1.98 · Cited by 276
Multilevel modeling of longitudinal data for psychotherapy researchers: I. The basics
Tasca, G. A., & Gallop, R. (2009) · Psychotherapy Research · doi:10.1080/10503300802641444 · Cited by 148 · cited 4×
“One of the main benefits of the MLM as discussed in the ‘‘basics’’ article (Tasca & Gallop, 2009) is that MLM includes all subjects in the analysis.”
BackgroundMethodInfluentialSelf-citation
Cited by
Therapeutic alliance predicts symptomatic improvement session by session.
Falkenström, F., Granström, F., & Holmqvist, R. (2013) · Journal of Counseling Psychology · doi:10.1037/a0032258 · Cited by 291
Newman, M. G., Castonguay, L. G., Borkovec, T. D., Fisher, A. J., Boswell, J. F., Szkodny, L. E., & Nordberg, S. S. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0022489 · Cited by 174
Dramatic pretend play games uniquely improve emotional control in young children
Goldstein, T. R., & Lerner, M. D. (2018) · Developmental Science · doi:10.1111/desc.12603 · Cited by 169
“The mean amount of missing data across all study variables was 7.1%, which is within tolerable limits for HLM using full maximum likelihood estimation (Gallop & Tasca, 2009).”
MethodInfluentialDiamond, G. S., Kobak, R. R., Krauthamer Ewing, E. S., Levy, S. A., Herres, J. L., Russon, J. M., & Gallop, R. J. (2019) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2018.10.006 · Cited by 147
“Hierarchical linear modeling tested treatment differences in rates of change across the full 16 weeks of treatment using the intent-to-treat sample.(24) For the SIQ-JR and the BDI, the main efficacy analysis modeled the rate of log-linear change from baseline to 16 weeks (end of active treatment).”
MethodSelf-citationA pilot randomized controlled trial of liraglutide 3.0 mg for binge eating disorder
Allison, K. C., Chao, A. M., Bruzas, M. B., McCuen‐Wurst, C., Jones, E., McAllister, C., Gruber, K., Berkowitz, R. I., Wadden, T. A., & Tronieri, J. S. (2023) · Obesity Science & Practice · doi:10.1002/osp4.619 · Cited by 75
Metadata
- Authors
- Gallop, R., & Tasca, G. A.
- Venue
- Psychotherapy Research
- Publisher
- Informa UK Limited
- Published
- 2009-07
- Volume
- 19
- Issue
- 4-5
- Pages
- 438-452
- ISSN
- 1050-3307 (ISSN-L)
- 1468-4381
- OpenAlex
- W2058515184
- Cited by
- 102
- References
- 20
- FWCI
- 2.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.