Intraclass Correlation Associated with Therapists: Estimates and Applications in Planning Psychotherapy Research
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Design and analysis of group-randomized trials: a review of recent methodological developments
Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.423 · Cited by 677 · cited 1×
“Previous reports discuss the design and analysis of nested designs generally (e.g. Cornfield, 1978;Murray et al., 2004; Pals et al., 2008; Roberts, 1999; Roberts & Roberts, 2005; Schnurr, Friedman, Lavori, & Hsieh, 2001; Zucker, 1990) and in psychotherapy research (e.g. Baldwin, Murray, & Shadish,…”
BackgroundSelf-citationBaldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.6.842 · Cited by 646 · cited 2×
“We located four new studies that published ICC estimates (Baldwin et al., 2007; Dinger, Strack, Leichsenring, Wilmers, & Schauenburg, 2008; Kim et al., 2006; Wampold & Brown, 2005).”
BackgroundSelf-citationLange, A., Rietdijk, D., Hudcovicova, M., van de Ven, J.-P., Schrieken, B., & Emmelkamp, P. M. G. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.5.901 · Cited by 495
Self-citationEhlers, A., Clark, D. M., Hackmann, A., McManus, F., Fennell, M., Herbert, C., & Mayou, R. (2003) · Archives of General Psychiatry · doi:10.1001/archpsyc.60.10.1024 · Cited by 485
Self-citationChristensen, A., Atkins, D. C., Berns, S., Wheeler, J., Baucom, D. H., & Simpson, L. E. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.2.176 · Cited by 380
Self-citationBrief treatments for cannabis dependence: Findings from a randomized multisite trial
Babor, T. F., & The Marijuana Treatment Project Research Group (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.3.455 · Cited by 313
Design and analysis of clinical trials with clustering effects due to treatment
Roberts, C., & Roberts, S. A. (2005) · Clinical Trials · doi:10.1191/1740774505cn076oa · Cited by 299 · cited 6×
“Third, the problems created by statistical dependence do not depend on the statistical significance of the ICC estimate (Kenny et al., 2002; Murray, 1998; Roberts & Roberts, 2005), and instead are a function of both the magnitude of the ICC and the number of patients treated by each therapist.”
BackgroundInfluentialAbramowitz, J. S., Foa, E. B., & Franklin, M. E. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.2.394 · Cited by 272
Self-citationCarlbring, P., Bohman, S., Brunt, S., Buhrman, M., Westling, B. E., Ekselius, L., & Andersson, G. (2006) · American Journal of Psychiatry · doi:10.1176/ajp.2006.163.12.2119 · Cited by 262 · cited 3×
“Becausewe wanted to calculate as many ICC estimates as possible, we included Carlbring et al. (2006) in our database.”
BackgroundInfluentialSelf-citation
Cited by
Everitt, H. A., Landau, S., O’Reilly, G., Sibelli, A., Hughes, S., Windgassen, S., Holland, R., Little, P., McCrone, P., Bishop, F., et al. (2019) · Gut · doi:10.1136/gutjnl-2018-317805 · Cited by 198
Paxling, B., Lundgren, S., Norman, A., Almlöv, J., Carlbring, P., Cuijpers, P., & Andersson, G. (2013) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465812000240 · Cited by 162
Self-citationEhlers, A., Grey, N., Wild, J., Stott, R., Liness, S., Deale, A., Handley, R., Albert, I., Cullen, D., Hackmann, A., et al. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.08.006 · Cited by 134
“In RCTs, Ehlers et al. (2003, 2005, see Baldwin et al., 2011) and Kubany et al. (2004) found no therapist effects, while Duffy et al. (2007) reported significantly worse outcome for one therapist who was inexperienced in delivering the treatment protocol.”
BackgroundResultSelf-citationAmiel, S. A., Potts, L., Goldsmith, K., Jacob, P., Smith, E. L., Gonder-Frederick, L., Heller, S., Toschi, E., Brooks, A., Kariyawasam, D., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-29488-x · Cited by 61
Everitt, H., Landau, S., Little, P., Bishop, F. L., McCrone, P., O'Reilly, G., Coleman, N., Logan, R., Chalder, T., & Moss-Morris, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008622 · Cited by 45
Metadata
- Authors
- Baldwin, S. A., Murray, D. M., Shadish, W. R., Pals, S. L., Holland, J. M., Abramowitz, J. S., Andersson, G., Atkins, D. C., Carlbring, P., Carroll, K. M., Christensen, A., Eddington, K. M., Ehlers, A., Feaster, D. J., Keijsers, G. P. J., Koch, E., Kuyken, W., Lange, A., Lincoln, T., Stephens, R. S., Taylor, S., Trepka, C., & Watson, J.
- Venue
- Cognitive Behaviour Therapy
- Publisher
- Informa UK Limited
- Published
- 2011-02-19
- Volume
- 40
- Issue
- 1
- Pages
- 15-33
- ISSN
- 1650-6073 (ISSN-L)
- 1651-2316
- OpenAlex
- W2126939865
- Cited by
- 81
- References
- 14
- FWCI
- 4.30
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Intraclass Correlation Associated with Therapists: Estimates and Applications in Planning Psychotherapy Research. https://app.cochat.ai/papers/10.1080/16506073.2010.520731
- The paper
- Baldwin, S. A., Murray, D. M., Shadish, W. R., Pals, S. L., Holland, J. M., Abramowitz, J. S., Andersson, G., Atkins, D. C., Carlbring, P., Carroll, K. M., Christensen, A., Eddington, K. M., Ehlers, A., Feaster, D. J., Keijsers, G. P. J., Koch, E., Kuyken, W., Lange, A., Lincoln, T., . . . Watson, J. (2011). Intraclass Correlation Associated with Therapists: Estimates and Applications in Planning Psychotherapy Research. Cognitive Behaviour Therapy. https://doi.org/10.1080/16506073.2010.520731
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.