Therapist variation within randomised trials of psychotherapy: implications for precision, internal and external validity
The DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index.
Abstract
Nesting of patients within therapists in psychotherapy trials creates an additional level within the design. The multilevel nature of this design has implications for the precision, internal and external validity of estimates of the treatment effect. Prior to or during a trial, psychotherapies are allocated to therapists and therapists are assigned to patients such that the therapist becomes part of the causal pathway from the intervention to the patient. It is therefore important to consider not only the relationship between interventions and patients but also relationships between interventions and therapists and between therapists and patients. Research designs comparing the effects of therapeutic approaches, therapist characteristics and packages of the two can be unified by viewing therapists as an important source of variability within psychotherapy outcome studies. Methodological considerations arising from therapist variation will be discussed, drawing together and building upon the associated psychotherapy and statistical literatures. Parallels will also be made with related designs and methods of analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To account for therapist clustering, we used a design effect of 1.87 for the intervention group, assuming an average of 30 carers per therapist and an intracluster correlation of 0.03.33”
In the citing paper
“Another limitation is related to potential therapist effects, which has implications for evaluating the treatment outcome (Thompson et al. 2012;Walwyn and Roberts 2010).”
From Trial-Based Cognitive Therapy: Efficacy of a New CBT Approach for Treating Social Anxiety Disorder with Comorbid Depression. (2018) · cites this paper as Walwyn and Roberts 2010
Citations
Cites
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,368
Experimental Design: Procedures for the Behavioral Sciences.
Addelman, S., & Kirk, R. E. (1969) · Journal of the American Statistical Association · doi:10.2307/2283497 · Cited by 5,159
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Statistical Principles in Experimental Design
Green, P. E., Winer, B. J., Brown, D. R., & Michels, K. M. (1992) · Journal of Marketing Research · doi:10.2307/3172747 · Cited by 4,333
Elkin, I. (1989) · Archives of General Psychiatry · doi:10.1001/archpsyc.1989.01810110013002 · Cited by 2,300
Boutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292
The Great Psychotherapy Debate: Models Methods and Findings
Horvath, A. O. (2002) · Psychotherapy Research · doi:10.1093/ptr/12.1.108 · Cited by 1,705
Goulias, K. (2002) · New Directions in Civil Engineering · doi:10.1201/9781420042283.ch9 · Cited by 1,270
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
Schnurr, P. P., Friedman, M. J., Engel, C. C., Foa, E. B., Shea, M. T., Chow, B. K., Resick, P. A., Thurston, V., Orsillo, S. M., Haug, R., et al. (2007) · JAMA · doi:10.1001/jama.297.8.820 · Cited by 813
Therapist Success and Its Determinants
Luborsky, L. (1985) · Archives of General Psychiatry · doi:10.1001/archpsyc.1985.01790290084010 · Cited by 759
Some myths of psychotherapy research and the search for a paradigm.
Kiesler, D. J. (1966) · Psychological Bulletin · doi:10.1037/h0022911 · Cited by 731
Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.6.842 · Cited by 646
A Treatment Outcome Study for Sexually Abused Preschool Children: Initial Findings
COHEN, J. A., & MANNARINO, A. P. (1996) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199601000-00011 · Cited by 534
NIMH Treatment of Depression Collaborative Research Program: Background and Research Plan
Elkin, I. (1985) · Archives of General Psychiatry · doi:10.1001/archpsyc.1985.01790260103013 · Cited by 520
Contamination in trials: is cluster randomisation the answer?
Torgerson, D. J. (2001) · BMJ · doi:10.1136/bmj.322.7282.355 · Cited by 508
Venning, P. (2000) · BMJ · doi:10.1136/bmj.320.7241.1048 · Cited by 494
Wampold, B. E., & Brown, G. S. (. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.914 · Cited by 476
Design and Analysis of Cluster Randomization Trials in Health Research
Davis, W. W. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s433 · Cited by 442
Pitfalls of and controversies in cluster randomization trials
Donner, A., & Klar, N. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.416 · Cited by 442
Need for expertise based randomised controlled trials
Devereaux, P. J., Bhandari, M., Clarke, M., Montori, V. M., Cook, D. J., Yusuf, S., Sackett, D. L., Cinà, C. S., Walter, S. D., Haynes, B., et al. (2005) · BMJ · doi:10.1136/bmj.330.7482.88 · Cited by 427
Multiple membership multiple classification (MMMC) models
Browne, W. J., Goldstein, H., & Rasbash, J. (2001) · Statistical Modelling · doi:10.1177/1471082x0100100202 · Cited by 376
Butler, G., Fennell, M., Robson, P., & Gelder, M. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.167 · Cited by 365
Waiting for supershrink: an empirical analysis of therapist effects
Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.383 · Cited by 330
Kim, D.-M., Wampold, B. E., & Bolt, D. M. (2006) · Psychotherapy Research · doi:10.1080/10503300500264911 · Cited by 330
Blanchard, E. B., Hickling, E. J., Devineni, T., Veazey, C. H., Galovski, T. E., Mundy, E., Malta, L. S., & Buckley, T. C. (2003) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(01)00131-0 · Cited by 300
Design and analysis of clinical trials with clustering effects due to treatment
Roberts, C., & Roberts, S. A. (2005) · Clinical Trials · doi:10.1191/1740774505cn076oa · Cited by 299
Self-citationCognitive Trauma Therapy for Battered Women With PTSD (CTT-BW).
Kubany, E. S., Hill, E. E., Owens, J. A., Iannce-Spencer, C., McCaig, M. A., Tremayne, K. J., & Williams, P. L. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.1.3 · Cited by 280
Okiishi, J. C., Lambert, M. J., Eggett, D., Nielsen, L., Dayton, D. D., & Vermeersch, D. A. (2006) · Journal of Clinical Psychology · doi:10.1002/jclp.20272 · Cited by 253
Comparative outcome studies of psychotherapy: Methodological issues and strategies.
Kazdin, A. E. (1986) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.54.1.95 · Cited by 249
Statistical evaluation of learning curve effects in surgical trials
Cook, J. A., Ramsaya, C. R., & Fayers, P. (2004) · Clinical Trials · doi:10.1191/1740774504cn042oa · Cited by 234
Therapist effects in outpatient psychotherapy: A three-level growth curve approach.
Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B. (2007) · Journal of Counseling Psychology · doi:10.1037/0022-0167.54.1.32 · Cited by 233
Statistical considerations in the design and analysis of community intervention trials
Donner, A., & Klar, N. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00511-0 · Cited by 230
Behavioral treatment of generalized anxiety disorder
Barlow, D. H., Rapee, R. M., & Brown, T. A. (1992) · Behavior Therapy · doi:10.1016/s0005-7894(05)80221-7 · Cited by 221
Luborsky, L., McLellan, A. T., Diguer, L., Woody, G., & Seligman, D. A. (1997) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.1997.tb00099.x · Cited by 212
The individual therapist's contribution to psychotherapy process and outcome
Lambert, M. J. (1989) · Clinical Psychology Review · doi:10.1016/0272-7358(89)90004-4 · Cited by 197
Durham, R. C., Murphy, T., Allan, T., Richard, K., Treliving, L. R., & Fenton, G. W. (1994) · British Journal of Psychiatry · doi:10.1192/bjp.165.3.315 · Cited by 192
The consequence of ignoring a nested factor on measures of effect size in analysis of variance.
Wampold, B. E., & Serlin, R. C. (2000) · Psychological Methods · doi:10.1037//1082-989x.5.4.425 · Cited by 186
Do therapists vary much in their success? Findings from four outcome studies.
Luborsky, L., Crits-Christoph, P., McLellan, A. T., Woody, G., Piper, W., Liberman, B., Imber, S., & Pilkonis, P. (1986) · American Journal of Orthopsychiatry · doi:10.1111/j.1939-0025.1986.tb03483.x · Cited by 184
rhDNase as an example of recurrent event analysis
Therneau, T. M., & Hamilton, S. A. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970930)16:18<2029::aid-sim637>3.0.co;2-h · Cited by 176
Psychiatrist effects in the psychopharmacological treatment of depression
McKay, K. M., Imel, Z. E., & Wampold, B. E. (2006) · Journal of Affective Disorders · doi:10.1016/j.jad.2006.01.020 · Cited by 169
Carlin, J. B., Wolfe, R., Coffey, C., & Patton, G. C. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991015)18:19<2655::aid-sim202>3.0.co;2-# · Cited by 166
Self-citationBorkovec, T. D., Mathews, A., Chambers, A., Ebrahimi, S., Lytle, R. A., & Nelson, R. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.6.883 · Cited by 164
Lovell, K., Cox, D., Haddock, G., Jones, C., Raines, D., Garvey, R., Roberts, C., & Hadley, S. (2006) · BMJ · doi:10.1136/bmj.38940.355602.80 · Cited by 162
Self-citationThe two-sample t test versus satterthwaite's approximate f test
Moser, B. K., Stevens, G. R., & Watts, C. L. (1989) · Communications in Statistics - Theory and Methods · doi:10.1080/03610928908830135 · Cited by 160
McLELLAN, A. T., WOODY, G. E., LUBORSKY, L., & GOEHL, L. (1988) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-198807000-00004 · Cited by 134
Bayesian methods of analysis for cluster randomized trials with binary outcome data
Turner, R. M., Omar, R. Z., & Thompson, S. G. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3<453::aid-sim803>3.0.co;2-l · Cited by 126
Kerry, S. M., & Martin Bland, J. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3<377::aid-sim799>3.0.co;2-n · Cited by 124
Clustering by health professional in individually randomised trials
Lee, K. J., & Thompson, S. G. (2005) · BMJ · doi:10.1136/bmj.330.7483.142 · Cited by 119
Elkin, I., Falconnier, L., Martinovich, Z., & Mahoney, C. (2006) · Psychotherapy Research · doi:10.1080/10503300500268540 · Cited by 110
Therapist effects on outcome and alliance in inpatient psychotherapy
Dinger, U., Strack, M., Leichsenring, F., Wilmers, F., & Schauenburg, H. (2008) · Journal of Clinical Psychology · doi:10.1002/jclp.20443 · Cited by 106
Analysis of cluster randomized cross‐over trial data: a comparison of methods
Turner, R. M., White, I. R., & Croudace, T. (2007) · Statistics in Medicine · doi:10.1002/sim.2537 · Cited by 106
A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies.
Elkin, I. (1999) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.6.1.10 · Cited by 104
Crits-Christoph, P., & Mintz, J. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.59.1.20 · Cited by 103
(1988) · American Journal of Psychiatry · doi:10.1176/ajp.145.8.909 · Cited by 99
Sample size calculation for cluster randomized cross‐over trials
Giraudeau, B., Ravaud, P., & Donner, A. (2008) · Statistics in Medicine · doi:10.1002/sim.3383 · Cited by 98
Thompson, S. G., Pyke, S. D. M., & Hardy, R. J. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970930)16:18<2063::aid-sim642>3.0.co;2-8 · Cited by 98
Sample Size Estimation in Cluster Randomized Studies with Varying Cluster Size
Manatunga, A. K., Hudgens, M. G., & Chen, S. (2001) · Biometrical Journal · doi:10.1002/1521-4036(200102)43:1<75::aid-bimj75>3.0.co;2-n · Cited by 95
United Kingdom Alcohol Treatment Trial (UKATT): hypotheses, design and methods
(2001) · Alcohol and Alcoholism · doi:10.1093/alcalc/36.1.11 · Cited by 92
Sample size formulae for trials comparing group and individual treatments in a multilevel model
Moerbeek, M., & Wong, W. K. (2008) · Statistics in Medicine · doi:10.1002/sim.3115 · Cited by 92
Generalised anxiety: A controlled treatment study
Blowers, C. M., Cobb, J. P., & Mathews, A. (1987) · Behaviour Research and Therapy · doi:10.1016/0005-7967(87)90057-x · Cited by 90
The use of random effects models to allow for clustering in individually randomized trials
Lee, K. J., & Thompson, S. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn082oa · Cited by 86
Cognitive therapy vs behaviour therapy in the treatment of chronic general anxiety
Durham, R. C., & Turvey, A. A. (1987) · Behaviour Research and Therapy · doi:10.1016/0005-7967(87)90051-9 · Cited by 84
Roberts, C. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991015)18:19<2605::aid-sim237>3.0.co;2-n · Cited by 80
Self-citationTherapist effects: Clever ways to make them (and everything else) disappear
Wampold, B. E., & Bolt, D. M. (2006) · Psychotherapy Research · doi:10.1080/10503300500265181 · Cited by 79
The therapist-as-fixed-effect fallacy in psychotherapy research.
Martindale, C. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.6.1526 · Cited by 77
The Subject-by-Formulation Interaction as a Criterion of Interchangeability of Drugs
Ekbohm, G., & Melander, H. (1989) · Biometrics · doi:10.2307/2531776 · Cited by 64
(1988) · American Journal of Psychiatry · doi:10.1176/ajp.145.9.1070 · Cited by 64
Design of Department of Veterans Affairs Cooperative Study No. 420
Schnurr, P. P., Friedman, M. J., Lavori, P. W., & Hsieh, F. Y. (2001) · Controlled Clinical Trials · doi:10.1016/s0197-2456(00)00118-5 · Cited by 63
Applied Statistics: Analysis of Variance and Regression
Ziegel, E. R., Dunn, O. J., & Clark, V. A. (1988) · Technometrics · doi:10.1080/00401706.1988.10488456 · Cited by 63
(2007) · Psychopathology · doi:10.4324/9780203935071-13 · Cited by 58
Crits-Christoph, P., & Gallop, R. (2006) · Psychotherapy Research · doi:10.1080/10503300500265025 · Cited by 57
Serlin, R. C., Wampold, B. E., & Levin, J. R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.524 · Cited by 57
Brien, C. J., & Bailey, R. A. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2006.00557.x · Cited by 56
Meehl, P. E. (1955) · Annual Review of Psychology · doi:10.1146/annurev.ps.06.020155.002041 · Cited by 55
Schnurr, P. P., Friedman, M. J., Engel, C. C., Foa, E. B., Shea, M. T., Resick, P. A., James, K. E., & Chow, B. K. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.09.001 · Cited by 49
Multilevel Models in the Study of Dynamic Household Structures
Goldstein, H., Rasbash, J., Browne, W. J., Woodhouse, G., & Poulain, M. (2000) · European Journal of Population / Revue européenne de Démographie · doi:10.1023/a:1006493723125 · Cited by 42
The Question of Therapists' Differential Effectiveness: A Sheffield Psychotherapy Project Addendum
Shapiro, D. A., Firth-Cozens, J., & Stiles, W. B. (1989) · British Journal of Psychiatry · doi:10.1192/bjp.154.3.383 · Cited by 42
Power and Measures of Effect Size in Analysis of Variance With Fixed Versus Random Nested Factors.
Siemer, M., & Joormann, J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.497 · Cited by 42
The clinical trial as a paradigm for epidemiologic research
Miettinen, O. S. (1989) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(89)90143-1 · Cited by 39
Clinical trials of behavioural interventions with heterogeneous teaching subgroup effects
Hoover, D. R. (2002) · Statistics in Medicine · doi:10.1002/sim.1139 · Cited by 34
Crits-Christoph, P., Tu, X., & Gallop, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.518 · Cited by 32
Models and meanings: Therapist effects and the stories we tell
Soldz, S. (2006) · Psychotherapy Research · doi:10.1080/10503300500264937 · Cited by 32
Howard, K. I., Orlinsky, D. E., & Perilstein, J. (1976) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.44.4.520 · Cited by 28
A comparison of power approximations for satterthwaite's test
DiSantostefano, R. L., & Muller, K. E. (1995) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919508813260 · Cited by 16
Siemer, M., & Joormann, J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.535 · Cited by 15
Rejoinder to commentaries by Stephen Soldz and Paul Crits-Christoph on therapist effects
Elkin, I., Falconnier, L., Martinovich, Z., & Mahoney, C. (2006) · Psychotherapy Research · doi:10.1080/10503300500282186 · Cited by 15
Staines, G. L. (2007) · Psychotherapy · doi:10.1037/0033-3204.44.2.161 · Cited by 14
Therapy?therapist confounds in psychotherapy research
Wilkins, W. (1986) · Cognitive Therapy and Research · doi:10.1007/bf01173378 · Cited by 13
Wampold, B. E., & Bolt, D. M. (2007) · Psychotherapy Research · doi:10.1080/10503300701250339 · Cited by 12
Howard, K. I., Orlinsky, D. E., & Perilstein, J. (1976) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.44.4.520 · Cited by 11
Paul, G. L., & Licht, M. H. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.46.6.1531 · Cited by 9
Nurse therapist trainee variability: the implications for selection and training
Brooker, C., & Wiggins, R. D. J. (1983) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.1983.tb00332.x · Cited by 8
Paul, G. L., & Licht, M. H. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.6.1531 · Cited by 8
(2007) · Psychotherapy Research · doi:10.1080/10503300601039626 · Cited by 1
Research works printed but not linked: 14
- Luborsky L., 1952, Menninger Quarterly, 6, 1
- Parloff M., Waskow I., Wolfe B. Research on therapist variables in relation to process and outcome . In Garfield SL, Bergin AE, eds. Handbook of psychotherapy and behavior change. Wiley, New York; 1978 : 233-82.
- Beutler LE, Malik M., Alimohamed S. et al. Therapist variables. In Lambert MJ ed. Bergin and Garfield’s handbook of psychotherapy and behavior change. Wiley & Sons, New York; 2004: 227-306.
- Ricks DF, 1974, Supershrink: Methods of a therapist judged successful on the basis of adult outcome of adolescent patients
- Orlinsky D. , Howard K. Gender and psychotherapeutic outcome. In Brodsky AM, Hare-Muslin RT, eds. Women and psychotherapy. Guilford, New York; 1980: 3-34.
- 10.1037/0022-006X.59.1.20 · DOI · Paper page
- MRC Health Services and Public Health Board., 2000, A Framework for Development and Evaluation of RCTs for Complex Interventions to Improve Health
- 10.1037/1082-989X.5.4.425 · doi:10.1037/1082-989x.5.4.425 · Paper page
- Elashof, J.D., 2002, nQuery Advisor Version 5.0 User’s Guide
- Rasbash J., 2004, A user’s guide to MLwiN version 2.0
- White PD, 2007, BMC Neurology, 7
- Higgins JPT, 2006, Cochrane handbook for systematic reviews of interventions 4.2.6 [Updated September 2006]
- Brien CJ, 2006, Series B, 68, 571
- 10.1037/0022-006X.54.1.95 · doi:10.1037/0022-006x.54.1.95 · Paper page
Cited by
Goodyer, I. M., Reynolds, S., Barrett, B., Byford, S., Dubicka, B., Hill, J., Holland, F., Kelvin, R., Midgley, N., Roberts, C., et al. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)30378-9 · Cited by 259
“25 For the cost-eff ectiveness analysis, we calculated incremental cost-eff ectiveness ratios (the diff erence in mean cost divided by the diff erence in mean eff ect) and explored uncertainty with cost-eff ectiveness acceptability curves, which show the probability that each of the treatments is the…”
InfluentialSelf-citationLivingston, G., Barber, J., Rapaport, P., Knapp, M., Griffin, M., King, D., Livingston, D., Mummery, C., Walker, Z., Hoe, J., et al. (2013) · BMJ · doi:10.1136/bmj.f6276 · Cited by 233
Crawford, M. J., Killaspy, H., Barnes, T. R. E., Barrett, B., Byford, S., Clayton, K., Dinsmore, J., Floyd, S., Hoadley, A., Johnson, T., et al. (2012) · BMJ · doi:10.1136/bmj.e846 · Cited by 230
Livingston, G., Barber, J., Rapaport, P., Knapp, M., Griffin, M., Romeo, R., King, D., Livingston, D., Lewis-Holmes, E., Mummery, C., et al. (2014) · Health Technology Assessment · doi:10.3310/hta18610 · Cited by 192
Fjorback, L. O., Arendt, M., Ørnbøl, E., Walach, H., Rehfeld, E., Schröder, A., & Fink, P. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2012.09.006 · Cited by 144
“We further adjusted for a number of variables where the choice of adjustment variables was made prior to the analysis and based on previous research [51].”
Method
Metadata
- Authors
- Walwyn, R., & Roberts, C.
- Affiliations
- King's College London (GB)
- University of Manchester (GB)
- Corresponding author
- Roberts, Chris
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2009-07-16
- Volume
- 19
- Issue
- 3
- Pages
- 291-315
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2164241727
- Cited by
- 93
- References
- 90
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 6 · 2023: 6 · 2022: 9 · 2021: 2 · 2020: 7 · 2019: 4 · 2018: 7 · 2017: 4 · 2016: 9 · 2015: 6 · 2014: 9 · 2013: 2 · 2012: 10
- FWCI
- 7.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Research Portal (King's College London) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- multi-level design, nested data, precision, internal validity, generalizability, treatment effect estimation, therapist characteristics, causal pathways, randomized controlled trials
- MeSH terms
- Mental Disorders, Psychotherapy, Reproducibility of Results, Models, Statistical, Randomized Controlled Trials as Topic, Treatment Outcome, Biostatistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.