Can Psychotherapists Function as Their Own Controls?: Meta-Analysis of the Crossed Therapist Design in Comparative Psychotherapy Trials
Open accessAbstract
OBJECTIVE: Clinical trials sometimes have the same therapists deliver more than 1 psychotherapy, ostensibly to control for therapist effects. This "crossed therapist" design makes controlling for therapist allegiance imperative, as therapists may prefer one treatment they deliver to the other(s). Research has established a strong relationship between principal investigators' allegiances and treatment outcome. Study therapists' allegiances probably also influence outcome, yet this moderating factor on outcome has never been studied. DATA SOURCES: English language abstracts in PsycINFO and MEDLINE from January 1985 to December 2011 were searched for keywords psychotherapy and randomized trial. STUDY SELECTION: The search yielded 990 abstracts that were searched manually. Trials using the same therapists in more than 1 condition were included. DATA EXTRACTION: Thirty-nine studies fulfilled inclusion criteria. Meta-regression analyses assessed the influence of researchers' allegiance on treatment outcome, testing the hypothesis that studies poorly controlling for therapist allegiance would show stronger influence of researcher allegiance on outcome. A single-item measure assessed researchers' reported attempts to control for therapist allegiance. RESULTS: Only 1 of 39 studies (3%) measured therapist treatment allegiance. Another 5 (13%) mentioned controlling for, without formally assessing, therapist allegiance. Most publications (67%) did not even mention therapist allegiance. In studies not controlling for therapist allegiance, researcher allegiance strongly influenced outcome, whereas studies reporting control for therapist allegiance showed no differential researcher allegiance. Researchers with cognitive-behavioral therapy allegiance described controlling for therapist allegiance less frequently than other researchers. CONCLUSIONS: The crossed therapist design is subject to bias due to differential psychotherapist allegiance. Worrisome results suggest that researchers strongly allied to a treatment may ignore therapist allegiance, potentially skewing outcomes. All clinical trials, and especially crossed therapist designs, should measure psychotherapist allegiance to evaluate this possible bias.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This bias will be minimized by randomizing patients (and not therapists) to the different treatment conditions in a crossed-therapist design.
“We will try to minimize this bias by randomizing patients (and not therapists) to the different treatment conditions in a crossed-therapist design (Falkenstrom et al. 2013).”
From The process-outcome mindfulness effects in trainees (PrOMET) study: Protocol of a pragmatic randomized controlled trial (2015) · cites this paper as Falkenstrom et al. 2013
The Discussion (page not provided) says randomizing patients helps distribute confounding factors and that crossed therapists might control for some therapist effects. But it also says therapist–treatment confounding remains, particularly when therapists differ in competence or allegiance across treatments. The source therefore does not support an unqualified expectation that this design will minimize therapist bias. (LLM, conf=0.86)
Experimental — accuracy not yet measuredA crossed-therapist design, in which a given therapist delivers the experimental and control conditions, includes specific limitations such as the increased problem of allegiance effects.
“This design includes specific limitations (e.g., the increased problem of allegiance effects (Falkenstrom et al. 2013)), but will be necessary to conduct the study in routine care.”
From The process-outcome mindfulness effects in trainees (PrOMET) study: Protocol of a pragmatic randomized controlled trial (2015) · cites this paper as Falkenstrom et al. 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2002) · Cited by 13,211 · see citing works
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771 · see citing works
Gill, J. (2005) · Encyclopedia of Social Measurement · doi:10.1016/b0-12-369398-5/00166-3 · Cited by 3,126 · see citing works
Defining empirically supported therapies.
Chambless, D. L., & Hollon, S. D. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.66.1.7 · Cited by 3,067 · see citing works
Defining empirically supported therapies.
Chambless, D. L., & Hollon, S. D. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.1.7 · Cited by 3,067 · cited 1× · see citing works
“The RCT is considered the optimal design to ensure internal validity of clinical trials, reducing the influence of variables other than the one(s) experimentally manipulated (8,9).”
BackgroundRobust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086 · see citing works
Comparative Studies of Psychotherapies: Is It True That "Everyone Has Won and All Must Have Prizes"?
Luborsky, L. (1975) · Archives of General Psychiatry · doi:10.1001/archpsyc.1975.01760260059004 · Cited by 1,485 · see citing works
Synthesizing standardized mean‐change measures
Becker, B. J. (1988) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1988.tb00901.x · Cited by 889 · see citing works
Two-Year Outcomes for Interpersonal and Social Rhythm Therapy in Individuals With Bipolar I Disorder
Frank, E., Kupfer, D. J., Thase, M. E., Mallinger, A. G., Swartz, H. A., Fagiolini, A. M., Grochocinski, V., Houck, P., Scott, J., Thompson, W., et al. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.9.996 · Cited by 770 · cited 1× · see citing works
“Two studies had to be excluded from the meta-regression analyses: one (34) because the effect size used was time-to-event (hazard ratio) from a survival analysis, which was not Falkenström et al.”
MethodAgras, W. S., Walsh, B. T., Fairburn, C. G., Wilson, G. T., & KRAEMER, H. C. (2000) · Archives of General Psychiatry · doi:10.1001/archpsyc.57.5.459 · Cited by 712 · see citing works
Robinson, L. A., Berman, J., & Neimeyer, R. (1990) · Psychological bulletin · doi:10.1037/0033-2909.108.1.30 · Cited by 691 · see citing works
Therapist adherence/competence and treatment outcome: A meta-analytic review.
Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0018912 · Cited by 657 · see citing works
Clark, D. M., Ehlers, A., Hackmann, A., McManus, F., Fennell, M., Grey, N., Waddington, L., & Wild, J. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.3.568 · Cited by 603 · see citing works
Wilfley, D. E., Welch, R. R., Stein, R. I., Spurrell, E. B., Cohen, L. R., Saelens, B. E., Dounchis, J. Z., Frank, M. A., Wiseman, C. V., & Matt, G. E. (2002) · Archives of General Psychiatry · doi:10.1001/archpsyc.59.8.713 · Cited by 597 · see citing works
Taylor, S. J., Thordarson, D. S., Maxfield, L., Fedoroff, I. C., Lovell, K., & Ogrodniczuk, J. S. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.2.330 · Cited by 563 · see citing works
Miklowitz, D. J., Otto, M. W., Frank, E., Reilly-Harrington, N. A., Wisniewski, S. R., Kogan, J. N., Nierenberg, A. A., Calabrese, J. R., Marangell, L. B., Gyulai, L., et al. (2007) · Archives of General Psychiatry · doi:10.1001/archpsyc.64.4.419 · Cited by 537 · see citing works
Three Psychological Treatments for Bulimia Nervosa
Fairburn, C. G. (1991) · Archives of General Psychiatry · doi:10.1001/archpsyc.1991.01810290075014 · Cited by 528 · see citing works
Three Psychotherapies for Anorexia Nervosa: A Randomized, Controlled Trial
McIntosh, V. V. W., Jordan, J., Carter, F. A., Luty, S. E., McKenzie, J. M., Bulik, C. M., FRAMPTON, C. M. A., & Joyce, P. R. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.4.741 · Cited by 487 · see citing works
Shapiro, D. A., Barkham, M., Rees, A., Hardy, G. E., Reynolds, S., & Startup, M. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.62.3.522 · Cited by 485 · see citing works
Wampold, B. E., & Brown, G. S. (. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.914 · Cited by 475 · see citing works
Tarrier, N., Pilgrim, H., Sommerfield, C., Faragher, B. E., Reynolds, M., Graham, E., & Barrowclough, C. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.1.13 · Cited by 458 · see citing works
Crits-Christoph, P. F., & Mintz, J. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.20 · Cited by 352 · see citing works
Grange, D. L., Crosby, R., Rathouz, P., & Leventhal, B. (2007) · Archives of General Psychiatry · doi:10.1001/archpsyc.64.9.1049 · Cited by 339 · see citing works
Kim, D., Wampold, B. E., & Bolt, D. M. (2006) · Psychotherapy Research · doi:10.1080/10503300500264911 · Cited by 330 · see citing works
Medication and psychotherapy in the treatment of bulimia nervosa
Walsh, B. T., Wilson, G. T., Loeb, K. L., Devlin, M. J., Pike, K. M., Roose, S. P., Fleiss, J. L., & WATERNAUX, C. M. (1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.4.523 · Cited by 321 · see citing works
Greenberg, L. S. (1998) · Psychotherapy Research · doi:10.1093/ptr/8.2.210 · Cited by 281 · see citing works
A comparison of two psychological treatments for bulimia nervosa
Fairburn, C. G., Kirk, J. W., O'Connor, M. E., & Cooper, P. (1986) · Behaviour Research and Therapy · doi:10.1016/0005-7967(86)90058-6 · Cited by 267 · see citing works
Masheb, R. M., Kerns, R. D., Lozano, C., Minkin, M. J., & Richman, S. M. (2008) · Pain · doi:10.1016/j.pain.2008.09.031 · Cited by 244 · see citing works
Goldman, R., Greenberg, L., & Angus, L. (2006) · Psychotherapy Research · doi:10.1080/10503300600589456 · Cited by 211 · see citing works
Researcher allegiance and meta-analysis: The case of cognitive therapy for depression.
Gaffan, E. A., Tsaousis, I., & Kemp-Wheeler, S. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.966 · Cited by 194 · see citing works
Hopko, D. R., Armento, M. E. A., Robertson, S. M. C., Ryba, M. M., Carvalho, J. P., Colman, L. K., Mullane, C., Gawrysiak, M., Bell, J. L., McNulty, J. K., et al. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0025450 · Cited by 193 · see citing works
Therapist effectiveness: Implications for accountability and patient care
Kraus, D. R., Castonguay, L. G., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011) · Psychotherapy Research · doi:10.1080/10503307.2011.563249 · Cited by 192 · see citing works
A crossover study of focused cognitive therapy for panic disorder
Beck, A. T., Sokol, L., Clark, D. A., Berchick, R. J., & Wright, F. (1992) · American Journal of Psychiatry · doi:10.1176/ajp.149.6.778 · Cited by 185 · see citing works
Luty, S. E., Carter, J. D., McKenzie, J. M., Rae, A. M., FRAMPTON, C. M. A., Mulder, R., & Joyce, P. R. (2007) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.106.024729 · Cited by 180 · see citing works
Hudson, J. L., Rapee, R. M., Deveney, C., Schniering, C. A., Lyneham, H. J., & Bovopoulos, N. (2009) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/chi.0b013e31819c2401 · Cited by 173 · see citing works
Borkovec, T. D., & Mathews, A. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.56.6.877 · Cited by 172 · see citing works
Barkham, M., Shapiro, D. A., Hardy, G. E., & Rees, A. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.2.201 · Cited by 159 · see citing works
A Randomised Controlled Trial of Psychotherapy in Patients with Refractory Irritable Bowel Syndrome
Guthrie, E., Creed, F. H., Dawson, D., & Tomenson, B. (1993) · The British Journal of Psychiatry · doi:10.1192/bjp.163.3.315 · Cited by 155 · see citing works
Borkovec, T. D., Mathews, A., Chambers, A., Ebrahimi, S., Lytle, R. A., & Nelson, R. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.6.883 · Cited by 151 · see citing works
Snyder, D. K., & Wills, R. M. (1989) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.57.1.39 · Cited by 149 · see citing works
Strauman, T. J., Vieth, A. Z., Eddington, K. M., Kolden, G. G., Woods, T. E., Klein, M. H., Papadakis, A. A., Schneider, K. L., & Kwapil, L. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.2.367 · Cited by 146 · see citing works
Barkham, M., Rees, A., Shapiro, D. A., Stiles, W. B., Agnew, R. M., Halstead, J. E., Culverwell, A., & Harrington, V. M. G. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.5.1079 · Cited by 143 · see citing works
Herbert, J. D., Gaudiano, B. A., Rheingold, A. A., Moitra, E., Myers, V. H., Dalrymple, K. L., & Brandsma, L. L. (2009) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2008.06.004 · Cited by 125 · see citing works
Gosselin, P., Ladouceur, R., Morin, C. M., Dugas, M. J., & Baillargeon, L. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.5.908 · Cited by 122 · see citing works
Crits-Christoph, P. F., & Mintz, J. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.59.1.20 · Cited by 113 · see citing works
Elkin, I., Falconnier, L., Martinovich, Z., & Mahoney, C. (2006) · Psychotherapy Research · doi:10.1080/10503300500268540 · Cited by 110 · see citing works
Prescriptive v. Exploratory Psychotherapy
Shapiro, D. A., & Firth, J. (1987) · British Journal of Psychiatry · doi:10.1192/bjp.151.6.790 · Cited by 105 · see citing works
Thoma, N. C., McKay, D., Gerber, A. J., Milrod, B. L., Edwards, A. R., & Kocsis, J. H. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11030433 · Cited by 104 · see citing works
A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies.
Elkin, I. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.10 · Cited by 104 · see citing works
Cognitive therapy versus rogerian supportive therapy in borderline personality disorder
Cottraux, J., Note, I. D., Boutitie, F., Milliery, M., Genouihlac, V., Yao, S. N., Note, B., Mollard, E., Bonasse, F., Gaillard, S., et al. (2009) · Psychotherapy and Psychosomatics · doi:10.1159/000229769 · Cited by 99 · see citing works
A randomized trial of interpersonal therapy versus supportive therapy for social anxiety disorder
Lipsitz, J. D., Gur, M., Vermes, D., Petkova, E., Cheng, J., Miller, N. L., Laino, J., Liebowitz, M. R., & Fyer, A. J. (2007) · Depression and Anxiety · doi:10.1002/da.20364 · Cited by 98 · see citing works
A new scale for assessing the quality of randomized clinical trials of psychotherapy
Kocsis, J. H., Gerber, A. J., Milrod, B. L., Roose, S. P., Barber, J. P., Thase, M. E., Perkins, P., & Leon, A. C. (2009) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2009.07.001 · Cited by 87 · see citing works
Cottraux, J. A., Note, I. D., Yao, S. N., Mey-Guillard, C. D., Bonasse, F., Djamoussian, D., Mollard, E., Note, B., & Chen, Y. S. (2008) · Psychotherapy and Psychosomatics · doi:10.1159/000112887 · Cited by 76 · see citing works
Markowitz, J. C., Kocsis, J. H., Christos, P. J., Bleiberg, K. L., & Carlin, A. (2008) · The Journal of Nervous and Mental Disease · doi:10.1097/nmd.0b013e31817738f1 · Cited by 70 · see citing works
de Jong, K., Moerbeek, M., & van der Leeden, R. (2009) · Psychotherapy Research · doi:10.1080/10503300903376320 · Cited by 59 · see citing works
Dunn, N. J., Rehm, L. P., Schillaci, J., Souchek, J., Mehta, P. D., Ashton, C. M., Yanasak, E. V., & Hamilton, J. D. (2007) · Journal of Traumatic Stress · doi:10.1002/jts.20214 · Cited by 56 · see citing works
Bjornsson, A. S., Bidwell, L. C., Brosse, A. L., Carey, G., Hauser, M., Seghete, K. L. M., Schulz‐Heik, R. J., Weatherley, D., Erwin, B. A., & Craighead, W. E. (2011) · Depression and Anxiety · doi:10.1002/da.20877 · Cited by 45 · see citing works
Castelnuovo, G., Manzoni, G. M., Villa, V., Cesa, G. L., & Molinari, E. (2011) · Clinical Practice and Epidemiology in Mental Health · doi:10.2174/1745017901107010029 · Cited by 37 · see citing works
Hedberg, E. C. (2011) · RePEc: Research Papers in Economics · Cited by 31 · see citing works
Marchand, A., Coutu, M., Dupuis, G., Fleet, R. P., Borgeat, F., Todorov, C., & Mainguy, N. (2008) · Cognitive Behaviour Therapy · doi:10.1080/16506070701743120 · Cited by 22 · see citing works
Staines, G. L. (2007) · Psychotherapy · doi:10.1037/0033-3204.44.2.161 · Cited by 14 · see citing works
Marks, I., Lovell, K., Noshirvani, H., Livanou, M., & Thrasher, S. (1998) · Archives of General Psychiatry · doi:10.1001/archpsyc.55.4.317 · Cited by 5 · see citing works
Group cognitive and contextual therapies in treatment of depression
Zettle, R. D., & Rains, J. C. (1989) · Journal of Clinical Psychology · doi:10.1002/1097-4679(198905)45:3
Flagged first, then by citations
Cited by
Sorted by citations
Small sample adjustments for robust variance estimation with meta-regression.
Tipton, E. (2015) · Psychological Methods · doi:10.1037/met0000011 · Cited by 810 · see citing works
Handling Complex Meta-analytic Data Structures Using Robust Variance Estimates: a Tutorial in R
Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016) · Journal of Developmental and Life-Course Criminology · doi:10.1007/s40865-016-0026-5 · Cited by 390 · see citing works
Is Exposure Necessary? A Randomized Clinical Trial of Interpersonal Psychotherapy for PTSD
Markowitz, J. C., Petkova, E., Neria, Y., Meter, P. E. V., Zhao, Y., Hembree, E. A., Lovell, K., Biyanova, T., & Marshall, R. D. (2015) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14070908 · Cited by 383 · see citing works
The processes of cognitive behavioral therapy: A review of meta-analyses
Kazantzis, N., Luong, H. K., Usatoff, A. S., Impala, T., Yew, R. Y., & Hofmann, S. G. (2018) · Cognitive Therapy and Research · doi:10.1007/s10608-018-9920-y · Cited by 218 · see citing works
Cochrane’s risk of bias tool in the context of psychotherapy outcome research
Munder, T., & Barth, J. (2018) · Psychotherapy Research · doi:10.1080/10503307.2017.1411628 · Cited by 174 · see citing works
Berger, T., Krieger, T., Sude, K., Meyer, B., & Maercker, A. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.11.021 · Cited by 153 · see citing works
Bricker, J. B., Bush, T., Zbikowski, S. M., Mercer, L. D., & Heffner, J. L. (2014) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntu102 · Cited by 100 · see citing works
A systematic appraisal of allegiance effect in randomized controlled trials of psychotherapy
Dragioti, E., Dimoliatis, I., Fountoulakis, K. N., & Evangelou, E. (2015) · Annals of General Psychiatry · doi:10.1186/s12991-015-0063-1 · Cited by 91 · see citing works
CRITICAL REVIEW OF OUTCOME RESEARCH ON INTERPERSONAL PSYCHOTHERAPY FOR ANXIETY DISORDERS
Markowitz, J. C., Lipsitz, J. D., & Milrod, B. L. (2014) · Depression and Anxiety · doi:10.1002/da.22238 · Cited by 87 · see citing works
Biases in research: risk factors for non-replicability in psychotherapy and pharmacotherapy research
Leichsenring, F., Abbass, A. A., Hilsenroth, M. J., Leweke, F., Luyten, P., Keefe, J. R., Midgley, N., Rabung, S., Salzer, S., & Steinert, C. (2016) · Psychological Medicine · doi:10.1017/s003329171600324x · Cited by 79 · see citing works
A Meta-Analysis of Mindfulness-Based Therapies for Insomnia and Sleep Disturbance
Rash, J. A., Kavanagh, V. A. J., & Garland, S. N. (2019) · Sleep Medicine Clinics · doi:10.1016/j.jsmc.2019.01.004 · Cited by 70 · see citing works
What is Supportive Psychotherapy?
Markowitz, J. C. (2014) · Focus · doi:10.1176/appi.focus.12.3.285 · Cited by 68 · see citing works
Mander, J., Blanck, P., Neubauer, A. B., Kröger, P., Flückiger, C., Lutz, W., Barnow, S., Bents, H., & Heidenreich, T. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22695 · Cited by 56 · see citing works
Ott, C. V., Vinberg, M., Kessing, L. V., Bowie, C. R., Forman, J. L., & Miskowiak, K. W. (2020) · Bipolar Disorders · doi:10.1111/bdi.13021 · Cited by 55 · see citing works
Interpersonal psychotherapy for eating disorders: current perspectives
Miniati, M., Callari, A., Maglio, A., & Calugi, S. (2018) · Psychology Research and Behavior Management · doi:10.2147/prbm.s120584 · Cited by 57 · see citing works
Predicting therapist effectiveness from their own practice-based evidence.
Kraus, D. R., Bentley, J. H., Alexander, P. C., Boswell, J. F., Constantino, M. J., Baxter, E. E., & Castonguay, L. G. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000083 · Cited by 63 · see citing works
Hoyer, J., Čolić, J., Pittig, A., Crawcour, S. C., Moeser, M., Ginzburg, D. M., Lin, J., Wiltink, J., Leibing, E., & Stangier, U. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.05.012 · Cited by 47 · see citing works
Dragioti, E., Dimoliatis, I., & Evangelou, E. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007206 · Cited by 44 · see citing works
Brief psychotherapy for maternal depression: impact on mothers and children
Swartz, H. A., Cyranowski, J. M., Cheng, Y., Zuckoff, A., Brent, D. A., Markowitz, J. C., Martin, S., Amole, M. C., Ritchey, F., & Frank, E. (2016) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2016.04.003 · Cited by 46 · see citing works
Zilcha‐Mano, S., Dolev, T., Leibovich, L., & Barber, J. P. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1934-1 · Cited by 46 · see citing works
Kleinstäuber, M., Gottschalk, J.-M., Berking, M., Rau, J., & Rief, W. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.12.003 · Cited by 42 · see citing works
Holtforth, M. G., Krieger, T., Zimmermann, J., Altenstein, D., Doerig, N., Meisch, L., & Hayes, A. M. (2017) · Psychotherapy Research · doi:10.1080/10503307.2017.1397796 · Cited by 41 · see citing works
Moderators in psychotherapy meta-analysis
Spielmans, G. I., & Flückiger, C. (2018) · Psychotherapy Research · doi:10.1080/10503307.2017.1422214 · Cited by 40 · see citing works
Falkenström, F. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102435 · Cited by 39 · see citing works
Self-citationImai, H., Tajika, A., Chen, P., Pompoli, A., & Furukawa, T. A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011170.pub2 · Cited by 38 · see citing works
Supportive Evidence: Brief Supportive Psychotherapy as Active Control and Clinical Intervention
Markowitz, J. C. (2022) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.2021.20210041 · Cited by 37 · see citing works
Zia, J. K., Barney, P., Cain, K. C., Jarrett, M. E., & Heitkemper, M. M. (2015) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2015.09.027 · Cited by 34 · see citing works
Krieger, T., Meyer, B., Sude, K., Urech, A., Maercker, A. R. A., & Berger, T. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0285-9 · Cited by 33 · see citing works
Enhancing psychotherapy process with common factors feedback: A randomized, clinical trial.
McClintock, A. S., Perlman, M. R., McCarrick, S. M., Anderson, T., & Himawan, L. K. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000188 · Cited by 33 · see citing works
Ekeblad, A., Falkenström, F., Andersson, G., Vestberg, R., & Holmqvist, R. (2016) · Depression and Anxiety · doi:10.1002/da.22495 · Cited by 33 · see citing works
Self-citationHeart, M. Y. H. B., Chase, J., Myers, O., Elkins, J., SKIPPER, B. J., Schmitt, C. L., Mootz, J. J., & Waldorf, V. A. (2019) · Psychotherapy · doi:10.1037/pst0000267 · Cited by 30 · see citing works
Arvidsdotter, T., Marklund, B. R. G., & Taft, C. (2013) · BMC Complementary and Alternative Medicine · doi:10.1186/1472-6882-13-308 · Cited by 31 · see citing works
The consequences of ignoring therapist effects in trials with longitudinal data: A simulation study.
Magnusson, K., Andersson, G., & Carlbring, P. (2018) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000333 · Cited by 30 · see citing works
What to do when a psychotherapy fails
Markowitz, J. C., & Milrod, B. L. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)00119-9 · Cited by 28 · see citing works
Mander, J., Neubauer, A. B., Schlarb, A., Teufel, M., Bents, H., Hautzinger, M., Zipfel, S., Wittorf, A., & Sammet, I. (2017) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12131 · Cited by 27 · see citing works
In support of supportive psychotherapy
Markowitz, J. C. (2022) · World Psychiatry · doi:10.1002/wps.20949 · Cited by 23 · see citing works
Kemmeren, L. L., van Schaik, A., Draisma, S., Kleiboer, A., Riper, H., & Smit, J. H. (2023) · Cognitive Therapy and Research · doi:10.1007/s10608-023-10363-y · Cited by 22 · see citing works
Arvidsdotter, T., Marklund, B. R. G., & Taft, C. (2014) · BMC Complementary and Alternative Medicine · doi:10.1186/1472-6882-14-210 · Cited by 21 · see citing works
¡Enjaulad a Dodo, por favor! El cuento de que todas las psicoterapias son igual de eficaces
González‐Blanch, C., & Carral-Fernández, L. (2017) · Papeles del Psicólogo - Psychologist Papers · doi:10.23923/pap.psicol2017.2828 · Cited by 20 · see citing works
Amole, M. C., Cyranowski, J. M., Conklin, L. R., Markowitz, J. C., Martin, S. E., & Swartz, H. A. (2016) · Journal of Psychotherapy Integration · doi:10.1037/int0000039 · Cited by 19 · see citing works
Podina, I. R., Vîslă, A., Fodor, L. A., & Flückiger, C. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2019.101774 · Cited by 18 · see citing works
Does cognitive behavioral therapy for anxiety disorders improve threat reappraisal?: A meta-analysis
Draheim, A. A., & Anderson, P. L. (2021) · Journal of Behavioral and Cognitive Therapy · doi:10.1016/j.jbct.2020.12.004 · Cited by 18 · see citing works
Wolf, N., van Oppen, P., Hoogendoorn, A. W., van den Heuvel, O. A., van Megen, H. J. G. M., Broekhuizen, A., Kampman, M., Cath, D. C., Schruers, K. R. J., van Es, S. M., et al. (2024) · Psychotherapy and Psychosomatics · doi:10.1159/000541508 · Cited by 15 · see citing works
Mander, J., Kröger, P., Heidenreich, T., Flückiger, C., Lutz, W., Bents, H., & Barnow, S. (2015) · BMC Psychology · doi:10.1186/s40359-015-0082-3 · Cited by 15 · see citing works
Markowitz, J. C., & Milrod, B. L. (2022) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.22com14647 · Cited by 15 · see citing works
Bricker, J. B., Sullivan, B. M., Mull, K. E., Torres, A. J., & Carpenter, K. (2022) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntac052 · Cited by 13 · see citing works
Babl, A., Holtforth, M. G., Heer, S., Lin, M., Stähli, A., Holstein, D. H., Belz, M., Egenolf, Y., Frischknecht, E., Ramseyer, F., et al. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-1136-7 · Cited by 13 · see citing works
Söderberg, A. K., Elfors, C., Larsson, M. H., Falkenström, F., & Holmqvist, R. (2013) · Psychotherapy Research · doi:10.1080/10503307.2013.826833 · Cited by 13 · see citing works
Self-citationA Meta-Analysis of Interpersonal and Psychodynamic Psychotherapies for Posttraumatic Stress Disorder
Keefe, J. R., Kimmel, D., & Weitz, E. S. (2024) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20230043 · Cited by 12 · see citing works
Luyten, T., Jacquemin, L., Looveren, N. V., Declau, F., Fransén, E., Cardon, E., Bodt, M. S. D., Topsakal, V., de Heyning, P. V., Rompaey, V. V., et al. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.02048 · Cited by 13 · see citing works
Analyzing Psychotherapy on Twitter: An 11-Year Analysis of Tweets From Major U.S. Media Outlets
Álvarez-Mon, M. Á., Fernandez-Lazaro, C. I., Ortega, M. A., Vidal, C., Molina-Ruiz, R. M., Álvarez‐Mon, M., & Martínez‐González, M. Á. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.871113 · Cited by 11 · see citing works
Rash, J. A., Kavanagh, V. A. J., & Garland, S. N. (2022) · Sleep Medicine Clinics · doi:10.1016/j.jsmc.2022.06.002 · Cited by 12 · see citing works
Lundh, L., Petersson, T., & Wolgast, M. (2016) · BMC Psychology · doi:10.1186/s40359-016-0151-2 · Cited by 10 · see citing works
Zilcha‐Mano, S., & Webb, C. A. (2023) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000868 · Cited by 9 · see citing works
Flückiger, C., Wolfer, C., Held, J., Hilpert, P., Rubel, J. A., Allemand, M., Zinbarg, R. E., & Vîslă, A. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1666-2 · Cited by 8 · see citing works
Spinelli, M. G., Endicott, J., & Goetz, R. R. (2015) · Journal of Psychiatric Practice · doi:10.1097/01.pra.0000462604.79606.4e · Cited by 6 · see citing works
Guarding the Guidelines: Objectivity in Treatment Recommendations
Markowitz, J. C., & Lowell, A. M. (2024) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20240009 · Cited by 5 · see citing works
PSYCHOLOGIES OF SMALL DIFFERENCES
Markowitz, J. C. (2016) · Depression and Anxiety · doi:10.1002/da.22488 · Cited by 5 · see citing works
Pedagogical stance in mentalization‐based treatment
Folmo, E. J., Langjord, T., Myhrvold, N. C. S., Stänicke, E., Lind, M., & Kvarstein, E. H. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23335 · Cited by 5 · see citing works
Treatment delivery of the community reinforcement approach in outpatient addiction treatment
Kraan, A. E. M., Dijkstra, B. A. G., & Markus, W. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.05.004 · Cited by 5 · see citing works
Miggiels, M. F., ten Klooster, P. M., Bremer-Hoeve, S., Dekker, J. J. M., Huibers, M. J. H., Reefhuis, E., Van, H. L., & van Dijk, M. K. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03156-8 · Cited by 5 · see citing works
“A switch of therapy may require a switch of therapist because the initial therapist does not have the necessary expertise, but possibly also because a change in clinical stance is necessary, which could be less credible if delivered by the same person [35].”
BackgroundFallstudien in der psychotherapeutischen Ausbildung
Neidhart, E., & Löffler‐Stastka, H. (2020) · Psychotherapie Forum · doi:10.1007/s00729-020-00137-2 · Cited by 4 · see citing works
Falkenström, F., & Holmqvist, R. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1998700 · Cited by 4 · see citing works
Self-citationPatterns of symptomatology over time and their relation to outcome.
Jiménez-Arista, L. E., Tracey, T. J. G., & Levy, R. (2018) · Canadian Psychology/Psychologie canadienne · doi:10.1037/cap0000160 · Cited by 3 · see citing works
Schiepek, G. K., & Wackernagel, S. (2025) · JMIR Research Protocols · doi:10.2196/78166 · Cited by 2 · see citing works
Larsson, M. H., Björkman, K., Nilsson, K., Falkenström, F., & Holmqvist, R. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22704 · Cited by 2 · see citing works
Self-citationThe Ethics of Neglecting Clinical Relationship and Alliance Building in Trauma-Focused Treatments
Gnaulati, E. (2019) · Ethical Human Psychology and Psychiatry · doi:10.1891/ehpp-d-19-00010 · Cited by 2 · see citing works
Stressrelaterad psykisk ohälsa; upplevelser och behandling inom primärvård
Arvidsdotter, T. (2014) · Gothenburg University Publications Electronic Archive (Gothenburg University) · Cited by 2 · see citing works
Ware, J. W., Venere, K., Miller, S. A., Freeman, H., Scalzitti, D. A., & Hoogeboom, T. J. (2023) · Physical Therapy · doi:10.1093/ptj/pzad023 · Cited by 1 · see citing works
Case studies in psychotherapy training using Austria as an example
Neidhart, E., & Löffler‐Stastka, H. (2020) · World Journal of Clinical Cases · doi:10.12998/wjcc.v8.i13.2787 · Cited by 1 · see citing works
Psychotherapy’s Role for Treatment-Resistant Depression: Reply to Bartova et al.
Markowitz, J. C., Wright, J. H., Peeters, F. P. M. L., Thase, M. E., Kocsis, J. H., & Sudak, D. M. (2022) · American Journal of Psychiatry · doi:10.1176/appi.ajp.20220212r · Cited by 1 · see citing works
Thérapies d'inspiration analytique chez les adolescents
Loisel, Y., Lamas, C., Jeammet, P., & Corcos, M. (2019)
Blause, S. D., Léonard, F., Willems, S. J., & Tirelli, E. (2025) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acaf048
Martínez‐García, L., Schaeuffele, C., Péris-Baquero, Ó., Torres-Alfosea, M. Á., & Osma, J. (2025) · Psychotherapy Research · doi:10.1080/10503307.2025.2541707
Markowitz, J. C., McCarthy, K. S., & Milrod, B. L. (2026) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20250053
Sunzeri, F. (2026) · Springer eBooks · doi:10.1007/978-3-032-14465-2_4
Riper, H. M., Kleiboer, A. M., Hoogendoorn, A. W., Bosmans, J. E., Schulte, M. H. J., Mathiasen, K., Hazo, J., Holtzmann, J., Chevreul, K., Ebert, D. D., et al. (2026) · Interactive Journal of Medical Research · doi:10.2196/72866
- Authors
- Falkenström, F., Markowitz, Jonker, H., Philips, B., & Holmqvist, R.
- Affiliations
- Linköping University (SE)
- New York State Psychiatric Institute (US)
- Utrecht University (NL)
- Corresponding author
- Falkenström, Fredrik
- Venue
- The Journal of Clinical Psychiatry
- Publisher
- Physicians Postgraduate Press, Inc
- Published
- 2012-10-30
- Volume
- 74
- Issue
- 05
- Pages
- 482-491
- ISSN
- 0160-6689 (ISSN-L)
- PMC
- PMC3683365
- OpenAlex
- W2055805583
- Cited by
- 78
- References
- 59
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 4 · 2023: 3 · 2022: 8 · 2021: 3 · 2020: 4 · 2019: 5 · 2018: 10 · 2017: 7 · 2016: 10 · 2015: 8 · 2014: 7 · 2013: 2
- FWCI
- 6.16
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- The Journal of Clinical Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH079078, MH-079078)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- cognitive behavioral therapy, meta-regression, randomized controlled trials
- MeSH terms
- Workforce, Humans, Psychotherapy, Research Design, Bias, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
