Allegiance in psychotherapy outcome research: Separating association from bias.
Abstract
Concern about the contamination of psychotherapy outcome studies by “allegiance bias”—distortion of findings because of investigators’ preferences—has led to the proposal that findings to date should not be used to make inferences about the relative efficacies of psychotherapies. It has also been proposed that results from all such studies should be adjusted to cancel the presumed distorting effects of allegiances. We argue that although much effort has been devoted towards establishing the existence of statistical associations between allegiances and outcomes, the causal implication—that investigators’ allegiances influence results—has gone virtually untested. We present a new vocabulary with the aim of sharpening the allegiance discourse, and we propose that research strategies markedly different from the ones used to date are needed to address some of the more serious limitations of allegiance bias research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The existence of the allegiance bias hypothesis, that advantageous findings result from the investigators' allegiances rather than from their inherent superiority, has yet to be proved.
“However, the existence of the allegiance bias hypothesis, that advantageous findings result from the investigators' allegiances rather than from their inherent superiority, has yet to be proved (Leykin 2009).”
From Psychological therapies for people with borderline personality disorder. (2012) · cites this paper as Leykin 2009
Researcher allegiance bias is problematic in psychotherapy research, including studies about EMDR therapy efficacy in anxiety disorders.
“Researcher allegiance bias is problematic in psychotherapy research [121, 122], including studies about EMDR therapy efficacy in anxiety disorders [38, 40].”
The study follows recommendations to reduce research allegiance bias by having a collaborative team with mixed allegiances.
“we follow recommendations to reduce research allegiance bias, such as having a collaborative team with mixed allegiances [121].”
Researcher allegiance can bias published results.
“Researcher allegiance can also bias published results.”
From Mindfulness-based Practices in Workers to Address Mental Health Conditions: A Systematic Review (2023) · cites this paper as [39]
Cites
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127 · see citing works
Smith, M. L., Glass, G. V., & Miller, T. I. (1980) · Medical Entomology and Zoology · Cited by 2,123 · see citing works
Empirically supported psychological interventions: Controversies and evidence
Chambless, D. L., & Ollendick, T. H. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.685 · Cited by 2,097 · see citing works
Comparative Studies of Psychotherapies: Is It True That "Everyone Has Won and All Must Have Prizes"?
Luborsky, L. (1975) · Archives of General Psychiatry · doi:10.1001/archpsyc.1975.01760260059004 · Cited by 1,485 · see citing works
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241 · see citing works
Foa, E. B., Liebowitz, M. R., Kozak, M. J., Davies, S., Campeas, R., Franklin, M. E., Huppert, J. D., Kjernisted, K., Rowan, V., Schmidt, A. B., et al. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.1.151 · Cited by 1,048 · see citing works
Cognitive Therapy vs Medications in the Treatment of Moderate to Severe Depression
DeRubeis, R. J., Hollon, S. D., Amsterdam, J. D., Shelton, R. C., Young, P. R., Salomon, R. M., O’Reardon, J. P., Lovett, M. L., Gladis, M. M., Brown, L. L., et al. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.4.409 · Cited by 989 · see citing works
Self-citationBarlow, D. H., Gorman, J. M., Shear, M. K., & Woods, S. W. (2000) · JAMA · doi:10.1001/jama.283.19.2529 · Cited by 953 · see citing works
Robinson, L. A., Berman, J., & Neimeyer, R. (1990) · Psychological bulletin · doi:10.1037/0033-2909.108.1.30 · Cited by 691 · see citing works
Cognitive Behavioral Group Therapy vs Phenelzine Therapy for Social Phobia: 12-Week Outcome
Heimberg, R. G., Liebowitz, M. R., Hope, D. A., Schneier, F. R., Holt, C. S., Welkowitz, L. A., Juster, H. R., Campeas, R., Bruch, M. A., Cloitre, M., et al. (1998) · Archives of General Psychiatry · doi:10.1001/archpsyc.55.12.1133 · Cited by 585 · see citing works
The Dodo bird verdict is alive and well-mostly.
Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T. P., Berman, J. S., Levitt, J. T., Seligman, D. A., & Krause, E. D. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.1.2 · Cited by 581 · see citing works
DeAngelis, C. D., Drazen, J. M., Frizelle, F., Haug, C. J., Hoey, J., Horton, R. C., Kotzin, S., Laine, C. M., Marušić, A., Overbeke, A. J. P. M., et al. (2004) · JAMA · doi:10.1001/jama.292.11.1363 · Cited by 579 · see citing works
The researcher's own therapy allegiances: A "wild card" in comparisons of treatment efficacy.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., Halperin, G., Bishop, M., Berman, J. S., & Schweizer, E. (1999) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.6.1.95 · Cited by 572 · see citing works
Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration
Mellers, B., Hertwig, R., & Kahneman, D. (2001) · Psychological Science · doi:10.1111/1467-9280.00350 · Cited by 448 · see citing works
Svartberg, M., Stiles, T. C., & Seltzer, M. H. (2004) · American Journal of Psychiatry · doi:10.1176/appi.ajp.161.5.810 · Cited by 398 · see citing works
Butler, G., Fennell, M. J. V., Robson, P., & Gelder, M. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.59.1.167 · Cited by 365 · see citing works
Butler, G., Fennell, M., Robson, P., & Gelder, M. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.167 · Cited by 365 · cited 2× · see citing works
“As for the second comparison, Butler et al. (1991) studied patients with generalized anxiety disorder, whereas the Wilson et al. (1983) study dealt with depressed patients.”
BackgroundResultTreatment of major depressive disorder in older adult outpatients with brief psychotherapies.
Gallagher-Thompson, D. E., & Thompson, L. W. (1982) · Psychotherapy · doi:10.1037/h0088461 · Cited by 243 · see citing works
Let's Face Facts: Common Factors Are More Potent Than Specific Therapy Ingredients
Messer, S. B. (2002) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/9.1.21 · Cited by 242 · see citing works
Researcher allegiance and meta-analysis: The case of cognitive therapy for depression.
Gaffan, E. A., Tsaousis, I., & Kemp-Wheeler, S. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.966 · Cited by 194 · see citing works
Siev, J., & Chambless, D. L. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.4.513 · Cited by 182 · cited 1× · see citing works
“Rather than resort to “hidden factors” to explain whatever differences might obtain between the pairs of studies, we suggest a more straightforward possibility: that different therapies are better suited to (and thus more effective with) different problems (Siev & Chambless, 2007).”
Background"Everyone Has Won and All Must Have Prizes"
Turk, D. C. (2000) · Clinical Journal of Pain · doi:10.1097/00002508-200006000-00001 · Cited by 141 · see citing works
Effectiveness of psychological and pharmacological treatments for nocturnal enuresis.
Houts, A. C., Berman, J. S., & Abramson, H. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.62.4.737 · Cited by 121 · see citing works
A conceptual and empirical review of the depressive realism hypothesis.
Dobson, K. S., & Franche, R. (1989) · Canadian Journal of Behavioural Science/Revue canadienne des sciences du comportement · doi:10.1037/h0079839 · Cited by 120 · see citing works
Effectiveness of psychological and pharmacological treatments for nocturnal enuresis.
Houts, A. C., Berman, J. S., & Abramson, H. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.62.4.737 · Cited by 120 · see citing works
Cognitive therapy versus systematic desensitization: Is one treatment superior?
Berman, J. S., Miller, R. C., & Massman, P. J. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.97.3.451 · Cited by 99 · see citing works
Comparative efficacy of behavioral and cognitive treatments of depression
Wilson, P. H., Goldin, J. C., & Charbonneau-Powis, M. (1983) · Cognitive Therapy and Research · doi:10.1007/bf01190064 · Cited by 98 · see citing works
Beutler, L. E. (2002) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.9.1.30 · Cited by 84 · see citing works
A Reevaluation of Estimates of Child Therapy Effectiveness
Shirk, S. R., & Russell, R. L. (1992) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199207000-00019 · Cited by 54 · see citing works
Lessons from psychotherapy research for psychological interventions for people with schizophrenia
Paley, G., & Shapiro, D. A. (2002) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1348/147608302169517 · Cited by 51 · see citing works
Short- and long-term follow-up of clients receiving insight-oriented therapy and behavior therapy.
Cross, D. G., Sheehan, P. W., & Khan, J. A. (1982) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.50.1.103 · Cited by 45 · see citing works
Allegiance effects in treatment research: A commentary.
Hollon, S. D. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.107 · Cited by 37 · see citing works
What is the investigator allegiance effect and what should we do about it?
Thase, M. E. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.113 · Cited by 27 · see citing works
Lambert, M. J. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.127 · Cited by 26 · see citing works
EFFECTIVENESS OF PSYCHOTHERAPY
Weiss, B. H., & Weisz, J. R. (1995) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199508000-00001 · Cited by 23 · see citing works
Luborsky, L. B. (2001) · Psychoanalytic Dialogues · doi:10.1080/10481881109348630 · Cited by 22 · see citing works
Commentary. Allegiance effects in treatment research: a commentary
Hollon, S. D. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/6.1.107 · Cited by 21 · see citing works
Commentary. What is the investigator allegiance effect and what should we do about it?
Thase, M. E. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/6.1.113 · Cited by 20 · see citing works
The role of the allegiance effect in psychotherapy research: Controlling and accounting for it.
Jacobson, N. S. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.116 · Cited by 19 · see citing works
Beyond allegiance to comparative outcome studies.
Shoham, V., & Rohrbaugh, M. J. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.120 · Cited by 12 · see citing works
Jacobson, N.-O. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/6.1.116 · Cited by 10 · see citing works
Dealing with the effects of therapy allegiances.
Klein, D. F. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.124 · Cited by 9 · see citing works
Commentary. Beyond allegiance to comparative outcome studies
Shoham, V. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/6.1.120 · Cited by 5 · see citing works
Commentary. Dealing with the effects of therapy allegiances
Klein, D. B. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/6.1.124 · Cited by 3 · see citing works
Empirically Supported Treatment
Doss, A. J., & Billy, T. M. (2008) · Encyclopedia of Special Education · doi:10.1002/9780470373699.speced0765 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Psychological therapies for people with borderline personality disorder.
Stoffers-Winterling, J. M., Völlm, B. A., Rücker, G., Timmer, A., Huband, N., & Lieb, K. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005652.pub2 · Cited by 895 · see citing works
The Role of Common Factors in Psychotherapy Outcomes
Cuijpers, P., Reijnders, M., & Huibers, M. J. H. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095424 · Cited by 748 · see citing works
Is cognitive–behavioral therapy more effective than other therapies?
Tolin, D. F. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.05.003 · Cited by 645 · see citing works
Dunning, D. L., Griffiths, K., Kuyken, W., Crane, C., Foulkes, L., Parker, J., & Dalgleish, T. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12980 · Cited by 610 · see citing works
Etzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356 · see citing works
Reid, J. E., Laws, K. R., Drummond, L., Vismara, M., Grancini, B., Mpavaenda, D., & Fineberg, N. A. (2021) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2021.152223 · Cited by 266 · see citing works
Researcher allegiance in psychotherapy outcome research: An overview of reviews
Munder, T., Brütsch, O., Leonhart, R., Gerger, H., & Barth, J. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.02.002 · Cited by 260 · see citing works
Targets and Outcomes of Psychotherapies for Mental Disorders: An Overview
Cuijpers, P. (2019) · World Psychiatry · doi:10.1002/wps.20661 · Cited by 206 · see citing works
Are the parts as good as the whole? A meta-analysis of component treatment studies.
Bell, E. C., Marcus, D. K., & Goodlad, J. K. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033004 · Cited by 192 · see citing works
Morrell, C. J., Sutcliffe, P., Booth, A., Stevens, J., Scope, A., Stevenson, M., Harvey, R., Bessey, A., Cantrell, A., Dennis, C.-L., et al. (2016) · Health Technology Assessment · doi:10.3310/hta20370 · Cited by 155 · see citing works
Efficacy of meaning-centered group psychotherapy for cancer survivors: a randomized controlled trial
van der Spek, N., Vos, J., van Uden-Kraan, C. F., Breitbart, W., Cuijpers, P., Holtmaat, K., Witte, B. I., Tollenaar, R. A. E. M., & Verdonck-de Leeuw, I. M. (2017) · Psychological Medicine · doi:10.1017/s0033291717000447 · Cited by 135 · see citing works
Andersen, P., Toner, P., Bland, M., & McMillan, D. (2016) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465816000229 · Cited by 128 · see citing works
How to prove that your therapy is effective, even when it is not: a guideline
Cuijpers, P., & Cristea, I. A. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796015000864 · Cited by 122 · see citing works
Evidence-Based Psychological Treatments: An Update and a Way Forward
Barlow, D. H., Bullis, J. R., Comer, J. S., & Ametaj, A. A. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185629 · Cited by 120 · see citing works
Outcomes associated with virtual reality in psychological interventions: where are we now?
Turner, W. A., & Casey, L. M. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.10.003 · Cited by 116 · see citing works
Coronado-Montoya, S., Levis, A. W., Kwakkenbos, L., Steele, R. J., Turner, E. H., & Thombs, B. D. (2016) · PLOS ONE · doi:10.1371/journal.pone.0153220 · Cited by 107 · see citing works
Knowles, M. M., Foden, P., El‐Deredy, W., & Wells, A. (2016) · Journal of Clinical Psychology · doi:10.1002/jclp.22312 · Cited by 94 · see citing works
Testing the allegiance bias hypothesis: a meta-analysis
Munder, T., Gerger, H., Trelle, S., & Barth, J. (2011) · Psychotherapy Research · doi:10.1080/10503307.2011.602752 · Cited by 91 · see citing works
A systematic appraisal of allegiance effect in randomized controlled trials of psychotherapy
Dragioti, E., Dimoliatis, I., Fountoulakis, K. N., & Evangelou, E. (2015) · Annals of General Psychiatry · doi:10.1186/s12991-015-0063-1 · Cited by 91 · see citing works
Munder, T., Flückiger, C., Gerger, H., Wampold, B. E., & Barth, J. (2012) · Journal of Counseling Psychology · doi:10.1037/a0029571 · Cited by 79 · see citing works
A common factors perspective on mindfulness-based interventions
Goldberg, S. B. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00090-8 · Cited by 75 · see citing works
Conflicts of interest and spin in reviews of psychological therapies: a systematic review
Lieb, K., Osten-Sacken, J. V. D., Stoffers-Winterling, J., Reiss, N., & Barth, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010606 · Cited by 69 · see citing works
Wang, L.-Q., Chien, W. T., Yip, L. K., & Karatzias, T. (2016) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s123239 · Cited by 54 · see citing works
Researcher Effects on Mortality Salience Research: A Meta-Analytic Moderator Analysis
Yen, C.-L., & Cheng, C.-P. (2013) · Death Studies · doi:10.1080/07481187.2012.682290 · Cited by 30 · see citing works
Hsu, T., Adamowicz, J. L., & Thomas, E. B. K. (2023) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2023.10.006 · Cited by 31 · see citing works
Allegiance effects in mindfulness-based interventions for psychiatric disorders: A meta-re-analysis
Goldberg, S. B., & Tucker, R. P. (2020) · Psychotherapy Research · doi:10.1080/10503307.2019.1664783 · Cited by 21 · see citing works
Yoder, W. R., Karyotaki, E., Cristea, I.-A., van Duin, D., & Cuijpers, P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024622 · Cited by 11 · see citing works
Endhoven, B., De Cort, K., Matthijssen, S. J. M. A., de Jongh, A., van Minnen, A., Duits, P., Schruers, K. R. J., van Dis, E. A. M., Krypotos, A. M., Gerritsen, L., et al. (2023) · BMC Psychiatry · doi:10.1186/s12888-022-04320-4 · Cited by 4 · see citing works
“Therefore, we follow recommendations to reduce research allegiance bias, such as having a collaborative team with mixed allegiances [121].”
BackgroundMethodMiggiels, M. F., ten Klooster, P. M., Bremer-Hoeve, S., Dekker, J. J. M., Huibers, M. J. H., Reefhuis, E., Van, H. L., & van Dijk, M. K. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03156-8 · Cited by 5 · see citing works
“The questionnaire is based on recommendations by Leykin and DeRubeis [51], which are partly based on research by Luborsky et al. [48, 94].”
BackgroundMethodInfluentialMindfulness-based Practices in Workers to Address Mental Health Conditions: A Systematic Review
Durand-Moreau, Q., Jackson, T., Deibert, D., Els, C., Kung, J. Y., & Straube, S. (2023) · Safety and Health at Work · doi:10.1016/j.shaw.2023.07.006
- Authors
- Leykin, Y., & DeRubeis, R. J.
- Affiliations
- University of California, San Francisco (US)
- University of Pennsylvania (US)
- Venue
- Clinical Psychology: Science and Practice
- Published
- 2009-03
- Volume
- 16
- Issue
- 1
- Pages
- 54-65
- ISSN
- 0969-5893 (ISSN-L)
- 1468-2850
- OpenAlex
- W2131433388
- Cited by
- 170
- References
- 45
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 6 · 2023: 7 · 2022: 11 · 2021: 11 · 2020: 7 · 2019: 8 · 2018: 42 · 2017: 5 · 2016: 15 · 2015: 12 · 2014: 7 · 2013: 13 · 2012: 7
- FWCI
- 7.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Psychology Science and Practice (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- psychotherapy outcome research, investigator bias, causal inference, bias correction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
