Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Objective: Psychotherapy research may suffer from factors such as a researcher’s own therapy allegiance. The aim of this study was to evaluate if researcher allegiance (RA) was reported in meta-analyses and randomised controlled trials (RCTs) of psychotherapeutic treatments. Design: Systematic approach using meta-analyses of different types of psychotherapies. Data sources: Medline, PsycINFO and Cochrane Database of Systematic Reviews. Methods: We evaluated meta-analyses of RCTs regarding various types of psychotherapies. Meta-analyses were eligible if they included at least one RCT with RA and they were published in journals in Medline, PsycINFO and Cochrane Database of Systematic Reviews with an impact factor larger than 5. Results: We identified 146 eligible meta-analyses that synthesised data from a total of 1198 unique RCTs. Only 25 of the meta-analyses (17.2%) reported allegiance and only 6 (4.1%) used a proper method to control its effect. Of the 1198 eligible primary RCTs, 793 (66.3%) were allegiant. Authors in 25 of these 793 RCTs (3.2%) reported their allegiance while only one study (0.2%) controlled for its effect. Conclusions: The vast majority among a group of published meta-analyses and RCTs of psychotherapeutic treatments seldom reported and evaluated the allegiance effect. The results of the present study highlight a major lack of this information in meta-analyses and their included studies, though meta-analyses perform slightly better than RCTs. Stringent guidelines should be adopted by journals in order to improve reporting and attenuate possible effects of RA in future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work
Wampold, B. E. (2015) · doi:10.4324/9780203582015 · Cited by 1,773
Comparative Studies of Psychotherapies: Is It True That "Everyone Has Won and All Must Have Prizes"?
Luborsky, L. (1975) · Archives of General Psychiatry · doi:10.1001/archpsyc.1975.01760260059004 · Cited by 1,485
Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.3.450 · Cited by 884 · cited 1×
“Statistical correction for the presence of allegiance is therefore pointless and may introduce bias.(25) This is supported by additional meta-analyses which have shown that RA did not divert the relative treatment effect, concluding that RA was not an important source of bias.”
ResultIs cognitive–behavioral therapy more effective than other therapies?
Tolin, D. F. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.05.003 · Cited by 645
The Dodo bird verdict is alive and well-mostly.
Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T. P., Berman, J. S., Levitt, J. T., Seligman, D. A., & Krause, E. D. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.1.2 · Cited by 581 · cited 1×
“11 Luborsky et al 12 13 have shown that RA accounted for two-thirds of the variance in treatment effect in favour of the preferred treatment.”
BackgroundThe researcher's own therapy allegiances: A "wild card" in comparisons of treatment efficacy.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., Halperin, G., Bishop, M., Berman, J. S., & Schweizer, E. (1999) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.6.1.95 · Cited by 572 · cited 1×
“11 Luborsky et al 12 13 have shown that RA accounted for two-thirds of the variance in treatment effect in favour of the preferred treatment.”
BackgroundWeisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.671 · Cited by 560
The efficacy of non-directive supportive therapy for adult depression: A meta-analysis
Cuijpers, P., Driessen, E., Hollon, S. D., van Oppen, P., Barth, J., & Andersson, G. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.01.003 · Cited by 369 · cited 1×
“It is also a fact that RA could affect the outcome whenever researchers select study therapists who share the RA according to the true efficacy hypothesis.(19) Similar mechanisms could occur regarding meta-analyses and studies selection.”
BackgroundResearcher allegiance in psychotherapy outcome research: An overview of reviews
Munder, T., Brütsch, O., Leonhart, R., Gerger, H., & Barth, J. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.02.002 · Cited by 260 · cited 1×
“association between allegiance and treatment effect.(23) Finally, although we found that the majority of allegiant authors of the primary studies had strong allegiance with the preferred treatment (eg, developed it or developed it and supervised/trained the therapist), we have not explored the role of funding sources…”
BackgroundResearcher allegiance and meta-analysis: The case of cognitive therapy for depression.
Gaffan, E. A., Tsaousis, I., & Kemp-Wheeler, S. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.966 · Cited by 194 · cited 2×
“34 31 35 Even in psychotherapy research where allegiance effects have been discussed and conceptualised very early, 7 11 – 13 14 – 18 24 25 28 there is a lack of sensitivity for such potential biases.”
BackgroundAllegiance in psychotherapy outcome research: Separating association from bias.
Leykin, Y., & DeRubeis, R. J. (2009) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2009.01143.x · Cited by 170 · cited 4×
“2 4 17 21 31 36 However, it is important to point out that reprint analysis as well as other such procedures including interviews with colleagues of the researchers of interest and interviews with the researchers themselves(27) or examinations of previous publications authored by the same work groups(31) are generally…”
BackgroundInfluentialPsychological treatments versus treatment as usual for obsessive compulsive disorder (OCD)
Gava, I., Barbui, C., Aguglia, E., Carlino, D., Churchill, R., De Vanna, M., & McGuire, H. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005333.pub2 · Cited by 151
Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments
Roseman, M. (2011) · JAMA · doi:10.1001/jama.2011.257 · Cited by 145 · cited 2×
“33 The selection criterion of IF has previously been applied to studies of reporting.(15) Meta-analyses of RCTs with at least one study with RA were eligible.”
BackgroundMethodWampold, B. E., Budge, S. L., Laska, K. M., Del Re, A. C., Baardseth, T. P., Flűckiger, C., Minami, T., Kivlighan, D. M., & Gunn, W. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.07.012 · Cited by 134
Direct comparisons of treatment modalities for youth disorders: a meta-analysis
Miller, S., Wampold, B., & Varhely, K. (2008) · Psychotherapy Research · doi:10.1080/10503300701472131 · Cited by 126 · cited 2×
“1 21 23 On the other hand, other”
BackgroundTesting the allegiance bias hypothesis: a meta-analysis
Munder, T., Gerger, H., Trelle, S., & Barth, J. (2011) · Psychotherapy Research · doi:10.1080/10503307.2011.602752 · Cited by 91 · cited 4×
“Even if some authors of meta-analyses are familiar with this factor and are willing to investigate RA effects, they have to rely on nonstandardised measuring methods(3) like reprint analysis (ie, analysis of the publication for the presence of attributes that may hint at allegiance) based on the limited information av…”
BackgroundInfluentialMunder, T., Flückiger, C., Gerger, H., Wampold, B. E., & Barth, J. (2012) · Journal of Counseling Psychology · doi:10.1037/a0029571 · Cited by 79
Falkenström, F., Markowitz, Jonker, H., Philips, B., & Holmqvist, R. (2013) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.12r07848 · Cited by 78 · cited 2×
“This procedure has been proposed as a criterion that gives information of moderate to strong allegiance.(31) We employed these methods because they allow measurement of all direct (eg, developed the therapy) and indirect (eg, author published supporting evidence for the treatment) levels of allegiance in meta-analyses…”
BackgroundMethodWilson, G. T., Wilfley, D. E., Agras, W. S., & Bryson, S. W. (2011) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2011.01243.x · Cited by 26
Cited by
Goodwin, G., Haddad, P., Ferrier, I., Aronson, J., Barnes, T., Cipriani, A., Coghill, D., Fazel, S., Geddes, Jr., Grunze, H., et al. (2016) · Journal of Psychopharmacology · doi:10.1177/0269881116636545 · Cited by 1,309
Steinert, C., Munder, T., Rabung, S., Hoyer, J., & Leichsenring, F. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.17010057 · Cited by 244
A systematic appraisal of allegiance effect in randomized controlled trials of psychotherapy
Dragioti, E., Dimoliatis, I., Fountoulakis, K. N., & Evangelou, E. (2015) · Annals of General Psychiatry · doi:10.1186/s12991-015-0063-1 · Cited by 91
Self-citationEffectiveness of multidisciplinary programmes for clinical pain conditions: an umbrella review.
Dragioti, E., Evangelou, E., Larsson, B., & Gerdle, B. (2018) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-2377 · Cited by 53
Self-citationIncrease value and reduce waste in research on psychological therapies
Cristea, I. A., & Naudet, F. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103479 · Cited by 22
Metadata
- Authors
- Dragioti, E., Dimoliatis, I., & Evangelou, E.
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2015-05
- Volume
- 5
- Issue
- 6
- Pages
- e007206
- ISSN
- 2044-6055 (ISSN-L)
- OpenAlex
- W2170815860
- Cited by
- 47
- References
- 19
- FWCI
- 5.68
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.