Working Alliance in Blended Versus Face-to-Face Cognitive Behavioral Treatment for Patients with Depression in Specialized Mental Health Care
Open accessAbstract
This study investigates working alliance in blended cognitive behavioral therapy (bCBT) for depressed adults in specialized mental health care. Patients were randomly allocated to bCBT (n = 47) or face-to-face CBT (n = 45). After 10 weeks of treatment, both patients and therapists in the two groups rated the therapeutic alliance on the Working Alliance Inventory Short-Form Revised (WAI-SR; Task, Bond, Goal, and composite scores). No between-group differences were found in relation to either patient or therapist alliance ratings, which were high in both groups. In the full sample, a moderate positive association was found between patient and therapist ratings on Task (ρ = 0.41, 95% CI 0.20; 0.59), but no significant associations emerged on other components or composite scores. At 30 weeks, within-and between-group associations between alliance and changes in depression severity (QIDS, Quick Inventory of Depressive Symptomatology) were analyzed with linear mixed models. The analyses revealed an association between depression over time, patient-rated alliance, and group (p < 0.001). In face-to-face CBT, but not in bCBT, lower depression scores were associated with higher alliance ratings. The online component in bCBT may have led patients to evaluate the working alliance differently from patients receiving face-to-face CBT only.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some studies have not reported meaningful correlations between ratings and clinical outcomes in the digital context.
“However, some studies have not reported meaningful correlations between ratings and clinical outcomes in the digital context (32, 36, 41–43).”
From Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review (2022) · cites this paper as [42]
Similar TA scores have been found for face-to-face therapy and Internet-based interventions.
“similar TA scores have been found for face-to-face therapy and IBIs [3,14,18,19,20]”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480 · see citing works
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · see citing works
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139 · see citing works
Kessler, R. C., Chiu, W. T., Demler, O., & Walters, E. E. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.6.617 · Cited by 12,052 · see citing works
(1998) · PubMed · Cited by 10,216 · see citing works
Cognitive Therapy for Depression
(2012) · PsycEXTRA Dataset · doi:10.1037/e675542012-001 · Cited by 9,678 · see citing works
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238 · see citing works
Rush, A. J., Trivedi, M. H., Ibrahim, H. M., Carmody, T. J., Arnow, B., Klein, D. N., Markowitz, J. C., Ninan, P. T., Kornstein, S., Manber, R., et al. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01866-8 · Cited by 3,829 · see citing works
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520 · see citing works
Alliance in individual psychotherapy
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011) · Psychotherapy · doi:10.1037/a0022186 · Cited by 2,072 · see citing works
The alliance in adult psychotherapy: A meta-analytic synthesis
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018) · Psychotherapy · doi:10.1037/pst0000172 · Cited by 1,763 · cited 4× · see citing works
“30) in psychotherapy for mental disorders [28] and r = 0.”
BackgroundMethodResultInfluentialHow important are the common factors in psychotherapy? An update
Wampold, B. E. (2015) · World Psychiatry · doi:10.1002/wps.20238 · Cited by 1,728 · cited 1× · see citing works
“This is relevant, as therapeutic alliance is considered an important common factor in psychotherapy [25].”
BackgroundTreatment gap in mental health care.
Abramowitz, M. Z., Greenberg, D., & Levav, I. (2008) · PubMed · Cited by 1,482 · see citing works
Computer-based psychological treatments for depression: A systematic review and meta-analysis
Richards, D., & Richardson, T. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.02.004 · Cited by 1,442 · see citing works
The treatment gap in mental health care.
(2004) · Europe PMC (PubMed Central) · Cited by 1,419 · see citing works
Development and validation of a revised short version of the working alliance inventory
Hatcher, R. L., & Gillaspy, J. A. (2006) · Psychotherapy Research · doi:10.1080/10503300500352500 · Cited by 1,355 · see citing works
Carlbring, P., Andersson, G., Cuijpers, P., Riper, H., & Hedman-Lagerlöf, E. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1401115 · Cited by 1,232 · see citing works
Self-citationAndersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014) · World Psychiatry · doi:10.1002/wps.20151 · Cited by 1,077 · see citing works
Self-citationAndrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922 · see citing works
Cuijpers, P., Donker, T., van Straten, A., Li, J., & Andersson, G. (2010) · Psychological Medicine · doi:10.1017/s0033291710000772 · Cited by 832 · see citing works
Munder, T., Wilmers, F., Leonhart, R., Linster, H. W., & Barth, J. (2010) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.658 · Cited by 538 · see citing works
Erbe, D., Eichert, H.-C., Riper, H., & Ebert, D. D. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6588 · Cited by 532 · see citing works
Self-citationMohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432 · see citing works
Berger, T. (2017) · Psychotherapy Research · doi:10.1080/10503307.2015.1119908 · Cited by 364 · cited 2× · see citing works
“A recent review has suggested that the association between alliance and outcome in online interventions may be stronger in terms of agreement on goals and tasks than in terms of the affective bond between patients and their therapists, partly because patients could have lower or different expectations of the bond that…”
Background(2007) · PubMed · Cited by 324 · see citing works
Zilcha-Mano, S. (2017) · American Psychologist · doi:10.1037/a0040435 · Cited by 303 · see citing works
Including all individuals is not enough: lessons for intention-to-treat analysis
White, I., Carpenter, J., & Horton, N. (2012) · Clinical Trials · doi:10.1177/1740774512450098 · Cited by 299 · see citing works
van der Vaart, R., Witting, M., Riper, H., Kooistra, L., Bohlmeijer, E. T., & van Gemert-Pijnen, L. J. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0355-z · Cited by 273 · cited 2× · see citing works
“The consequent modifications in the amounts and forms of contact between therapists and patients in online and blended interventions, as compared with face-to-face treatments, have led some therapists to worry that therapeutic alliance might be negatively impacted [18] and that this could weaken treatment outcomes.”
BackgroundResultSelf-citationHow much psychotherapy is needed to treat depression? A metaregression analysis
Cuijpers, P., Huibers, M., Daniel Ebert, D., Koole, S. L., & Andersson, G. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.02.030 · Cited by 258 · see citing works
Self-guided psychological treatment for depressive symptoms: a meta-analysis
Cuijpers, P., Donker, T., Johansson, R., Mohr, D. C., van Straten, A., & Andersson, G. (2011) · PLoS ONE · doi:10.1371/journal.pone.0021274 · Cited by 250 · see citing works
Hesser, H. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.02.003 · Cited by 192 · see citing works
Preschl, B., Maercker, A., & Wagner, B. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-189 · Cited by 158 · cited 2× · see citing works
“Initial evaluations showed patient-rated alliance in online interventions to be high and comparable to ratings in face-to-face treatment [32,33].”
BackgroundResultHof, E. V., Cuijpers, P., & Stein, D. J. (2009) · CNS Spectrums · doi:10.1017/s1092852900027279 · Cited by 156 · see citing works
Høifødt, R. S., Lillevoll, K. R., Griffiths, K. M., Wilsgaard, T., Eisemann, M., Waterloo, K., & Kolstrup, N. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2714 · Cited by 154 · see citing works
Berger, T., Krieger, T., Sude, K., Meyer, B., & Maercker, A. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.11.021 · Cited by 153 · see citing works
Kleiboer, A., Smit, J., Bosmans, J., Ruwaard, J., Andersson, G., Topooco, N., Berger, T., Krieger, T., Botella, C., Baños, R., et al. (2016) · Trials · doi:10.1186/s13063-016-1511-1 · Cited by 133 · see citing works
Self-citationImproving the Efficiency of Psychotherapy for Depression: Computer-Assisted Versus Standard CBT
Thase, M. E., Wright, J. H., Eells, T. D., Barrett, M. S., Wisniewski, S. R., Balasubramani, G. K., McCrone, P., & Brown, G. K. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.17010089 · Cited by 131 · see citing works
Cameron, S. K., Rodgers, J., & Dagnan, D. (2018) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2180 · Cited by 128 · see citing works
Long-term effects of internet-supported cognitive behaviour therapy
Andersson, G., Rozental, A., Shafran, R., & Carlbring, P. (2018) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2018.1400381 · Cited by 117 · see citing works
Kenter, R. M. F., van de Ven, P. M., Cuijpers, P., Koole, G., Niamat, S., Gerrits, R. S., Willems, M., & van Straten, A. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.01.001 · Cited by 105 · see citing works
Internet interventions for depression: new developments
Schröder, J., Berger, T., Westermann, S., Klein, J. P., & Moritz, S. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.2/jschroeder · Cited by 105 · see citing works
Kooistra, L. C., Ruwaard, J., Wiersma, J. E., van Oppen, P., van der Vaart, R., van Gemert-Pijnen, J. E. W. C., & Riper, H. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.01.003 · Cited by 92 · see citing works
Self-citationKooistra, L. C., Wiersma, J. E., Ruwaard, J., Neijenhuijs, K., Lokkerbol, J., van Oppen, P., Smit, F., & Riper, H. (2019) · Journal of Medical Internet Research · doi:10.2196/14261 · Cited by 84 · see citing works
Self-citationKooistra, L. C., Wiersma, J. E., Ruwaard, J., van Oppen, P., Smit, F., Lokkerbol, J., Cuijpers, P., & Riper, H. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0290-z · Cited by 82 · see citing works
Self-citationLaws, H. B., Constantino, M. J., Sayer, A. G., Klein, D. N., Kocsis, J. H., Manber, R., Markowitz, J. C., ROTHBAUM, B. O., Steidtmann, D. K., Thase, M. E., et al. (2016) · Psychotherapy Research · doi:10.1080/10503307.2015.1114687 · Cited by 77 · see citing works
Working alliance as a predictor of change in depression during blended cognitive behaviour therapy
Vernmark, K., Hesser, H., Topooco, N., Berger, T., Riper, H., Luuk, L., Backlund, L., Carlbring, P., & Andersson, G. (2019) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1533577 · Cited by 59 · see citing works
Self-citationInternet interventions for depressive disorders: an overview
Cuijpers, P., & Riper, H. (2015) · Revista de Psicopatología y Psicología Clínica · doi:10.5944/rppc.vol.19.num.3.2014.13902 · Cited by 32 · see citing works
Self-citationLorenzo‐Luaces, L., Driessen, E., DeRubeis, R. J., Van, H. L., Keefe, J. R., Hendriksen, M., & Dekker, J. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.11.011 · Cited by 29 · see citing works
Protocollaire behandelingen in de ambulante GGZ
Keijsers, G. P. J., van Minnen, A., & Hoogduin, C. A. L. (2004) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-313-7117-4_1 · Cited by 17 · see citing works
Protocollaire behandeling van patiënten met een depressieve stoornis
Bockting, C. L. H., & Huibers, M. J. H. (2011) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 5 · see citing works
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [37]Van Vliet I.M. De Beurs E. Het Mini Internationaal Neuropsychiatrisch Interview (MINI). Een kort gestructureerd diagnostisch psychiatrisch interview voor DSM-IV en ICD-10-stoornissen Tijdschr. Psychiatr. 2007 49 393 397 17614093
Cited by
Sorted by citations
Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review
Tong, F., Lederman, R., D'Alfonso, S., Berry, K., & Bucci, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.819623 · Cited by 106 · see citing works
Herrero, R., Vara, M. D., Miragall, M., Botella, C., García-Palacios, A., Riper, H., Kleiboer, A., & Baños, R. M. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17176169 · Cited by 81 · see citing works
Self-citationAlavi, N., Moghimi, E., Stephenson, C., Gutierrez, G., Jagayat, J., Kumar, A., Shao, Y., Miller, S., Yee, C. S., Stefatos, A., et al. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1113956 · Cited by 36 · see citing works
(2024) · doi:10.2196/47515 · Cited by 24 · see citing works
Self-citationAskjer, S., & Mathiasen, K. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100404 · Cited by 18 · see citing works
Doukani, A., Free, C., Michelson, D., Araya, R., Montero‐Marín, J., Smith, S. C., Pashoja, A. C., & Kakuma, R. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036299 · Cited by 17 · see citing works
Schuster, R., Fischer, E., Jansen, C., Napravnik, N., Rockinger, S., Steger, N., & Laireiter, A.-R. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100551 · Cited by 15 · see citing works
Doukani, A., Free, C., Araya, R., Michelson, D., Cerga-Pashoja, A., & Kakuma, R. (2022) · BJPsych Open · doi:10.1192/bjo.2022.546 · Cited by 15 · see citing works
Innovative Technology Based Interventions for Psychological Treatment of Common Mental Disorders
Donker, T., & Kleiboer, A. M. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9103075 · Cited by 15 · see citing works
Wu, M. S., Wickham, R. E., Chen, S.-Y., Varra, A. A., Chen, C., & Lungu, A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0313112 · Cited by 12 · see citing works
Adherence to online psychotherapy during the COVID-19: a scoping review
Cardoso, N. D. O., Tagliapietra, K. D. R., Salvador, E. Z., & Machado, W. D. L. (2023) · Psico-USF · doi:10.1590/1413-82712023280110 · Cited by 4 · see citing works
Current trends in e-mental health interventions: Health care providers' attitudes and usage patterns
Weitzel, E. C., Schladitz, K., Schwenke, M., Welzel, F. D., Schomerus, G., Schönknecht, P., Bleckwenn, M., Mehnert, A., Riedel-Heller, S. G., & Löbner, M. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100808 · Cited by 3 · see citing works
Amarti, K., Schulte, M. H. J., Kleiboer, A. M., van Genugten, C., Oudega, M. L., Sonnenberg, C., Gonçalves, G., Rocha, A., & Riper, H. M. (2022) · JMIR Research Protocols · doi:10.2196/41445 · Cited by 3 · see citing works
Self-citationvan Seters, M. L. F. M., Rasing, S. P. A., Huvenaars, M. J., Vermulst, A. A., Bodden, D. H. M., & Stikkelbroek, Y. A. J. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100776 · Cited by 3 · see citing works
The present and future of blended care: current research and introduction to the B-FIT framework
Bielinski, L. L., Büscher, R., Hennemann, S., Henning, C., Kohl, S. H., Meyer, C., Reinhold, A. S., Steubl, L., Titzler, I., Vogel, L., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02526-4 · Cited by 2 · see citing works
Čihařová, M., Cuijpers, P. J., Harrer, M., Hirshler, Y., Panagiotopoulou, O.-M., Tong, L., Miguel, C., Frankova, I., Gemmill, A. W., Buntrock, C., et al. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100993 · Cited by 1 · see citing works
Innovative Technology Based Interventions for Psychological Treatment of Common Mental Disorders
Donker, T., & Kleiboer, A. M. (2020) · MDPI eBooks · doi:10.3390/books978-3-03943-736-8 · Cited by 1 · see citing works
Telemedizin und digitale Hilfen in der psychiatrischen Praxis
Krenn, M. (2022) · psychopraxis neuropraxis · doi:10.1007/s00739-022-00791-y
Internet psychoeducation and psychotherapy for obsessive-compulsive disorder - review
Pasztor, J., Praško, J., Vaněk, J., Višňovský, J., Zaťková, M., Šlepecký, M., Sollár, T., & Nesnídal, V. (2025) · Psychiatrie pro praxi · doi:10.36290/psy.2025.018
West, J. C., Vance, M. C., & Ursano, R. J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-824067-0.00040-2
Velthuizen, S. L. M., Lutz, M. C., van den Bos, E., Miers, A. C., & Westenberg, P. M. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2583114
Gallegos-Carrillo, K., Ramírez‐Palacios, P., Chávez‐Iñiguez, A., Cupertino, A. P., Cartujano‐Barrera, F., & Rodríguez-Bolaños, R. (2026) · Tobacco Induced Diseases · doi:10.18332/tid/213755
Behr, S., Bohn, J., Fenski, F., Hammelrath, L., Bellu, L., Boettcher, J., Knaevelsrud, C., & Schaeuffele, C. (2026) · Psychotherapy Research · doi:10.1080/10503307.2026.2638268
Amarti, K., Schulte, M. H. J., Kleiboer, A., van Genugten, C. V., Oudega, M., Rocha, A., & Riper, H. (2025) · JMIR Formative Research · doi:10.2196/83316
Self-citationJanus, I., & Piepiora, P. A. (2026) · Advances in Environmental and Engineering Research · doi:10.21926/aeer.2602015
Orobosheri, C. (2026) · Postgraduate Quarterly · doi:10.53841/bpspgq.2026.1.1.41
Internet CBT for panic disorder - a review
Praško, J., Vaněk, J., Bělohradová, K., Sollár, T., Šlepecký, M., Mikula, M., & Ocisková, M. (2026) · Psychiatrie pro praxi · doi:10.36290/psy.2026.021
Zhou, X., Huang, H., Bi, R., Verdeli, L., & Lyu, H. (2026) · Journal of Zhejiang University SCIENCE B · doi:10.1631/jzus.b2500701
Riper, H. M., Kleiboer, A. M., Hoogendoorn, A. W., Bosmans, J. E., Schulte, M. H. J., Mathiasen, K., Hazo, J., Holtzmann, J., Chevreul, K., Ebert, D. D., et al. (2026) · Interactive Journal of Medical Research · doi:10.2196/72866
Self-citation
- Authors
- Kooistra, L., Ruwaard, J., Wiersma, J., van Oppen, P., & Riper, H.
- Affiliations
- Amsterdam Public Health (NL)
- GGZ inGeest (NL)
- Department of Research and Innovation, GGZ inGeest Specialized Mental Health Care, Oldenaller 1, 1081 HJ Amsterdam, The Netherlands
- Corresponding author
- Kooistra, Lisa
- Venue
- Journal of Clinical Medicine
- Publisher
- MDPI AG
- Published
- 2020-01-27
- Volume
- 9
- Issue
- 2
- Pages
- 347
- ISSN
- 2077-0383 (ISSN-L)
- PMC
- PMC7073833
- OpenAlex
- W3002605402
- Cited by
- 30
- References
- 52
- Citations by year
- 2026: 10 · 2025: 3 · 2024: 4 · 2023: 2 · 2022: 6 · 2021: 1 · 2020: 4
- FWCI
- 1.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Clinical Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Pure Amsterdam UMC (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Netherlands Organisation for Health Research and Development (ZonMw) (837001007)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- therapeutic relationship, depression, specialized mental health care, online therapy, depression severity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
