The Association of Therapeutic Alliance With Long-Term Outcome in a Guided Internet Intervention for Depression: Secondary Analysis From a Randomized Control Trial
Open accessPublished version of a preprint
The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index; 2 of the 26 sources its claims cite were checked for support.
Abstract
BACKGROUND: Therapeutic alliance has been well established as a robust predictor of face-to-face psychotherapy outcomes. Although initial evidence positioned alliance as a relevant predictor of internet intervention success, some conceptual and methodological concerns were raised regarding the methods and instruments used to measure the alliance in internet interventions and its association with outcomes. OBJECTIVE: The aim of this study was to explore the alliance-outcome association in a guided internet intervention using a measure of alliance especially developed for and adapted to guided internet interventions, showing evidence of good psychometric properties. METHODS: A sample of 223 adult participants with moderate depression received an internet intervention (ie, Deprexis) and email support. They completed the Working Alliance Inventory for Guided Internet Intervention (WAI-I) and a measure of treatment satisfaction at treatment termination and measures of depression severity and well-being at termination and 3- and 9-month follow-ups. For data analysis, we used two-level hierarchical linear modeling that included two subscales of the WAI-I (ie, tasks and goals agreement with the program and bond with the supporting therapist) as predictors of the estimated values of the outcome variables at the end of follow-up and their rate of change during the follow-up period. The same models were also used controlling for the effect of patient satisfaction with treatment. RESULTS: =2.72, P=.007) at the end of follow-up. A greater score in this subscale was related to lower levels of residual depressive symptoms and a higher level of well-being. However, there were no significant effects of the tasks and goals subscale on the rate of change in these variables during follow-up (depressive symptoms, P=.48; patient well-being, P=.26). The effects of the bond subscale were also nonsignificant when predicting the estimated values of depressive symptoms and well-being at the end of follow-up and the rate of change during that period (depressive symptoms, P=.08; patient well-being, P=.68). CONCLUSIONS: The results of this study point out the importance of attuning internet interventions to patients' expectations and preferences in order to enhance their agreement with the tasks and goals of the treatment. Thus, the results support the notion that responsiveness to a patient's individual needs is crucial also in internet interventions. Nevertheless, these findings need to be replicated to establish if they can be generalized to different diagnostic groups, internet interventions, and supporting formats.
Overview
CoChat Summary
TL;DR
In a secondary analysis of 223 adults with moderate depression receiving Deprexis plus email support, agreement on the intervention’s tasks and goals at posttreatment predicted better 9-month outcomes. The bond with the supporting therapist was not significantly associated with long-term depressive symptoms or well-being.
Key findings
- The sample included 223 German-speaking adults aged 18 to 65 years (mean 44.48, SD 10.68); 157/223 (70.4%) were female.
- Higher WAI-I tasks and goals scores at posttreatment were associated with lower PHQ-9 residual depressive symptoms at 9-month follow-up (γ02 = −1.74, SE 0.40, 95% CI −2.52 to −0.96, P < .001) and higher SF-P well-being (γ02 = 3.10, SE 1.14, 95% CI 0.87-5.33, P = .007).
- Tasks-and-goals scores did not predict change during follow-up for either outcome (PHQ-9 P = .48; SF-P P = .26).
- The WAI-I bond subscale was not significant for PHQ-9 or SF-P at follow-up or over time (PHQ-9 P = .07 and .08; SF-P P = .41 and .68).
Methods
- Secondary analysis of the EVIDENT randomized controlled trial; subsample of participants who received Deprexis plus weekly email support and completed the WAI-I at treatment termination.
- Outcomes were PHQ-9 depression severity and SF-12 mental well-being (SF-P) at posttreatment and 3- and 9-month follow-ups; treatment satisfaction was measured with ZUF-8.
- Main analysis used two-level hierarchical linear models to estimate outcome levels at 9-month follow-up and change during follow-up, with WAI-I subscales entered as predictors; additional models controlled for patient satisfaction and participant-supporter interaction indicators.
Limitations
- The paper states that alliance was assessed only once at posttreatment, so it could not separate between-patient from within-patient alliance effects, and a single measure may be less reliable than repeated assessments.
- The authors note that the limited contact with the supporter may have made it hard to fully capture the bond component of alliance, which may be less relevant here than tasks/goals.
- Evident from the text: the analysis was restricted to a subsample of 223 patients with moderate depression who received Deprexis with email support, so generalizability to other diagnoses, severity levels, interventions, or support formats is uncertain.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 16 Not checked · 29 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“They lack an exploration of the psychometric properties (ie, validity and reliability) of the measuring instruments used in the specific context of guided online interventions.”
§ The Association of Therapeutic Alliance With Long-Term Outcome in a Guided Internet Intervention for Depression: Secondary Analysis From a Randomized Control Trial > Introduction Several meta-analyses positioned therapeutic alliance as a robust predictor of outcomes in face-to-face psychotherapy [1-3]. However, alliance effects do not seem to be limited to the field of traditional psychotherapy. Alliance also predicted outcomes in other health-related interventions, such as pharmacotherapy [4,5]. The increasing development of internet interventions and the evidence for their efficacy and effectiveness in treating diverse mental disorders [6-8] raised the question of what role therapeutic alliance might play in such treatments, especially in those providing guidance from trained supporters (guided internet interventions) [9]. In the last years, several studies addressed this question scientifically. Although some authors found that the alliance could be less important in internet interventions than in traditional face-to-face therapies [10], a recent meta-analysis reported similar effect sizes (r=0.275) for the alliance-outcome relationship in online interventions as in traditional face-to-face therapies [1]. Among the 18 studies included in that meta-analysis [1], 15 analyzed the alliance specifically in guided internet interventions. To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient and the supporting therapist [10,12-21]. This approach of measuring alliance follows its classical conceptualization in psychotherapy research as tripartite, consisting of (1) the patient-therapist emotional bond, (2) patient agreement with the tasks of therapy, and (3) patient agreement with the therapeutic goals that the patient and therapist seek in treatment [22,23]. However, it was not considered that in these trials, treatment tasks and goals were not set in collaboration with the supporting therapist but were proposed by the online program, which might be a limiting factor. Some studies did this the other way around; they used adapted instruments to measure the alliance between the patient and the online intervention only [18,24-26]. Regardless of whether the measuring instruments focused on the supporting therapist or the program, all the abovementioned studies have one thing in common. They lack an exploration of the psychometric properties (ie, validity and reliability) of the measuring instruments used in the specific context of guided online interventions. Only Kiluk et al [24] presented an exploratory analysis of the adapted version of the WAI (ie, WAI-Tech) with some evidence of internal consistency (Cronbach alpha) and external validity (ie, significant correlations with the original WAI and with patient treatment satisfaction). However, the small sample size of that study (n=34) limited the interpretability of the findings and prevented the provision of further evidence for the psychometric properties of the scale (eg, construct validity). Thus, beside conceptual issues identified in most previous alliance-outcome studies of guided internet interventions, concerns might be raised regarding the validity and reliability of the instruments used to measure the alliance construct. In this context, Berger et al [27], as well as Scherer et al [28], presented a compromise between the two previous approaches of analyzing alliance-outcome associations in guided internet interventions (focus on the supporting therapist or the online program). The authors took the original version of the WAI and adapted it to guided internet interventions, exploring the bond with the supporting therapist but the tasks and goals with the online program. With this version of the instrument, they captured the most relevant aspects of the alliance, considering both the importance of the relationship with the supporting therapist and patient attunement with the online program. Recently, Gómez Penedo et al [29] systematized the efforts by Berger et al [27] and Scherer et al [28], presenting the Working Alliance Inventory for Guided Internet Interventions (WAI-I) and exploring the psychometric properties of the scale. The findings provide evidence for adequate internal consistency, external validity, and construct validity (based on a confirmatory factor analysis) of the WAI-I. Given the cumulative evidence showing that residual depressive symptoms are some of the main predictors of relapse [30-32], in this study, we will analyze how the alliance during treatment is associated with long-term outcomes (ie, 9-month follow-up) after a guided internet intervention for patients with moderate depressive symptoms. We will focus on both analyzing the effects of the alliance on the changes produced during the follow-up period (ie, deterioration or further improvement) and evaluating the residual depressive symptoms at the 9-month follow-up. Furthermore, we will analyze the same effects on the well-being of patients. Beside responding to a general call for further studies clarifying the role of alliance in online interventions [1,9], especially with adapted and psychometrically sound instruments [29], the aim of this study was to explore the alliance-outcome association using the WAI-I, a measure of alliance especially developed for and adapted to guided internet interventions.
What the sources say
- Partly supportedcites [24] sourceExperimental — accuracy not yet measured
The DISCUSSION of Kiluk et al. (2014) states that prior work measuring a working alliance with a specific technology-based intervention 'did not include an evaluation of the psychometric characteristics of the adapted measure' (citing Bickmore et al., 2005; Ormrod et al., 2010), and the INTRODUCTION frames the study as one of the first psychometric evaluations (reliability and construct validity) of an alliance measure adapted to a technology-based program — this supports the general gap the citing sentence describes. However, the source restricts the gap to alliance with the technology-based intervention itself measured with an adapted instrument; the studies in guided/counselor-supported online interventions (Andersson et al., 2012; Holmes & Foster, 2012; Sucala et al., 2012; Kay-Lambkin et al., 2011) are described only as having measured alliance with the clinician, without any statement that they lacked psychometric exploration of their measures. The citing sentence's qualifier 'in the specific context of guided online interventions' and its reference to 'the measuring instruments used' generally is therefore not what this source says. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found significant effects of the tasks and goals subscale of the WAI-I on the estimated values of residual depressive symptoms (γ02=−1.74, standard error [SE]=0.40, 95% CI −2.52 to −0.96, t206=−4.37, P<.001) and patient well-being (γ02=3.10, SE=1.14, 95% CI 0.87-5.33, t198=2.72, P=.007) at the end of follow-up.”
- Also stated in: § Conditional Models: Alliance Main Effects · plain
“A 1-unit greater tasks and goals score at posttreatment was associated with a 3.10-unit higher score in the SF-P at the end of the 9-month follow-up.”
§ The Association of Therapeutic Alliance With Long-Term Outcome in a Guided Internet Intervention for Depression: Secondary Analysis From a Randomized Control Trial > Abstract > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Conditional Models: Alliance Main Effects · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“or the evolution of the SF-P during follow-up (γ11=−0.36, SE=0.89, 95% CI −2.10 to 1.38, t191=0.41, P=.68).”
§ The Association of Therapeutic Alliance With Long-Term Outcome in a Guided Internet Intervention for Depression: Secondary Analysis From a Randomized Control Trial > Results > Conditional Models: Alliance Main Effects
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The conditional model with the PHQ-9 as an outcome variable and the alliance subscales as level 2 predictors significantly improved the fit of the time-as-only predictor model (χ24=26.71, P<.001).”
§ The Association of Therapeutic Alliance With Long-Term Outcome in a Guided Internet Intervention for Depression: Secondary Analysis From a Randomized Control Trial > Results > Conditional Models: Alliance Main Effects
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, the models controlling for participant-supporter interaction did not improve the conditional model fit when predicting the SF-P (χ26=6.43, P=.38).”
§ The Association of Therapeutic Alliance With Long-Term Outcome in a Guided Internet Intervention for Depression: Secondary Analysis From a Randomized Control Trial > Results > Conditional Models: Alliance Main Effects Controlling for Patient Satisfaction and Patient-Supporter Interaction The results of the conditional models estimating the alliance effects controlling for either patient satisfaction or patient-supporter interaction are presented in Multimedia Appendix 2. When running the same conditional models to predict PHQ-9 scores presented above on controlling for patient satisfaction with treatment, there was a significant improvement in the conditional model fit (χ22=8.61, P=.01). This model comparison test suggested the importance of controlling for patient satisfaction when estimating the alliance effects on the PHQ-9. The results of this model showed that there was still a significant effect of the tasks and goals subscale on the estimated PHQ-9 value at the end of follow-up (γ02=−1.78, SE=0.58, 95% CI −2.92 to −0.64, t203=−3.09, P=.002). The other effects of the alliance were nonsignificant as in the previous models. Furthermore, the effect of patient satisfaction was not significant when predicting the estimated PHQ-9 scores at the 9-month follow-up (γ03=0.09, SE=0.85, 95% CI −1.58 to 1.76, t200=0.11, P=.91) but was significant when predicting the change produced in the PHQ-9 during follow-up (γ13=2.00, SE=0.81, 95% CI 0.41-3.59, t193=2.48, P=.58). When controlling for the alliance subscale effects, a 1-unit greater score in the ZUF-8 at posttreatment (ie, patient satisfaction with treatment) was associated with a 2-unit increase in the PHQ-9 score during the follow-up period. However, when running a conditional model exploring alliance effects on the PHQ controlling for patient-supporter interaction indicators (ie, number of messages sent by the participant, number of messages sent by the supporter, and number of messages read by the participant), there was no significant improvement in the model fit (χ26=0.56, P=.99). This test suggested not to include participant-supporter interactions as covariates in conditional alliance models. Nevertheless, it is worth highlighting that the model controlling for participant-supporter interaction indicators still showed significant effects of the tasks and goals subscale on the estimated PHQ-9 value at the end of follow-up (γ02=−1.97, SE=0.44, 95% CI −2.82 to −1.12, t141=−4.49, P<.001).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
20 more claims
Create a free account to see all 47 claims and to verify the 24 references still open.
- Probably not a substantial claim. 20
- It may state two separate claims as one. 9
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“since some studies found that ratings of task and goal link with clinical outcomes in the context of fully automated mental health apps (80, 81).”
From Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review (2022) · cites this paper as [80]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Moreover, patients with higher scores on TA were more satisfied with the intervention after the treatment.”
From Working Alliance Inventory for Online Interventions-Short Form (WAI-TECH-SF): The Role of the Therapeutic Alliance between Patient and Online Program in Therapeutic Outcomes (2020) · cites this paper as [24]
Experimental — accuracy not yet measuredIn the citing paper
“Results of the WAI for guided Internet interventions (i.e., WAI-I) showed a two-factor solution (“tasks-goals” and “bonds”) with adequate internal consistency and external validity in a sample of patients with mild to moderate depression.”
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · cited 2×
“Previous studies showed that it is a reliable and valid instrument to measure depression severity [41].”
BackgroundMethod(2014) · Climate Change 2013 – The Physical Science Basis · doi:10.1017/cbo9781107415324.004 · Cited by 23,880
WARE, J. E., KOSINSKI, M., & KELLER, S. D. (1996) · Medical Care · doi:10.1097/00005650-199603000-00003 · Cited by 17,392 · cited 2×
“The Short-Form Health Survey-12 (SF-12) is an instrument for measuring health-related quality of life [42].”
BackgroundMethodThe generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238 · cited 1×
“This approach of measuring alliance follows its classical conceptualization in psychotherapy research as tripartite, consisting of (1) the patient-therapist emotional bond, (2) patient agreement with the tasks of therapy, and (3) patient agreement with the therapeutic goals that the patient and therapist seek in treat…”
MethodKroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2010.03.006 · Cited by 4,057 · cited 1×
“Of these 509 participants, 317 presented with moderate depressive symptoms (ie, score between 10 and 14 in the Patient Health Questionnaire [PHQ]) [36] and therefore additionally received weekly email support from trained clinicians.”
BackgroundDevelopment and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodRelation of the therapeutic alliance with outcome and other variables: a meta-analytic review
Martin, D. J., Garske, J. P., & Davis, M. K. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.3.438 · Cited by 3,355 · cited 1×
“Several meta-analyses positioned therapeutic alliance as a robust predictor of outcomes in face-to-face psychotherapy [1-3].”
BackgroundAlliance in individual psychotherapy
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011) · Psychotherapy · doi:10.1037/a0022186 · Cited by 2,072
The alliance in adult psychotherapy: A meta-analytic synthesis
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018) · Psychotherapy · doi:10.1037/pst0000172 · Cited by 1,763 · cited 6×
“275) for the alliance-outcome relationship in online interventions as in traditional face-to-face therapies [1].”
BackgroundMethodResultInfluentialCarlbring, P., Andersson, G., Cuijpers, P., Riper, H., & Hedman-Lagerlöf, E. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1401115 · Cited by 1,232 · cited 1×
“The increasing development of internet interventions and the evidence for their efficacy and effectiveness in treating diverse mental disorders [6-8] raised the question of what role therapeutic alliance might play in such treatments, especially in those providing guidance from trained supporters (guided internet inte…”
BackgroundPsychotherapy Relationships That Work
Norcross, J. C. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199737208.001.0001 · Cited by 947
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922 · cited 1×
“The increasing development of internet interventions and the evidence for their efficacy and effectiveness in treating diverse mental disorders [6-8] raised the question of what role therapeutic alliance might play in such treatments, especially in those providing guidance from trained supporters (guided internet inte…”
BackgroundKessler, R. C., Zhao, S., Blazer, D. G., & Swartz, M. (1997) · Journal of Affective Disorders · doi:10.1016/s0165-0327(97)00056-6 · Cited by 790 · cited 1×
“Considering the high rates of comorbidities in patients with depression [47], one extension that could enhance the intervention is incorporation of modules that address other relevant symptomatologies in these patients (eg, anxiety) [27].”
BackgroundKrupnick, J. L., Sotsky, S. M., Simmens, S., Moyer, J., Elkin, I., Watkins, J., & Pilkonis, P. A. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.3.532 · Cited by 605 · cited 1×
“Alliance also predicted outcomes in other health-related interventions, such as pharmacotherapy [4,5].”
BackgroundMunder, T., Wilmers, F., Leonhart, R., Linster, H. W., & Barth, J. (2010) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.658 · Cited by 538 · cited 1×
“The WAI-I is an instrument derived from the Working Alliance Inventory-Short revised [39], and it was specifically adapted to guided internet interventions [27,29].”
MethodResponsiveness in psychotherapy.
Stiles, W. B., Honos-Webb, L., & Surko, M. (1998) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.1998.tb00166.x · Cited by 442 · cited 1×
“Being responsive to patients’ needs has been presented as a fundamental process of change in face-to-face psychotherapy [44-46].”
BackgroundBerger, T., Hämmerli, K., Gubser, N., Andersson, G., & Caspar, F. (2011) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2011.616531 · Cited by 383 · cited 1×
“The instructions provided to the supporters were in line with those used in a similar previous trial [38].”
ResultSelf-citationBerger, T. (2017) · Psychotherapy Research · doi:10.1080/10503307.2015.1119908 · Cited by 364 · cited 5×
“Responding to a general call for further studies clarifying the role of alliance in online interventions [1,9], particularly with psychometrically sound instruments [29], the aim of this study was to analyze the alliance-outcome association in a guided internet intervention for participants with moderate depression, u…”
BackgroundMethodResultInfluentialSelf-citationMeyer, B., Berger, T., Caspar, F., Beevers, C. G., Andersson, G., & Weiss, M. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1151 · Cited by 363 · cited 9×
“Furthermore, the analyses of the study were conducted using a subsample of the EVIDENT randomized controlled trial (ie, patients with moderate depression who received Deprexis with email support).”
MethodInfluentialSelf-citationAndersson, G., Carlbring, P., Titov, N., & Lindefors, N. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719839381 · Cited by 324 · cited 1×
“The increasing development of internet interventions and the evidence for their efficacy and effectiveness in treating diverse mental disorders [6-8] raised the question of what role therapeutic alliance might play in such treatments, especially in those providing guidance from trained supporters (guided internet inte…”
BackgroundZilcha-Mano, S. (2017) · American Psychologist · doi:10.1037/a0040435 · Cited by 303 · cited 1×
“For instance, with only one assessment point for the alliance, differential effects of the general level of the alliance during treatment (ie, between-patient effects or trait-like components of alliance) and effects of the modifications of the alliance during the intervention (ie, within-patient effects or state-like…”
Background(1989) · PubMed · Cited by 298
Johansson, R., Sjöberg, E., Sjögren, M., Johnsson, E., Carlbring, P., Andersson, T., Rousseau, A., & Andersson, G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036905 · Cited by 268 · cited 1×
“In the last years, there were several developments in this direction, tailoring internet interventions to the symptom profile of the patient and offering individually prescribed treatment modules accordingly [27,48-50].”
BackgroundIndividually-tailored, internet-based treatment for anxiety disorders: a randomized controlled trial
Carlbring, P., Maurin, L., Törngren, C., Linna, E., Eriksson, T., Sparthan, E., Strååt, M., Marquez von Hage, C., Bergman-Nordgren, L., & Andersson, G. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.10.002 · Cited by 227 · cited 1×
“In the last years, there were several developments in this direction, tailoring internet interventions to the symptom profile of the patient and offering individually prescribed treatment modules accordingly [27,48-50].”
BackgroundCrits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0023668 · Cited by 220
Zuroff, D. C., & Blatt, S. J. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.1.130 · Cited by 209 · cited 1×
“Alliance also predicted outcomes in other health-related interventions, such as pharmacotherapy [4,5].”
BackgroundAndersson, G., Paxling, B., Wiwe, M., Vernmark, K., Felix, C. B., Lundborg, L., Furmark, T., Cuijpers, P., & Carlbring, P. (2012) · Behaviour Research and Therapy · doi:10.1016/j.brat.2012.05.003 · Cited by 202 · cited 2×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodResultMeyer, B., Bierbrodt, J., Schröder, J., Berger, T., Beevers, C. G., Weiss, M., Jacob, G., Späth, C., Andersson, G., Lutz, W., et al. (2015) · Internet Interventions · doi:10.1016/j.invent.2014.12.003 · Cited by 188 · cited 3×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodResultSelf-citationKnaevelsrud, C., & Maercker, A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.4.e31 · Cited by 184 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodBerger, T., Boettcher, J., & Caspar, F. (2014) · Psychotherapy · doi:10.1037/a0032527 · Cited by 158 · cited 8×
“Considering the high rates of comorbidities in patients with depression [47], one extension that could enhance the intervention is incorporation of modules that address other relevant symptomatologies in these patients (eg, anxiety) [27].”
BackgroundMethodResultInfluentialSelf-citationPreschl, B., Maercker, A., & Wagner, B. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-189 · Cited by 158 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodDynamic models of individual change in psychotherapy process research.
Falkenström, F., Finkel, S., Sandell, R., Rubel, J. A., & Holmqvist, R. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000203 · Cited by 147
Klein, J. P., Berger, T., Schröder, J., Späth, C., Meyer, B., Caspar, F., Lutz, W., Arndt, A., Greiner, W., Gräfe, V., et al. (2016) · Psychotherapy and Psychosomatics · doi:10.1159/000445355 · Cited by 146 · cited 3×
“For this study, we drew on a dataset from the EVIDENT study [33,34], a large multicenter randomized controlled trial analyzing the effects of the internet intervention Deprexis [35].”
MethodSelf-citationThe working alliance: Where have we been, where are we going?
Doran, J. M. (2016) · Psychotherapy Research · doi:10.1080/10503307.2014.954153 · Cited by 128
Krupnick, J. L., Sotsky, S. M., Elkin, I., Simmens, S., Moyer, J., Watkins, J., & Pilkonis, P. A. (2006) · Focus · doi:10.1176/foc.4.2.269 · Cited by 111
Wojnarowski, C., Firth, N., Finegan, M., & Delgadillo, J. (2019) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465819000080 · Cited by 108 · cited 1×
“Given the cumulative evidence showing that residual depressive symptoms are some of the main predictors of relapse [30-32], in this study, we will analyze how the alliance during treatment is associated with long-term outcomes (ie, 9-month follow-up) after a guided internet intervention for patients with moderate depr…”
BackgroundTwomey, C., O’Reilly, G., & Meyer, B. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.06.081 · Cited by 103 · cited 1×
“Deprexis is an internet intervention that demonstrated effectiveness when treating depression, showing medium effect sizes at posttreatment [37].”
BackgroundSelf-citationAndersson, G., Estling, F., Jakobsson, E., Cuijpers, P., & Carlbring, P. (2011) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2010.529457 · Cited by 102 · cited 1×
“In the last years, there were several developments in this direction, tailoring internet interventions to the symptom profile of the patient and offering individually prescribed treatment modules accordingly [27,48-50].”
BackgroundAnderson, R. E. E., Spence, S. H., Donovan, C. L., March, S., Prosser, S., & Kenardy, J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1848 · Cited by 102 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodThe Working Alliance Inventory for guided Internet interventions (WAI‐I)
Gómez Penedo, J. M., Berger, T., grosse Holtforth, M., Krieger, T., Schröder, J., Hohagen, F., Meyer, B., Moritz, S., & Klein, J. P. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.22823 · Cited by 100 · cited 5×
“Responding to a general call for further studies clarifying the role of alliance in online interventions [1,9], particularly with psychometrically sound instruments [29], the aim of this study was to analyze the alliance-outcome association in a guided internet intervention for participants with moderate depression, u…”
MethodInfluentialSelf-citationBergman Nordgren, L., Carlbring, P., Linna, E., & Andersson, G. (2013) · JMIR Research Protocols · doi:10.2196/resprot.2292 · Cited by 92 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
Wagner, B., Brand, J., Schulz, W., & Knaevelsrud, C. (2012) · Depression and Anxiety · doi:10.1002/da.21962 · Cited by 84 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodBockting, C., Spinhoven, P., Koeter, M. W. J., Wouters, L. F., & Schene, A. H. (2006) · PubMed · Cited by 84
Only connect: The working alliance in computer-based cognitive behavioral therapy
Kiluk, B. D., Serafini, K., Frankforter, T., Nich, C., & Carroll, K. M. (2014) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.10.003 · Cited by 80 · cited 3×
“Only Kiluk et al [24] presented an exploratory analysis of the adapted version of the WAI (ie, WAI-Tech) with some evidence of internal consistency (Cronbach alpha) and external validity (ie, significant correlations with the original WAI and with patient treatment satisfaction).”
BackgroundMethodBockting, C. L. H., Spinhoven, P., Koeter, M. W. J., Wouters, L. F., Schene, A. H., & Depression Evaluation Longitudinal Therapy Assessment (DELTA) Study Group (2006) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v67n0508 · Cited by 78
Jasper, K., Weise, C., Conrad, I., Andersson, G., Hiller, W., & Kleinstäuber, M. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.04.002 · Cited by 68 · cited 2×
“We decided to measure alliance only at the end of therapy, as some studies showed that patients might have difficulties to complete it early in treatment, because of the limited interaction with the supporter during the intervention [16].”
MethodResultHedman, E., Andersson, E., Lekander, M., & Ljótsson, B. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.11.009 · Cited by 56 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodOrmrod, J. A., Kennedy, L., Scott, J., & Cavanagh, K. (2010) · Cognitive Behaviour Therapy · doi:10.1080/16506071003675614 · Cited by 55 · cited 1×
“Some studies did this the other way around; they used adapted instruments to measure the alliance between the patient and the online intervention only [18,24-26].”
MethodRichards, D., Timulak, L., & Hevey, D. (2013) · Counselling and Psychotherapy Research · doi:10.1080/14733145.2012.733715 · Cited by 55
Andersson, E., Ljótsson, B., Hedman, E., Enander, J., Kaldo, V., Andersson, G., Lindefors, N., & Rück, C. (2015) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2014.10.003 · Cited by 52 · cited 1×
“To measure the alliance, most of the studies used the same instruments usually administered in face-to-face psychotherapy (ie, the Working Alliance Inventory [WAI]) [11] with very slight modifications (eg, talking about a treatment instead of therapy) and focused on the effects of the relationship between the patient…”
MethodScherer, S., Alder, J., Gaab, J., Berger, T., Ihde, K., & Urech, C. (2016) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2015.10.011 · Cited by 51 · cited 3×
“Recently, Gómez Penedo et al [29] systematized the efforts by Berger et al [27] and Scherer et al [28], presenting the Working Alliance Inventory for Guided Internet Interventions (WAI-I) and exploring the psychometric properties of the scale.”
BackgroundMethodResultSelf-citationKlein, J. P., Berger, T., Schröder, J., Späth, C., Meyer, B., Caspar, F., Lutz, W., Greiner, W., Hautzinger, M., Rose, M., et al. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-239 · Cited by 51 · cited 3×
“For this study, we drew on a dataset from the EVIDENT study [33,34], a large multicenter randomized controlled trial analyzing the effects of the internet intervention Deprexis [35].”
MethodSelf-citationWorking alliance with an avatar: How far can we go with internet interventions?
Heim, E., Rötger, A., Lorenz, N., & Maercker, A. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.01.005 · Cited by 41 · cited 2×
“Some studies did this the other way around; they used adapted instruments to measure the alliance between the patient and the online intervention only [18,24-26].”
MethodResultGomez Penedo, J. M., Constantino, M. J., Coyne, A. E., Westra, H. A., & Antony, M. M. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000233 · Cited by 37 · cited 1×
“Being responsive to patients’ needs has been presented as a fundamental process of change in face-to-face psychotherapy [44-46].”
BackgroundVerhoeven, F. E. A., Wardenaar, K. J., Ruhé, H. G. E., Conradi, H. J., & de Jonge, P. (2018) · Depression and Anxiety · doi:10.1002/da.22695 · Cited by 31 · cited 1×
“Given the cumulative evidence showing that residual depressive symptoms are some of the main predictors of relapse [30-32], in this study, we will analyze how the alliance during treatment is associated with long-term outcomes (ie, 9-month follow-up) after a guided internet intervention for patients with moderate depr…”
Background
Research works printed but not linked: 4
- 10.1037//0022-006x.64.3.532 · doi:10.1037//0022-006x.64.3.532 · Paper page
- Schmidt, J, 1989, Psychother Psychosom Med Psychol, 39, 248
- Raudenbush, S, 2001, Hierarchical Linear Models: Applications And Data Analysis Methods (Advanced Quantitative Techniques in The Social Sciences)
- Constantino, M, 2013, J Unified Psychother Clin Sci, 2, 1
Cited by
Andersson, G., Carlbring, P., Titov, N., & Lindefors, N. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719839381 · Cited by 324
Bickman, L. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01065-8 · Cited by 184
“Penedo et al. (2020), in their study of a guided internet-based treat-ment, showed that it was important to align with the client’s expectations and goals because these were related to outcomes, but no such relationship existed with the traditional third component of TA, bond with the supporting…”
BackgroundDigital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review
Tong, F., Lederman, R., D'Alfonso, S., Berry, K., & Bucci, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.819623 · Cited by 106
Herrero, R., Vara, M. D., Miragall, M., Botella, C., García-Palacios, A., Riper, H., Kleiboer, A., & Baños, R. M. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17176169 · Cited by 81
“[24] explored the reliability and validity of the WAI-SF in guided IBIs, taking into account the relational aspects involved in this type of intervention (e.”
BackgroundLevin, A. W., Lancelotta, R., Sepeda, N. D., Gukasyan, N., Nayak, S., Wagener, T. L., Barrett, F. S., Griffiths, R. R., & Davis, A. K. (2024) · PLoS ONE · doi:10.1371/journal.pone.0300501 · Cited by 82
Metadata
- Authors
- Gómez Penedo, J. M., Babl, A. M., grosse Holtforth, M., Hohagen, F., Krieger, T., Lutz, W., Meyer, B., Moritz, S., Klein, J. P., & Berger, T.
- Affiliations
- University of Bern (CH)
- University of Lübeck (DE)
- Universität Trier (DE)
- City, University of London (GB)
- University Medical Center Hamburg-Eppendorf (DE)
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-03-24
- Volume
- 22
- Issue
- 3
- Pages
- e15824
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7139432
- OpenAlex
- W3000810609
- Semantic Scholar
- 78da1d5718f34dda7f352a8587ea7b0e9b890542
- Cited by
- 33
- References
- 56
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 6 · 2023: 7 · 2022: 5 · 2021: 7 · 2020: 3
- FWCI
- 4.97
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- therapeutic relationship, depression, treatment satisfaction, email support, Deprexis, patient well-being, residual depressive symptoms, long-term outcomes
- MeSH terms
- Therapeutic Alliance, Internet-Based Intervention, Depression, Female, Humans, Male, Psychometrics, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.