Working Alliance Inventory for Online Interventions-Short Form (WAI-TECH-SF): The Role of the Therapeutic Alliance between Patient and Online Program in Therapeutic Outcomes
Open accessAbstract
Background: Therapeutic alliance (TA) between the patient and therapist has been related to positive therapeutic outcomes. Because Internet-based interventions are increasingly being implemented, a tool is needed to measure the TA with Internet-based self-guided programs. The Working Alliance Inventory for online interventions (WAI-TECH-SF) was adapted based on the WAI Short Form (Hatcher & Gillaspy, 2006). The objectives of this study were: (1) to analyse the psychometric properties of the WAI-TECH-SF; (2) to explore the differences in the WAI-TECH-SF scores according to different categories of the sample; and (3) to analyse whether the WAI-TECH-SF can predict therapeutic outcomes and satisfaction with the treatment. Methods: 193 patients diagnosed with depression were included and received blended Cognitive-Behavioural Therapy. Measures of preferences, satisfaction, and credibility about the treatment, TA with the online program, depressive symptoms, and satisfaction with the treatment were administered. Results: An exploratory factor analysis revealed a one-dimensional structure with adequate internal consistency. Linear regression analyses showed that the WAI-TECH-SF predicted changes in depressive symptoms and satisfaction with the treatment. Conclusions: WAI-TECH-SF is a reliable questionnaire to assess the TA between the patient and the online program, which is associated with positive therapeutic outcomes and satisfaction with the treatment.
AI snapshot
Written from the paper’s full text.
TL;DR
The study adapted the Working Alliance Inventory for online interventions (WAI-TECH-SF) to measure therapeutic alliance between patients and a self-guided online program in blended CBT for depression. It found a unidimensional, internally consistent scale whose scores predicted greater depressive symptom reduction and higher treatment satisfaction.
Key findings
- Exploratory factor analysis supported a one-dimensional WAI-TECH-SF structure explaining 73.49% of total variance, with Cronbach’s α = 0.97; the retained factor had a raw eigenvalue of 9.08 versus 1.53 at the 95th percentile from parallel analysis.
- WAI-TECH-SF scores predicted changes in depressive symptoms, F(1,188) = 14.42, p < 0.001, explaining 6.7% of the variance, and predicted treatment satisfaction, F(1,187) = 185.53, p < 0.001, explaining 49.7% of the variance.
- There were no significant WAI-TECH-SF differences by sex, age-range, initial depression severity, treatment preference, expectations, or credibility; patients with high versus low education scored higher, p = 0.042.
- In exploratory regression, level of education predicted WAI-TECH-SF scores (β = 0.180, p = 0.015; 2.7% variance), and a second model added age (education β = 0.205, p = 0.006; age β = 0.167, p = 0.024; 4.9% variance).
Methods
- 193 patients diagnosed with depression (ages 19–69; M = 40.44, SD = 12.79; 64.2% women) received blended CBT in the European e-Compared project across primary or specialized care.
- Measures included the WAI-TECH-SF at post-assessment, PHQ-9 at pre- and post-assessment, MINI diagnosis, treatment preference, CEQ expectancy/credibility, and CSQ-8 satisfaction.
- Analyses used SPSS v.26: missing data imputation, EFA with Maximum Likelihood extraction and parallel analysis, Cronbach’s alpha, t-tests/ANOVAs, and simple/stepwise linear regressions; power analysis was also reported.
Limitations
- The paper states the WAI-TECH-SF was administered only at the end of treatment, so it could not establish whether alliance preceded symptom and satisfaction changes or support causal conclusions.
- The paper states the sample was only depressive patients, so replication in other diagnoses is needed; adherence or number of sessions was not registered; and analyses included only patients who completed the questionnaire after the self-guided IBI.
- Evident from the text: the study used a clinical-trial sample with exclusion criteria and no control condition for the alliance analyses, which may limit generalizability and causal interpretation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 45 cited sources checked · 0 supported · 0 flagged
- Findingown data
Higher WAI-TECH-SF scores predicted a greater decrease in depression symptoms.
“Higher scores on the WAI-TECH-SF predicted a greater decrease in depression symptoms.”
Quote in the paperSubstantial claim - Findingown data
Patients with high education levels achieved higher scores on the WAI-TECH-SF than patients with low education levels, p = 0.042.
“However, there were significant differences in the WAI-TECH-SF scores based on the level of education. Patients with high (vs. low) education levels achieved higher scores on the WAI-TECH-SF, p = 0.042.”
Quote in the paperNot a substantial claim - Findingown data
Exclusion of any item increased the alpha value for the overall scale.
“We analysed the item–total correlation, and the exclusion of any item increased the alpha value for the overall scale.”
Quote in the paperNot a substantial claim - Findingown data
Cronbach's alpha coefficient for the WAI-TECH-SF was high for the overall scale (α = 0.97).
“Cronbach’s alpha coefficient for the WAI-TECH-SF was high for the overall scale (α = 0.97).”
Quote in the paperSubstantial claim - Findingown data
One dimension explained 73.49% of the total variance.
“Factorial rotation with one dimension was performed using the Maximum Likelihood extraction method, which showed that one dimension explained 73.49% of the total variance.”
Quote in the paperSubstantial claim
- Findingown data
Parallel Analysis showed that one factor had to be retained because only one factor had an eigenvalue of 9.08, greater than the 95th percentile eigenvalue of 1.53 for randomly generated data.
“Parallel Analysis [40] showed that one factor had to be retained because only one factor had an eigenvalue (raw data eigenvalue = 9.08) greater than the eigenvalue at the 95th percentile for randomly generated data (95th percentile eigenvalue = 1.53) [48].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [40] Schafer, J.L. (1997). Analysis of Incomplete Multivariate Data. Monographs on Statistics and Applied Probability, Chapman & Hall. source
Not checked in the free run.
- cites [48] Fabrigar, L.R., and Wegener, D.T. (2012). Exploratory Factor Analysis, Oxford University Press. source
Not checked in the free run.
- Findingown data
All items had minimum factor loadings and communalities above or equal to 0.30.
“The factorial solution showed that all the items had minimum factor loadings and communalities above ≥0.30 (see Table 2).”
Quote in the paperSubstantial claim - Findingown data
In a stepwise linear regression, level of education was a positive significant predictor of WAI-TECH-SF scores, with model F(1,181)=6.059, p=0.015, explaining 2.7% of variance.
“The first model included level of education (β = 0.180, t = 2.462, p = 0.015) as a positive significant predictor of the WAI-TECH-SF scores. This model was significant, F(1,181) = 6.059, p = 0.015, explaining 2.7% of the variance.”
Quote in the paperNot a substantial claim - Findingown data
The model predicting satisfaction with treatment from WAI-TECH-SF scores was statistically significant, F(1,187)=185.53, p<0.001, explaining 49.7% of variance.
“Similarly, the model in which satisfaction with the treatment was predicted by the WAI-TECH-SF scores was statistically significant, F(1,187) = 185.53, p < 0.001, explaining 49.7% of the variance.”
- Also stated in: § 5. Conclusions · plain
“Finally, this study highlights the importance of considering the “patient and online program TA” because the WAI-TECH-SF score was a significant predictor variable of both the change in depressive symptoms and satisfaction at the end of the treatment.”
Quote in the paperSubstantial claim - Also stated in: § 5. Conclusions · plain
- Findingown data
The WAI-TECH-SF predicts changes in depressive symptoms and satisfaction with treatment.
“Linear regression analyses showed that the WAI-TECH-SF predicted changes in depressive symptoms and satisfaction with the treatment.”
Quote in the paperSubstantial claim - Findingown data
The model predicting changes in PHQ pre-post intervention scores from WAI-TECH-SF scores was statistically significant, F(1,188)=14.42, p<0.001, explaining 6.7% of variance.
“The model where changes in PHQ pre-post intervention scores were predicted by WAI-TECH-SF scores was statistically significant, F(1,188) = 14.42, p < 0.001, explaining 6.7% of the variance.”
- Also stated in: § 5. Conclusions · plain
“Finally, this study highlights the importance of considering the “patient and online program TA” because the WAI-TECH-SF score was a significant predictor variable of both the change in depressive symptoms and satisfaction at the end of the treatment.”
Quote in the paperSubstantial claim - Also stated in: § 5. Conclusions · plain
- Findingown data
In a stepwise linear regression, level of education and age were positive significant predictors of WAI-TECH-SF scores, with model F(2,181)=5.696, p=0.004, explaining 4.9% of variance.
“The second model included level of education (β = 0.205, t = 2.798, p = 0.006) and age (β = 0.167, t = 2.279, p = 0.024) as positive significant predictors of the WAI-TECH-SF scores. This model was significant, F(2,181) = 5.696, p = 0.004, explaining 4.9% of the variance.”
Quote in the paperNot a substantial claim - Findingown data
Patients with a higher level of education scored higher on TA with the online program than patients with a low level of education.
“Nevertheless, patients with a higher level of education scored higher on TA with the online program than patients with a low level of education.”
Quote in the paperNot a substantial claim - Findingown data
Higher WAI-TECH-SF scores predicted higher satisfaction with treatment.
“Higher scores on the WAI-TECH-SF predicted higher scores on satisfaction with the treatment (see Table 4).”
Quote in the paperSubstantial claim - Findingown data
In exploratory factor analysis, a unidimensional structure emerged for the WAI-TECH-SF, accounting for 73.49% of explained variance.
“With regard to the psychometric properties of the WAI-TECH-SF, a unidimensional structure emerged in the EFA that accounted for 73.49% of the explained variance.”
Quote in the paperSubstantial claim - Findingown data
The overall WAI-TECH-SF scale had excellent internal consistency.
“All the factors had high factor loadings, and the overall scale had excellent internal consistency.”
Quote in the paperSubstantial claim - Findingreasoning
Older patients achieved higher TA in this study.
“Nevertheless, older patients achieved higher TA in our study.”
Quote in the paperNot a substantial claim - Findingown data
All factors of the WAI-TECH-SF had high factor loadings.
“All the factors had high factor loadings, and the overall scale had excellent internal consistency.”
Quote in the paperSubstantial claim - Interpretationown data
The WAI-TECH-SF is a reliable questionnaire to assess the therapeutic alliance between the patient and the online program.
“WAI-TECH-SF is a reliable questionnaire to assess the TA between the patient and the online program”
Quote in the paperSubstantial claim - Premisecitation
Similar TA scores have been found for face-to-face therapy and Internet-based interventions.
“similar TA scores have been found for face-to-face therapy and IBIs [3,14,18,19,20]”
Quote in the paperNot a substantial claim0 of 5 sources checked- cites [3] Wampold, 2018, The alliance in adult psychotherapy: A meta-analytic synthesis, Psychotherapy source
Not checked in the free run.
- cites [14] Kooistra, L., Ruwaard, J., Wiersma, J., van Oppen, P., and Riper, H. (2020). Working alliance in blended versus face-to-face cognitive behavioral treatment for patients with depression in specialized mental health care. J. Clin. Med., 9. source
Not checked in the free run.
- cites [18] Jasper, 2014, The working alliance in a randomized controlled trial comparing internet-based self-help and face-to-face cognitive behavior therapy for chronic tinnitus, Internet Interv. source
Not checked in the free run.
- cites [19] Knaevelsrud, C., and Maercker, A. (2007). Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: A randomized controlled clinical trial. BMC Psychiatry, 7. source
Not checked in the free run.
- cites [20] Preschl, B., Maercker, A., and Wagner, B. (2011). The working alliance in a randomized controlled trial comparing online with face-to-face cognitive-behavioral therapy for depression. BMC Psychiatry, 11. source
Not checked in the free run.
- Premisecitation
Interventions fully or partially delivered through the Internet, especially those based on Cognitive Behavioural Therapy, are effective in treating different psychological disorders.
“Well-established evidence shows that interventions fully or partially delivered through the Internet (especially those based on Cognitive Behavioural Therapy (CBT)) are effective in treating different psychological disorders [8,9,10]”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [8] Richards, 2012, Computer-based psychological treatments for depression: A systematic review and meta-analysis, Clin. Psychol. Rev. source
Not checked in the free run.
- cites [9] Berger, 2016, Internet interventions for depression: New developments, Dialogues Clin. Neurosci. source
Not checked in the free run.
- cites [10] Sztein, 2018, Efficacy of cognitive behavioural therapy delivered over the Internet for depressive symptoms: A systematic review and meta-analysis, J. Telemed. Telecare source
Not checked in the free run.
- Premisecitation
Similar moderate effect sizes have been found for the relationship between TA and therapeutic outcomes in Internet-based interventions.
“as well as similar moderate effect sizes for the relationship between TA and therapeutic outcomes [3,21].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [3] Wampold, 2018, The alliance in adult psychotherapy: A meta-analytic synthesis, Psychotherapy source
Not checked in the free run.
- cites [21] Probst, G.H., Berger, T., and Flückiger, C. (2019). The alliance-outcome relation in internet-based interventions for psychological disorders: A correlational meta-analysis. Verhaltenstherapie, 1–12. source
Not checked in the free run.
- Premisecitation
Previous e-Compared findings found that only therapist-rated TA, but not patient-rated TA, was predictive of changes in depression scores during blended treatment in a sample of 73 patients.
“previous findings found that only therapist-rated TA (but not patient-rated TA) was predictive of changes in depression scores during a blended treatment in a sample of 73 patients [30].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [30] Vernmark, 2019, Working alliance as a predictor of change in depression during blended cognitive behaviour therapy, Cogn. Behav. source
Not checked in the free run.
- Premisecitation
Internet-based interventions can be as effective as face-to-face therapy.
“and can be as effective as face-to-face therapy [11,12].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [11] Andersson, 2014, Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: A systematic review and meta-analysis, World Psychiatry source
Not checked in the free run.
- cites [12] Wagner, 2014, Internet-based versus face-to-face cognitive-behavioral intervention for depression: A randomized controlled non-inferiority trial, J. Affect. Disord. source
Not checked in the free run.
- Interpretationown data
In the context of IBIs, a three-dimensional structure of the WAI-TECH-SF cannot be assumed a priori.
“According to our findings, a three-dimensional structure cannot be assumed a priori in the context of IBIs.”
Quote in the paperSubstantial claim - Premisecitation
Miragall et al. [25] found a significant relationship between the TA with the technology and therapeutic outcomes.
“Hence, so far, only the present study and Miragall et al. [25] found a significant relationship between the TA with the technology (i.e., the TA between the patient and virtual and augmented reality) and therapeutic outcomes.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25] Miragall, 2015, Working alliance inventory applied to virtual and augmented reality (WAI-VAR): Psychometrics and therapeutic outcomes, Front Psychol. source
Not checked in the free run.
- Premisecitation
Kiluk et al. [29] did not find that the total scores on the long form of the WAI-Tech were associated with the change in therapeutic outcomes.
“By contrast, Kiluk et al. [29] did not find that the total scores on the long form of the WAI-Tech were associated with the change in therapeutic outcomes.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [29] Kiluk, 2014, Only connect: The working alliance in computer-based cognitive behavioral therapy, Behav. Res. source
Not checked in the free run.
- Interpretationown data
To our knowledge, this is the first study to confirm the relationship between patient and online program TA and therapeutic outcomes.
“However, to our knowledge, this is the first study to confirm the relationship between “patient and online program TA” and therapeutic outcomes.”
Quote in the paperSubstantial claim - Interpretationown data
The relationship between patient-online program TA and therapeutic outcomes is in line with the positive relationship found between patient-therapist TA and therapeutic outcomes in face-to-face therapy and IBIs.
“Consequently, the relationship between “patient-online program TA” and therapeutic outcomes is also in line with the positive relationship found between the “patient-therapist TA” and the therapeutic outcomes in face-to-face therapy [1,2] and IBIs [3,21].”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [1] Cameron, 2018, The relationship between the therapeutic alliance and clinical outcomes in cognitive behaviour therapy for adults with depression: A meta-analytic review, Clin. Psychol. Psychother. source
Not checked in the free run.
- cites [2] Horvath, 2011, Alliance in individual psychotherapy, Psychotherapy source
Not checked in the free run.
- cites [3] Wampold, 2018, The alliance in adult psychotherapy: A meta-analytic synthesis, Psychotherapy source
Not checked in the free run.
- cites [21] Probst, G.H., Berger, T., and Flückiger, C. (2019). The alliance-outcome relation in internet-based interventions for psychological disorders: A correlational meta-analysis. Verhaltenstherapie, 1–12. source
Not checked in the free run.
- Interpretationown data
The WAI-TECH-SF is a reliable questionnaire to measure TA with an online program.
“According to our exploratory factor analysis, the WAI-TECH-SF is a reliable questionnaire to measure this construct, but it would be advisable to calculate an overall score for the total scale, rather than using the traditional theoretical three-dimensional “task-goals-bonds” structure of TA.”
Quote in the paperSubstantial claim - Interpretationown data
Patients with major depression can develop TA with an online program during a self-guided IBI in the context of primary care.
“In conclusion, this study reveals that patients with major depression can develop TA with an online program during a self-guided IBI in the context of primary care.”
Quote in the paperSubstantial claim - Interpretationcitation
The unidimensional structure of the WAI-TECH-SF is in line with the structure found in the validation of the WAI applied to virtual and augmented reality (WAI-VAR).
“This unidimensionality is in line with the structure found in the validation of the WAI applied to virtual and augmented reality (WAI-VAR, [25]).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [25] Miragall, 2015, Working alliance inventory applied to virtual and augmented reality (WAI-VAR): Psychometrics and therapeutic outcomes, Front Psychol. source
Not checked in the free run.
- Interpretationown data
The unidimensional structure of the WAI-TECH-SF is inconsistent with the three-dimensional structure of Bordin's theory and the original validation of the WAI-SF.
“However, this structure is inconsistent with the three-dimensional structure of Bordin’s [4] theory and the original validation of the WAI-SF carried out by Hatcher and Gallispy [16] to measure TA in the face-to-face context, distinguishing three separate factors: tasks, goals, and bonds.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [4] Bordin, 1979, The generalizability of the psychoanalytic concept of the working alliance, Psychol. Psychother.
Not checked in the free run.
- cites [16] Hatcher, 2007, Development and validation of a revised short version of the working alliance inventory, Psychother. Res. source
Not checked in the free run.
- Interpretationcitation
So far, only the present study and Miragall et al. [25] found a significant relationship between the TA with the technology and therapeutic outcomes.
“Hence, so far, only the present study and Miragall et al. [25] found a significant relationship between the TA with the technology (i.e., the TA between the patient and virtual and augmented reality) and therapeutic outcomes.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [25] Miragall, 2015, Working alliance inventory applied to virtual and augmented reality (WAI-VAR): Psychometrics and therapeutic outcomes, Front Psychol. source
Not checked in the free run.
- Premisecitation
There is a positive relationship between patient-therapist TA and therapeutic outcomes in face-to-face therapy and IBIs.
“Consequently, the relationship between “patient-online program TA” and therapeutic outcomes is also in line with the positive relationship found between the “patient-therapist TA” and the therapeutic outcomes in face-to-face therapy [1,2] and IBIs [3,21].”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [1] Cameron, 2018, The relationship between the therapeutic alliance and clinical outcomes in cognitive behaviour therapy for adults with depression: A meta-analytic review, Clin. Psychol. Psychother. source
Not checked in the free run.
- cites [2] Horvath, 2011, Alliance in individual psychotherapy, Psychotherapy source
Not checked in the free run.
- cites [3] Wampold, 2018, The alliance in adult psychotherapy: A meta-analytic synthesis, Psychotherapy source
Not checked in the free run.
- cites [21] Probst, G.H., Berger, T., and Flückiger, C. (2019). The alliance-outcome relation in internet-based interventions for psychological disorders: A correlational meta-analysis. Verhaltenstherapie, 1–12. source
Not checked in the free run.
- May combine two claims. 16
- Classified as probably not a substantial claim. 14
May combine two claims.
Findingown dataThere were no significant differences in WAI-TECH-SF scores based on sex, age-range, initial severity of depression, preferences for any of the treatments offered, expectations about the treatment, and credibility of the treatment.
“Independent-sample t-tests and one-way ANOVAs showed that there were no significant differences in the WAI-TECH-SF scores based on sex, age-range, initial severity of depression, preferences for any of the treatments offered, expectations about the treatment, and credibility of the treatment.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe WAI-TECH-SF has a one-dimensional structure with adequate internal consistency.
“An exploratory factor analysis revealed a one-dimensional structure with adequate internal consistency.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe average WAI-TECH-SF score was around 58 (on a scale ranging from 12 to 84).
“The average score on the WAI-TECH-SF was around 58 (on a scale ranging from 12 to 84).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationClarke et al. did not find a relationship between TA and therapeutic outcomes.
“In this study, they did not find a relationship between TA and therapeutic outcomes.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [22] Clarke, 2016, Therapeutic alliance with a fully automated mobile phone and web-based intervention: Secondary analysis of a randomized controlled trial, JMIR Ment. Health source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductnone givenAll patients were diagnosed with depression using the MINI.
“All the patients were diagnosed with depression using the MINI.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationIndividuals have the ability to form a bond and be open with an online application.
“Previous studies have found that individuals have the ability to form a bond and be open with an online application [28].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [28] Berry, 2018, Assessing therapeutic alliance in the context of mHealth interventions for mental health problems: Development of the mobile Agnew relationship measure (mARM) questionnaire, J. Med. Internet Res. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationClarke et al. found that TA is high even when the intervention does not involve human support.
“they found that TA is high even when the intervention does not involve human support.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [22] Clarke, 2016, Therapeutic alliance with a fully automated mobile phone and web-based intervention: Secondary analysis of a randomized controlled trial, JMIR Ment. Health source
Not checked in the free run.
May combine two claims.
Interpretationown dataThe WAI-TECH-SF is associated with positive therapeutic outcomes and satisfaction with treatment.
“which is associated with positive therapeutic outcomes and satisfaction with the treatment.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationTherapeutic alliance (TA) between the patient and therapist has a relevant influence on therapeutic outcomes.
“Evidence shows that the therapeutic alliance (TA) (also called the working alliance) has a relevant influence on therapeutic outcomes [1].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] Cameron, 2018, The relationship between the therapeutic alliance and clinical outcomes in cognitive behaviour therapy for adults with depression: A meta-analytic review, Clin. Psychol. Psychother. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe adaptation of the patient version of the WAI-SF was carried out following the recommendations of Hambleton and Patsula.
“An adaptation of the patient version of the WAI-SF [16] was carried out following the recommendations of Hambleton and Patsula [39].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [16] Hatcher, 2007, Development and validation of a revised short version of the working alliance inventory, Psychother. Res. source
Not checked in the free run.
- cites [39] Hambleton, 1999, Increasing the validity of adapted tests: Myths to be avoided and guidelines for improving test adaptation practices, J. Appl. Test. Technol.
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study was approved by the corresponding ethical committee in each country.
“The study was approved by the corresponding ethical committee in each country:”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWAI-Tech total scores were not associated with change in therapeutic outcomes.
“Results showed lower scores on the bond subscale, and the total scores on WAI-Tech were not associated with the change in therapeutic outcomes.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [29] Kiluk, 2014, Only connect: The working alliance in computer-based cognitive behavioral therapy, Behav. Res. source
Not checked in the free run.
May combine two claims.
PremisecitationA systematic review showed an effect of TA on anxiety and depressive outcomes, with higher levels of TA related to better clinical outcomes.
“a systematic review showed an effect of TA on anxiety and depressive outcomes, pointing out that higher levels of TA were related to better clinical outcomes [23].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23] Pihlaja, 2018, Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders–a systematic review, Internet Interv. source
Not checked in the free run.
May combine two claims.
Study design and conductown dataMissing values in the WAI-TECH-SF, PHQ-9, and CEQ ranged from 0% to 1.6%, were missing at random according to Little's MCAR test (p > 0.05), and were imputed using the Expectation-Maximization Algorithm method.
“The percentage of missing values in the WAI-TECH-SF, PHQ-9, and CEQ scores ranged from 0% to 1.6%. After testing that the values were missing at random using Little’s MCAR test (p > 0.05), they were imputed using the Expectation–Maximization Algorithm method [40].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [40] Schafer, J.L. (1997). Analysis of Incomplete Multivariate Data. Monographs on Statistics and Applied Probability, Chapman & Hall. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenAll participants were informed about the study and gave informed consent before the trial, in accordance with the Declaration of Helsinki.
“All participants were informed about the study and gave their informed consent before the beginning of the trial, in accordance with the Declaration of Helsinki.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPatients with higher scores on TA were more satisfied with the intervention after treatment.
“Moreover, patients with higher scores on TA were more satisfied with the intervention after the treatment.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24] Babl, 2020, The association of therapeutic alliance with long-term outcome in a guided Internet Intervention for depression: Secondary analysis from a randomized control trial, J. Med. Internet Res. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTA is moderately associated with better treatment outcomes in face-to-face therapy, regardless of the therapeutic framework or patient characteristics.
“Several meta-analyses have found that TA is moderately associated with better treatment outcomes in face-to-face therapy [1,2], regardless of the therapeutic framework or patient characteristics, among other aspects [3].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [1] Cameron, 2018, The relationship between the therapeutic alliance and clinical outcomes in cognitive behaviour therapy for adults with depression: A meta-analytic review, Clin. Psychol. Psychother. source
Not checked in the free run.
- cites [2] Horvath, 2011, Alliance in individual psychotherapy, Psychotherapy source
Not checked in the free run.
- cites [3] Wampold, 2018, The alliance in adult psychotherapy: A meta-analytic synthesis, Psychotherapy source
Not checked in the free run.
May combine two claims.
PremisecitationGómez-Penedo et al. found that the WAI for guided Internet interventions had a two-factor solution ("tasks-goals" and "bonds") with adequate internal consistency and external validity in a sample of patients with mild to moderate depression.
“Results of the WAI for guided Internet interventions (i.e., WAI-I) showed a two-factor solution (“tasks-goals” and “bonds”) with adequate internal consistency and external validity in a sample of patients with mild to moderate depression.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [24] Babl, 2020, The association of therapeutic alliance with long-term outcome in a guided Internet Intervention for depression: Secondary analysis from a randomized control trial, J. Med. Internet Res. source
Not checked in the free run.
May combine two claims.
Study design and conductown dataThe current study had 69.38% and 62.68% power for one-way ANOVAs with three and four groups, respectively, 75.49% for the t-test, and 99.77% for the regression analyses with one predictor to detect a medium effect size at p < 0.05.
“Results indicated that the current study had 69.38% and 62.68% power for one-way ANOVAs with three and four groups, respectively, 75.49% for the t-test, and 99.77% for the regression analyses with one predictor to detect a medium effect size at p < 0.05.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationKiluk et al.'s WAI-Tech showed similar psychometric properties to the original WAI scale.
“Kiluk et al. [29] conducted an adaptation of the 36-item original version of the WAI called WAI-Tech, which was designed to measure the TA between cocaine-dependent patients and the online program, and it showed similar psychometric properties to the original scale (i.e., the WAI).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [29] Kiluk, 2014, Only connect: The working alliance in computer-based cognitive behavioral therapy, Behav. Res. source
Not checked in the free run.
May combine two claims.
Study design and conductown dataThe KMO value was 0.96 and Bartlett's Test of Sphericity was significant, showing that it was appropriate to perform a factor analysis.
“The KMO value was (0.96), and the Barlett’s Test of Sphericity value, χ2(66) = 2587.26, p < 0.001, showed that it was appropriate to perform a factor analysis.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe therapists were trained in how to deliver the blended treatment.
“All of them were trained in how to deliver the blended treatment.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationIn a WAI adapted to measure TA between patients and the virtual environment, the three dimensions proposed by Bordin did not arise in exploratory factor analysis, and only one general dimension was found.
“in the WAI adapted to measure the TA between the patients and the virtual environment, where the word “therapist” was replaced by “virtual environment”, the three dimensions proposed by Bordin [4] did not arise in the exploratory factor analysis, and only one general dimension was found [25].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25] Miragall, 2015, Working alliance inventory applied to virtual and augmented reality (WAI-VAR): Psychometrics and therapeutic outcomes, Front Psychol. source
Not checked in the free run.
May combine two claims.
Study design and conductown dataThe WAI-TECH-SF was only administered at the end of the treatment, which did not allow exploration of whether patient-online program TA preceded symptoms and satisfaction throughout therapeutic sessions.
“First, the WAI-TECH-SF was only administered at the end of the treatment, which did not allow us to explore whether the “patient and online program TA” preceded the symptoms and satisfaction throughout the therapeutic sessions.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe adherence or number of sessions performed by the patients was not registered.
“Third, the adherence or number of sessions performed by the patients was not registered.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe study sample was only composed of depressive patients, limiting generalizability.
“Second, the study sample was only composed of depressive patients.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe statistical analyses of TA were only conducted with patients who accepted to fill in the questionnaire after the self-guided IBI was finished, but the normal distribution and wide range of variability of WAI-TECH-SF scores allowed reliable conclusions.
“Fourth, the statistical analyses of TA were only conducted with the patients that accepted to fill in the questionnaire after the self-guided IBI was finished. However, the normal distribution (e.g., skewness = −0.56; kurtosis = −0.83) and the wide range of variability of the WAI-TECH-SF scores (i.e., from 12 to 84) allowed us to draw reliable conclusions.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataIn the case of TA with an online program during a self-guided IBI, the theoretical distinction between task, goals, and bond with the online program was not psychometrically significant, and a single factor could explain the majority of the explained variance of the TA between the patient and the online program.
“More specifically, in the case of the TA with an online program during a self-guided IBI, the theoretical distinction between task, goals, and bond with the online program was not psychometrically significant, and a single factor could explain the majority of the explained variance of the TA between the patient and the online program.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningPatients can feel that the program is "taking care" of them, in terms of allowing them to achieve therapeutic goals, proposing appropriate tasks to achieve these goals, and making them feel "embraced" and "cared for" by the program.
“Thus, patients can feel that the program is “taking care” of them, in terms of allowing them to achieve therapeutic goals, proposing appropriate tasks to achieve these goals, and making them feel “embraced” and “cared for” by the program.”
Quote in the paperSubstantial claim
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some studies created DTA instruments by editing WAI or ARM.
“Most quantitative studies have used the traditional in-person TA scales, such as the WAI (32) and Agnew Relationship Measure (14), to analyze the TA in the digital context, while some studies created DTA instruments by editing WAI or ARM (33–37).”
From Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review (2022) · cites this paper as [35]
Modifying the words in TA scales is not sufficient to develop new DTA scales.
“Herrero et al. (35) suggested that modifying the words in TA scales is not sufficient to develop new DTA scales.”
From Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review (2022) · cites this paper as Herrero et al. (35)
Herrero et al. found good psychometric properties of the WAI-TECH-SF and its association with positive therapeutic outcomes (changes in depressive symptoms) and satisfaction with treatment in a sample of 193 participants with depressed diagnosis.
“Authors found good psychometric properties of the WAI-TECH-SF and its association with positive therapeutic outcomes (changes in depressive symptoms) and satisfaction with the treatment in a sample of 193 participants with depressed diagnosis.”
From Using Information and Communication Technologies (ICT) for Mental Health Prevention and Treatment (2021) · cites this paper as [14]
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentχ²(33) = 14.27reported p = .998 · recomputed p = .998
“χ2(33) = 14.27, p = .998”
- Consistentχ²(66) = 2587.26reported p < .001 · recomputed p < .001
“χ2(66) = 2587.26, p < .001”
- ConsistentF(1, 181) = 6.059reported p = .015 · recomputed p = .015
“F(1,181) = 6.059, p = .015”
- ConsistentF(2, 181) = 5.696reported p = .004 · recomputed p = .004
“F(2,181) = 5.696, p = .004”
- ConsistentF(1, 188) = 14.42reported p < .001 · recomputed p < .001
“F(1,188) = 14.42, p < .001”
- ConsistentF(1, 187) = 185.53reported p < .001 · recomputed p < .001
“F(1,187) = 185.53, p < .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Nested regression models imply different sample sizes
MinorThe first model's F(1,181) implies 183 observations, whereas the second model's F(2,181) implies 184. Adding age to the first model cannot increase the number of observations while retaining its education predictor.
“The first model included level of education (β = 0.180, t = 2.462, p = 0.015) as a positive significant predictor of the WAI-TECH-SF scores. This model was significant, F(1,181) = 6.059, p = 0.015”
“The second model included level of education (β = 0.205, t = 2.798, p = 0.006) and age (β = 0.167, t = 2.279, p = 0.024) as positive significant predictors of the WAI-TECH-SF scores. This model was significant, F(2,181) = 5.696, p = 0.004”
A scale described as 10-point ranges only from 1 to 9
MinorA scale with scores from 1 through 9 has nine points, not ten.
“three questions rated on a 10-point scale”
“ranging from 1 to 9”
Twelve-item scale lists item 10 twice and omits item 12
MinorThe three four-item dimension lists name only 11 distinct items: item 10 appears under both goals and tasks, while item 12 appears under none.
“(1) therapeutic goals (items 1, 2, 8, 10), (2) tasks (items 4, 6, 10, 11), and (3) bonds (items 3, 5, 7, 9).”
“It is a 12-item self-report questionnaire designed to assess the TA with the online program”
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · see citing works
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139 · see citing works
(1998) · PubMed · Cited by 10,216 · see citing works
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587 · see citing works
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,844 · see citing works
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012 · see citing works
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627 · see citing works
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238 · see citing works
O’connor, B. P. (2000) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200807 · Cited by 4,116 · see citing works
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520 · see citing works
Psychometric properties of the credibility/expectancy questionnaire
Devilly, G. J., & Borkovec, T. D. (2000) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/s0005-7916(00)00012-4 · Cited by 2,295 · cited 1× · see citing works
“Specifically, the following question was asked: “If you had the chance to choose your depression treatment, which one would you prefer to receive?” Credibility and Expectancy Questionnaire (CEQ; [37]) was used to assess the prior predisposition of patients to the proposed intervention.”
MethodAlliance in individual psychotherapy
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011) · Psychotherapy · doi:10.1037/a0022186 · Cited by 2,072 · cited 2× · see citing works
“associated with better treatment outcomes in face-to-face therapy [1,2], regardless of the therapeutic framework or patient characteristics, among other aspects [3].”
BackgroundResult(1995) · Cited by 1,801 · see citing works
The alliance in adult psychotherapy: A meta-analytic synthesis
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018) · Psychotherapy · doi:10.1037/pst0000172 · Cited by 1,763 · cited 16× · see citing works
“However, these results should be interpreted with caution because IBIs are continuously evolving, and a more personalized treatment that uses algorithms to provide personalized feedback or set individualized goals or tasks depending on the emotional state or unique needs of each patient throughout the treatment could…”
BackgroundResultInfluentialLevis, B., Benedetti, A., & Thombs, B. D. (2019) · BMJ · doi:10.1136/bmj.l1476 · Cited by 1,718 · see citing works
Factor structure of the working alliance inventory
Tracey, T. J., & Kokotovic, A. M. (1989) · Psychological Assessment: A Journal of Consulting and Clinical Psychology · doi:10.1037/1040-3590.1.3.207 · Cited by 1,501 · see citing works
Computer-based psychological treatments for depression: A systematic review and meta-analysis
Richards, D., & Richardson, T. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.02.004 · Cited by 1,442 · see citing works
Fabrigar, L. R., & Wegener, D. T. (2011) · doi:10.1093/acprof:osobl/9780199734177.001.0001 · Cited by 1,408 · see citing works
Development and validation of a revised short version of the working alliance inventory
Hatcher, R. L., & Gillaspy, J. A. (2006) · Psychotherapy Research · doi:10.1080/10503300500352500 · Cited by 1,355 · see citing works
Rebooting psychotherapy research and practice to reduce the burden of mental illness
Kazdin, A. E., & Blase, S. L. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393527 · Cited by 1,254 · see citing works
Andersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014) · World Psychiatry · doi:10.1002/wps.20151 · Cited by 1,077 · see citing works
Self-citationBrysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996 · see citing works
Russell, D. W. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/014616702237645 · Cited by 954 · see citing works
Assessment of patient satisfaction: Development and refinement of a Service Evaluation Questionnaire
Nguyen, T. D., Attkisson, C. C., & Stegner, B. L. (1983) · Evaluation and Program Planning · doi:10.1016/0149-7189(83)90010-1 · Cited by 736 · see citing works
Erbe, D., Eichert, H.-C., Riper, H., & Ebert, D. D. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6588 · Cited by 532 · see citing works
Self-citationDiagnostic accuracy of the mood module of the patient health questionnaire: a systematic review
Wittkampf, K. A., Naeije, L., Schene, A. H., Huyser, J., & van Weert, H. C. (2007) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2007.06.004 · Cited by 493 · see citing works
Hambleton, R. K., & Patsula, L. N. (1999) · Cited by 423 · see citing works
Knaevelsrud, C., & Maercker, A. (2007) · BMC Psychiatry · doi:10.1186/1471-244x-7-13 · Cited by 398 · see citing works
Wagner, B., Horn, A. B., & Maercker, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.06.032 · Cited by 311 · see citing works
Technology Adoption by Older Adults: Findings From the PRISM Trial
Mitzner, T. L., Savla, J., Boot, W. R., Sharit, J., Charness, N., Czaja, S. J., & Rogers, W. A. (2019) · The Gerontologist · doi:10.1093/geront/gny113 · Cited by 311 · see citing works
IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety
Mohr, D. C., Tomasino, K. N., Lattie, E. G., Palac, H. L., Kwasny, M. J., Weingardt, K., Karr, C. J., Kaiser, S. M., Rossom, R. C., Bardsley, L. R., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6645 · Cited by 301 · see citing works
Attitudes towards digital treatment for depression: A European stakeholder survey
Topooco, N., Riper, H., Araya, R., Berking, M., Brunn, M., Chevreul, K., Cieslak, R., Ebert, D. D., Etchmendy, E., Herrero, R., et al. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.01.001 · Cited by 262 · see citing works
Self-citationBatterham, P. J., & Calear, A. L. (2017) · JMIR Mental Health · doi:10.2196/mental.7722 · Cited by 161 · see citing works
Berger, T., Boettcher, J., & Caspar, F. (2014) · Psychotherapy · doi:10.1037/a0032527 · Cited by 158 · see citing works
Preschl, B., Maercker, A., & Wagner, B. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-189 · Cited by 158 · see citing works
Sztein, D. M., Koransky, C. E., Fegan, L., & Himelhoch, S. (2018) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x17717402 · Cited by 135 · see citing works
Pihlaja, S., Stenberg, J.-H., Joutsenniemi, K., Mehik, H., Ritola, V., & Joffe, G. (2018) · Internet Interventions · doi:10.1016/j.invent.2017.11.005 · Cited by 134 · see citing works
Kleiboer, A., Smit, J., Bosmans, J., Ruwaard, J., Andersson, G., Topooco, N., Berger, T., Krieger, T., Botella, C., Baños, R., et al. (2016) · Trials · doi:10.1186/s13063-016-1511-1 · Cited by 133 · see citing works
Self-citationCameron, S. K., Rodgers, J., & Dagnan, D. (2018) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2180 · Cited by 128 · see citing works
Internet interventions for depression: new developments
Schröder, J., Berger, T., Westermann, S., Klein, J. P., & Moritz, S. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.2/jschroeder · Cited by 105 · see citing works
Berry, K., Salter, A., Morris, R., James, S., & Bucci, S. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8252 · Cited by 103 · see citing works
Clarke, J., Proudfoot, J., Whitton, A., Birch, M.-R., Boyd, M., Parker, G., Manicavasagar, V., Hadzi-Pavlovic, D., & Fogarty, A. (2016) · JMIR Mental Health · doi:10.2196/mental.4656 · Cited by 86 · see citing works
Only connect: The working alliance in computer-based cognitive behavioral therapy
Kiluk, B. D., Serafini, K., Frankforter, T., Nich, C., & Carroll, K. M. (2014) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.10.003 · Cited by 80 · cited 2× · see citing works
“[29] did not find that the total scores on the long form of the WAI-Tech were associated with the change in therapeutic outcomes.”
BackgroundMethodJasper, K., Weise, C., Conrad, I., Andersson, G., Hiller, W., & Kleinstäuber, M. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.04.002 · Cited by 68 · see citing works
Miloff, A., Carlbring, P., Hamilton, W., Andersson, G., Reuterskiöld, L., & Lindner, P. (2020) · Journal of Medical Internet Research · doi:10.2196/16660 · Cited by 60 · see citing works
Working alliance as a predictor of change in depression during blended cognitive behaviour therapy
Vernmark, K., Hesser, H., Topooco, N., Berger, T., Riper, H., Luuk, L., Backlund, L., Carlbring, P., & Andersson, G. (2019) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1533577 · Cited by 59 · see citing works
Self-citationStaying connected during the COVID-19 pandemic
Ng, Q. X., Chee, K. T., Deyn, M. L. Z. Q. D., & Chua, Z. (2020) · International Journal of Social Psychiatry · doi:10.1177/0020764020926562 · Cited by 56 · see citing works
Probst, G. H., Berger, T., & Flückiger, C. (2022) · Verhaltenstherapie · doi:10.1159/000503432 · Cited by 54 · see citing works
Moret‐Tatay, C., Beneyto-Arrojo, M. J., Gutierrez, E., Boot, W. R., & Charness, N. H. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01165 · Cited by 53 · see citing works
Predicting outcome of Internet-based treatment for depressive symptoms
Warmerdam, L., van Straten, A., Twisk, J. W. R., & Cuijpers, P. J. (2013) · Psychotherapy Research · doi:10.1080/10503307.2013.807377 · Cited by 49 · see citing works
Considering the therapeutic alliance in digital mental health interventions
Henson, P., Peck, P., & Torous, J. (2019) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000224 · Cited by 43 · see citing works
Working alliance with an avatar: How far can we go with internet interventions?
Heim, E., Rötger, A., Lorenz, N., & Maercker, A. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.01.005 · Cited by 41 · see citing works
Gómez Penedo, J. M., Babl, A. M., grosse Holtforth, M., Hohagen, F., Krieger, T., Lutz, W., Meyer, B., Moritz, S., Klein, J. P., & Berger, T. (2020) · Journal of Medical Internet Research · doi:10.2196/15824 · Cited by 33 · see citing works
Kooistra, L., Ruwaard, J., Wiersma, J., van Oppen, P., & Riper, H. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020347 · Cited by 30 · see citing works
Self-citationMiragall, M., Baños, R. M., Cebolla, A., & Botella, C. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01531 · Cited by 29 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review
Tong, F., Lederman, R., D'Alfonso, S., Berry, K., & Bucci, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.819623 · Cited by 106 · see citing works
“Herrero et al. (35) suggested that modifying the words in TA scales is not sufficient to develop new DTA scales.”
BackgroundAlliance with an unguided smartphone app: validation of the digital working alliance inventory
Goldberg, S. B., Baldwin, S. A., Riordan, K. M., Torous, J., Dahl, C. J., Davidson, R. J., & Hirshberg, M. J. (2022) · Assessment · doi:10.1177/10731911211015310 · Cited by 74 · see citing works
Linares, R., Aranda, M., García-Domingo, M., Amezcua, T., Fuentes, V., & Moreno-Padilla, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0253180 · Cited by 51 · see citing works
Domhardt, M., Engler, S., Nowak, H., Lutsch, A., Baumel, A., & Baumeister, H. (2021) · Journal of Medical Internet Research · doi:10.2196/29742 · Cited by 41 · see citing works
Using Information and Communication Technologies (ICT) for Mental Health Prevention and Treatment
Fonseca, A., & Osma, J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18020461 · Cited by 30 · see citing works
Braun, L., Terhorst, Y., Titzler, I., Freund, J., Thielecke, J., Ebert, D. D., & Baumeister, H. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192113858 · Cited by 17 · see citing works
Saxler, E., Schindler, T., Philipsen, A., Schulze, M., & Lux, S. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1293851 · Cited by 17 · see citing works
Barakat, S., Touyz, S., Maloney, D., Russell, J., Hay, P., Cunich, M., Lymer, S., Kim, M., Madden, S., Miskovic-Wheatley, J., et al. (2021) · Journal of Eating Disorders · doi:10.1186/s40337-021-00482-w · Cited by 14 · see citing works
- Authors
- Herrero, R., Vara, M. D., Miragall, M., Botella, C., García-Palacios, A., Riper, H., Kleiboer, A., & Baños, R. M.
- Affiliations
- Universitat de València (ES)
- Polibienestar Research Institute, University of Valencia, 46022 Valencia, Spain
- Universitat Jaume I (ES)
- CIBERObn Physiopathology of Obesity and Nutrition, Instituto de Salud Carlos III, 28029 Madrid, Spain
- GGZ inGeest (NL)
- Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Miragall, Marta
- Venue
- International Journal of Environmental Research and Public Health
- Publisher
- MDPI AG
- Published
- 2020-08-25
- Volume
- 17
- Issue
- 17
- Pages
- 6169
- ISSN
- 1660-4601 (ISSN-L)
- PMC
- PMC7503297
- OpenAlex
- W3080571288
- Semantic Scholar
- 11de240f031ba3e5c76b0e8d8788b0ba01f36bbe
- Cited by
- 81
- References
- 56
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 12 · 2023: 5 · 2022: 9 · 2021: 32 · 2020: 6
- FWCI
- 5.20
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- International Journal of Environmental Research and Public Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (publishedVersion, other-oa)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- Repositori UJI (Universitat Jaume I) (submittedVersion, cc-by-sa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion, other-oa)
- VU Research Portal (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- European Commission (FP7-Health-2013-Innovation-N603098)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- therapeutic relationship, online intervention, psychometric properties, depression, treatment satisfaction, treatment credibility, depressive symptoms, questionnaire adaptation
- MeSH terms
- Therapeutic Alliance, Internet-Based Intervention, Adolescent, Adult, Aged, Behavior Therapy, Depression, Female, Humans, Male, Middle Aged, Professional-Patient Relations, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Treatment Outcome, Patient Satisfaction, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
