Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index; 10 of the 65 sources its claims cite were checked for support.
Abstract
Background: Automated virtual reality exposure therapies (VRETs) are self-help treatments conducted by oneself and supported by a virtual therapist embodied visually and/or with audio feedback. This simulates many of the nonspecific relational elements and common factors present in face-to-face therapy and may be a means of improving adherence to and efficacy of self-guided treatments. However, little is known about alliance toward the virtual therapist, despite alliance being an important predictor of treatment outcome. Objective: In this study, we aimed to evaluate the first alliance instrument developed for use with embodied virtual therapists in an automated treatment format—the Virtual Therapist Alliance Scale (VTAS)—by (1) assessing its psychometric properties, (2) verifying the dimensionality of the scale, and (3) determining the predictive ability of the scale with treatment outcome. Methods: A psychometric evaluation and exploratory factor analysis of the VTAS was conducted using data from two samples of spider-fearful patients treated with VRET and the help of an embodied, voice-based virtual therapist (n=70). Multiple regression models and bivariate correlations were used to assess the VTAS relationship with treatment outcome, according to self-reported fear and convergence with presence and user-friendliness process measures. Results: The VTAS showed a sound two-factor solution composed of a primary factor covering task, goal, and copresence; adequate internal consistency; and good convergent validity, including moderate correlation (r=.310, P=.01) with outcomes over follow-up. Conclusions: These preliminary results suggest that alliance toward a virtual therapist is a significant predictor of treatment outcome, favors the importance of a task-goal over bond-factor, and should be explored in studies with larger sample sizes and in additional forms of embodiment.
Overview
CoChat Summary
## TL;DR
This study developed and psychometrically evaluated the Virtual Therapist Alliance Scale (VTAS) for automated virtual reality exposure therapy (VRET) with embodied virtual therapists. In two spider-fearful patient samples, the VTAS showed a two-factor structure and good overall reliability, and alliance scores were positively related to follow-up outcome.
## Key findings
- The VTAS had a 17-item format and showed a two-factor solution: factor 1 covered task, goal, and copresence items, and factor 2 covered bond and empathy items.
- Internal consistency was high for the full scale: Cronbach alpha = .921 and McDonald omega = .921 overall; by study, alpha/omega were .915/.917 and .936/.938.
- Factor 1 explained 38.8% of total variance and factor 2 explained 14.0%.
- VTAS total scores correlated with outcome at follow-up: the abstract reports r = .310, P = .01, while the discussion describes a posttreatment correlation of r = .213 and a follow-up correlation of r = .310.
## Methods
- Psychometric evaluation and exploratory factor analysis of the VTAS were conducted using data from two independent samples of spider-fearful patients treated with automated VRET and an embodied, voice-based virtual therapist.
- The combined final sample with VTAS data was 70 participants; the two studies had 50 eligible and 25 eligible participants, respectively, and were conducted in a university laboratory environment.
- Analyses included maximum likelihood exploratory factor analysis with oblimin rotation, internal consistency estimates (Cronbach alpha, McDonald omega), Pearson correlations, Steiger tests, and multiple regression models to assess convergent validity and prediction of outcome.
## Limitations
- The paper states that the sample size was relatively small and that the two treatments differed in small ways (for example, language and whether a technician was present), so measurement invariance had to be assumed but could not be formally tested.
- The single-session treatment format prevented testing cross-lagged and mediation models for symptom reduction.
- The study examined only client-reported alliance ratings; it did not include therapist, observer, or friend/family ratings.
- Evidence from the text also suggests a limitation: the virtual therapist was primarily voice-based and noninteractive, with limited embodiment, so findings may not generalize to other embodiment styles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
7 Valid · 1 Not valid · 50 Not checked · 53 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“In a systematic review of VRET process and outcome research, only two studies were identified that captured alliance information [15]: a study of fear of flying, but not an acrophobia study, found a significant correlation with outcome.”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Disputedcites [15] sourceExperimental — accuracy not yet measured
The Abstract says only one study explicitly addressed therapeutic alliance in VRET. The section “Therapeutic Alliance and Expectancy of Therapeutic Gains” explains that the two process studies examined different mechanisms—therapeutic alliance and treatment-outcome expectancy—not that both captured alliance information. It does support the sentence’s later distinction: alliance predicted outcome in the fear-of-flying group but not the acrophobia group. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, previous research suggests that treatments conducted without therapist support suffer from lower adherence and efficacy as compared to treatments that include contact [19,20].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Partly supportedcites [19] sourceExperimental — accuracy not yet measured
The Treatment Approach section says guided internet-delivered CBT tends to be more effective than self-guided CBT. The Evidence Base to Date section reports high dropout in early unguided studies, but does not directly compare their adherence with guided treatments, so the lower-adherence comparison is not established by this paper. (LLM, conf=0.85)
- Partly supportedcites [20] sourceExperimental — accuracy not yet measured
The Discussion (Implications for Future Work; page unavailable) reports that telephone and email reminders were likely important for retention and improvement, and that case management may support adherence. It does not establish that treatments without therapist support have lower adherence and efficacy than otherwise comparable treatments with therapist contact. (LLM, conf=0.84)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Few studies, however, have assessed alliance during VRET treatments, perhaps due to an expectation that alliance will be poor in treatments that prevent face-to-face contact with a therapist [9].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Supportedcites [15] source
The Abstract says only one study had explicitly addressed therapeutic alliance in VRET. The Concluding Remarks likewise describe research on alliance during VRET as scarce. Page numbers are not provided in our copy. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Alliance is considered a generic common factor, with meaning and usefulness expected to be preserved across theoretical orientations [2] and evidence suggests that a range of technology-based interventions can facilitate alliance [36].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Supportedcites [2] source
The Introduction describes alliance as a “common factor,” a generic variable that, at least in theory, has similar utility and meaning across theoretical orientations. The claim preserves that qualification by saying its meaning and usefulness are expected to carry across orientations. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“however, a high coefficient alpha is difficult to obtain in scales with few items, given its relationship to length (ie, 12 versus five items on factor 2) [49].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Discussion
What the sources say
- Supportedcites [49] source
In “Psychometric evaluation: An iterative process,” Clark and Watson state that coefficient alpha depends on scale length and average interitem correlation, and that many items make a high reliability estimate difficult to avoid. This supports the claim that obtaining a high alpha is harder with fewer items; the 12-versus-five-item comparison is the citing paper’s illustration, not a result reported by the source. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
36 more claims
Create a free account to see all 111 claims and to verify the 55 references still open.
- Probably not a substantial claim. 46
- It may state two separate claims as one. 43
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“and that the quality of this relationship predicted long-term improvements (Miloff et al. 2020).”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Miloff et al. 2020
The cited source says what the claim attributes to it.
In the citing paper
“Although initial research has revealed a correlation between alliance and outcomes [53]”
From Experiences of gamified and automated virtual reality exposure therapy for spider phobia: qualitative study (2020) · cites this paper as [53]
In the citing paper
“Precisely how common factors investigated in traditional psychotherapy translate into the mHealth context, particularly unguided mHealth, is an area of active research122,123.”
From A common factors perspective on mindfulness-based interventions (2022) · cites this paper as [123]
In the citing paper
“showing that a relationship similar to a working alliance does seem to form with the virtual therapist”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Miloff et al. 2020
In the citing paper
“Recent research has shown that users can develop a relationship similar to a working alliance with either the VR intervention itself [52] or a VR therapist [53].”
From Experiences of gamified and automated virtual reality exposure therapy for spider phobia: qualitative study (2020) · cites this paper as [53]
Citations
Cites
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238 · cited 1×
“Alliance is conceptualized as requiring three distinct processes: agreement on therapeutic goals, consensus on tasks that make up therapy, and bond between therapist and patient [3].”
BackgroundAn Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · cited 1×
“The System Usability Scale (SUS) is a 10-item scale for assessing user friendliness [43] and appropriateness of a tool for a given context [44]; it is considered technology agnostic, showing utility across a range of interface types, including graphical and speech-based systems.”
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520 · cited 16×
“Contrary to this expectation, studies of patients treated with VRET and in vivo exposure therapy for social anxiety disorder found no significant difference between interventions as measured by the WAI [10-12].”
BackgroundMethodInfluentialFactor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis
Hayton, J. C., Allen, D. G., & Scarpello, V. (2004) · Organizational Research Methods · doi:10.1177/1094428104263675 · Cited by 3,104 · cited 1×
“Parallel analysis [48] was used to extract factor loading of items using maximum likelihood explorative factor analysis with oblimin rotation.”
MethodThe Experience of Presence: Factor Analytic Insights
Schubert, T., Friedmann, F., & Regenbrecht, H. (2001) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474601300343603 · Cited by 2,303 · cited 6×
“Presence according to the IPQ, and the Gatineau Presence Questionnaire (GPQ) below, were taken as a measure of convergent validity with VTAS copresence items.”
MethodInfluentialThe alliance in adult psychotherapy: A meta-analytic synthesis
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018) · Psychotherapy · doi:10.1037/pst0000172 · Cited by 1,763 · cited 2×
“Nevertheless, overall results in this study compare favorably to current meta-analytic best evidence for the relationship between alliance and outcome [57].”
BackgroundResultFactor structure of the working alliance inventory
Tracey, T. J., & Kokotovic, A. M. (1989) · Psychological Assessment: A Journal of Consulting and Clinical Psychology · doi:10.1037/1040-3590.1.3.207 · Cited by 1,501 · cited 3×
“The WAI-VAR [31], however, was developed to address comfort and trust with the virtual environment itself, replacing “my therapist” with “virtual environment” for all items of the Spanish version of the WAI-S.”
BackgroundInfluentialConstructing validity: New developments in creating objective measuring instruments.
Clark, L. A., & Watson, D. (2019) · Psychological Assessment · doi:10.1037/pas0000626 · Cited by 1,470 · cited 6×
“As recommended by Clark and Watson [49], a sensitivity analysis was carried out that removed similarly cross-loading items, with the threshold set at 0.”
BackgroundMethodInfluentialVirtual Reality in the Assessment, Understanding, and Treatment of Mental Health Disorders
Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017) · Psychological Medicine · doi:10.1017/s003329171700040x · Cited by 1,428 · cited 2×
“Automated VRET treatments—replacing in vivo stimuli with virtual stimuli in addition to replacing human therapist with virtual therapist [16]—are an example of autonomous digital treatments, and being self-guided may be among the most scalable of psychological interventions [17,18].”
BackgroundDevelopment and validation of a revised short version of the working alliance inventory
Hatcher, R. L., & Gillaspy, J. A. (2006) · Psychotherapy Research · doi:10.1080/10503300500352500 · Cited by 1,355 · cited 2×
“Inspiration for VTAS items came primarily from the WAI 12-item short report [37] and 32-item long report [1], covering dimensions of bond, task, and goal.”
BackgroundMethodPersuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308 · cited 1×
“There is also evidence that more contact and better contact (ie, frequency of interactions and persuasive design elements, such as dialogue support) may result in more patients completing treatment [21].”
BackgroundThe role of the therapeutic alliance in psychotherapy.
Horvath, A. O., & Luborsky, L. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.561 · Cited by 1,279 · cited 1×
“Alliance is also understood to occur in stages, first with identifying the therapist as a source of encouragement, warmth, and support and, in later stages, the development of a client’s faith and investment in the therapeutic process, a collaborative working together [4].”
BackgroundInterpreting the magnitudes of correlation coefficients.
Hemphill, J. F. (2003) · American Psychologist · doi:10.1037/0003-066x.58.1.78 · Cited by 1,237 · cited 2×
“50; see Cohen 1988 as cited in Hemphill [51]; see Table 2).”
BackgroundAdherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · cited 1×
“However, previous research suggests that treatments conducted without therapist support suffer from lower adherence and efficacy as compared to treatments that include contact [19,20].”
BackgroundA Systematic Review of Social Presence: Definition, Antecedents, and Implications
Oh, C. S., Bailenson, J. N., & Welch, G. F. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00114 · Cited by 1,032
Carl, E., Stein, A. T., Levihn-Coon, A., Pogue, J. R., Rothbaum, B., Emmelkamp, P., Asmundson, G. J. G., Carlbring, P., & Powers, M. B. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.08.003 · Cited by 889 · cited 1×
“Virtual reality exposure therapy (VRET) is a technology-based method of delivering exposure-based treatments; it has been tested in clinical trials for some 20 years, with positive results [6], but has only recently become available to consumers [7].”
MethodSelf-citationInterpersonal Distance in Immersive Virtual Environments
Bailenson, J. N., Blascovich, J., Beall, A. C., & Loomis, J. M. (2003) · Personality and Social Psychology Bulletin · doi:10.1177/0146167203029007002 · Cited by 807 · cited 1×
“The Social Presence Survey [40] was used to construct items relevant to work with a virtual therapist.”
MethodInternet-delivered psychological treatments: from innovation to implementation
Andersson, G., Titov, N., Dear, B. F., Rozental, A., & Carlbring, P. (2019) · World Psychiatry · doi:10.1002/wps.20610 · Cited by 750 · cited 1×
“However, previous research suggests that treatments conducted without therapist support suffer from lower adherence and efficacy as compared to treatments that include contact [19,20].”
BackgroundSelf-citationFear of spiders’ questionnaire
Szymanski, J., & O'Donohue, W. (1995) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)00072-t · Cited by 450 · cited 8×
“Of the four covariates, user friendliness, presence, and follow-up FSQ change scores were all significantly associated with the VTAS.”
MethodInfluentialThe impact of digital technology on psychological treatments and their dissemination
Fairburn, C. G., & Patel, V. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.08.012 · Cited by 448 · cited 1×
“Automated VRET treatments—replacing in vivo stimuli with virtual stimuli in addition to replacing human therapist with virtual therapist [16]—are an example of autonomous digital treatments, and being self-guided may be among the most scalable of psychological interventions [17,18].”
BackgroundEmbodied conversational agents in clinical psychology: a scoping review
Provoost, S., Lau, H. M., Ruwaard, J., & Riper, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6553 · Cited by 433 · cited 1×
“Therefore, simulating key elements of face-to-face therapist interaction with a virtual agent manifesting visually and/or through auditory instruction may improve adherence and bridge the efficacy gap between guided and unguided interventions [22].”
BackgroundThe Revised Helping Alliance Questionnaire (HAq-II) : Psychometric Properties.
(1996) · PubMed Central · Cited by 394
Freeman, D., Haselton, P., Freeman, J., Spanlang, B., Kishore, S., Albery, E., Denne, M., Brown, P., Slater, M., & Nickless, A. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30226-8 · Cited by 377 · cited 2×
“To date, three automated VRET treatments have been tested in clinical trials: two for use with acrophobia [26,27] and one for use with spider phobia [28].”
BackgroundMethodVirtual reality exposure therapy for social anxiety disorder: A randomized controlled trial.
Anderson, P. L., Price, M., Edwards, S. M., Obasaju, M. A., Schmertz, S. K., Zimand, E., & Calamaras, M. R. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033559 · Cited by 371 · cited 16×
“Contrary to this expectation, studies of patients treated with VRET and in vivo exposure therapy for social anxiety disorder found no significant difference between interventions as measured by the WAI [10-12].”
ResultInfluentialBerger, T. (2017) · Psychotherapy Research · doi:10.1080/10503307.2015.1119908 · Cited by 364 · cited 2×
“A recent narrative review of therapeutic alliance in Internet interventions found no other study that identified a relationship between the bond-factor and outcome, but many with a meaningful and statistically significant relationship between task-goal and outcome [9].”
BackgroundBouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, É., Forget, H., Loranger, C., & Roucaut, F. X. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.184234 · Cited by 354 · cited 16×
“Contrary to this expectation, studies of patients treated with VRET and in vivo exposure therapy for social anxiety disorder found no significant difference between interventions as measured by the WAI [10-12].”
ResultInfluentialReporting Mental Health Symptoms: Breaking Down Barriers to Care with Virtual Human Interviewers
Lucas, G. M., Rizzo, A., Gratch, J., Scherer, S., Stratou, G., Boberg, J., & Morency, L.-P. (2017) · Frontiers in Robotics and AI · doi:10.3389/frobt.2017.00051 · Cited by 308
Meyerbröker, K., & Emmelkamp, P. M. G. (2010) · Depression and Anxiety · doi:10.1002/da.20734 · Cited by 290 · cited 1×
“In a systematic review of VRET process and outcome research, only two studies were identified that captured alliance information [15]: a study of fear of flying, but not an acrophobia study, found a significant correlation with outcome.”
BackgroundTaking the time to care: empowering low health literacy hospital patients with virtual nurse agents
Bickmore, T. W., Pfeifer, L. M., & Jack, B. W. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518891 · Cited by 289 · cited 2×
“Relational behaviors, such as calling the patient by name, use of humor when appropriate, including information discussed during past interactions, and engaging in a social chat at the beginning of a new interaction, have all been shown to increase how much a patient feels cared for and how useful the information they…”
BackgroundBaus, O., & Bouchard, S. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00112 · Cited by 260 · cited 1×
“Using augmented reality exposure therapy (ARET), a related technology [13], the same lack of difference in WAI scores was found following treatment for small-animal phobia [14].”
BackgroundFeatures and functionality of the Iterapi platform for internet-based psychological treatment
Vlaescu, G., Alasjö, A., Miloff, A., Carlbring, P., & Andersson, G. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.09.006 · Cited by 229 · cited 1×
“Outcome data were collected using an online tool [41] for all follow-up in the second study and for follow-up in the first study only if a participant was unavailable to meet in person.”
MethodSelf-citationLing, Y., Nefs, H. T., Morina, N., Heynderickx, I., & Brinkman, W.-P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096144 · Cited by 215
Lindner, P., Miloff, A., Hamilton, W., Reuterskiöld, L., Andersson, G., Powers, M. B., & Carlbring, P. (2017) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1280843 · Cited by 187 · cited 1×
“Virtual reality exposure therapy (VRET) is a technology-based method of delivering exposure-based treatments; it has been tested in clinical trials for some 20 years, with positive results [6], but has only recently become available to consumers [7].”
MethodSelf-citationDonker, T., Cornelisz, I., van Klaveren, C., van Straten, A., Carlbring, P., Cuijpers, P., & van Gelder, J.-L. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0219 · Cited by 187 · cited 2×
“In the study by Donker et al [26], patients were given background information about phobias, provided case examples, and given motivation using a 2D, animated, mobile phone app-based avatar called Tara.”
MethodSelf-citationThe factor structure of the working alliance inventory in cognitive-behavioral therapy.
Andrusyna, T. P., Tang, T., DeRubeis, R. J., & Luborsky, L. B. (2001) · PubMed Central · Cited by 174
Miloff, A., Lindner, P., Dafgård, P., Deak, S., Garke, M., Hamilton, W., Heinsoo, J., Kristoffersson, G., Rafi, J., Sindemark, K., et al. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.04.004 · Cited by 155 · cited 7×
“We designed our automated treatment for spider phobia [28] with a virtual therapist to deliver guidance and support from within the VRET application using primarily voice-based instructions and, in one version, a graphical representation.”
BackgroundMethodInfluentialSelf-citationTechnology-Based Mental Health Treatment and the Impact on the Therapeutic Alliance
Lopez, A., Schwenk, S., Schneck, C. D., Griffin, R. J., & Mishkind, M. C. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1055-7 · Cited by 140 · cited 1×
“Alliance is considered a generic common factor, with meaning and usefulness expected to be preserved across theoretical orientations [2] and evidence suggests that a range of technology-based interventions can facilitate alliance [36].”
BackgroundResearch on the alliance: Knowledge in search of a theory
Horvath, A. O. (2018) · Psychotherapy Research · doi:10.1080/10503307.2017.1373204 · Cited by 137 · cited 3×
“Some alliance measures, such as the Helping Alliance Questionnaire (HAq) [5], are an authoritative operationalization of their authors’ key theories; however, the boundaries of the popular WAI were never formalized [2] and researchers continue to refine the measure as they seek to develop a holistic understanding of a…”
BackgroundFalkenström, F., Ekeblad, A., & Holmqvist, R. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000119 · Cited by 103 · cited 1×
“The use of a single-session treatment format prevented the exploration of cross-lagged and mediation models to predict symptom reduction following a session [58].”
BackgroundBerry, K., Salter, A., Morris, R., James, S., & Bucci, S. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8252 · Cited by 103 · cited 2×
“Relational bond between patient and mobile self-help apps was explored recently in a mixed qualitative-quantitative study [33].”
BackgroundThe Working Alliance Inventory for guided Internet interventions (WAI‐I)
Gómez Penedo, J. M., Berger, T., grosse Holtforth, M., Krieger, T., Schröder, J., Hohagen, F., Meyer, B., Moritz, S., & Klein, J. P. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.22823 · Cited by 100 · cited 1×
“A recently evaluated WAI scale for guided Internet-based treatments had a similar task-goal and bond two-factor solution [53].”
BackgroundFalkenström, F., Hatcher, R. L., & Holmqvist, R. (2015) · Assessment · doi:10.1177/1073191114552472 · Cited by 96 · cited 1×
“Falkenström, Hatcher, and Holmqvist [52] conducted a confirmatory factor analysis of the scale among three large samples in Sweden and the United States and argued that, given the high intercorrelation between task and goal factors, a two-factor solution was more defensible psychometrically.”
BackgroundSimCoach: an intelligent virtual human system for providing healthcare information and support
Rizzo, A., Lange, B., Buckwalter, J. G., Forbell, E., Kim, J., Sagae, K., Williams, J., Difede, J., Rothbaum, B. O., Reger, G., et al. (2011) · International Journal on Disability and Human Development · doi:10.1515/ijdhd.2011.046 · Cited by 94 · cited 1×
“Previous research on embodied graphically generated agents have included SimCoach [61], which used rich visual representations of humans, gestures, and emotions, as well as natural conversational speech to comfort, guide, and motivate armed service members toward psychological health and to solicit anonymous informati…”
Background(Inter)personal Computing: The Role of the Therapeutic Relationship in E-mental Health
Cavanagh, K., & Millings, A. (2013) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-013-9242-z · Cited by 88 · cited 1×
“This is particularly so if they are successful in replicating common factors in therapy and nonspecific relational elements, such as empathy and therapist attention [23], which some have suggested are better conceived of as an active ingredient and should be intentionally used in treatment to facilitate better outcome…”
BackgroundVîslă, A., Constantino, M. J., Newkirk, K., Ogrodniczuk, J. S., & Söchting, I. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1218089 · Cited by 83 · cited 1×
“Heim et al [30] suggest this may be possible through improved credibility and expectancy, which mediation models indicate function in a bidirectional relationship with alliance to improve outcomes [60].”
BackgroundComputerized cognitive behavioural therapy and the therapeutic alliance: A qualitative enquiry
Barazzone, N., Cavanagh, K., & Richards, D. A. (2012) · British Journal of Clinical Psychology · doi:10.1111/j.2044-8260.2012.02035.x · Cited by 67 · cited 1×
“The SUS was taken as a measure of convergent validity with bond and empathy; the SUS captured characteristics similar to alliance common factors, such as accessibility, empowerment, guidance, and delivery of a sense of security [45].”
MethodAn Intelligent Virtual Human System for Providing Healthcare Information and Support
Rizzo Albert, A., Belinda, L., Buckwalter John, G., Eric, F., Julia, K., Kenji, S., Josh, W., Rothbaum Barbara, O., JoAnn, D., Greg, R., et al. (2011) · Studies in Health Technology and Informatics · doi:10.3233/978-1-60750-706-2-503 · Cited by 67
Tielman, M. L., Neerincx, M. A., van Meggelen, M., Franken, I., & Brinkman, W.-P. (2018) · Technology and Health Care · doi:10.3233/thc-170899 · Cited by 63
Laforest, M., Bouchard, S., Crétu, A.-M., & Mesly, O. (2016) · Frontiers in ICT · doi:10.3389/fict.2016.00018 · Cited by 60
Working alliance as a predictor of change in depression during blended cognitive behaviour therapy
Vernmark, K., Hesser, H., Topooco, N., Berger, T., Riper, H., Luuk, L., Backlund, L., Carlbring, P., & Andersson, G. (2019) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1533577 · Cited by 59 · cited 1×
“Finally, this study explored only client-reported ratings of alliance, versus therapist, observer, and friend or family ratings, whereas other studies, such as a recent blended Internet-based CBT treatment for depression, found only therapist-rated alliance results to be significant with outcome [59].”
ResultSelf-citationExploring users’ social responses to computer counseling interviewers’ behavior
Kang, S., & Gratch, J. S. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.01.006 · Cited by 58 · cited 1×
“Disclosure of personal backstories and intimate personal details by an automated therapist has been shown to increase self-disclosure by highly anxious patients [63].”
BackgroundZilcha-Mano, S., Roose, S. P., Brown, P. J., & Rutherford, B. R. (2019) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2018.00293 · Cited by 53 · cited 1×
“This is particularly so if they are successful in replicating common factors in therapy and nonspecific relational elements, such as empathy and therapist attention [23], which some have suggested are better conceived of as an active ingredient and should be intentionally used in treatment to facilitate better outcome…”
BackgroundLindner, P., Miloff, A., Hamilton, W., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00132 · Cited by 47 · cited 1×
“Automated VRET treatments—replacing in vivo stimuli with virtual stimuli in addition to replacing human therapist with virtual therapist [16]—are an example of autonomous digital treatments, and being self-guided may be among the most scalable of psychological interventions [17,18].”
BackgroundSelf-citationWorking alliance with an avatar: How far can we go with internet interventions?
Heim, E., Rötger, A., Lorenz, N., & Maercker, A. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.01.005 · Cited by 41 · cited 7×
“It is conceivable that bond is strengthened in interventions with an embodied avatar as compared to text-based, self-help applications [35] or, as suggested by Heim et al [30], bond may occur earlier in treatment.”
BackgroundResultInfluentialMcClintock, A. S., Anderson, T., Patterson, C. L., & Wing, E. H. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22568 · Cited by 40
Ngai, I., Tully, E. C., & Anderson, P. L. (2015) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s135246581300088x · Cited by 35 · cited 16×
“Contrary to this expectation, studies of patients treated with VRET and in vivo exposure therapy for social anxiety disorder found no significant difference between interventions as measured by the WAI [10-12].”
ResultInfluentialMiragall, M., Baños, R. M., Cebolla, A., & Botella, C. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01531 · Cited by 29
A Virtual Therapist That Responds Empathically to Your Answers
Pontier, M. A., & Siddiqui, G. F. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85483-8_42 · Cited by 28 · cited 2×
“Therefore, categories for measuring empathic behavior from a study entitled A Virtual Therapist That Responds Empathically to Your Answers [34] were used to construct novel items.”
BackgroundWrzesien, M., Bretón-López, J., Botella, C., Burkhardt, J.-M., Alcañiz, M., Pérez-Ara, M. Á., & del Amo, A. R. (2013) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465813000088 · Cited by 26 · cited 1×
“Using augmented reality exposure therapy (ARET), a related technology [13], the same lack of difference in WAI scores was found following treatment for small-animal phobia [14].”
BackgroundInfluentialVirtual Reality and Other Realities
Botella, C., Baños, R. M., García-Palacios, A., & Quero, S. (2017) · The Science of Cognitive Behavioral Therapy · doi:10.1016/b978-0-12-803457-6.00022-2 · Cited by 16
Research works printed but not linked: 5
- Luborsky, L, 1996, J Psychother Pract Res, 5, 260
- HartantoDBrinkmanWPKampmannIlMorinaNEmmelkampPGNeerincxMAHome-based virtual reality exposure therapy with virtual health agent supportProceedings of the International Symposium on Pervasive Computing Paradigms for Mental Health, MindCare 20152016International Symposium on Pervasive Computing Paradigms for Mental Health, MindCare 2015September 24-25, 2015Milan, ItalyCham, SwitzerlandSpringer International Publishing8598
- Andrusyna, TP, 2001, J Psychother Pract Res, 10, 173
- Law, M, 2015, J Acad Bus Ethics, 10, 1
- Bangor, A, 2009, J Usability Stud, 4, 114
Cited by
Dincelli, E., & Yayla, A. (2022) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2022.101717 · Cited by 490
Artificial intelligence technologies and compassion in healthcare: A systematic scoping review
Morrow, E., Zidaru, T., Ross, F., Mason, C., Patel, K. D., Ream, M., & Stockley, R. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.971044 · Cited by 250
“← Automated VR exposure therapies (VRETs) for patient adherence and efficacy of self-guided treatments (Brandt et al., 2018; Miloff et al., 2020)”
BackgroundVirtual reality in the diagnostic and therapy for mental disorders: A systematic review
Wiebe, A., Kannen, K., Selaskowski, B., Mehren, A., Thöne, A.-K., Pramme, L., Blumenthal, N., Li, M., Asché, L., Jonas, S., et al. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102213 · Cited by 223
A new era in Internet interventions: The advent of Chat-GPT and AI-assisted therapist guidance
Carlbring, P., Hadjistavropoulos, H., Kleiboer, A., & Andersson, G. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100621 · Cited by 118
Self-citationBetter, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy
Lindner, P. (2021) · International Journal of Cognitive Therapy · doi:10.1007/s41811-020-00090-7 · Cited by 120
Self-citation
Metadata
- Authors
- Miloff, A., Carlbring, P., Hamilton, W., Andersson, G., Reuterskiöld, L., & Lindner, P.
- Affiliations
- Stockholm University (SE)
- Mimerse (Sweden) (SE)
- Linköping University (SE)
- Karolinska Institutet (SE)
- Corresponding author
- Miloff, Alexander
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-03-24
- Volume
- 22
- Issue
- 3
- Pages
- e16660
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7139418
- OpenAlex
- W3012939091
- Semantic Scholar
- 9ada8fdce5d8438189224235500df93001408030
- Cited by
- 60
- References
- 61
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 10 · 2023: 14 · 2022: 12 · 2021: 7 · 2020: 3
- FWCI
- 3.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- virtual reality exposure therapy, spider phobia, treatment outcome prediction, co-presence, presence, ease of use, scale development
- MeSH terms
- Factor Analysis, Statistical, Female, Humans, Male, Psychometrics, Surveys and Questionnaires, Treatment Outcome, Telemedicine, Virtual Reality Exposure Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.