The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index; 10 of the 65 sources its claims cite were checked for support.
Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation
Open accessAbstract
Background: Automated virtual reality exposure therapies (VRETs) are self-help treatments conducted by oneself and supported by a virtual therapist embodied visually and/or with audio feedback. This simulates many of the nonspecific relational elements and common factors present in face-to-face therapy and may be a means of improving adherence to and efficacy of self-guided treatments. However, little is known about alliance toward the virtual therapist, despite alliance being an important predictor of treatment outcome. Objective: In this study, we aimed to evaluate the first alliance instrument developed for use with embodied virtual therapists in an automated treatment format—the Virtual Therapist Alliance Scale (VTAS)—by (1) assessing its psychometric properties, (2) verifying the dimensionality of the scale, and (3) determining the predictive ability of the scale with treatment outcome. Methods: A psychometric evaluation and exploratory factor analysis of the VTAS was conducted using data from two samples of spider-fearful patients treated with VRET and the help of an embodied, voice-based virtual therapist (n=70). Multiple regression models and bivariate correlations were used to assess the VTAS relationship with treatment outcome, according to self-reported fear and convergence with presence and user-friendliness process measures. Results: The VTAS showed a sound two-factor solution composed of a primary factor covering task, goal, and copresence; adequate internal consistency; and good convergent validity, including moderate correlation (r=.310, P=.01) with outcomes over follow-up. Conclusions: These preliminary results suggest that alliance toward a virtual therapist is a significant predictor of treatment outcome, favors the importance of a task-goal over bond-factor, and should be explored in studies with larger sample sizes and in additional forms of embodiment.
Overview
CoChat Summary
## TL;DR
This study developed and psychometrically evaluated the Virtual Therapist Alliance Scale (VTAS) for automated virtual reality exposure therapy (VRET) with embodied virtual therapists. In two spider-fearful patient samples, the VTAS showed a two-factor structure and good overall reliability, and alliance scores were positively related to follow-up outcome.
## Key findings
- The VTAS had a 17-item format and showed a two-factor solution: factor 1 covered task, goal, and copresence items, and factor 2 covered bond and empathy items.
- Internal consistency was high for the full scale: Cronbach alpha = .921 and McDonald omega = .921 overall; by study, alpha/omega were .915/.917 and .936/.938.
- Factor 1 explained 38.8% of total variance and factor 2 explained 14.0%.
- VTAS total scores correlated with outcome at follow-up: the abstract reports r = .310, P = .01, while the discussion describes a posttreatment correlation of r = .213 and a follow-up correlation of r = .310.
## Methods
- Psychometric evaluation and exploratory factor analysis of the VTAS were conducted using data from two independent samples of spider-fearful patients treated with automated VRET and an embodied, voice-based virtual therapist.
- The combined final sample with VTAS data was 70 participants; the two studies had 50 eligible and 25 eligible participants, respectively, and were conducted in a university laboratory environment.
- Analyses included maximum likelihood exploratory factor analysis with oblimin rotation, internal consistency estimates (Cronbach alpha, McDonald omega), Pearson correlations, Steiger tests, and multiple regression models to assess convergent validity and prediction of outcome.
## Limitations
- The paper states that the sample size was relatively small and that the two treatments differed in small ways (for example, language and whether a technician was present), so measurement invariance had to be assumed but could not be formally tested.
- The single-session treatment format prevented testing cross-lagged and mediation models for symptom reduction.
- The study examined only client-reported alliance ratings; it did not include therapist, observer, or friend/family ratings.
- Evidence from the text also suggests a limitation: the virtual therapist was primarily voice-based and noninteractive, with limited embodiment, so findings may not generalize to other embodiment styles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
7 Valid · 1 Not valid · 50 Not checked · 53 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“In a systematic review of VRET process and outcome research, only two studies were identified that captured alliance information [15]: a study of fear of flying, but not an acrophobia study, found a significant correlation with outcome.”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Disputedcites [15] sourceExperimental — accuracy not yet measured
The Abstract says only one study explicitly addressed therapeutic alliance in VRET. The section “Therapeutic Alliance and Expectancy of Therapeutic Gains” explains that the two process studies examined different mechanisms—therapeutic alliance and treatment-outcome expectancy—not that both captured alliance information. It does support the sentence’s later distinction: alliance predicted outcome in the fear-of-flying group but not the acrophobia group. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, previous research suggests that treatments conducted without therapist support suffer from lower adherence and efficacy as compared to treatments that include contact [19,20].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Partly supportedcites [19] sourceExperimental — accuracy not yet measured
The Treatment Approach section says guided internet-delivered CBT tends to be more effective than self-guided CBT. The Evidence Base to Date section reports high dropout in early unguided studies, but does not directly compare their adherence with guided treatments, so the lower-adherence comparison is not established by this paper. (LLM, conf=0.85)
- Partly supportedcites [20] sourceExperimental — accuracy not yet measured
The Discussion (Implications for Future Work; page unavailable) reports that telephone and email reminders were likely important for retention and improvement, and that case management may support adherence. It does not establish that treatments without therapist support have lower adherence and efficacy than otherwise comparable treatments with therapist contact. (LLM, conf=0.84)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Few studies, however, have assessed alliance during VRET treatments, perhaps due to an expectation that alliance will be poor in treatments that prevent face-to-face contact with a therapist [9].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Supportedcites [15] source
The Abstract says only one study had explicitly addressed therapeutic alliance in VRET. The Concluding Remarks likewise describe research on alliance during VRET as scarce. Page numbers are not provided in our copy. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Alliance is considered a generic common factor, with meaning and usefulness expected to be preserved across theoretical orientations [2] and evidence suggests that a range of technology-based interventions can facilitate alliance [36].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Introduction
What the sources say
- Supportedcites [2] source
The Introduction describes alliance as a “common factor,” a generic variable that, at least in theory, has similar utility and meaning across theoretical orientations. The claim preserves that qualification by saying its meaning and usefulness are expected to carry across orientations. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“however, a high coefficient alpha is difficult to obtain in scales with few items, given its relationship to length (ie, 12 versus five items on factor 2) [49].”
§ Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation > Discussion
What the sources say
- Supportedcites [49] source
In “Psychometric evaluation: An iterative process,” Clark and Watson state that coefficient alpha depends on scale length and average interitem correlation, and that many items make a high reliability estimate difficult to avoid. This supports the claim that obtaining a high alpha is harder with fewer items; the 12-versus-five-item comparison is the citing paper’s illustration, not a result reported by the source. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
36 more claims
Create a free account to see all 111 claims and to verify the 55 references still open.
- Probably not a substantial claim. 46
- It may state two separate claims as one. 43
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“and that the quality of this relationship predicted long-term improvements (Miloff et al. 2020).”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Miloff et al. 2020
The cited source says what the claim attributes to it.
In the citing paper
“Although initial research has revealed a correlation between alliance and outcomes [53]”
From Experiences of gamified and automated virtual reality exposure therapy for spider phobia: qualitative study (2020) · cites this paper as [53]
In the citing paper
“Precisely how common factors investigated in traditional psychotherapy translate into the mHealth context, particularly unguided mHealth, is an area of active research122,123.”
From A common factors perspective on mindfulness-based interventions (2022) · cites this paper as [123]
In the citing paper
“showing that a relationship similar to a working alliance does seem to form with the virtual therapist”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Miloff et al. 2020
In the citing paper
“Recent research has shown that users can develop a relationship similar to a working alliance with either the VR intervention itself [52] or a VR therapist [53].”
From Experiences of gamified and automated virtual reality exposure therapy for spider phobia: qualitative study (2020) · cites this paper as [53]
Citations
Cites
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238 · cited 1×
“Alliance is conceptualized as requiring three distinct processes: agreement on therapeutic goals, consensus on tasks that make up therapy, and bond between therapist and patient [3].”
BackgroundAn Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · cited 1×
“The System Usability Scale (SUS) is a 10-item scale for assessing user friendliness [43] and appropriateness of a tool for a given context [44]; it is considered technology agnostic, showing utility across a range of interface types, including graphical and speech-based systems.”
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520 · cited 16×
“Contrary to this expectation, studies of patients treated with VRET and in vivo exposure therapy for social anxiety disorder found no significant difference between interventions as measured by the WAI [10-12].”
BackgroundMethodInfluentialFactor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis
Hayton, J. C., Allen, D. G., & Scarpello, V. (2004) · Organizational Research Methods · doi:10.1177/1094428104263675 · Cited by 3,104 · cited 1×
“Parallel analysis [48] was used to extract factor loading of items using maximum likelihood explorative factor analysis with oblimin rotation.”
MethodThe Experience of Presence: Factor Analytic Insights
Schubert, T., Friedmann, F., & Regenbrecht, H. (2001) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474601300343603 · Cited by 2,303 · cited 6×
“Presence according to the IPQ, and the Gatineau Presence Questionnaire (GPQ) below, were taken as a measure of convergent validity with VTAS copresence items.”
MethodInfluentialThe alliance in adult psychotherapy: A meta-analytic synthesis
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018) · Psychotherapy · doi:10.1037/pst0000172 · Cited by 1,763 · cited 2×
“Nevertheless, overall results in this study compare favorably to current meta-analytic best evidence for the relationship between alliance and outcome [57].”
BackgroundResultFactor structure of the working alliance inventory
Tracey, T. J., & Kokotovic, A. M. (1989) · Psychological Assessment: A Journal of Consulting and Clinical Psychology · doi:10.1037/1040-3590.1.3.207 · Cited by 1,501 · cited 3×
“The WAI-VAR [31], however, was developed to address comfort and trust with the virtual environment itself, replacing “my therapist” with “virtual environment” for all items of the Spanish version of the WAI-S.”
BackgroundInfluentialConstructing validity: New developments in creating objective measuring instruments.
Clark, L. A., & Watson, D. (2019) · Psychological Assessment · doi:10.1037/pas0000626 · Cited by 1,470 · cited 6×
“As recommended by Clark and Watson [49], a sensitivity analysis was carried out that removed similarly cross-loading items, with the threshold set at 0.”
BackgroundMethodInfluentialVirtual Reality in the Assessment, Understanding, and Treatment of Mental Health Disorders
Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017) · Psychological Medicine · doi:10.1017/s003329171700040x · Cited by 1,428 · cited 2×
“Automated VRET treatments—replacing in vivo stimuli with virtual stimuli in addition to replacing human therapist with virtual therapist [16]—are an example of autonomous digital treatments, and being self-guided may be among the most scalable of psychological interventions [17,18].”
Background
Research works printed but not linked: 5
- Luborsky, L, 1996, J Psychother Pract Res, 5, 260
- HartantoDBrinkmanWPKampmannIlMorinaNEmmelkampPGNeerincxMAHome-based virtual reality exposure therapy with virtual health agent supportProceedings of the International Symposium on Pervasive Computing Paradigms for Mental Health, MindCare 20152016International Symposium on Pervasive Computing Paradigms for Mental Health, MindCare 2015September 24-25, 2015Milan, ItalyCham, SwitzerlandSpringer International Publishing8598
- Andrusyna, TP, 2001, J Psychother Pract Res, 10, 173
- Law, M, 2015, J Acad Bus Ethics, 10, 1
- Bangor, A, 2009, J Usability Stud, 4, 114
Cited by
Dincelli, E., & Yayla, A. (2022) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2022.101717 · Cited by 490
Artificial intelligence technologies and compassion in healthcare: A systematic scoping review
Morrow, E., Zidaru, T., Ross, F., Mason, C., Patel, K. D., Ream, M., & Stockley, R. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.971044 · Cited by 250
“← Automated VR exposure therapies (VRETs) for patient adherence and efficacy of self-guided treatments (Brandt et al., 2018; Miloff et al., 2020)”
BackgroundVirtual reality in the diagnostic and therapy for mental disorders: A systematic review
Wiebe, A., Kannen, K., Selaskowski, B., Mehren, A., Thöne, A.-K., Pramme, L., Blumenthal, N., Li, M., Asché, L., Jonas, S., et al. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102213 · Cited by 223
A new era in Internet interventions: The advent of Chat-GPT and AI-assisted therapist guidance
Carlbring, P., Hadjistavropoulos, H. D., Kleiboer, A. M., & Andersson, G. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100621 · Cited by 157
Self-citationBetter, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy
Lindner, P. (2021) · International Journal of Cognitive Therapy · doi:10.1007/s41811-020-00090-7 · Cited by 120
Self-citation
Metadata
- Authors
- Miloff, A., Carlbring, P., Hamilton, W., Andersson, G., Reuterskiöld, L., & Lindner, P.
- Affiliations
- Stockholm University (SE)
- Mimerse (Sweden) (SE)
- Linköping University (SE)
- Karolinska Institutet (SE)
- Corresponding author
- Miloff, Alexander
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-03-24
- Volume
- 22
- Issue
- 3
- Pages
- e16660
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7139418
- OpenAlex
- W3012939091
- Semantic Scholar
- 9ada8fdce5d8438189224235500df93001408030
- Cited by
- 60
- References
- 61
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 10 · 2023: 14 · 2022: 12 · 2021: 7 · 2020: 3
- FWCI
- 3.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- virtual reality exposure therapy, spider phobia, treatment outcome prediction, co-presence, presence, ease of use, scale development
- MeSH terms
- Factor Analysis, Statistical, Female, Humans, Male, Psychometrics, Surveys and Questionnaires, Treatment Outcome, Telemedicine, Virtual Reality Exposure Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation. https://app.cochat.ai/papers/10.2196/16660
- The paper
- Miloff, A., Carlbring, P., Hamilton, W., Andersson, G., Reuterskiöld, L., & Lindner, P. (2020). Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation. Journal of Medical Internet Research. https://doi.org/10.2196/16660
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.