Improving User Experience of Virtual Health Assistants: Scoping Review
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Background: Virtual assistants can be used to deliver innovative health programs that provide appealing, personalized, and convenient health advice and support at scale and low cost. Design characteristics that influence the look and feel of the virtual assistant, such as visual appearance or language features, may significantly influence users’ experience and engagement with the assistant. Objective: This scoping review aims to provide an overview of the experimental research examining how design characteristics of virtual health assistants affect user experience, summarize research findings of experimental research examining how design characteristics of virtual health assistants affect user experience, and provide recommendations for the design of virtual health assistants if sufficient evidence exists. Methods: We searched 5 electronic databases (Web of Science, MEDLINE, Embase, PsycINFO, and ACM Digital Library) to identify the studies that used an experimental design to compare the effects of design characteristics between 2 or more versions of an interactive virtual health assistant on user experience among adults. Data were synthesized descriptively. Health domains, design characteristics, and outcomes were categorized, and descriptive statistics were used to summarize the body of research. Results for each study were categorized as positive, negative, or no effect, and a matrix of the design characteristics and outcome categories was constructed to summarize the findings. Results: The database searches identified 6879 articles after the removal of duplicates. We included 48 articles representing 45 unique studies in the review. The most common health domains were mental health and physical activity. Studies most commonly examined design characteristics in the categories of visual design or conversational style and relational behavior and assessed outcomes in the categories of personality, satisfaction, relationship, or use intention. Over half of the design characteristics were examined by only 1 study. Results suggest that empathy and relational behavior and self-disclosure are related to more positive user experience. Results also suggest that if a human-like avatar is used, realistic rendering and medical attire may potentially be related to more positive user experience; however, more research is needed to confirm this. Conclusions: There is a growing body of scientific evidence examining the impact of virtual health assistants’ design characteristics on user experience. Taken together, data suggest that the look and feel of a virtual health assistant does affect user experience. Virtual health assistants that show empathy, display nonverbal relational behaviors, and disclose personal information about themselves achieve better user experience. At present, the evidence base is broad, and the studies are typically small in scale and highly heterogeneous. Further research, particularly using longitudinal research designs with repeated user interactions, is needed to inform the optimal design of virtual health assistants.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0611-x · Cited by 11,385
Machines and Mindlessness: Social Responses to Computers
Nass, C., & Moon, Y. (2000) · Journal of Social Issues · doi:10.1111/0022-4537.00153 · Cited by 4,022 · cited 1×
“The finding that users respond positively to the autobiographical stories of a virtual health assistant supports the computers are social actors paradigm, where users display social responses to computers, although they know they are not human [73,74].”
BackgroundThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931 · cited 1×
“Eligibility criteria were designed using the population, intervention, comparator, and outcome framework (population: adults; intervention: virtual health assistant; comparator: design characteristics; and outcome: user experience) [18].”
MethodFitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×
“The finding that users respond positively to the autobiographical stories of a virtual health assistant supports the computers are social actors paradigm, where users display social responses to computers, although they know they are not human [73,74].”
Establishing and maintaining long-term human-computer relationships
Bickmore, T. W., & Picard, R. W. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1067860.1067867 · Cited by 1,333 · cited 4×
“Although some effects were nonsignificant [27,40,48,49,51], more effects were positive or mixed positive, with 71% (5/7) of studies showing at least some positive effect [27,31,39,48,51].”
BackgroundInfluentialThe Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review
Milne-Ives, M., de Cock, C., Lim, E., Shehadeh, M. H., de Pennington, N., Mole, G., Normando, E., & Meinert, E. (2020) · Journal of Medical Internet Research · doi:10.2196/20346 · Cited by 626 · cited 1×
“Virtual assistants using natural language processing have been commonly used in health care [77] and, with rapid advancements in artificial intelligence, are likely to become increasingly sophisticated.”
MethodNass, C., Steuer, J., & Tauber, E. R. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260288 · Cited by 579
Spoken dialogue technology: enabling the conversational user interface
McTear, M. F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505285 · Cited by 571
Establishing the computer-patient working alliance in automated health behavior change interventions
Bickmore, T., Gruber, A., & Picard, R. (2005) · Patient Education and Counseling · doi:10.1016/j.pec.2004.09.008 · Cited by 494 · cited 1×
“Multimedia Appendix 2 [21-68] provides an overview of the participant characteristics and study designs for all studies included in the review.”
BackgroundShould Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot
Liu, B., & Sundar, S. S. (2018) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2018.0110 · Cited by 447
"I Hear You, I Feel You": Encouraging Deep Self-disclosure through a Chatbot
Lee, Y.-C., Yamashita, N., Huang, Y., & Fu, W. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376175 · Cited by 339 · cited 2×
“Results were similar whether the autobiographical information was framed as being about the virtual assistant’s experience as a computer agent [44] or included human experiences that could not actually be true [29,47].”
BackgroundMAINTAINING ENGAGEMENT IN LONG-TERM INTERVENTIONS WITH RELATIONAL AGENTS
Bickmore, T., Schulman, D., & Yin, L. (2010) · Applied Artificial Intelligence · doi:10.1080/08839514.2010.492259 · Cited by 246
I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions
Lisetti, C., Amini, R., Yasavur, U., & Rishe, N. (2013) · ACM Transactions on Management Information Systems · doi:10.1145/2544103 · Cited by 242
Design Features of Embodied Conversational Agents in eHealth: a Literature Review
ter Stal, S., Kramer, L. L., Tabak, M., op den Akker, H., & Hermens, H. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102409 · Cited by 188 · cited 3×
“Notably, the review by ter Stal et al [15] included studies using any research design and studies where participants viewed stimuli but did not necessarily interact with a virtual assistant.”
BackgroundMethodInfluentialMaher, C. A., Davis, C. R., Curtis, R. G., Short, C. E., & Murphy, K. J. (2020) · JMIR mHealth and uHealth · doi:10.2196/17558 · Cited by 165 · cited 1×
“Virtual assistants are also increasingly being designed for various health applications, such as delivering cognitive behavior therapy for depression and anxiety [2], improving diet and physical activity [3], and conducting remote patient monitoring [4].”
BackgroundSelf-citationEMMA: an emotion-aware wellbeing chatbot
Ghandeharioun, A., McDuff, D., Czerwinski, M., & Rowan, K. (2019) · 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2019.8925455 · Cited by 126 · cited 3×
“Although some effects were nonsignificant [27,40,48,49,51], more effects were positive or mixed positive, with 71% (5/7) of studies showing at least some positive effect [27,31,39,48,51].”
BackgroundSocial support agents for older adults: longitudinal affective computing in the home
Ring, L., Shi, L., Totzke, K., & Bickmore, T. (2015) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-014-0157-0 · Cited by 97
Hey Google, Can I Ask You Something in Private?
Cho, E. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300488 · Cited by 88 · cited 3×
“Multimedia Appendix 2 [21-68] provides an overview of the participant characteristics and study designs for all studies included in the review.”
BackgroundArtificial Intelligence: power for civilisation – and for better healthcare
Horgan, D., Romao, M., Morré, S. A., & Kalra, D. (2019) · Public Health Genomics · doi:10.1159/000504785 · Cited by 86 · cited 1×
“Advancements in machine learning and artificial intelligence offer promise for delivering automated, tailored, convenient health assistance with an unprecedented level of sophistication and personalization and are already contributing to the transformation of health care [1].”
BackgroundBellur, S., & Sundar, S. S. (2017) · Human Communication Research · doi:10.1111/hcre.12094 · Cited by 69
Agent-User Concordance and Satisfaction with a Virtual Hospital Discharge Nurse
Zhou, S., Bickmore, T., Paasche-Orlow, M., & Jack, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-09767-1_63 · Cited by 57
Chatbot Personalities Matters: Improving the User Experience of Chatbot Interfaces
Smestad, T. L., & Volden, F. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-17705-8_15 · Cited by 52 · cited 5×
“Approximately 7% (3/41) of studies examined personality [36,60,61].”
BackgroundInfluentialLooking the Part: The Effect of Attire and Setting on Perceptions of a Virtual Health Counselor
Parmar, D., Olafsson, S., Utami, D., & Bickmore, T. (2018) · Proceedings of the 18th International Conference on Intelligent Virtual Agents · doi:10.1145/3267851.3267915 · Cited by 50
Effects of Patient Care Assistant Embodiment and Computer Mediation on User Experience
Kim, K., Norouzi, N., Losekamp, T., Bruder, G., Anderson, M., & Welch, G. (2019) · 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) · doi:10.1109/aivr46125.2019.00013 · Cited by 39 · cited 1×
“An additional study not included in Table 3 concluded that virtual assistants with an animated avatar were preferred over voice-only assistants; however, the analyses included both real and virtual assistants [45].”
BackgroundAdapting a Persuasive Conversational Agent for the Chinese Culture
Zhou, S., Zhang, Z., & Bickmore, T. (2017) · 2017 International Conference on Culture and Computing (Culture and Computing) · doi:10.1109/culture.and.computing.2017.42 · Cited by 31 · cited 5×
“Notably, although 75% (3/4) of studies included participants who were born overseas (in China [68], India [50], or a Spanish-speaking Latin-American country [65]), participants in all the studies lived in the United States.”
BackgroundResultInfluential
Cited by
Revolutionizing healthcare: the role of artificial intelligence in clinical practice
Alowais, S. A., Alghamdi, S. S., Alsuhebany, N., Alqahtani, T., Alshaya, A. I., Almohareb, S. N., Aldairem, A., Alrashed, M., Bin Saleh, K., Badreldin, H. A., et al. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04698-z · Cited by 3,146
“It is designed to simulate human conversation to offer personalized patient care based on input from the patient [83].”
BackgroundThe Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century
Maleki Varnosfaderani, S., & Forouzanfar, M. (2024) · Bioengineering · doi:10.3390/bioengineering11040337 · Cited by 931
van Bussel, M. J. P., Odekerken–Schröder, G. J., Ou, C., Swart, R. R., & Jacobs, M. J. G. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08189-7 · Cited by 60
“To summarize, VAs are scalable, easily accessible and operate at low cost at any time [8, 9].”
BackgroundAdvancing health care via artificial intelligence: From concept to clinic
Sandeep Ganesh, G., Kolusu, A. S., Prasad, K., Samudrala, P. K., & Nemmani, K. V. S. (2022) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2022.175320 · Cited by 43
REVIEW OF AI APPLICATIONS IN HEALTHCARE: COMPARATIVE INSIGHTS FROM THE USA AND AFRICA
Babarinde, A. O., Ayo-Farai, O., Maduka, C. P., Okongwu, C. C., Ogundairo, O., & Sodamade, O. (2023) · International Medical Science Research Journal · doi:10.51594/imsrj.v3i3.641 · Cited by 42
Metadata
- Authors
- Curtis, R. G., Bartel, B., Ferguson, T., Blake, H. T., Northcott, C., Virgara, R., & Maher, C. A.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-12-21
- Volume
- 23
- Issue
- 12
- Pages
- e31737
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4200013702
- Cited by
- 115
- References
- 28
- FWCI
- 15.58
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.