Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review
Open accessPublished version of a preprint
The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index; 129 of the 325 sources its claims cite were checked for support.
Abstract
BACKGROUND: Chronic and mental health conditions are increasingly prevalent worldwide. As devices in our everyday lives offer more and more voice-based self-service, voice-based conversational agents (VCAs) have the potential to support the prevention and management of these conditions in a scalable manner. However, evidence on VCAs dedicated to the prevention and management of chronic and mental health conditions is unclear. OBJECTIVE: This study provides a better understanding of the current methods used in the evaluation of health interventions for the prevention and management of chronic and mental health conditions delivered through VCAs. METHODS: We conducted a systematic literature review using PubMed MEDLINE, Embase, PsycINFO, Scopus, and Web of Science databases. We included primary research involving the prevention or management of chronic or mental health conditions through a VCA and reporting an empirical evaluation of the system either in terms of system accuracy, technology acceptance, or both. A total of 2 independent reviewers conducted the screening and data extraction, and agreement between them was measured using Cohen kappa. A narrative approach was used to synthesize the selected records. RESULTS: Of 7170 prescreened papers, 12 met the inclusion criteria. All studies were nonexperimental. The VCAs provided behavioral support (n=5), health monitoring services (n=3), or both (n=4). The interventions were delivered via smartphones (n=5), tablets (n=2), or smart speakers (n=3). In 2 cases, no device was specified. A total of 3 VCAs targeted cancer, whereas 2 VCAs targeted diabetes and heart failure. The other VCAs targeted hearing impairment, asthma, Parkinson disease, dementia, autism, intellectual disability, and depression. The majority of the studies (n=7) assessed technology acceptance, but only few studies (n=3) used validated instruments. Half of the studies (n=6) reported either performance measures on speech recognition or on the ability of VCAs to respond to health-related queries. Only a minority of the studies (n=2) reported behavioral measures or a measure of attitudes toward intervention-targeted health behavior. Moreover, only a minority of studies (n=4) reported controlling for participants' previous experience with technology. Finally, risk bias varied markedly. CONCLUSIONS: The heterogeneity in the methods, the limited number of studies identified, and the high risk of bias show that research on VCAs for chronic and mental health conditions is still in its infancy. Although the results of system accuracy and technology acceptance are encouraging, there is still a need to establish more conclusive evidence on the efficacy of VCAs for the prevention and management of chronic and mental health conditions, both in absolute terms and in comparison with standard health care.
Overview
CoChat Summary
## TL;DR
This systematic review identified 12 studies on voice-based conversational agents (VCAs) for chronic and mental health conditions out of 7170 prescreened papers, and found that the evidence is still preliminary. The studies were all nonexperimental, highly heterogeneous, and generally focused on technology acceptance and system accuracy rather than actual clinical effectiveness.
## Key findings
- Of 7170 prescreened papers, 12 met the inclusion criteria; all 12 studies were nonexperimental.
- VCAs were used for behavioral support (n=5), health monitoring (n=3), or both (n=4), and were delivered via smartphones (n=5), tablets (n=2), smart speakers (n=3), or unspecified devices (n=2).
- Technology acceptance was assessed in 7 studies, but only 3 used validated instruments; system performance was reported in 6 studies, with 3 describing good or very good speech recognition performance and 1 reporting mediocre recognition accuracy.
- The included studies targeted a range of conditions, including cancer (n=3 VCAs), diabetes (n=2), heart failure (n=2), hearing impairment, asthma, Parkinson disease, dementia, autism, intellectual disability, and depression.
- Risk bias varied markedly, with scores ranging from 1 to 11.25 and a median of 6.36.
## Methods
- Systematic literature review following PRISMA, searching PubMed MEDLINE, Embase, PsycINFO, Scopus, and Web of Science; the search was conducted using literature available in July 2020.
- Included primary research on prevention, treatment, or management of chronic diseases or mental disorders delivered through a voice-based conversational agent, with empirical evaluation of system accuracy and/or technology acceptance.
- Two independent reviewers screened titles, abstracts, and full texts; two reviewers extracted data; disagreement was resolved with a third investigator; synthesis was narrative, and risk of bias was assessed using an adapted CONSORT-based tool.
## Limitations
- The paper states that its search strategy used nonspecific terms such as “health,” which may have retrieved many irrelevant papers; this was an explicit tradeoff to avoid missing relevant literature.
- The review covers research evidence only and does not include industry developments; this is stated in the paper.
- The authors state that their bias tool may be unsuitable because none of the included papers were experimental trials; this is a limitation stated in the paper.
- Evident from the text: the evidence base is small (12 studies), all studies were nonexperimental and mostly laboratory/short-term evaluations, so the review cannot establish actual effectiveness of VCAs in improving health outcomes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
53 Valid · 6 Not valid · 39 Not checked · 73 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67]”
§ Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review > Results > Methodology of the Included Studies
What the sources say
- Supportedcites [59] Amith, M, 2019, Stud Health Technol Inform source
The Methods (p. 4) describe participant surveys of usability, speech-interface quality, and HPV vaccine beliefs; the Results (pp. 4–6) report survey scores rather than technical system-performance measures. This supports citing the paper as one study that did not report system measures. The paper alone cannot establish the review’s aggregate count of six studies. (LLM, conf=0.82)
- Disputedcites [60] Amith, M, 2020, AMIA Jt Summits Transl Sci Proc sourceExperimental — accuracy not yet measured
The cited paper reports measures of its conversational agent rather than reporting none. Its Methods (p. 4) identify the SUS and SUISQ instruments, and its Results (pp. 4–5) report system-usability and speech-interface scores. The paper cannot establish how many of the other included studies reported measures. (LLM, conf=0.80)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [62] source
- Not matchedcites [65] Ireland, D, 2016, Stud Health Technol Inform
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Disputedcites [66] sourceExperimental — accuracy not yet measured
The Evaluation Method section lists chatbot quality and system usability among kBot’s evaluation criteria. The Evaluation Result section reports scores for quality metrics and system usability (SUS), contrary to the attribution that this study reported no system measures. The paper does not address the citing review’s count of six studies. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [67] source
plainn = 6- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On the same note, conversational agents have been proven a valuable tool to deliver digital health interventions [7-9].”
§ Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review > Introduction > Background
What the sources say
- Partly supportedcites [7] sourceExperimental — accuracy not yet measured
The Results describe conversational agents used for health tasks, including self-care, and report one efficacy trial that reduced depression symptoms. But the Conclusions say efficacy was rarely evaluated and the evidence remains limited, so the review does not establish that conversational agents have been proven valuable for digital health interventions generally. Page numbers were not available in our copy. (LLM, conf=0.87)
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says conversational agents have demonstrated benefits and are used to deliver digital health interventions. The Discussion’s Principal Findings (page unavailable) reports generally positive results but notes that evidence on health outcomes came from only two studies and that most studies were quasi-experimental. “Proven” overstates the strength of the evidence this review reports. (LLM, conf=0.83)
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
The Results (Conversational Agent Evaluation; page unavailable) report positive outcomes for some conversational-agent interventions, including improvements in food consumption, physical activity, and diabetes measures. But the Discussion (Principal Findings; page unavailable) says most studies were case studies and notes gaps in evaluating adverse effects. The source supports their potential value for delivering digital health interventions, but “proven” overstates the evidence. (LLM, conf=0.88)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“whereas 1 study [60] described better usability of its VCA for human papillomavirus (HPV) in comparison with industry standards (ie, SUS score of mean 72/100)”
§ Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review > Results > Main Findings
What the sources say
- Supportedcites [60] Amith, M, 2020, AMIA Jt Summits Transl Sci Proc source
The Results (p. 4) compare the agent with an industry mean SUS score of 72 and describe its score as on par with or slightly above that benchmark. The Discussion (p. 6) gives the agent’s score as 77 versus 72, while noting 94% confidence in the comparison. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A total of 3 studies [60,66,67] reported technology acceptance through a System Usability Survey (SUS).”
§ Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review > Results > Main Findings
What the sources say
- Supportedcites [60] Amith, M, 2020, AMIA Jt Summits Transl Sci Proc source
The Methods (p. 4) identify the System Usability Survey (SUS) as one of the instruments used, and the Results (p. 4) report a SUS score for the conversational agent. This supports citing the paper as one study that assessed usability with SUS; the total of three studies comes from the citing document, not this source alone. (LLM, conf=0.91)
- Partly supportedcites [66] sourceExperimental — accuracy not yet measured
The Evaluation Method identifies technology acceptance and system usability as separate criteria. The Evaluation Result reports technology acceptance from the first questionnaire and SUS scores from a second questionnaire; this paper did not measure technology acceptance through SUS. The cited paper also cannot establish the citing sentence’s total of three studies. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [67] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“whereas 3 [63,68,69] reported good or very good speech recognition performance”
§ Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review > Results > Main Findings
What the sources say
- Supportedcites [63] source
The Speech Recognition Results (p. 4) report that nine of fourteen participants had checkup word error rates below 21%, and the Discussion (p. 5) calls the speech-recognition performance “quite encouraging.” This supports describing this paper as one reporting good performance, although results were poor for the other five participants. The count of three studies belongs to the citing document, not to this source alone. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [68] source
- Not checked — no full text of the sourcecites [69] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
49 more claims · 12 with their cited sources checked
Create a free account to see all 171 claims and every source verdict.
- Probably not a substantial claim. 69
- It may state two separate claims as one. 79
- The quote only partly states this claim. 1
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 1×
“The interpretation of the Cohen kappa coefficient was based on the categories developed by Douglas Altman: 0.00-0.20 (poor), 0.21-0.40 (fair), 0.41-0.60 (moderate), 0.61-0.80 (good), and 0.81-1.00 (very good) [56,57].”
InfluentialPRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · cited 1×
“This study is compliant with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist [48] (an overview of the study protocol is given in Multimedia Appendix 1 [49-55]).”
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012) · MIS Quarterly · doi:10.2307/41410412 · Cited by 15,756 · cited 2×
“However, this aspect has an important influence on the technology acceptance [71].”
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · cited 1×
“This study is compliant with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist [48] (an overview of the study protocol is given in Multimedia Appendix 1 [49-55]).”
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review
Martin, D. J., Garske, J. P., & Davis, M. K. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.3.438 · Cited by 3,355 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
The Technology Acceptance Model: Its past and its future in health care
Holden, R. J., & Karsh, B.-T. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.002 · Cited by 2,669 · cited 1×
“Thus, we include evidence from both journals and conference papers and provide an overview of aspects affecting technology adoption, that is, system and user performance, ease of use, and attitude toward the target health behavior [47].”
CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000251 · Cited by 2,296
The nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433 · cited 1×
“This study is compliant with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist [48] (an overview of the study protocol is given in Multimedia Appendix 1 [49-55]).”
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388 · cited 2×
“These databases were chosen as they cover relevant aspects in the fields of medicine, technology, and interdisciplinary research and have also been used in other systematic reviews covering similar topics [7,8].”
The role of the therapeutic alliance in psychotherapy.
Horvath, A. O., & Luborsky, L. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.561 · Cited by 1,279 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
PRACTICAL STATISTICS FOR MEDICAL RESEARCH
Ludbrook, J. (1991) · Australian and New Zealand Journal of Surgery · doi:10.1111/j.1445-2197.1991.tb00019.x · Cited by 1,213
Hamine, S., Gerth-Guyette, E., Faulx, D., Green, B. B., & Ginsburg, A. S. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3951 · Cited by 1,149 · cited 1×
“As the health care system is currently unfit to sustain the prevention and management of chronic and mental health conditions while containing its costs, continuous and personalized smartphone-based interventions have been developed to provide scaled-up behavioral support [3-6].”
Alexa, Siri, Cortana, and more: An introduction to voice assistants.
Hoy, M. B. (2018) · Medical Reference Services Quarterly · doi:10.1080/02763869.2018.1404391 · Cited by 965 · cited 1×
“This modality of interaction allows for hands-free access to some basic functions, such as searching for information on the internet, managing calendars, playing media content, calling, texting, emails, controlling internet-of-things devices and telling jokes [10,11].”
Maher, C. A., Lewis, L. K., Ferrar, K., Marshall, S., De Bourdeaudhuij, I., & Vandelanotte, C. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2952 · Cited by 733 · cited 2×
“After extensive team discussions, the investigators decided to follow the approach of Maher et al [58], who devised a risk of bias assessment tool based on the CONSORT (Consolidated Standards of Reporting Trials) checklist [51].”
(2019) · Cited by 633
Patient-centred consultations and outcomes in primary care: a review of the literature
Mead, N., & Bower, P. (2002) · Patient Education and Counseling · doi:10.1016/s0738-3991(02)00099-x · Cited by 613 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260288 · Cited by 579
Conversational Agents in Health Care: Scoping Review and Conceptual Analysis
Tudor Car, L., Dhinagaran, D. A., Kyaw, B. M., Kowatsch, T., Joty, S., Theng, Y.-L., & Atun, R. (2020) · Journal of Medical Internet Research · doi:10.2196/17158 · Cited by 553 · cited 2×
“Text-based conversational agents have been extensively studied in the domain of digital health interventions [80-83] and can be considered as a precursor to VCAs [9].”
Self-citationEstablishing the computer-patient working alliance in automated health behavior change interventions
Bickmore, T., Gruber, A., & Picard, R. (2005) · Patient Education and Counseling · doi:10.1016/j.pec.2004.09.008 · Cited by 494 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
Pradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448 · cited 1×
“Moreover, voice modality may differ in their appropriateness of app, compared with text modality, depending on the health-related context (eg, public spaces [84,85] and type of user [24-26,86,87]).”
Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot
Liu, B., & Sundar, S. S. (2018) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2018.0110 · Cited by 447 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
Persuasive computers: perspectives and research directions
Fogg, B. (1998) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '98 · doi:10.1145/274644.274677 · Cited by 434 · cited 1×
“First, voice-based interaction leverages the naturalness [20,21] and social presence [22,23] of human-to-human conversation.”
Miner, A. S., Milstein, A., Schueller, S., Hegde, R., Mangurian, C., & Linos, E. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0400 · Cited by 368 · cited 1×
“Although there is still room for improvement [41-43], curiosity in using VCAs for health care is growing.”
Music, Search, and IoT: How People (Really) Use Voice Assistants
Ammari, T., Kaye, J., Tsai, J. Y., & Bentley, F. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3311956 · Cited by 356
Survey of conversational agents in health
Montenegro, J. L. Z., da Costa, C. A., & da Rosa Righi, R. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.054 · Cited by 322 · cited 1×
“Moreover, voice modality may differ in their appropriateness of app, compared with text modality, depending on the health-related context (eg, public spaces [84,85] and type of user [24-26,86,87]).”
Bickmore, T. W., Trinh, H., Olafsson, S., O'Leary, T. K., Asadi, R., Rickles, N. M., & Cruz, R. (2018) · Journal of Medical Internet Research · doi:10.2196/11510 · Cited by 308 · cited 1×
“However, these products may be of privacy [76] or safety concerns [77].”
Wang, K., Varma, D. S., & Prosperi, M. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.10.006 · Cited by 308 · cited 1×
“As the health care system is currently unfit to sustain the prevention and management of chronic and mental health conditions while containing its costs, continuous and personalized smartphone-based interventions have been developed to provide scaled-up behavioral support [3-6].”
Use of Intelligent Voice Assistants by Older Adults with Low Technology Use
Pradhan, A., Lazar, A., & Findlater, L. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3373759 · Cited by 285 · cited 1×
“VCAs are used to retrieve health-related information (eg, symptoms, medication, nutrition, and health care facilities) [32,44].”
López, G., Quesada, L., & Guerrero, L. A. (2018) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-60366-7_23 · Cited by 277
Schachner, T., Keller, R., & v Wangenheim, F. (2020) · Journal of Medical Internet Research · doi:10.2196/20701 · Cited by 267 · cited 2×
“These databases were chosen as they cover relevant aspects in the fields of medicine, technology, and interdisciplinary research and have also been used in other systematic reviews covering similar topics [7,8].”
Self-citationHoermann, S., McCabe, K. L., Milne, D. N., & Calvo, R. A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7023 · Cited by 263 · cited 1×
“Text-based conversational agents have been extensively studied in the domain of digital health interventions [80-83] and can be considered as a precursor to VCAs [9].”
Rapport: A key to treatment success
Leach, M. J. (2005) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2005.05.005 · Cited by 257 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
Privacy Concerns for Use of Voice Activated Personal Assistant in the Public Space
Easwara Moorthy, A., & Vu, K.-P. L. (2015) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2014.986642 · Cited by 250 · cited 1×
“Moreover, voice modality may differ in their appropriateness of app, compared with text modality, depending on the health-related context (eg, public spaces [84,85] and type of user [24-26,86,87]).”
Chung, H., Iorga, M., Voas, J., & Lee, S. (2017) · Computer · doi:10.1109/mc.2017.3571053 · Cited by 230
Digital medicine's march on chronic disease
Kvedar, J. C., Fogel, A. L., Elenko, E., & Zohar, D. (2016) · Nature Biotechnology · doi:10.1038/nbt.3495 · Cited by 218 · cited 1×
“As the health care system is currently unfit to sustain the prevention and management of chronic and mental health conditions while containing its costs, continuous and personalized smartphone-based interventions have been developed to provide scaled-up behavioral support [3-6].”
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214 · cited 1×
“Text-based conversational agents have been extensively studied in the domain of digital health interventions [80-83] and can be considered as a precursor to VCAs [9].”
Key Considerations for Incorporating Conversational AI in Psychotherapy
Miner, A. S., Shah, N., Bullock, K. D., Arnow, B. A., Bailenson, J., & Hancock, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00746 · Cited by 166 · cited 2×
“Second, it facilitates input for users with low literacy or with visual [24], intellectual [25], motor, linguistic, and cognitive disabilities [26] and can support more natural health routine tasks when in-person health care is not possible [19,27].”
Mota, N. B., Copelli, M., & Ribeiro, S. (2017) · npj Schizophrenia · doi:10.1038/s41537-017-0019-3 · Cited by 157
Readiness for voice assistants to support healthcare delivery during a health crisis and pandemic
Sezgin, E., Huang, Y., Ramtekkar, U., & Lin, S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00332-0 · Cited by 155 · cited 1×
“Second, it facilitates input for users with low literacy or with visual [24], intellectual [25], motor, linguistic, and cognitive disabilities [26] and can support more natural health routine tasks when in-person health care is not possible [19,27].”
Talking to machines about personal mental health problems
Miner, A. S., Milstein, A., & Hancock, J. T. (2017) · JAMA · doi:10.1001/jama.2017.14151 · Cited by 153 · cited 1×
“Although there is still room for improvement [41-43], curiosity in using VCAs for health care is growing.”
Technical metrics used to evaluate health care chatbots: scoping review
Abd-Alrazaq, A., Safi, Z., Alajlani, M., Warren, J., Househ, M., & Denecke, K. (2020) · Journal of Medical Internet Research · doi:10.2196/18301 · Cited by 150 · cited 1×
“Text-based conversational agents have been extensively studied in the domain of digital health interventions [80-83] and can be considered as a precursor to VCAs [9].”
A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology
Bickmore, T. W., Schulman, D., & Sidner, C. L. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2010.12.006 · Cited by 147 · cited 1×
“For instance, text-based and embodied conversational agents can currently be implemented using existing frameworks dedicated to digital health interventions [72-75]; however, to the best of our knowledge, there is no such framework for VCAs.”
Detecting Dementia Through Interactive Computer Avatars
Tanaka, H., Adachi, H., Ukita, N., Ikeda, M., Kazui, H., Kudo, T., & Nakamura, S. (2017) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2017.2752152 · Cited by 127 · cited 1×
“Third, it opens the door to voice or speech analysis, whereas features of the patient’s utterances can be passively monitored to derive health states [28-31].”
Alzheimer's disease and automatic speech analysis: A review
Pulido, M. L. B., Hernández, J. B. A., Ballester, M. Á. F., González, C. M. T., Mekyska, J., & Smékal, Z. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113213 · Cited by 120 · cited 1×
“Third, it opens the door to voice or speech analysis, whereas features of the patient’s utterances can be passively monitored to derive health states [28-31].”
Falkenström, F., Hatcher, R. L., Skjulsvik, T., Larsson, M. H., & Holmqvist, R. (2015) · Psychological Assessment · doi:10.1037/pas0000038 · Cited by 117 · cited 1×
“Future research should favor the use of multiple standardized questionnaires dedicated to voice user interfaces [78] to further explore the factors potentially influencing their effectiveness (eg, rapport [79] and intention of use [71]).”
Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study
Masina, F., Orso, V., Pluchino, P., Dainese, G., Volpato, S., Nelini, C., Mapelli, D., Spagnolli, A., & Gamberini, L. (2020) · Journal of Medical Internet Research · doi:10.2196/18431 · Cited by 105 · cited 2×
“Second, it facilitates input for users with low literacy or with visual [24], intellectual [25], motor, linguistic, and cognitive disabilities [26] and can support more natural health routine tasks when in-person health care is not possible [19,27].”
kBot: Knowledge-Enabled Personalized Chatbot for Asthma Self-Management
Kadariya, D., Venkataramanan, R., Yip, H. Y., Kalra, M., Thirunarayanan, K., & Sheth, A. (2019) · 2019 IEEE International Conference on Smart Computing (SMARTCOMP) · doi:10.1109/smartcomp.2019.00043 · Cited by 102 · cited 10×
“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67], whereas the other half reported either speech recognition performance measures (n=4) [63,64,68,69] or criterion-based evaluation of the goodness of the VCA’s response (n=2) [61,70].”
InfluentialAcoustic differences between healthy and depressed people: a cross-situation study
Wang, J., Zhang, L., Liu, T., Pan, W., Hu, B., & Zhu, T. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2300-7 · Cited by 98 · cited 1×
“Third, it opens the door to voice or speech analysis, whereas features of the patient’s utterances can be passively monitored to derive health states [28-31].”
Hey Google, Can I Ask You Something in Private?
Cho, E. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300488 · Cited by 88 · cited 1×
“First, voice-based interaction leverages the naturalness [20,21] and social presence [22,23] of human-to-human conversation.”
Sezgin, E., Militello, L. K., Huang, Y., & Lin, S. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz141 · Cited by 83 · cited 10×
“In particular, voice-based conversational agents (VCAs) have been shown to provide high user satisfaction in delivering interventions to influence healthy lifestyles [6].”
InfluentialCheng, A., Raghavaraju, V., Kanugo, J., Handrianto, Y. P., & Shang, Y. (2018) · 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC) · doi:10.1109/ccnc.2018.8319283 · Cited by 82 · cited 10×
“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67], whereas the other half reported either speech recognition performance measures (n=4) [63,64,68,69] or criterion-based evaluation of the goodness of the VCA’s response (n=2) [61,70].”
InfluentialKowatsch, T., Nißen, M. K., Shih, C.-H. I., Rüegger, D., Volland, D., Filler, A., Künzler, F., Barata, F., Hung, S., Büchter, D., et al. (2017) · ETH Zurich · doi:10.3929/ethz-b-000218776 · Cited by 74
Self-citationHealth and Fitness Apps for Hands-Free Voice-Activated Assistants: Content Analysis
Chung, A. E., Griffin, A. C., Selezneva, D., & Gotz, D. (2018) · JMIR mHealth and uHealth · doi:10.2196/mhealth.9705 · Cited by 70 · cited 1×
“Moreover, numerous health-related apps of VCAs are available [34].”
Measuring User Experience in Conversational Interfaces: A Comparison of Six Questionnaires
Kocaballi, A. B., Laranjo, L., & Coiera, E. W. (2018) · Electronic workshops in computing · doi:10.14236/ewic/hci2018.21 · Cited by 67 · cited 1×
“Future research should favor the use of multiple standardized questionnaires dedicated to voice user interfaces [78] to further explore the factors potentially influencing their effectiveness (eg, rapport [79] and intention of use [71]).”
Filler, A., Kowatsch, T., Haug, S., Wahle, F., Staake, T., & Fleisch, E. (2015) · 2015 Wireless Telecommunications Symposium (WTS) · doi:10.1109/wts.2015.7117255 · Cited by 63 · cited 1×
“For instance, text-based and embodied conversational agents can currently be implemented using existing frameworks dedicated to digital health interventions [72-75]; however, to the best of our knowledge, there is no such framework for VCAs.”
Self-citationUse of voice activated interfaces by people with intellectual disability
Balasuriya, S. S., Sitbon, L., Bayor, A. A., Hoogstrate, M., & Brereton, M. (2018) · Proceedings of the 30th Australian Conference on Computer-Human Interaction · doi:10.1145/3292147.3292161 · Cited by 63 · cited 2×
“Second, it facilitates input for users with low literacy or with visual [24], intellectual [25], motor, linguistic, and cognitive disabilities [26] and can support more natural health routine tasks when in-person health care is not possible [19,27].”
Hello Harlie: Enabling Speech Monitoring Through Chat-Bot Conversations
David, I., Christina, A., Jacki, L., Dana, B., Helen, L., Olivia, R., Thomas, M., Dan, A., Janet, W., Simon, M., et al. (2016) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-666-8-55 · Cited by 56 · cited 10×
“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67], whereas the other half reported either speech recognition performance measures (n=4) [63,64,68,69] or criterion-based evaluation of the goodness of the VCA’s response (n=2) [61,70].”
InfluentialReis, A., Paulino, D., Paredes, H., Barroso, I., Monteiro, M. J., Rodrigues, V., & Barroso, J. (2018) · 2018 2nd International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW) · doi:10.1109/tishw.2018.8559503 · Cited by 56 · cited 10×
“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67], whereas the other half reported either speech recognition performance measures (n=4) [63,64,68,69] or criterion-based evaluation of the goodness of the VCA’s response (n=2) [61,70].”
InfluentialBoyd, M., & Wilson, N. (2018) · PLOS ONE · doi:10.1371/journal.pone.0194811 · Cited by 40 · cited 10×
“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67], whereas the other half reported either speech recognition performance measures (n=4) [63,64,68,69] or criterion-based evaluation of the goodness of the VCA’s response (n=2) [61,70].”
InfluentialEffects of Patient Care Assistant Embodiment and Computer Mediation on User Experience
Kim, K., Norouzi, N., Losekamp, T., Bruder, G., Anderson, M., & Welch, G. (2019) · 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) · doi:10.1109/aivr46125.2019.00013 · Cited by 39 · cited 1×
“First, voice-based interaction leverages the naturalness [20,21] and social presence [22,23] of human-to-human conversation.”
Medical Instructed Real-Time Assistant for Patient with Glaucoma and Diabetic Conditions
Rehman, U. U., Chang, D. J., Jung, Y., Akhtar, U., Razzaq, M. A., & Lee, S. (2020) · Applied Sciences · doi:10.3390/app10072216 · Cited by 37 · cited 9×
“In total, 4 of those studies [63,64,68,69] described precise speech recognition performance, whereas 3 [63,68,69] reported good or very good speech recognition performance, and 1 [64] study found mediocre recognition accuracy, with single-letter responses being slightly better recognized than word-based responses (det…”
InfluentialEarly Usability Assessment of a Conversational Agent for HPV Vaccination
Muhammad, A., Anna, Z., Rachel, C., Rebecca, L., Lara, S., Laura, S., Yong, C., Yang, G., Julie, B., Kirk, R., et al. (2019) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-951-5-17 · Cited by 35 · cited 10×
“Thus, 1 study [59] reported good ease of use (mean 5.”
InfluentialAndroid based voice assistant for blind people
Barata, M., Galih Salman, A., Faahakhododo, I., & Kanigoro, B. (2018) · Library Hi Tech News · doi:10.1108/lhtn-11-2017-0083 · Cited by 22 · cited 2×
“Second, it facilitates input for users with low literacy or with visual [24], intellectual [25], motor, linguistic, and cognitive disabilities [26] and can support more natural health routine tasks when in-person health care is not possible [19,27].”
Advances in Human Factors and Systems Interaction
(2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-94334-3 · Cited by 20
Speech recognition in a dialog system for patient health monitoring
Galescu, L., Allen, J., Ferguson, G., Quinn, J., & Swift, M. (2009) · 2009 IEEE International Conference on Bioinformatics and Biomedicine Workshop · doi:10.1109/bibmw.2009.5332111 · Cited by 18 · cited 9×
“Among the papers developing and evaluating a VCA, 6 [59,60,62,65-67] assessed the technology acceptance of the VCA, whereas 3 [63,64,68] assessed the system accuracy.”
InfluentialA narrative review of alliance formation and outcome in text-based telepsychotherapy.
Schwartzman, C. M., & Boswell, J. F. (2020) · Practice Innovations · doi:10.1037/pri0000120 · Cited by 18 · cited 1×
“Just as text-based [12,13] and embodied [14] conversational agents, VCAs have the potential to form an alliance [15] or rapport [16] with the patient through conversation, which is beneficial to treatment outcomes [17-19].”
CARMIE: A Conversational Medication Assistant for Heart Failure
Lobo, J., Ferreira, L., & Ferreira, A. J. (2017) · International Journal of E-Health and Medical Communications · doi:10.4018/ijehmc.2017100102 · Cited by 15 · cited 10×
“One study [67] reported a relatively high usability score (SUS score of mean 88/100), whereas 1 study [60] described better usability of its VCA for human papillomavirus (HPV) in comparison with industry standards (ie, SUS score of mean 72/100).”
InfluentialNew Directions for the Consideration of HIV: Heterogeneity and the Cognition of Time
Tan, M. (2020) · New Directions for Child and Adolescent Development · doi:10.1002/cad.20327 · Cited by 15
Amith, M., Lin, R., Cunningham, R., Wu, Q. L., Savas, L. S., Gong, Y., Boom, J. A., Tang, L., & Tao, C. (2020) · arXiv · doi:10.48550/arxiv.2001.02306 · Cited by 14 · cited 10×
“One study [67] reported a relatively high usability score (SUS score of mean 88/100), whereas 1 study [60] described better usability of its VCA for human papillomavirus (HPV) in comparison with industry standards (ie, SUS score of mean 72/100).”
Influential“Computer, Test My Hearing”: Accurate Speech Audiometry with Smart Speakers
Ooster, J., Moreta, P. N. P., Bach, J.-H., Holube, I., & Meyer, B. T. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2118 · Cited by 10 · cited 9×
“Among the papers developing and evaluating a VCA, 6 [59,60,62,65-67] assessed the technology acceptance of the VCA, whereas 3 [63,64,68] assessed the system accuracy.”
InfluentialGreuter, S., Balandin, S., & Watson, J. (2019) · Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts · doi:10.1145/3341215.3356308 · Cited by 10 · cited 9×
“Support Smart speaker Intellectual disability Adults with lifelong intellectual disability (n=9) Development and performance evaluation Greuter and Balandin (2019) [64]”
Influential(2014) · Oxford Music Online · doi:10.1093/gmo/9781561592630.article.l2267688 · Cited by 1
Research works printed but not linked: 7
- 2020, World Health Statistics 2020: monitoring health for the SDGs, sustainable development goals, 1
- 2019, Document number: WHO/MSD/MER/19.3
- 10.1136/bmj.g7647 · DOI · Paper page
- Amith, M, 2019, Stud Health Technol Inform, 257, 17
- Amith, M, 2020, AMIA Jt Summits Transl Sci Proc, 2020, 43
- Ireland, D, 2016, Stud Health Technol Inform, 227, 55
- Moorthy, AE, 2014, Information and Knowledge in Applications and Services, 324
Cited by
Mariani, M. M., Hashemi, N., & Wirtz, J. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.113838 · Cited by 343
Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review (Preprint)
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · doi:10.2196/preprints.56930 · Cited by 302
Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · Journal of Medical Internet Research · doi:10.2196/56930 · Cited by 305
“…in mental health support for various age groups, reflecting the pressing need for accessible mental health services highlighted by others [4,8,12-17, Furthermore, chatbots have emerged as tools for reducing stigma [12,265], linking users to health services [266-268], and protecting sensitive…”
Huq, S. M., Maskeliūnas, R., & Damaševičius, R. (2022) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2022.2146768 · Cited by 107
Imran, M., Zaman, U., Imran, I., Imtiaz, J., Fayaz, M., & Gwak, J. (2021) · Electronics · doi:10.3390/electronics10202501 · Cited by 91
“Mobile application has been introduced based on machine learning algorithms by university students from the DY Patil Institute, Mumbai, India, to detect COVID-19 patients [220].”
Method
Metadata
- Authors
- Bérubé, C., Schachner, T., Keller, R., Fleisch, E., v Wangenheim, F., Barata, F., & Kowatsch, T.
- Affiliations
- ETH Zurich (CH)
- Singapore-ETH Centre (SG)
- University of St.Gallen (CH)
- National University of Singapore (SG)
- Corresponding author
- Bérubé, Caterina
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-03-29
- Volume
- 23
- Issue
- 3
- Pages
- e25933
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC8042539
- OpenAlex
- W3135619344
- Semantic Scholar
- f96d2964c795a9a9ff0de4b46abb73310566597c
- Cited by
- 133
- References
- 74
- Citations by year
- 2026: 22 · 2025: 33 · 2024: 34 · 2023: 15 · 2022: 18 · 2021: 11
- FWCI
- 3.13
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- National University of Singapore (submittedVersion, cc-by)
- Alexandria (UniSG) (University of St.Gallen) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- ETH Zürich Research Collection (cc-by)
- Funders
- National Research Foundation
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- voice assistants, chronic disease prevention, mental health, health behavior change, technology acceptance, speech recognition accuracy, health monitoring, behavioral support, health intervention evaluation, risk of bias, cancer, diabetes mellitus, heart failure, depression, dementia, autism spectrum disorder, hearing loss, Parkinson's disease, asthma, intellectual disability, smart speakers, smartphones, tablets, risk of bias assessment tools
- MeSH terms
- Smartphone, Asthma, Communication, Humans, Mental Health, Health Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
