An Embodied Conversational Agent for Unguided Internet-Based Cognitive Behavior Therapy in Preventative Mental Health: Feasibility and Acceptability Pilot Trial
Open accessPublished version of a preprint
Abstract
Background: Recent years have seen an increase in the use of internet-based cognitive behavioral therapy in the area of mental health. Although lower effectiveness and higher dropout rates of unguided than those of guided internet-based cognitive behavioral therapy remain critical issues, not incurring ongoing human clinical resources makes it highly advantageous. Objective: Current research in psychotherapy, which acknowledges the importance of therapeutic alliance, aims to evaluate the feasibility and acceptability, in terms of mental health, of an application that is embodied with a conversational agent. This application was enabled for use as an internet-based cognitive behavioral therapy preventative mental health measure. Methods: Analysis of the data from the 191 participants of the experimental group with a mean age of 38.07 (SD 10.75) years and the 263 participants of the control group with a mean age of 38.05 (SD 13.45) years using a 2-way factorial analysis of variance (group × time) was performed. Results: There was a significant main effect (P=.02) and interaction for time on the variable of positive mental health (P=.02), and for the treatment group, a significant simple main effect was also found (P=.002). In addition, there was a significant main effect (P=.02) and interaction for time on the variable of negative mental health (P=.005), and for the treatment group, a significant simple main effect was also found (P=.001). Conclusions: This research can be seen to represent a certain level of evidence for the mental health application developed herein, indicating empirically that internet-based cognitive behavioral therapy with the embodied conversational agent can be used in mental health care. In the pilot trial, given the issues related to feasibility and acceptability, it is necessary to pursue higher quality evidence while continuing to further improve the application, based on the findings of the current research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Design and deployment of conversational agents as virtual coaching solutions to deliver psychoeducational interventions for mental health and well-being have been shown to be ideal in maintaining intuitiveness and naturalness of dialogue-based interaction while exploiting full automation.
“The design and deployment of conversational agents, such as chatbots, as virtual coaching solutions to deliver psychoeducational interventions for mental health and well-being have so far been shown to be ideal in maintaining the intuitiveness and naturalness of dialogue-based interaction, while exploiting the benefits of full automation [19,20].”
From Engagement and effectiveness of a healthy-coping intervention via chatbot for university students during the COVID-19 pandemic: Mixed methods proof-of-concept study (2021) · cites this paper as [19]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredA growing number of studies report effectiveness of chatbots in reducing perceived stress.
“as well as their effectiveness in reducing perceived stress [16-25]”
From Engagement and effectiveness of a healthy-coping intervention via chatbot for university students during the COVID-19 pandemic: Mixed methods proof-of-concept study (2021) · cites this paper as [19]
The Methods (Measures; page not provided) identify K10 as a measure of psychological distress, and the Results (Table 1; page not provided) show lower K10 scores after the conversational-agent intervention. The paper does not report a perceived-stress measure, so it supports a related reduction in distress rather than the specific claim about perceived stress. (LLM, conf=0.84)
Experimental — accuracy not yet measuredRelational agents can enhance the DTA.
“They further argued that the RAs can enhance the DTA.”
From Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review (2022) · cites this paper as [75]
In Suganuma et al [35], distress level in the chatbot group significantly improved from before treatment (mean 23.58) to after treatment (mean 21.65).
“Further, there was a statistically significant improvement in distress level among the chatbot group after treatment (mean 21.65) compared with before treatment (mean 23.58).”
From Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis (2020) · cites this paper as [35]
In Suganuma et al [35], chatbot use resulted in statistically significantly lower distress levels after treatment compared with no intervention (mean 21.65 vs 23.97, P=.005).
“According to Suganuma et al [35], there was a statistically significant difference (P=.005) favoring chatbot use (mean 21.65) over no intervention (mean 23.97) on distress levels after treatment.”
From Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis (2020) · cites this paper as [35]
Pretest-posttest quasiexperiments are not as reliable as RCTs for finding the effect of an intervention due to low internal validity resulting from selection bias.
“First, pretest-posttest quasiexperiments are not as reliable as RCTs for finding the effect of an intervention due to low internal validity resulting from selection bias [35,41].”
From Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis (2020) · cites this paper as [35]
The third subjective psychological wellbeing study found no statistically significant difference (P=.32) between the chatbot and control groups on subjective psychological wellbeing after treatment.
“the third study found no statistically significant difference (P=.32) between the chatbot and control groups on subjective psychological wellbeing after treatment [35].”
From Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis (2020) · cites this paper as [35]
Five controlled studies reported significant posttreatment improvements in psychological distress in the intervention group compared with a no treatment or information control group.
“Of the 13 studies included, 5 controlled studies reported significant posttreatment improvements in psychological distress in the intervention group compared with a no treatment or information control group [23,25,26,31,34].”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
Effects ranged from small (d=−0.24) to very large (d=2.0).
“Effects ranged from small (d=−0.24 [34]) to very large (d=2.0 [23]).”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
Significant improvements were observed on measures of depression, psychological distress, anxiety, fear of heights, and positive affect.
“Significant improvements were observed on measures of depression [25,26,31], psychological distress [34], anxiety [26], fear of heights [23] and positive affect [26,34].”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
In the study by Suganuma et al, 236 out of 427 (55.2%) of intervention participants did not complete 15 or more days of the intervention and were excluded from the analysis.
“In the study by Suganuma et al [34], 236 out of 427 (55.2%) of intervention participants did not complete 15 or more days of the intervention and were excluded from the analysis.”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
This provides some support for the utility of conversational agents in treating mild-to-moderate psychological distress in adults.
“This provides some support for the utility of conversational agents in treating mild-to-moderate psychological distress in adults [23,25,26,31,34].”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
The use of conversational agents for treating mental health problems appears to be limited but is growing quickly, with 5 of the included studies published in 2018 alone.
“The use of conversational agents for treating mental health problems appears to be limited but is growing quickly, with 5 of the included studies published in 2018 alone [23,24,26,29,34].”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
Studies that used more flexible recruitment routes such as online adverts and app stores recruited greater numbers of participants.
“Studies that used more flexible recruitment routes such as online adverts [34] and app stores [29] recruited greater numbers of participants.”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [34]
Using an identical search strategy, the authors updated the review in January 2020 and identified a further 11 studies; most recent studies were aimed at mental health self-management (n=5).
“Using an identical search strategy, we updated this review in January 2020 and identified a further 11 studies. Similar to the previous review, most recent studies were aimed at mental health self-management (n=5) [10-12].”
From A Physical Activity and Diet Program Delivered by Artificially Intelligent Virtual Health Coach: Proof-of-Concept Study (2020) · cites this paper as [12]
Suganuma et al. showed high acceptability of an automated conversational agent with a graphical representation enabling users to communicate over a text-based dialogue.
“Similarly, Suganuma et al. (21) showed a high acceptability of an automated CA with a graphical representation enabling the users to communicate over a text-based dialogue.”
From Computer-assisted avatar-based treatment for dysfunctional beliefs in depressive inpatients: A pilot study (2021) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Need for a New Medical Model: A Challenge for Biomedicine
Engel, G. L. (1977) · Science · doi:10.1126/science.847460 · Cited by 12,887 · see citing works
KESSLER, R. C., ANDREWS, G., COLPE, L. J., HIRIPI, E., MROCZEK, D. K., NORMAND, S.-L. T., WALTERS, E. E., & ZASLAVSKY, A. M. (2002) · Psychological Medicine · doi:10.1017/s0033291702006074 · Cited by 10,327 · see citing works
The WHO-5 well-being index: A systematic review of the literature
Topp, C. W., Østergaard, S. D., Søndergaard, S., & Bech, P. (2015) · Psychotherapy and Psychosomatics · doi:10.1159/000376585 · Cited by 4,708 · see citing works
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1× · see citing works
“In fact, its efficacy in reducing symptoms of depression has been reported by Fitzpatrick et al [15] in their research, utilizing a completely automated text-based response agent based on CBT principles.”
MethodGarnefski, N., Kraaij, V., Schroevers, M. J., & Somsen, G. A. (2008) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-008-9136-5 · Cited by 2,487 · see citing works
The performance of the Japanese version of the K6 and K10 in the World Mental Health Survey Japan
Furukawa, T. A., Kawakami, N., Saitoh, M., Ono, Y., Nakane, Y., Nakamura, Y., Tachimori, H., Iwata, N., Uda, H., Nakane, H., et al. (2008) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.257 · Cited by 1,478 · see citing works
Internet-Delivered Psychological Treatments
Andersson, G. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093006 · Cited by 595 · see citing works
The conceptualization and measurement of therapeutic alliance: An empirical review
Elvins, R., & Green, J. (2008) · Clinical Psychology Review · doi:10.1016/j.cpr.2008.04.002 · Cited by 534 · see citing works
Kanter, J. W., Mulick, P. S., Busch, A. M., Berlin, K. S., & Martell, C. R. (2007) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-006-9038-5 · Cited by 493 · see citing works
Embodied conversational agents in clinical psychology: a scoping review
Provoost, S., Lau, H. M., Ruwaard, J., & Riper, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6553 · Cited by 433 · see citing works
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328 · see citing works
Designing mobile health technology for bipolar disorder
Bardram, J. E., Frost, M. M., Szántó, K., Faurholt‐Jepsen, M., Vinberg, M., & Kessing, L. V. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481364 · Cited by 304 · see citing works
Awata, S., Bech, P., YOSHIDA, S., HIRAI, M., Suzuki, S., Yamashita, M., Ohara, A., Hinokio, Y., Matsuoka, H., & Oka, Y. (2007) · Psychiatry and Clinical Neurosciences · doi:10.1111/j.1440-1819.2007.01619.x · Cited by 296 · see citing works
Doherty, G., Coyle, D., & Sharry, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208602 · Cited by 198 · cited 1× · see citing works
“In recent years, internet-based CBT (ICBT) and computerized CBT have increasingly been used as a means to fill the service gap and resolve various problems related to mental health, such as bipolar disorder [4,5], anxiety disorder [6], depression [7], treatment adherence [8], and common mental health problems [9].”
BackgroundBarriers to mental health service utilization
STEFL, M. E., & Prosperi, D. C. (1985) · Community Mental Health Journal · doi:10.1007/bf00754732 · Cited by 164 · see citing works
Matthews, M. L., & Doherty, G. J. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979379 · Cited by 129 · see citing works
Artificial Intelligence in Behavioral and Mental Health Care
(2016) · Elsevier eBooks · doi:10.1016/c2013-0-12824-3 · Cited by 117 · see citing works
Kanter, J. W., Rusch, L. C., Busch, A. M., & Sedivy, S. K. (2008) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-008-9088-y · Cited by 114 · see citing works
Exploratory evaluations of a computer game supporting cognitive behavioural therapy for adolescents
Coyle, D., McGlade, N., Doherty, G. J., & O’Reilly, G. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979378 · Cited by 109 · see citing works
The MONARCA self-assessment system
Bardram, J. E., Frost, M. M., Szántó, K., & Marcu, G. M. (2012) · International Health Informatics Symposium · doi:10.1145/2110363.2110370 · Cited by 104 · see citing works
Sebastian, J., & Richards, D. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.03.071 · Cited by 96 · see citing works
Health Technologies 'In the Wild'
Rennick‐Egglestone, S., Knowles, S. E., Toms, G. R., Bee, P., Lovell, K., & Bower, P. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858128 · Cited by 68 · see citing works
Computerized cognitive behavioural therapy and the therapeutic alliance: A qualitative enquiry
Barazzone, N., Cavanagh, K., & Richards, D. A. (2012) · British Journal of Clinical Psychology · doi:10.1111/j.2044-8260.2012.02035.x · Cited by 67 · cited 1× · see citing works
“The therapeutic alliance, as formed in unguided ICBT, is noted to be lacking in terms of elements of “development” and “maintenance” [14].”
BackgroundMartínez-Miranda, J. (2017) · Journal of Medical Systems · doi:10.1007/s10916-017-0784-6 · Cited by 51 · see citing works
Yokotani, K., Takagi, G., & Wakashima, K. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.03.045 · Cited by 51 · see citing works
Blümle, A., Schandelmaier, S., Oeller, P., Kasenda, B., Briel, M., von Elm, E. B., & on behalf of the DISCO study group (2016) · PLoS ONE · doi:10.1371/journal.pone.0165605 · Cited by 25 · see citing works
The Impact of Multiple Roles on Psychological Distress among Japanese Workers
Honda, A., Abe, Y., Date, Y., & Honda, S. (2015) · Safety and Health at Work · doi:10.1016/j.shaw.2014.12.004 · Cited by 22 · see citing works
Konishi, M., Tachibana, Y., Tang, J., Takehara, K., Kubo, T., Hashimoto, K., Kitazawa, H., Saito, H., & Ohya, Y. (2015) · Community Mental Health Journal · doi:10.1007/s10597-015-9931-z · Cited by 13 · see citing works
Designing behavioral self-regulation application for preventive personal mental healthcare
Hirano, M., Ogura, K., Kitahara, M., Sakamoto, D., & Shimoyama, H. (2017) · Health Psychology Open · doi:10.1177/2055102917707185 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- [2]Westbrook D Kennerley H Kirk J An introduction to cognitive behaviour therapy: skills and applications, second edition 2011 London SAGE Publications
- [10]Andersson G The Internet and CBT: A Clinical Guide 2014 Boca Raton CRC Press
- [16]Beun RJ de Vos E Witteman C Rist T Aylett R Ballin D Embodied Conversational Agents: Effects on Memory Performance and Anthropomorphisation Intelligent Virtual Agents. IVA 2003. Lecture Notes in Computer Science 2003 Berlin Springer 315 319
Cited by
Sorted by citations
Fiske, A., Henningsen, P., & Buyx, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13216 · Cited by 857 · see citing works
Nadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810 · see citing works
Abd-Alrazaq, A. A., Rababeh, A., Alajlani, M., Bewick, B. M., & Househ, M. (2020) · Journal of Medical Internet Research · doi:10.2196/16021 · Cited by 503 · see citing works
“Similarly, the third study found no statistically significant difference ( P =.32) between the chatbot and control groups on subjective psychological wellbeing after treatment [35].”
InfluentialPhysicians’ perceptions of chatbots in health Care: cross-sectional web-based survey
Palanica, A., Flaschner, P., Thommandram, A., Li, M., & Fossat, Y. (2019) · Journal of Medical Internet Research · doi:10.2196/12887 · Cited by 384 · see citing works
Bickmore, T. W., Trinh, H., Olafsson, S., O'Leary, T. K., Asadi, R., Rickles, N. M., & Cruz, R. (2018) · Journal of Medical Internet Research · doi:10.2196/11510 · Cited by 308 · see citing works
Conversational agents in the treatment of mental health problems: mixed-method systematic review
Gaffney, H., Mansell, W., & Tai, S. (2019) · JMIR Mental Health · doi:10.2196/14166 · Cited by 275 · see citing works
Liu, H., Peng, H., Song, X., Xu, C., & Zhang, M. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100495 · Cited by 296 · see citing works
“Previous randomized controlled trials (Burton et al., 2016; Shinichiro et al., 2018) validated the efficacy of psychological treatments delivered by embodied agents.”
BackgroundA critical review of state‐of‐the‐art chatbot designs and applications
Luo, B., Lau, R. Y. K., Li, C., & Si, Y. (2021) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1434 · Cited by 244 · see citing works
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214 · see citing works
He, Y., Yang, L., Qian, C., Li, T., Su, Z., Zhang, Q., & Hou, X. (2023) · Journal of Medical Internet Research · doi:10.2196/43862 · Cited by 213 · see citing works
He, Y., Yang, L., Zhu, X., Wu, B., Zhang, S., Qian, C., & Tian, T. (2022) · Journal of Medical Internet Research · doi:10.2196/40719 · Cited by 178 · see citing works
Bardhan, I. R., Chen, H., & Karahanna, E. (2020) · MIS Quarterly · doi:10.25300/misq/2020/14644 · Cited by 171 · see citing works
Maher, C. A., Davis, C. R., Curtis, R. G., Short, C. E., & Murphy, K. J. (2020) · JMIR mHealth and uHealth · doi:10.2196/17558 · Cited by 165 · see citing works
Influence of Rapport and Social Presence with an AI Psychotherapy Chatbot on Users’ Self-Disclosure
Lee, J., Lee, D., & Lee, J.-G. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2146227 · Cited by 130 · see citing works
Gabrielli, S., Rizzi, S., Bassi, G., Carbone, S., Maimone, R., Marchesoni, M., & Forti, S. (2021) · JMIR mHealth and uHealth · doi:10.2196/27965 · Cited by 125 · see citing works
“The design and deployment of conversational agents, such as chatbots, as virtual coaching solutions to deliver psychoeducational interventions for mental health and well-being have so far been shown to be ideal in maintaining the intuitiveness and naturalness of dialogue-based interaction, while exploiting the benefit…”
BackgroundDigital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review
Tong, F., Lederman, R., D'Alfonso, S., Berry, K., & Bucci, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.819623 · Cited by 106 · see citing works
“(75) used an embodied relational agent in a fully automated mental health app and found it was effective in bringing positive outcomes.”
BackgroundGooding, P., & Kariotis, T. (2021) · JMIR Mental Health · doi:10.2196/24668 · Cited by 100 · see citing works
Preliminary Evaluation of the Engagement and Effectiveness of a Mental Health Chatbot
Daley, K., Hungerbuehler, I., Cavanagh, K., Claro, H. G., Swinton, P. A., & Kapps, M. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.576361 · Cited by 101 · see citing works
“Within mental health, chatbots have been used to deliver psychoeducation, self care strategies and skills training and show high satisfaction rates (14, 15).”
BackgroundSecurity, privacy, and healthcare-related conversational agents: a scoping review
May, R., & Denecke, K. (2021) · Informatics for Health and Social Care · doi:10.1080/17538157.2021.1983578 · Cited by 91 · see citing works
Deploying a robotic positive psychology coach to improve college students’ psychological well-being
Jeong, S., Aymerich-Franch, L., Arias, K., Alghowinem, S., Lapedriza, À., Picard, R. W., Park, H. W., & Breazeal, C. (2022) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-022-09337-8 · Cited by 90 · see citing works
Vaidyam, A. N., Linggonegoro, D. W., & Torous, J. B. (2020) · The Canadian Journal of Psychiatry · doi:10.1177/0706743720966429 · Cited by 83 · see citing works
A Robotic Positive Psychology Coach to Improve College Students’ Wellbeing
Jeong, S., Alghowinem, S., Aymerich-Franch, L., Arias, K., Lapedriza, À., Picard, R. W., Park, H. W., & Breazeal, C. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223588 · Cited by 86 · see citing works
Rehabilitation, the Great Absentee of Virtual Coaching in Medical Care: Scoping Review
Tropea, P., Schlieter, H., Sterpi, I., Judica, E., Gand, K., Caprino, M., Gabilondo, I., Gomez-Esteban, J. C., Busnatu, S., Sinescu, C., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/12805 · Cited by 73 · see citing works
Anisha, S. A., Sen, A., & Bain, C. (2024) · Journal of Medical Internet Research · doi:10.2196/56114 · Cited by 78 · see citing works
Jabir, A. I., Martinengo, L., Lin, X., Torous, J., Subramaniam, M., & Tudor Car, L. (2023) · Journal of Medical Internet Research · doi:10.2196/44548 · Cited by 63 · see citing works
“%) included embodied CAs [24,29-31, 39,43-50 43-50], 11 (”
InfluentialLimpanopparat, S., Gibson, E., & Harris, D. A. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100081 · Cited by 71 · see citing works
Human-Human vs Human-AI Therapy: An Empirical Study
Kuhail, M. A., Alturki, N. M., Thomas, J., Alkhalifa, A. K., & Alshardan, A. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2385001 · Cited by 68 · see citing works
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2019) · Verhaltenstherapie · doi:10.1159/000499492 · Cited by 62 · see citing works
Ding, H., Simmich, J., Vaezipour, A., Andrews, N. E., & Russell, T. G. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad222 · Cited by 57 · see citing works
Strika, Z., Petković, K., Likić, R., & Batenburg, R. S. (2024) · Postgraduate Medical Journal · doi:10.1093/postmj/qgae122 · Cited by 59 · see citing works
Martinengo, L., Jabir, A. I., Goh, W. W. T., Lo, N. Y. W., Ho, M. R., Kowatsch, T., Atun, R. A., Michie, S., & Car, L. T. (2022) · Journal of Medical Internet Research · doi:10.2196/39243 · Cited by 48 · see citing works
Kang, B., & Hong, M. (2024) · JMIR Medical Informatics · doi:10.2196/63538 · Cited by 42 · see citing works
Conversational agents and the making of mental health recovery
Meadows, R. P., Hine, C., & Suddaby, E. (2020) · Digital Health · doi:10.1177/2055207620966170 · Cited by 41 · see citing works
Kolenik, T., & Gams, M. (2021) · Electronics · doi:10.3390/electronics10111250 · Cited by 49 · see citing works
You'll be Great: Virtual Agent-based Cognitive Restructuring to Reduce Public Speaking Anxiety
Kimani, E., Bickmore, T., Trinh, H., & Pedrelli, P. (2019) · 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2019.8925438 · Cited by 41 · see citing works
Cho, Y., Rai, S., Ungar, L., Sedoc, J., & Guntuku, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.698 · Cited by 33 · see citing works
Talking Mental Health: a Battle of Wits Between Humans and AI
Lopes, E., Jain, G., Carlbring, P., & Pareek, S. (2023) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-023-00359-6 · Cited by 31 · see citing works
User experience with a parenting chatbot micro intervention
Entenberg, G. A., Dosovitsky, G., Aghakhani, S., Mostovoy, K., Carre, N., Marshall, Z., Benfica, D., Mizrahi, S., Testerman, A., Rousseau, A., et al. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.989022 · Cited by 29 · see citing works
Schillings, C., Meißner, E., Erb, B., Bendig, E., Schultchen, D., & Pollatos, O. (2024) · JMIR Mental Health · doi:10.2196/50454 · Cited by 44 · see citing works
Tong, A. C. Y., Wong, K. T. Y., Chung, W. W. T., & Mak, W. W. S. (2025) · Journal of Medical Internet Research · doi:10.2196/70436 · Cited by 28 · see citing works
Venning, A., Herd, M. C. E., Oswald, T., Razmi, S., Glover, F., Hawke, T., Quartermain, V., & Redpath, P. (2021) · Health Informatics Journal · doi:10.1177/1460458221994873 · Cited by 21 · see citing works
Wagner, D. N. (2020) · Telecommunications Policy · doi:10.1016/j.telpol.2020.101954 · Cited by 26 · see citing works
Automatic Thoughts and Facial Expressions in Cognitive Restructuring With Virtual Agents
Shidara, K., Tanaka, H., Adachi, H., Kanayama, D., Sakagami, Y., Kudo, T., & Nakamura, S. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.762424 · Cited by 23 · see citing works
The Implications of Embodied Artificial Intelligence in Mental Healthcare for Digital Wellbeing
Fiske, A., Henningsen, P., & Buyx, A. (2020) · Philosophical studies series · doi:10.1007/978-3-030-50585-1_10 · Cited by 22 · see citing works
Chatbots in the Field of Mental Health: Challenges and Opportunities
Viduani, A. C., Cosenza, V., Araújo, R., & Kieling, C. (2023) · Springer eBooks · doi:10.1007/978-3-031-10698-9_8 · Cited by 22 · see citing works
Provoost, S., Kleiboer, A. M., Ornelas, J., Bosse, T., Ruwaard, J., Rocha, A., Cuijpers, P. J., & Riper, H. M. (2020) · Trials · doi:10.1186/s13063-020-04777-2 · Cited by 19 · see citing works
Huang, M.-Y., Weng, C.-S., Kuo, H.-L., & Su, Y. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e15798 · Cited by 19 · see citing works
Depression intervention using AI chatbots with social cues: a randomized trial of effectiveness
Xu, S., & Ma, T. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119760 · Cited by 21 · see citing works
Fekih‐Romdhane, F., Cherif, W., Alhuwailah, A., Fawaz, M. A., Shuwiekh, H. A. M., Helmy, M. S., Hassan, I. H. M., Naser, A. Y., Zarrouq, B., Chebli, M., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2988215/v1 · Cited by 18 · see citing works
Schillings, C., Meißner, E., Erb, B., Schultchen, D., Bendig, E., & Pollatos, O. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1046202 · Cited by 17 · see citing works
Kocur, M., Dechant, M., Wolff, C., Nothdurfter, C., Wetter, T. C., Rupprecht, R., & Shiban, Y. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.608997 · Cited by 15 · see citing works
Lara‐Cabrera, M. L., Bjørkly, S., De las Cuevas, C., Pedersen, S. A., & Mundal, I. P. (2020) · Journal of Advanced Nursing · doi:10.1111/jan.14445 · Cited by 15 · see citing works
Influence of Rapport and Social Presence with an Ai Psychotherapy Chatbot on Users’ Self-Disclosure
Lee, J., Lee, J.-G., & Lee, D. (2022) · doi:10.2139/ssrn.4063508 · Cited by 127 · see citing works
Mancone, S., Diotaiuti, P., Valente, G., Corrado, S., Bellizzi, F., Vilarino, G. T., & Andrade, A. (2023) · Behavioral Sciences · doi:10.3390/bs13070550 · Cited by 14 · see citing works
Shidara, K., Tanaka, H., Adachi, H., Kanayama, D., Sakagami, Y., Kudo, T., & Nakamura, S. (2020) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3395035.3425223 · Cited by 14 · see citing works
Armaou, M. (2024) · Social Sciences · doi:10.3390/socsci13020114 · Cited by 10 · see citing works
A Review of Virtual Therapists in Anxiety and Phobias Alleviating Applications
Mitruț, O., Moldoveanu, A. D. B., Petrescu, L., Petrescu, C., & Moldoveanu, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77599-5_6 · Cited by 9 · see citing works
Conversational agents on smartphones and the web
Bickmore, T., & O’Leary, T. K. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90045-4.00010-1 · Cited by 9 · see citing works
Yi, Y., Kaiser‐Jackson, L., Bather, J. R., Goodman, M. S., Chavez‐Yenter, D., Bradshaw, R. L., Chambers, R. L., Espinel, W., Hess, R., Mann, D. M., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/73391 · Cited by 8 · see citing works
Gong, B., Yao, N., Xie, H., Huang, C., Tomoko, K., Berenbaum, H., & Mu, W. (2026) · Journal of Medical Internet Research · doi:10.2196/82677 · Cited by 18 · see citing works
A Review on Trustworthiness of Digital Assistants for Personal Healthcare
Bailoni, T., & Dragoni, M. (2025) · ACM Computing Surveys · doi:10.1145/3714999 · Cited by 7 · see citing works
Insights and lessons learned from trialling a mental health chatbot in the wild
Potts, C., Bond, R., Mulvenna, M. D., Ennis, E., Bickerdike, A., Coughlan, E. K., Broderick, T., Burns, C., McTear, M., Kuosmanen, L., et al. (2021) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc53001.2021.9631395 · Cited by 7 · see citing works
Multiple Composite Scenarios: A Game-Based Methodology for the Prevention of Mental Disorders
Lepe-Salazar, F., & Salgado-Torres, S. (2022) · Entertainment Computing · doi:10.1016/j.entcom.2022.100519 · Cited by 11 · see citing works
CuentosIE: can a chatbot about “tales with a message” help to teach emotional intelligence?
Ferrández, A., Lavigne-Cerván, R., Peral, J., Sória, I. N., Lloret, A. T., Gil, D., & Rocamora, C. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1866 · Cited by 10 · see citing works
Find Supports for the Post about Mental Issues: More Than Semantic Matching
Yun, Z., Liu, D., Wan, C., Liu, X., Qiu, X., & Nie, J. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3508373 · Cited by 6 · see citing works
Conversational Agents to Support Couple Therapy
Yuksel, B., & Kocaballi, A. B. (2022) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3572921.3572922 · Cited by 6 · see citing works
Wiratunga, N., Cooper, K., Wijekoon, A., Palihawadana, C., Mendham, V., Reiter, E., & Martin, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14067 · Cited by 19 · see citing works
Digital Companions for Well-being: Challenges and Opportunities
Nieves, J. C., Osorio, M., Rojas-Velázquez, D., Magallanes, Y., & Brännström, A. (2024) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-219336 · Cited by 5 · see citing works
Say what you want, I’m not listening!
González, A. L., Geiskkovitch, D. Y., & Young, J. E. (2023) · i-com · doi:10.1515/icom-2022-0047 · Cited by 5 · see citing works
Hopman, K., Richards, D., & Norberg, M. M. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2477757 · Cited by 4 · see citing works
van Velsen, L., Schokking, L., Siderakis, E., Knospe, G.-M., Brandl, L., Mooser, B., Madörin, S., Jacinto, S., Gouveia, A., & Brodbeck, J. (2023) · Death Studies · doi:10.1080/07481187.2023.2203680 · Cited by 5 · see citing works
Virtual Agents in DTx: Focusing on Usability and Therapeutic Effectiveness
Jeong, H., Yoo, J. H., Goh, M., & Song, H. (2023) · Electronics · doi:10.3390/electronics13010014 · Cited by 4 · see citing works
He, Y., Yang, L., Zhu, X., Wu, B., Zhang, S., Qian, C., & Tian, T. (2022) · doi:10.2196/preprints.40719 · Cited by 3 · see citing works
Lewis-Fung, S. E., Tchao, D., Gray, H. G., Nguyen, E., Pardini, S., Harris, L. R., Calabia, D., & Appel, L. (2023) · Frontiers in Virtual Reality · doi:10.3389/frvir.2023.1209535 · Cited by 3 · see citing works
Hopman, K., Richards, D., & Norberg, M. M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-58226-4_13 · Cited by 3 · see citing works
A Cognitive Agent for University Student Support
Schatten, M., Đurić, B. O., & Peharda, T. (2021) · doi:10.1109/temscon-eur52034.2021.9488627 · Cited by 3 · see citing works
Achieving CSR with Artificially Intelligent Nudging
Wagner, D. N. (2023) · CSR, sustainability, ethics & governance · doi:10.1007/978-3-031-09245-9_15 · Cited by 3 · see citing works
AI applications in diagnostic technologies and services
Catania, L. J. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-824477-7.00005-5 · Cited by 3 · see citing works
A Cognitive Agent's Infrastructure for Smart Mobility
Schatten, M., Đurić, B. O., Peharda, T., & Tomičić, I. (2022) · Transportation research procedia · doi:10.1016/j.trpro.2022.09.024 · Cited by 3 · see citing works
An Online Therapeutic Intervention for Veterans Patients Suffering with Chronic Pain
Reilly, E. D., Volonte, M., & Bickmore, T. (2023) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw58643.2023.00039 · Cited by 3 · see citing works
A Robotic Positive Psychology Coach to Improve College Students' WellbeingPreprint
Jeong, S., Alghowinem, S., Aymerich-Franch, L., Arias, K., Lapedriza, À., Picard, R. W., Park, H. W., & Breazeal, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.03829 · Cited by 3 · see citing works
Hayashi, Y., & Kiuchi, K. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610978.3640733 · Cited by 2 · see citing works
Robot-Mediated Multi-Party Conversation Aimed at Affect Improvement for Psychiatric Patients
Ochi, K., Lala, D., Inoue, K., Kawahara, T., & Kumazaki, H. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3571427 · Cited by 2 · see citing works
Kim, S. J., Swanson, V. A., Collier, G. H., Rabinowitz, A. R., Zondervan, D. K., & Reinkensmeyer, D. J. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3428915 · Cited by 3 · see citing works
Identifying Synthetic Voices’ Qualities for Conversational Agents
Cuciniello, M., Amorese, T., Cordasco, G., Marrone, S., Marulli, F., Cavallo, F., Gordeeva, O., Callejas, Z., & Esposito, A. A. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-24801-6_24 · Cited by 2 · see citing works
How Social Cues in Depression Intervention AI Chatbots Affect Human-Computer Trust
Xu, S., & Ma, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93502-2_7 · Cited by 1 · see citing works
AI-Based Chatbots for Mental Health: Comparative Study
Ech-Cheikh, S., Kharmoum, N., & Ziti, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-01970-7_6 · Cited by 1 · see citing works
Naumann, E. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.100952 · Cited by 1 · see citing works
Chen, X., Cheng, C.-Y., & Hartanto, A. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101267 · Cited by 1 · see citing works
Cognitive computing systems and public value: The case of chatbots and public service delivery
Makasi, T. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.230002 · Cited by 1 · see citing works
Human-Like Avatar Embodiment: Advantage or Disadvantage in Digital Emotion Regulation Intervention?Preprint
Naumann, E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5440041 · Cited by 1 · see citing works
Augmented Human-Centered Management – Personalentwicklung für hochautomatisierte Geschäftsfelder
Wagner, D. N. (2021) · Management-Reihe Corporate Social Responsibility · doi:10.1007/978-3-662-63223-9_16
Huang, M.-Y., Weng, C.-S., Kuo, H.-L., & Su, Y. (2021) · doi:10.2196/preprints.33824
Huang, M.-Y., Weng, C.-S., Kuo, H.-L., & Su, Y. (2021) · doi:10.2196/preprints.32984
Automated interventions for public speaking anxiety
Kimani, E. (2021) · doi:10.17760/d20439207
Duduciuc, A. C., Barbaros, M. C., & Ionescu, A. (2022) · The International Encyclopedia of Health Communication · doi:10.1002/9781119678816.iehc0728
A Review of the Use of Neural Models of Language and Conversation to Support Mental Health
Callejas, Z., Fernández-Martínez, F., Esposito, A. A., & Griol, D. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-99-3592-5_31
TBHEB 2023-4 Vol 80 Full Printed Journal
ERCÖMERT, U. (2023) · Turkish Bulletin of Hygiene and Experimental Biology · doi:10.5505/turkhijyen.2023.97572
An Intensive Role of Chatbot Services in the field of Healthcare
Edison, D., Goswami, S., Janakiraman, V., Almusawi, M., Kuppuraj, T., & Ramesh, A. (2024) · International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) · doi:10.1109/icacite60783.2024.10617084
- Authors
- Suganuma, S., Sakamoto, D., & Shimoyama, H.
- Affiliations
- The University of Tokyo (JP)
- Hokkaido University (JP)
- Corresponding author
- Suganuma, Shinichiro
- Venue
- JMIR Mental Health
- Publisher
- JMIR Publications Inc.
- Published
- 2018-07-31
- Volume
- 5
- Issue
- 3
- Pages
- e10454
- ISSN
- 2368-7959 (ISSN-L)
- PMC
- PMC6092592
- OpenAlex
- W2869048854
- Cited by
- 112
- References
- 29
- Citations by year
- 2026: 10 · 2025: 12 · 2024: 13 · 2023: 20 · 2022: 16 · 2021: 14 · 2020: 18 · 2019: 8 · 2018: 1
- FWCI
- 12.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Mental Health (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- University of Tokyo
- Japan Society for the Promotion of Science (18K13342)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Keywords
- internet-based cognitive behavioral therapy, therapeutic relationship, feasibility, acceptability, positive mental health, negative mental health, dropout
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
