The Challenges in Designing a Prevention Chatbot for Eating Disorders: Observational Study
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Background: Chatbots have the potential to provide cost-effective mental health prevention programs at scale and increase interactivity, ease of use, and accessibility of intervention programs. Objective: The development of chatbot prevention for eating disorders (EDs) is still in its infancy. Our aim is to present examples of and solutions to challenges in designing and refining a rule-based prevention chatbot program for EDs, targeted at adult women at risk for developing an ED. Methods: Participants were 2409 individuals who at least began to use an EDs prevention chatbot in response to social media advertising. Over 6 months, the research team reviewed up to 52,129 comments from these users to identify inappropriate responses that negatively impacted users’ experience and technical glitches. Problems identified by reviewers were then presented to the entire research team, who then generated possible solutions and implemented new responses. Results: The most common problem with the chatbot was a general limitation in understanding and responding appropriately to unanticipated user responses. We developed several workarounds to limit these problems while retaining some interactivity. Conclusions: Rule-based chatbots have the potential to reach large populations at low cost but are limited in understanding and responding appropriately to unanticipated user responses. They can be most effective in providing information and simple conversations. Workarounds can reduce conversation errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prevalence of eating disorders over the 2000–2018 period: a systematic literature review
Galmiche, M., Déchelotte, P., Lambert, G., & Tavolacci, M. P. (2019) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqy342 · Cited by 1,712 · cited 1×
“A recent meta-analysis found EDs prevalence rates of about 8% in women in the Western world, as well as evidence that these disorders are prevalent worldwide [2].”
BackgroundChatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape
Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719828977 · Cited by 1,048 · cited 2×
“While chatbots in various forms are becoming widespread, few studies have evaluated their effectiveness in the prevention of mental health problems [14,22,23].”
BackgroundFulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018) · doi:10.2196/preprints.9782 · Cited by 715
Eisenberg, D., Nicklett, E. J., Roeder, K., & Kirz, N. E. (2011) · Journal of American College Health · doi:10.1080/07448481.2010.546461 · Cited by 634
An overview of the features of chatbots in mental health: A scoping review
Abd-alrazaq, A. A., Alajlani, M., Alalwan, A. A., Bewick, B. M., Gardner, P., & Househ, M. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103978 · Cited by 605 · cited 1×
“A systematic review in 2019 found that only 1 out of 41 mental health chatbots targeted EDs [33].”
BackgroundSurvey on Chatbot Design Techniques in Speech Conversation Systems
A., S., & John, D. (2015) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2015.060712 · Cited by 479
Prevention of Eating Disorders in At-Risk College-Age Women
Taylor, C. B., Bryson, S., Luce, K. H., Cunning, D., Doyle, A. C., Abascal, L. B., Rockwell, R., Dev, P., Winzelberg, A. J., & Wilfley, D. E. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.8.881 · Cited by 354 · cited 7×
“On the assumption that internet-based programs can provide easy and convenient access to EDs prevention, we developed a cognitive behavior–based program called Student Bodies (C).”
BackgroundMethodInfluentialSelf-citationA Survey on Conversational Agents/Chatbots Classification and Design Techniques
Hussain, S., Ameri Sianaki, O., & Ababneh, N. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-15035-8_93 · Cited by 279
Mohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216 · cited 1×
“Consistent with Mohr et al [45], our general approach was not to change any of the core content or activities of the program but to improve the users’experience by fixing bugs and reducing problematic chatbot responses through transcript review.”
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214 · cited 1×
“While chatbots in various forms are becoming widespread, few studies have evaluated their effectiveness in the prevention of mental health problems [14,22,23].”
BackgroundLearning from Dialogue after Deployment: Feed Yourself, Chatbot!
Hancock, B., Bordes, A., Mazare, P.-E., & Weston, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1358 · Cited by 161
An Overview of Machine Learning in Chatbots
Suta, P., Lan, X., Wu, B., Mongkolnam, P., & Chan, J. H. (2020) · International Journal of Mechanical Engineering and Robotics Research · doi:10.18178/ijmerr.9.4.502-510 · Cited by 152
Technical metrics used to evaluate health care chatbots: scoping review
Abd-Alrazaq, A., Safi, Z., Alajlani, M., Warren, J., Househ, M., & Denecke, K. (2020) · Journal of Medical Internet Research · doi:10.2196/18301 · Cited by 150
Beintner, I., Jacobi, C., & Taylor, C. B. (2012) · European Eating Disorders Review · doi:10.1002/erv.1130 · Cited by 139 · cited 2×
“Human-moderated (guided) versions of Student Bodies have been shown to be associated with moderate improvements in ED-related attitudes, including reductions in negative body image and the desire to be thin [9-12].”
BackgroundSelf-citationArtificial Intelligence Chatbot for Depression: Descriptive Study of Usage.
Dosovitsky, G., Pineda, B. S., Jacobson, N. C., Chang, C., Escoredo, M., & Bunge, E. L. (2020) · JMIR Formative Research · doi:10.2196/17065 · Cited by 118 · cited 1×
“In mental health chatbots, they can be used in areas outside the scope of the core content, such as suicidality [29,30].”
BackgroundGraff Low, K., Charanasomboon, S., Lesser, J., Reinhalter, K., Martin, R., Jones, H., Winzelberg, A., Abascal, L., & Taylor, C. B. (2006) · Eating Disorders · doi:10.1080/10640260500403816 · Cited by 96
Self-citationKass, A. E., Trockel, M., Safer, D. L., Sinton, M. M., Cunning, D., Rizk, M. T., Genkin, B. H., Weisman, H. L., Bailey, J. O., Jacobi, C., et al. (2014) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.09.010 · Cited by 71
Self-citationTaylor, C. B., Kass, A. E., Trockel, M., Cunning, D., Weisman, H., Bailey, J., Sinton, M., Aspen, V., Schecthman, K., Jacobi, C., et al. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000077 · Cited by 70
Self-citationA Chatbot System for Mental Healthcare Based on SAT Counseling Method
Kamita, T., Ito, T., Matsumoto, A., Munakata, T., & Inoue, T. (2019) · Mobile Information Systems · doi:10.1155/2019/9517321 · Cited by 69 · cited 1×
“Research has found that, relative to an internet browser–based program, a chatbot-based program was associated with higher ease of use and increased response rate [20,21].”
Background
Cited by
Stade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00056-z · Cited by 391
Use of AI in Mental Health Care: Community and Mental Health Professionals Survey
Cross, S., Bell, I., Nicholas, J., Valentine, L., Mangelsdorf, S., Baker, S., Titov, N., & Alvarez-Jimenez, M. (2024) · JMIR Mental Health · doi:10.2196/60589 · Cited by 113
Does the Digital Therapeutic Alliance Exist? Integrative Review
Malouin-Lachance, A., Capolupo, J., Laplante, C., & Hudon, A. (2025) · JMIR Mental Health · doi:10.2196/69294 · Cited by 103
Ulrich, S., Gantenbein, A. R., Zuber, V., Von Wyl, A., Kowatsch, T., & Künzli, H. (2024) · Journal of Medical Internet Research · doi:10.2196/50132 · Cited by 36
“Natural language processing and large language models enable the CA to interpret user inputs more dynamically, yet existing AI-based chatbots struggle with unforeseen user responses [133,134].”
Ebner, P., & Szczuka, J. (2025) · arXiv · doi:10.48550/arxiv.2503.00195 · Cited by 33
Metadata
- Authors
- Chan, W. W., Fitzsimmons-Craft, E. E., Smith, A. C., Firebaugh, M.-L., Fowler, L. A., DePietro, B., Topooco, N., Wilfley, D. E., Taylor, C. B., & Jacobson, N. C.
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2022-01-19
- Volume
- 6
- Issue
- 1
- Pages
- e28003
- ISSN
- 2561-326X (ISSN-L)
- OpenAlex
- W4205111021
- Cited by
- 50
- References
- 19
- FWCI
- 8.68
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.