Attrition in Conversational Agent–Delivered Mental Health Interventions: Systematic Review and Meta-Analysis
Open accessThe DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
Background: Conversational agents (CAs) or chatbots are computer programs that mimic human conversation. They have the potential to improve access to mental health interventions through automated, scalable, and personalized delivery of psychotherapeutic content. However, digital health interventions, including those delivered by CAs, often have high attrition rates. Identifying the factors associated with attrition is critical to improving future clinical trials. Objective: This review aims to estimate the overall and differential rates of attrition in CA-delivered mental health interventions (CA interventions), evaluate the impact of study design and intervention-related aspects on attrition, and describe study design features aimed at reducing or mitigating study attrition. Methods: We searched PubMed, Embase (Ovid), PsycINFO (Ovid), Cochrane Central Register of Controlled Trials, and Web of Science, and conducted a gray literature search on Google Scholar in June 2022. We included randomized controlled trials that compared CA interventions against control groups and excluded studies that lasted for 1 session only and used Wizard of Oz interventions. We also assessed the risk of bias in the included studies using the Cochrane Risk of Bias Tool 2.0. Random-effects proportional meta-analysis was applied to calculate the pooled dropout rates in the intervention groups. Random-effects meta-analysis was used to compare the attrition rate in the intervention groups with that in the control groups. We used a narrative review to summarize the findings. Results: The systematic search retrieved 4566 records from peer-reviewed databases and citation searches, of which 41 (0.90%) randomized controlled trials met the inclusion criteria. The meta-analytic overall attrition rate in the intervention group was 21.84% (95% CI 16.74%-27.36%; I2=94%). Short-term studies that lasted ≤8 weeks showed a lower attrition rate (18.05%, 95% CI 9.91%- 27.76%; I2=94.6%) than long-term studies that lasted >8 weeks (26.59%, 95% CI 20.09%-33.63%; I2=93.89%). Intervention group participants were more likely to attrit than control group participants for short-term (log odds ratio 1.22, 95% CI 0.99-1.50; I2=21.89%) and long-term studies (log odds ratio 1.33, 95% CI 1.08-1.65; I2=49.43%). Intervention-related characteristics associated with higher attrition include stand-alone CA interventions without human support, not having a symptom tracker feature, no visual representation of the CA, and comparing CA interventions with waitlist controls. No participant-level factor reliably predicted attrition. Conclusions: Our results indicated that approximately one-fifth of the participants will drop out from CA interventions in short-term studies. High heterogeneities made it difficult to generalize the findings. Our results suggested that future CA interventions should adopt a blended design with human support, use symptom tracking, compare CA intervention groups against active controls rather than waitlist controls, and include a visual representation of the CA to reduce the attrition rate. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42022341415; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022341415
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cumpston, M., Li, T., Page, M. J., Chandler, J., Welch, V. A., Higgins, J. P., & Thomas, J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.ed000142 · Cited by 7,218
McGuinness, L. A., & Higgins, J. P. T. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1411 · Cited by 5,579
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
Gulliver, A., Griffiths, K. M., & Christensen, H. (2010) · BMC Psychiatry · doi:10.1186/1471-244x-10-113 · Cited by 3,292
Clement, S., Schauman, O., Graham, T., Maggioni, F., Evans-Lacko, S., Bezborodovs, N., Morgan, C., Rüsch, N., Brown, J. S. L., & Thornicroft, G. (2015) · Psychological Medicine · doi:10.1017/s0033291714000129 · Cited by 3,152
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524
The role of google scholar in evidence reviews and its applicability to grey literature searching
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138237 · Cited by 2,387
Bramer, W. M., Rethlefsen, M. L., Kleijnen, J., & Franco, O. H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0644-y · Cited by 2,114
Doing Meta-Analysis with R: A Hands-On Guide
Harrer, M., Cuijpers, P., Furukawa, T. A., & Ebert, D. D. (2021) · doi:10.1201/9781003107347 · Cited by 1,850
Virtual Reality in the Assessment, Understanding, and Treatment of Mental Health Disorders
Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017) · Psychological Medicine · doi:10.1017/s003329171700040x · Cited by 1,428
Moltu, C., Stefansen, J., Svisdahl, M., & Veseth, M. (2012) · Disability and Rehabilitation · doi:10.3109/09638288.2012.656792 · Cited by 1,247
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157
Undertreatment of people with major depressive disorder in 21 countries
Thornicroft, G., Chatterji, S., Evans-Lacko, S., Gruber, M., Sampson, N., Aguilar-Gaxiola, S., Al-Hamzawi, A., Alonso, J., Andrade, L., Borges, G., et al. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.188078 · Cited by 1,121
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063
Torous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102891 · Cited by 790
Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018) · doi:10.2196/preprints.9782 · Cited by 715
Haddaway, N. R., Grainger, M. J., & Gray, C. T. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1563 · Cited by 604
Linardon, J., & Fuller-Tyszkiewicz, M. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000459 · Cited by 573
Conversational Agents in Health Care: Scoping Review and Conceptual Analysis
Tudor Car, L., Dhinagaran, D. A., Kyaw, B. M., Kowatsch, T., Joty, S., Theng, Y.-L., & Atun, R. (2020) · Journal of Medical Internet Research · doi:10.2196/17158 · Cited by 553
Self-citationTorous, J., Lipschitz, J., Ng, M., & Firth, J. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.167 · Cited by 546
Standalone smartphone apps for mental health—a systematic review and meta-analysis
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0188-8 · Cited by 447
Jakob, R., Harperink, S., Rudolf, A. M., Fleisch, E., Haug, S., Mair, J. L., Salamanca-Sanabria, A., & Kowatsch, T. (2022) · Journal of Medical Internet Research · doi:10.2196/35371 · Cited by 446
Berger, T., Hämmerli, K., Gubser, N., Andersson, G., & Caspar, F. (2011) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2011.616531 · Cited by 383
Ly, K. H., Ly, A.-M., & Andersson, G. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.10.002 · Cited by 368
Meyer, B., Berger, T., Caspar, F., Beevers, C. G., Andersson, G., & Weiss, M. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1151 · Cited by 363
Liu, H., Peng, H., Song, X., Xu, C., & Zhang, M. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100495 · Cited by 296
Greer, S., Ramo, D., Chang, Y.-J., Fu, M., Moskowitz, J., & Haritatos, J. (2019) · JMIR mHealth and uHealth · doi:10.2196/15018 · Cited by 265
Cooper, A. A., & Conklin, L. R. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.05.001 · Cited by 264
revtools: An R package to support article screening for evidence synthesis
Westgate, M. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1374 · Cited by 261
Prochaska, J. J., Vogel, E. A., Chieng, A., Kendra, M., Baiocchi, M., Pajarito, S., & Robinson, A. (2021) · Journal of Medical Internet Research · doi:10.2196/24850 · Cited by 257
Linardon, J., Shatte, A., Messer, M., Firth, J., & Fuller-Tyszkiewicz, M. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000575 · Cited by 218
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214
Klos, M. C., Escoredo, M., Joerin, A., Lemos, V. N., Rauws, M., & Bunge, E. L. (2021) · JMIR Formative Research · doi:10.2196/20678 · Cited by 200
Meyer, B., Bierbrodt, J., Schröder, J., Berger, T., Beevers, C. G., Weiss, M., Jacob, G., Späth, C., Andersson, G., Lutz, W., et al. (2015) · Internet Interventions · doi:10.1016/j.invent.2014.12.003 · Cited by 188
Lim, S. M., Shiau, C. W. C., Cheng, L. J., & Lau, Y. (2022) · Behavior Therapy · doi:10.1016/j.beth.2021.09.007 · Cited by 163
Berger, T., Krieger, T., Sude, K., Meyer, B., & Maercker, A. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.11.021 · Cited by 153
Talking to machines about personal mental health problems
Miner, A. S., Milstein, A., & Hancock, J. T. (2017) · JAMA · doi:10.1001/jama.2017.14151 · Cited by 153
Linardon, J., Hindle, A., & Brennan, L. (2018) · International Journal of Eating Disorders · doi:10.1002/eat.22850 · Cited by 146
Fischer, A., Schröder, J., Vettorazzi, E., Wolf, O. T., Pöttgen, J., Lau, S., Heesen, C., Moritz, S., & Gold, S. M. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)00049-2 · Cited by 143
A randomized controlled trial of internet-based therapy in depression
Moritz, S., Schilling, L., Hauschildt, M., Schröder, J., & Treszl, A. (2012) · Behaviour Research and Therapy · doi:10.1016/j.brat.2012.04.006 · Cited by 140
Burton, C., Szentagotai Tatar, A., McKinstry, B., Matheson, C., Matu, S., Moldovan, R., Macnab, M., Farrow, E., David, D., Pagliari, C., et al. (2016) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x15609793 · Cited by 125
Oh, J., Jang, S., Kim, H., & Kim, J.-J. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104171 · Cited by 124
Zwerenz, R., Becker, J., Knickenberg, R. J., Siepmann, M., Hagen, K., & Beutel, M. E. (2017) · Psychotherapy and Psychosomatics · doi:10.1159/000481177 · Cited by 108
Virtual training and coaching of health behavior: Example from mindfulness meditation training
Hudlicka, E. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2013.05.007 · Cited by 104
Twomey, C., O’Reilly, G., & Meyer, B. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.06.081 · Cited by 103
Taylor, H., Strauss, C., & Cavanagh, K. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102078 · Cited by 102
Berger, T., Urech, A., Krieger, T., Stolz, T., Schulz, A., Vincent, A., Moser, C. T., Moritz, S., & Meyer, B. (2017) · Psychological Medicine · doi:10.1017/s0033291716002270 · Cited by 98
Scholten, M. R., Kelders, S. M., & Van Gemert-Pijnen, J. E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7351 · Cited by 97
Schröder, J., Brückner, K., Fischer, A., Lindenau, M., Köther, U., Vettorazzi, E., & Moritz, S. (2014) · Epilepsia · doi:10.1111/epi.12833 · Cited by 96
Creating New Technologies for Companionable Agents to Support Isolated Older Adults
Sidner, C. L., Bickmore, T., Nooraie, B., Rich, C., Ring, L., Shayganfar, M., & Vardoulakis, L. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3213050 · Cited by 89
Gersh, E., Hallford, D. J., Rice, S. M., Kazantzis, N., Gersh, H., Gersh, B., & McCarty, C. A. (2017) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2017.10.001 · Cited by 82
The Singapore Mental Health Study: an overview of the methodology
Subramaniam, M., Vaingankar, J., Heng, D., Kwok, K. W., Lim, Y. W., Yap, M., & Chong, S. A. (2012) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1351 · Cited by 80
Brain drain: a challenge to global mental health
Oladeji, B. D., & Gureje, O. (2016) · BJPsych. International · doi:10.1192/s2056474000001240 · Cited by 79
Danieli, M., Ciulli, T., Mousavi, S. M., & Riccardi, G. (2021) · JMIR Formative Research · doi:10.2196/30053 · Cited by 64
Jabir, A. I., Martinengo, L., Lin, X., Torous, J., Subramaniam, M., & Tudor Car, L. (2023) · Journal of Medical Internet Research · doi:10.2196/44548 · Cited by 63
Self-citationBücker, L., Bierbrodt, J., Hand, I., Wittekind, C., & Moritz, S. (2018) · PLoS ONE · doi:10.1371/journal.pone.0198859 · Cited by 50
Bendig, E., Erb, B., Meißner, E., Bauereiß, N., & Baumeister, H. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100377 · Cited by 24
Cited by
Jiang, X.-H., Yuan, X.-H., Zhao, H., & Peng, J.-S. (2025) · Journal of Medical Internet Research · doi:10.2196/76968 · Cited by 10
User-centered development of a chatbot for diverse adolescents at high risk for eating disorders
Gullo, N. A., Goldberg, J., Howe, C. P., Smith, A. C., Firebaugh, M., Horton, M., Kouveliotes, M., Larson, G., Wilfley, D. E., Fowler, L. A., et al. (2026) · International Journal of Eating Disorders · doi:10.1002/eat.70008 · Cited by 2
Li, M., Kong, L., & Wang, C. (2026) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-026-07384-3
Metadata
- Authors
- Jabir, A. I., Lin, X., Martinengo, L., Sharp, G., Theng, Y.-L., & Tudor Car, L.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-02-27
- Volume
- 26
- Pages
- e48168
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4389316278
- Cited by
- 66
- References
- 59
- FWCI
- 2.29
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.