Understanding Client Support Strategies to Improve Clinical Outcomes in an Online Mental Health Intervention
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
Online mental health interventions are increasingly important in providing access to, and supporting the effectiveness of, mental health treatment. While these technologies are effective, user attrition and early disengagement are key challenges. Evidence suggests that integrating a human supporter into such services mitigates these challenges, however, it remains under-studied how supporter involvement benefits client outcomes, and how to maximize such effects. We present our analysis of 234,735 supporter messages to discover how different support strategies correlate with clinical outcomes. We describe our machine learning methods for: (i) clustering supporters based on client outcomes; (ii) extracting and analyzing linguistic features from supporter messages; and (iii) identifying context-specific patterns of support. Our findings indicate that concrete, positive and supportive feedback from supporters that reference social behaviors are strongly associated with better outcomes; and show how their importance varies dependent on different client situations. We discuss design implications for personalized support and supporter interfaces.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Whiteford, H. A., Degenhardt, L., Rehm, J., Baxter, A. J., Ferrari, A. J., Erskine, H. E., Charlson, F. J., Norman, R. E., Flaxman, A. D., Johns, N., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)61611-6 · Cited by 6,006
Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., & Herzberg, P. Y. (2008) · Medical Care · doi:10.1097/mlr.0b013e318160d093 · Cited by 4,527
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524
Estimating the true global burden of mental illness
Vigo, D., Thornicroft, G., & Atun, R. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00505-2 · Cited by 2,494
Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., & Murphy, S. A. (2018) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9830-8 · Cited by 2,171 · cited 1×
“Other recent trends include the development of just-in-time adaptive interventions (JITAIs) that utilize algorithms to provide the right type or amount of support, at the right time, in response to an individuals’ internal and contextual state [60].”
MethodNock, M. K., Hwang, I., Sampson, N. A., & Kessler, R. C. (2010) · Molecular Psychiatry · doi:10.1038/mp.2009.29 · Cited by 1,125 · cited 1×
“It presents the largest cause of disability worldwide and is the strongest predictor of suicide [64, 99].”
BackgroundWang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116 · cited 2×
“As is common in ML approaches for mental health, user privacy should be carefully addressed to preserve anonymity (cf. phone data [10, 101], sensors [77], social media [68, 85, 105]), and analysis should occur in a context where there is a clear case for the prospective benefits that could arise,…”
BackgroundMohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · cited 1×
“…the important relationship and genuine human connection that is formed between supporter and client and crucial to their alliance and positive outcomes [9, 58]; and (ii) ensure supporters feel that their input and expertise is valued rather than made redundant or replaced in favor of data science.”
BackgroundMachine learning in mental health: a scoping review of methods and applications
Shatte, A. B. R., Hutchinson, D. M., & Teague, S. J. (2019) · Psychological Medicine · doi:10.1017/s0033291719000151 · Cited by 1,047
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922 · cited 2×
“Over the last decade, this has brought forward developments of mobile apps [19, 29, 55], and computerized psycho-educational and psycho-therapeutic interventions [5, 17, 81, 103], or chat-based [27, 48, 88] programs to complement, and expand access to, psychotherapy.”
BackgroundMichie, S., Yardley, L., West, R., Patrick, K., & Greaves, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7126 · Cited by 865
Internet-delivered psychological treatments: from innovation to implementation
Andersson, G., Titov, N., Dear, B. F., Rozental, A., & Carlbring, P. (2019) · World Psychiatry · doi:10.1002/wps.20610 · Cited by 750 · cited 1×
“Further, extensive research has evidenced the clinical effectiveness of internet-delivered CBT (iCBT) with sustainable results comparable to face-to-face therapy [4, 5, 100, 103].”
BackgroundAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Working Alliance in Online Therapy as Compared to Face-to-Face Therapy: Preliminary Results
Cook, J. E., & Doyle, C. (2002) · CyberPsychology & Behavior · doi:10.1089/109493102753770480 · Cited by 503
Newman, M. G., Szkodny, L. E., Llera, S. J., & Przeworski, A. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.09.008 · Cited by 480 · cited 2×
“However, existing research on the effectiveness of supported-interventions has primarily assessed the impact of support duration and frequency [32, 45, 61, 96]; and to a lesser extent, different types of supporter behaviors [36, 69, 87].”
BackgroundAlthoff, T., Clark, K., & Leskovec, J. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00111 · Cited by 322 · cited 3×
“This surprisingly contradicts previous findings by [3], and thus, requires further research.”
BackgroundResultInfluentialWagner, B., Horn, A. B., & Maercker, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.06.032 · Cited by 311
Johansson, R., Sjöberg, E., Sjögren, M., Johnsson, E., Carlbring, P., Andersson, T., Rousseau, A., & Andersson, G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036905 · Cited by 268 · cited 1×
“To measure clinical outcomes over a period of time, Randomized Controlled Trials (RCTs) tend to assess the difference between the clients’ initial and final clinical scores [97, 41].”
BackgroundBringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250
Passive and In-Situ assessment of mental and physical well-being using mobile sensors
Rabbi, M., Ali, S., Choudhury, T., & Berke, E. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030164 · Cited by 243
Computer-Assisted Cognitive-Behavior Therapy for Depression: A Systematic Review and Meta-Analysis
Wright, J. H., Owen, J. J., Richards, D., Eells, T. D., Richardson, T., Brown, G. K., Barrett, M., Rasku, M. A., Polser, G., & Thase, M. E. (2019) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.18r12188 · Cited by 200
Self-citationDoherty, G., Coyle, D., & Sharry, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208602 · Cited by 198
Self-citationPocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy
Schroeder, J., Wilkes, C., Rowan, K., Toledo, A., Paradiso, A., Czerwinski, M., Mark, G., & Linehan, M. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173972 · Cited by 175
Paxling, B., Lundgren, S., Norman, A., Almlöv, J., Carlbring, P., Cuijpers, P., & Andersson, G. (2013) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465812000240 · Cited by 162 · cited 7×
“This includes quantitative and qualitative content analysis of therapist emails to clients receiving iCBT for depression [36], anxiety [69] or bulimia [87].”
BackgroundInfluentialInternet-delivered psychotherapy for depression in adults
Titov, N. (2011) · Current Opinion in Psychiatry · doi:10.1097/yco.0b013e32833ed18f · Cited by 158 · cited 4×
“The literature however is less conclusive on how differences in communication medium [51], frequency [16] and duration [96] of support impact outcomes.”
BackgroundInfluentialMental Health Support and its Relationship to Linguistic Accommodation in Online Communities
Sharma, E., & De Choudhury, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174215 · Cited by 156
Sánchez-Ortiz, V. C., Munro, C., Stahl, D., House, J., Startup, H., Treasure, J., Williams, C., & Schmidt, U. (2011) · Psychological Medicine · doi:10.1017/s0033291710000711 · Cited by 154
PopTherapy: Coping with Stress through Pop-Culture
Paredes, P., Giald-Bachrach, R., Czerwinski, M., Roseway, A., Rowan, K., & Hernandez, J. (2014) · Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare · doi:10.4108/icst.pervasivehealth.2014.255070 · Cited by 152 · cited 1×
“[67] applied reinforcement learning for recommending stress coping interventions tailored to the person.”
BackgroundModerated online social therapy: Designing and evaluating technology for mental health
Lederman, R., Wadley, G., Gleeson, J., Bendall, S., & Álvarez-Jiménez, M. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2513179 · Cited by 151
Nonparametric bootstrapping for hierarchical data
Ren, S., Lai, H., Tong, W., Aminzadeh, M., Hou, X., & Lai, S. (2010) · Journal of Applied Statistics · doi:10.1080/02664760903046102 · Cited by 148
DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection
Cao, B., Zheng, L., Zhang, C., Yu, P. S., Piscitello, A., Zulueta, J., Ajilore, O., Ryan, K., & Leow, A. D. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098086 · Cited by 145 · cited 2×
“As is common in ML approaches for mental health, user privacy should be carefully addressed to preserve anonymity (cf. phone data [10, 101], sensors [77], social media [68, 85, 105]), and analysis should occur in a context where there is a clear case for the prospective benefits that could arise,…”
BackgroundModeling Stress with Social Media Around Incidents of Gun Violence on College Campuses
Saha, K., & De Choudhury, M. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134727 · Cited by 124
Mohr, D. C., Schueller, S. M., Tomasino, K. N., Kaiser, S. M., Alam, N., Karr, C., Vergara, J. L., Gray, E. L., Kwasny, M. J., & Lattie, E. G. (2019) · Journal of Medical Internet Research · doi:10.2196/13609 · Cited by 116
Richards, D., Timulak, L., O'Brien, E., Hayes, C., Vigano, N., Sharry, J., & Doherty, G. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.10.005 · Cited by 109
Self-citationRooksby, J., Morrison, A., & Murray-Rust, D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300655 · Cited by 87
Schueller, S. M., Tomasino, K. N., & Mohr, D. C. (2017) · Clinical Psychology: Science and Practice · doi:10.1037/h0101740 · Cited by 70
Therapist behaviours in internet-based cognitive behaviour therapy (ICBT) for depressive symptoms
Holländare, F., Gustafsson, S. A., Berglind, M., Grape, F., Carlbring, P., Andersson, G., Hadjistavropoulos, H., & Tillfors, M. (2016) · Internet Interventions · doi:10.1016/j.invent.2015.11.002 · Cited by 65
Chen, A. T., Wu, S., Tomasino, K. N., Lattie, E. G., & Mohr, D. C. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103187 · Cited by 58 · cited 1×
“Clustering & Text Mining in (Large-Scale) Mental Health Data Clustering is an unsupervised ML technique to identify behavioral patterns in data through commonalities in each data piece; it is often used to identify features in unstructured data ( e.g. [2, 14, 68]).”
MethodRichards, D., & Timulak, L. (2012) · Counselling Psychology Quarterly · doi:10.1080/09515070.2012.703129 · Cited by 40 · cited 4×
“The research suggests that such an alliance can be formed within iCBT environments [65] with consistent evidence of the benefits of support in those interventions [40, 82, 89, 103].”
BackgroundMethodInfluentialSelf-citationLindner, P., Olsson, E. L., Johnsson, A., Dahlin, M., Andersson, G., & Carlbring, P. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.09.001 · Cited by 39
Mining Free-Text Medical Notes for Suicide Risk Assessment
Adamou, M., Antoniou, G., Greasidou, E., Lagani, V., Charonyktakis, P., & Tsamardinos, I. (2018) · Proceedings of the 10th Hellenic Conference on Artificial Intelligence · doi:10.1145/3200947.3201020 · Cited by 22 · cited 3×
“Outside of social media, text mining is used to predict suicide risks from SMS messages [63], suicide notes [71], and EHRs [2] to aid care provision.”
BackgroundPatient experience of supported computerized CBT in an inner-city IAPT service: a qualitative study
Perera-Delcourt, R. P., & Sharkey, G. (2019) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x18000284 · Cited by 14 · cited 1×
“This often means increasing opportunities for: (i) interactivity ; (ii) personalized experiences ; and (iii) social support through a community of (peer) moderators, trained supporters, or remote therapists [21, 50, 70, 79].”
Background
Cited by
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
The promise of machine learning in predicting treatment outcomes in psychiatry
Chekroud, A. M., Bondar, J., Delgadillo, J., Doherty, G., Wasil, A., Fokkema, M., Cohen, Z., Belgrave, D., DeRubeis, R., Iniesta, R., et al. (2021) · World Psychiatry · doi:10.1002/wps.20882 · Cited by 531
“Chikersal et al 88 analyzed 234,735 messages sent from clini cal supporters to clients within an iCBT platform, examining how support strategies correlate with clinical outcomes.”
BackgroundSelf-citationThieme, A., Belgrave, D., & Doherty, G. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3398069 · Cited by 406
“[34] Prerna Chikersal, Danielle Belgrave, Gavin Doherty, Angel Enrique, Jorge E.”
BackgroundSelf-citationWhat is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“…around 20% of the total papers researched these human-AI collaborative teams in various capacities and domains [3, 14, 23, 24, 36, 49, 58, 61, 64, 66, 76, 82, 100, 116, 117, 128, 139, 143, 149, 163, 164, 179, 181, 184, 189, 201, 205, 217, 218, 220, 223, 254, 255, 260, 266–270, 275, 276, 287,…”
BackgroundInfluentialChien, I., Enrique, A., Palacios, J., Regan, T., Keegan, D., Carter, D., Tschiatschek, S., Nori, A., Thieme, A., Richards, D., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.10791 · Cited by 111
Self-citation
Metadata
- Authors
- Chikersal, P., Belgrave, D., Doherty, G., Enrique, A., Palacios, J. E., Richards, D., & Thieme, A.
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-16
- OpenAlex
- W3002082332
- Cited by
- 63
- References
- 45
- FWCI
- 19.33
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.