CoChat

Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring

Open access
Sharma, Ashish, Rushton, Kevin, Lin, Inna Wanyin, Nguyen, Theresa, Althoff, Tim · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems
Cited by 102

The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.

Abstract

Self-guided mental health interventions, such as “do-it-yourself” tools to learn and practice coping strategies, show great promise to improve access to mental health care. However, these interventions are often cognitively demanding and emotionally triggering, creating accessibility barriers that limit their wide-scale implementation and adoption. In this paper, we study how human-language model interaction can support self-guided mental health interventions. We take cognitive restructuring, an evidence-based therapeutic technique to overcome negative thinking, as a case study. In an IRB-approved randomized field study on a large mental health website with 15,531 participants, we design and evaluate a system that uses language models to support people through various steps of cognitive restructuring. Our findings reveal that our system positively impacts emotional intensity for 67% of participants and helps 65% overcome negative thoughts. Although adolescents report relatively worse outcomes, we find that tailored interventions that simplify language model generations improve overall effectiveness and equity.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642761

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 35 references

Research works printed but not linked: 63

  1. 10.1176/appi.ps.56.9.1061 · DOI · Paper page
  2. Tal August Katharina Reinecke and Noah A Smith. 2022. Generating scientific definitions with controllable complexity. In ACL. 8298–8317. · DOI
  3. 10.1111/j.1467-9841.2008.00353.x · DOI · Paper page
  4. Tom L Beauchamp and James F Childress. 2001. Principles of biomedical ethics. Oxford University Press, USA.
  5. Aaron T Beck. 1976. Cognitive therapy and the emotional disorders.International Universities Press.
  6. Aaron T Beck, Denise D Davis, and Arthur Freeman. 2015. Cognitive therapy of personality disorders. Guilford Publications.
  7. 10.1001/jama.1975.03260240050026 · DOI · Paper page
  8. Christopher G Beevers, Tony T Wells, and Ivan W Miller. 2007. Predicting response to depression treatment: The role of negative cognition.Journal of Consulting and Clinical Psychology 75, 3 (2007), 422.
  9. Robert A Bjork. 2017. Creating desirable difficulties to enhance learning. Carmarthen: Crown House Publishing (2017).
  10. Robert A Bjork and Elizabeth L Bjork. 2020. Desirable difficulties in theory and practice.Journal of Applied research in Memory and Cognition 9, 4 (2020), 475.
  11. Ann Blandford, Jo Gibbs, Nikki Newhouse, Olga Perski, Aneesha Singh, and Elizabeth Murray. 2018. Seven lessons for interdisciplinary research on interactive digital health interventions. Digital health 4 (2018), 2055207618770325.
  12. David D Burns. 1980. Feeling Good: The New Mood Therapy. Avon New York.
  13. Andrew C Butler, Jason E Chapman, Evan M Forman, and Aaron T Beck. 2006. The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical psychology review 26, 1 (2006), 17–31.
  14. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183–186.
  15. 10.1177/20552076231183542 · DOI · Paper page
  16. Meri Coleman and Ta Lin Liau. 1975. A computer readability formula designed for machine scoring.Journal of Applied Psychology 60, 2 (1975), 283.
  17. Sona Dimidjian, Manuel Barrera Jr, Christopher Martell, Ricardo F Muñoz, and Peter M Lewinsohn. 2011. The origins and current status of behavioral activation treatments for depression. Annual review of clinical psychology 7 (2011), 1–38.
  18. 10.18653/v1/2022.naacl-srw.9 · DOI
  19. Luciano Floridi and Josh Cowls. 2022. A Unified Framework of Five Principles for AI in Society. Harvard Data Science Review (2022), 535–545.
  20. Sandra Garrido, Chris Millington, Daniel Cheers, Katherine Boydell, Emery Schubert, Tanya Meade, and Quang Vinh Nguyen. 2019. What works and what doesn’t work? A systematic review of digital mental health interventions for depression and anxiety in young people. Frontiers in psychiatry 10 (2019), 759.
  21. Manas Gaur Amanuel Alambo Joy Prakash Sain Ugur Kursuncu Krishnaprasad Thirunarayan Ramakanth Kavuluru Amit Sheth Randy Welton and Jyotishman Pathak. 2019. Knowledge-aware assessment of severity of suicide risk for early intervention. In WWW. 514–525. · DOI · Paper page
  22. Liesl M Heinrich and Eleonora Gullone. 2006. The clinical significance of loneliness: A literature review. Clinical psychology review 26, 6 (2006), 695–718.
  23. Stefan G Hofmann, Anu Asnaani, Imke JJ Vonk, Alice T Sawyer, and Angela Fang. 2012. The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive therapy and research 36 (2012), 427–440.
  24. Ari Holtzman Jan Buys Li Du Maxwell Forbes and Yejin Choi. 2020. The curious case of neural text degeneration. In ICLR.
  25. Esther Howe Jina Suh Mehrab Bin Morshed Daniel McDuff Kael Rowan Javier Hernandez Marah Ihab Abdin Gonzalo Ramos Tracy Tran and Mary P Czerwinski. 2022. Design of digital workplace stress-reduction intervention systems: Effects of intervention type and timing. In CHI. 1–16. · DOI
  26. 10.1089/tmj.2016.0114 · DOI · Paper page
  27. Natalie E Hundt, Joseph Mignogna, Cathy Underhill, and Jeffrey A Cully. 2013. The relationship between use of CBT skills and depression treatment outcome: A theoretical and methodological review of the literature. Behavior therapy 44, 1 (2013), 12–26.
  28. Eirini Karyotaki, Annet Kleiboer, Filip Smit, David Trevor Turner, A Mira Pastor, Gerhard Andersson, Thomas Berger, Cristina Botella, JM Breton, Per Carlbring, 2015. Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’meta-analysis. Psychological medicine 45, 13 (2015), 2717–2726.
  29. Rachel Kornfield, Jonah Meyerhoff, Hannah Studd, Ananya Bhattacharjee, Joseph Jay Williams, Madhu Reddy, and David C Mohr. 2022. Meeting Users Where They Are: User-centered Design of an Automated Text Messaging Tool to Support the Mental Health of Young Adults. In CHI. 1–16.
  30. Kaylee P Kruzan, Ellen E Fitzsimmons-Craft, Mallory Dobias, Jessica L Schleider, and Abhishek Pratap. 2022. Developing, deploying, and evaluating digital mental health interventions in spaces of online help-and information-seeking. Procedia computer science 206 (2022), 6–22.
  31. Kaylee Payne Kruzan Jonah Meyerhoff Theresa Nguyen Madhu Reddy David C Mohr and Rachel Kornfield. 2022. “I Wanted to See How Bad it Was”: Online Self-screening as a Critical Transition Point Among Young Adults with Common Mental Health Conditions. In CHI. 1–16. · DOI
  32. Harsh Kumar, Yiyi Wang, Jiakai Shi, Ilya Musabirov, Norman AS Farb, and Joseph Jay Williams. 2023. Exploring the Use of Large Language Models for Improving the Awareness of Mindfulness. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–7.
  33. 10.18653/v1/W19-3002 · DOI
  34. Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard-Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, 2022. Evaluating human-language model interaction. arXiv preprint arXiv:2212.09746 (2022). · arXiv
  35. Ron C Li, Steven M Asch, and Nigam H Shah. 2020. Developing a delivery science for artificial intelligence in healthcare. NPJ digital medicine 3, 1 (2020), 107.
  36. Kien Hoa Ly, Ann-Marie Ly, and Gerhard Andersson. 2017. A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods. Internet interventions 10 (2017), 39–46.
  37. Tara Matthews Kathleen O’Leary Anna Turner Manya Sleeper Jill Palzkill Woelfer Martin Shelton Cori Manthorne Elizabeth F Churchill and Sunny Consolvo. 2017. Stories from survivors: Privacy & security practices when coping with intimate partner abuse. In CHI. 2189–2201. · DOI · Paper page
  38. 10.2196/40242 · DOI · Paper page
  39. World Health Organization 2022. Mental health and COVID-19: early evidence of the pandemic’s impact: scientific brief, 2 March 2022. Technical Report. World Health Organization.
  40. Verónica Pérez-Rosas, Kenneth Resnicow, Rada Mihalcea, 2022. PAIR: Prompt-Aware margIn Ranking for Counselor Reflection Scoring in Motivational Interviewing. In EMNLP. 148–158.
  41. Verónica Pérez-Rosas Xinyi Wu Kenneth Resnicow and Rada Mihalcea. 2019. What Makes a Good Counselor? Learning to Distinguish between High-quality and Low-quality Counseling Conversations. In ACL. 926–935. · DOI
  42. 10.1007/s13142-013-0214-3 · DOI · Paper page
  43. Yada Pruksachatkun Sachin R Pendse and Amit Sharma. 2019. Moments of Change: Analyzing Peer-Based Cognitive Support in Online Mental Health Forums. In CHI. 1–13. · DOI · Paper page
  44. Stefan Rennick-Egglestone, Sarah Knowles, Gill Toms, Penny Bee, Karina Lovell, and Peter Bower. 2016. Health Technologies’ In the Wild’ Experiences of Engagement with Computerised CBT. In CHI. 2124–2135.
  45. 10.1177/10598405060220010401 · DOI · Paper page
  46. 10.1145/3421937.3421975 · DOI
  47. Stephen M Schueller, Martha Neary, Jocelyn Lai, and Daniel A Epstein. 2021. Understanding people’s use of and perspectives on mood-tracking apps: interview study. JMIR 8, 8 (2021), e29368.
  48. Raj Sanjay Shah, Faye Holt, Shirley Anugrah Hayati, Aastha Agarwal, Yi-Chia Wang, Robert E Kraut, and Diyi Yang. 2022. Modeling Motivational Interviewing Strategies On An Online Peer-to-Peer Counseling Platform. CSCW 6, 2 (2022), 1–24.
  49. Ashish Sharma Monojit Choudhury Tim Althoff and Amit Sharma. 2020. Engagement Patterns of Peer-to-Peer Interactions on Mental Health Platforms. In ICWSM Vol. 14. 614–625. · DOI
  50. 10.1145/3442381.3450097 · DOI · Paper page
  51. Ashish Sharma, Kevin Rushton, Inna Lin, David Wadden, Khendra Lucas, Adam Miner, Theresa Nguyen, and Tim Althoff. 2023. Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction. In ACL, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 9977–10000.
  52. 10.2196/46062 · DOI · Paper page
  53. Chaitali Sinha, Abby L Cheng, and Madhura Kadaba. 2022. Adherence and Engagement With a Cognitive Behavioral Therapy–Based Conversational Agent (Wysa for Chronic Pain) Among Adults With Chronic Pain: Survival Analysis. JMIR 6, 5 (2022), e37302.
  54. 10.1145/3479561 · DOI
  55. 10.1080/15374416.2021.1875325 · DOI
  56. Wally Smith Greg Wadley Sarah Webber Benjamin Tag Vassilis Kostakos Peter Koval and James J Gross. 2022. Digital emotion regulation in everyday life. In CHI. 1–15. · DOI
  57. Jiyoung Song, Boris Litvin, Ryan Allred, Shiyu Chen, Thomas D Hull, and Patricia A Areán. 2023. Comparing Message-Based Psychotherapy to Once-Weekly, Video-Based Psychotherapy for Moderate Depression: Randomized Controlled Trial. JMIR 25 (2023), e46052.
  58. 10.1145/3543507.3583380 · DOI
  59. Elizabeth Stade, Shannon Wiltsey Stirman, Lyle H Ungar, David Bryce Yaden, H Andrew Schwartz, João Sedoc, Robb Willer, Robert DeRubeis, 2023. Artificial Intelligence Will Change the Future of Psychotherapy: A Proposal for Responsible, Psychologist-led Development. (2023).
  60. Daniel R Strunk, Shannon N Hollars, Abby D Adler, Lizabeth A Goldstein, and Justin D Braun. 2014. Assessing patients’ cognitive therapy skills: Initial evaluation of the competencies of cognitive therapy scale. Cognitive therapy and research 38 (2014), 559–569.
  61. Steven Tate Sajjad Fouladvand Jonathan H Chen and Chwen-Yuen Angie Chen. 2023. The ChatGPT therapist will see you now: Navigating generative artificial intelligence’s potential in addiction medicine research and patient care. 2249–2251 pages. · DOI
  62. Dave Wadden Tal August Qisheng Li and Tim Althoff. 2021. The Effect of Moderation on Online Mental Health Conversations. In ICWSM Vol. 15. 751–763. · DOI
  63. Caleb Ziems Minzhi Li Anthony Zhang and Diyi Yang. 2022. Inducing Positive Perspectives with Text Reframing. In ACL. 3682–3700. · DOI · Paper page

Cited by

See all 102

Metadata

Authors
Sharma, A., Rushton, K., Lin, I. W., Nguyen, T., & Althoff, T.
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-29
OpenAlex
W4396832979
Cited by
102
References
35
FWCI
42.05
Type
conference paper
Language
en
Open access
hybrid
Funders
  • NIH (National Institutes of Health) (R01MH125179)
  • NSF (National Science Foundation) (CAREER IIS-2142794 and IIS-1901386)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring. https://app.cochat.ai/papers/10.1145/3613904.3642761
The paper
Sharma, A., Rushton, K., Lin, I. W., Nguyen, T., & Althoff, T. (2024). Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642761

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.