Modeling Motivational Interviewing Strategies on an Online Peer-to-Peer Counseling Platform
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Millions of people participate in online peer-to-peer support sessions, yet there has been little prior research on systematic psychology-based evaluations of fine-grained peer-counselor behavior in relation to client satisfaction. This paper seeks to bridge this gap by mapping peer-counselor chat-messages to motivational interviewing (MI) techniques. We annotate 14,797 utterances from 734 chat conversations using 17 MI techniques and introduce four new interviewing codes such as ''chit-chat'' and ''inappropriate'' to account for the unique conversational patterns observed on online platforms. We automate the process of labeling peer-counselor responses to MI techniques by fine-tuning large domain-specific language models and then use these automated measures to investigate the behavior of the peer counselors via correlational studies. Specifically, we study the impact of MI techniques on the conversation ratings to investigate the techniques that predict clients' satisfaction with their counseling sessions. When counselors use techniques such as reflection and affirmation, clients are more satisfied. Examining volunteer counselors' change in usage of techniques suggest that counselors learn to use more introduction and open questions as they gain experience. This work provides a deeper understanding of the use of motivational interviewing techniques on peer-to-peer counselor platforms and sheds light on how to build better training programs for volunteer counselors on online platforms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review
Martin, D. J., Garske, J. P., & Davis, M. K. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.3.438 · Cited by 3,355
A Meta-Analysis of Motivational Interviewing: Twenty-Five Years of Empirical Studies
Lundahl, B. W., Kunz, C., Brownell, C., Tollefson, D., & Burke, B. L. (2010) · Research on Social Work Practice · doi:10.1177/1049731509347850 · Cited by 1,086 · cited 2×
“Similar to previous works onMI and counseling success in offline settings [2, 29, 30], we also found that MI-consistent techniques were predictive of better outcomes (defined here as client satisfaction).”
BackgroundResultPeer Support/Peer Provided Services Underlying Processes, Benefits, and Critical Ingredients.
Solomon, P. (2004) · Psychiatric Rehabilitation Journal · doi:10.2975/27.2004.392.401 · Cited by 1,060
The Descriptive Epidemiology of Commonly Occurring Mental Disorders in the United States
Kessler, R. C., & Wang, P. S. (2008) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.29.020907.090847 · Cited by 888 · cited 1×
“Additionally, peer-support can offer a solution to the historical inaccessibility of traditional mental health services, which has resulted in a majority of mental problems being untreated [23].”
BackgroundArdito, R. B., & Rabellino, D. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00270 · Cited by 647
Dropout and therapeutic alliance: A meta-analysis of adult individual psychotherapy.
Sharf, J., Primavera, L. H., & Diener, M. J. (2010) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/a0021175 · Cited by 352
Althoff, T., Clark, K., & Leskovec, J. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00111 · Cited by 322 · cited 6×
“We utilize client satisfaction as an outcome metric to explore the value of individual MI techniques, following prior work [1, 5, 25].”
BackgroundMethodInfluentialArtificial Intelligence-Assisted Online Social Therapy for Youth Mental Health
D'Alfonso, S., Santesteban-Echarri, O., Rice, S., Wadley, G., Lederman, R., Miles, C., Gleeson, J., & Alvarez-Jimenez, M. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00796 · Cited by 192 · cited 1×
“Researchers are starting to build integrated artificial intelligence (AI) application to improve the effectiveness of those participating in online support platforms [12, 41, 46].”
BackgroundWang, X., Zhao, K., & Street, N. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6834 · Cited by 182
Mental Health Support and its Relationship to Linguistic Accommodation in Online Communities
Sharma, E., & De Choudhury, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174215 · Cited by 156 · cited 1×
“While the direct incorporation of AI applications into OMHCs is still in preliminary stages, there have been several applications of AI to analyze linguistic patterns and explore factors that might be responsible for counseling success [1, 51].”
BackgroundMetaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146
Peng, Z., Guo, Q., Tsang, K. W., & Ma, X. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376695 · Cited by 81 · cited 1×
“Researchers are starting to build integrated artificial intelligence (AI) application to improve the effectiveness of those participating in online support platforms [12, 41, 46].”
BackgroundBoldt, K., Coenen, M., Movsisyan, A., Voss, S., Rehfuess, E., Kunzler, A. M., Lieb, K., & Jung-Sievers, C. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18052361 · Cited by 72 · cited 1×
“The number of support seekers using online peer counseling platforms and online mental health communities (OMHCs) has grown significantly in the last few years [53] and has seen a surge in demand due to the COVID-19 pandemic [6].”
BackgroundCausal Factors of Effective Psychosocial Outcomes in Online Mental Health Communities
Saha, K., & Sharma, A. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7326 · Cited by 72
Baumel, A. (2015) · Journal of Mental Health · doi:10.3109/09638237.2015.1079308 · Cited by 44 · cited 5×
“Prior research indicates that peer support services can be an integral part of mental health recovery [48] and benefits support seekers [5], even though most peer counselors have not received professional training [13].”
BackgroundMethodInfluential
Cited by
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
Self-citationCan Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
Self-citationStade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00056-z · Cited by 391
“Asanotherexample,LLMapplicationshavebeenusedtoidentifytherapists ’ and clients ’ behaviors in a motivational interviewing framework 8,9 .”
BackgroundUsing large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369
Self-citationTherapy as an NLP Task: Comparing LLMs and Human Peers Behaviors in CBT Sessions
Iftikhar, Z., Ransom, S., Xiao, A., Nugent, N., & Huang, J. (2024) · arXiv · doi:10.48550/arxiv.2409.02244 · Cited by 6
Metadata
- Authors
- Shah, R. S., Holt, F., Hayati, S. A., Agarwal, A., Wang, Y.-C., Kraut, R. E., & Yang, D.
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2022-11-07
- Volume
- 6
- Issue
- CSCW2
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4308827648
- Cited by
- 49
- References
- 16
- FWCI
- 6.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.