"Rate My Therapist": Automated Detection of Empathy in Drug and Alcohol Counseling via Speech and Language Processing
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
The technology for evaluating patient-provider interactions in psychotherapy-observational coding-has not changed in 70 years. It is labor-intensive, error prone, and expensive, limiting its use in evaluating psychotherapy in the real world. Engineering solutions from speech and language processing provide new methods for the automatic evaluation of provider ratings from session recordings. The primary data are 200 Motivational Interviewing (MI) sessions from a study on MI training methods with observer ratings of counselor empathy. Automatic Speech Recognition (ASR) was used to transcribe sessions, and the resulting words were used in a text-based predictive model of empathy. Two supporting datasets trained the speech processing tasks including ASR (1200 transcripts from heterogeneous psychotherapy sessions and 153 transcripts and session recordings from 5 MI clinical trials). The accuracy of computationally-derived empathy ratings were evaluated against human ratings for each provider. Computationally-derived empathy scores and classifications (high vs. low) were highly accurate against human-based codes and classifications, with a correlation of 0.65 and F-score (a weighted average of sensitivity and specificity) of 0.86, respectively. Empathy prediction using human transcription as input (as opposed to ASR) resulted in a slight increase in prediction accuracies, suggesting that the fully automatic system with ASR is relatively robust. Using speech and language processing methods, it is possible to generate accurate predictions of provider performance in psychotherapy from audio recordings alone. This technology can support large-scale evaluation of psychotherapy for dissemination and process studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kuhn, M., & Johnson, K. (2013) · doi:10.1007/978-1-4614-6849-3 · Cited by 6,788 · cited 1×
“[37] Here the empathy prediction model was trained on all but one counselor’s sessions and then the model predicted the empathy of the one, left-out counselor’s sessions based on the words generated by the ASR transcripts for those sessions, meaning that predictions of empathy are fully automatic and independent to th…”
Method(1992) · doi:10.5555/176313.176316 · Cited by 2,892
Conditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443 · cited 1×
“[43] However, specific, immediate feedback is precisely what is missing from the practice of psychotherapy.”
BackgroundToward a theory of motivational interviewing
Miller, W. R., & Rose, G. S. (2009) · American Psychologist · doi:10.1037/a0016830 · Cited by 1,603 · cited 1×
“[9] There is an international organization devoted to the training and dissemination of MI (The Motivational Interviewing Network of Trainers [MINT]), and MI developers have been at the forefront of research on training and dissemination.”
BackgroundAssessing competence in the use of motivational interviewing
Moyers, T. B., Martin, T., Manuel, J. K., Hendrickson, S. M. L., & Miller, W. R. (2005) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2004.11.001 · Cited by 738 · cited 1×
“segment (5–6:1) [11]–and this cost does not scale up as more sessions are coded due to concerns with coder turnover (e.”
BackgroundCognitive therapy versus medication for depression: treatment outcomes and neural mechanisms
DeRubeis, R. J., Siegle, G. J., & Hollon, S. D. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2345 · Cited by 690 · cited 1×
“[6] Imaging techniques have documented the effects of psychotherapy on the brain [7], and computerized adaptations of psychotherapies now have the potential to augment human providers.”
BackgroundDetecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494
Sustaining motivational interviewing: A meta-analysis of training studies
Schwalbe, C. S., Oh, H. Y., & Zweben, A. (2014) · Addiction · doi:10.1111/add.12558 · Cited by 343
Internet-delivered treatment for substance abuse: a multisite randomized controlled trial
Campbell, A. N. C., Nunes, E. V., Matthews, A. G., Stitzer, M., Miele, G. M., Polsky, D., Turrigiano, E., Walters, S., McClure, E. A., Kyle, T. L., et al. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.13081055 · Cited by 263
Psychological treatments: A call for mental-health science
Holmes, E. A., Craske, M. G., & Graybiel, A. M. (2014) · Nature · doi:10.1038/511287a · Cited by 214 · cited 1×
“[6] Imaging techniques have documented the effects of psychotherapy on the brain [7], and computerized adaptations of psychotherapies now have the potential to augment human providers.”
Background
Cited by
Morris, R. R., Kouddous, K., Kshirsagar, R., & Schueller, S. M. (2018) · Journal of Medical Internet Research · doi:10.2196/10148 · Cited by 273
Illuminating the dark spaces of healthcare with ambient intelligence
Haque, A., Milstein, A., & Fei-Fei, L. (2020) · Nature · doi:10.1038/s41586-020-2669-y · Cited by 274
“Although this is lower than the 90% accuracy of a single human assessor 120 , ambient intelligence can more readily be applied to a larger number of patients.”
BackgroundMethodInfluentialA Scoping Review of Machine Learning in Psychotherapy Research
Aafjes-van Doorn, K., Kamsteeg, C., Bate, J., & Aafjes, M. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1808729 · Cited by 179
Key Considerations for Incorporating Conversational AI in Psychotherapy
Miner, A. S., Shah, N., Bullock, K. D., Arnow, B. A., Bailenson, J., & Hancock, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00746 · Cited by 166
Tanana, M. J., Soma, C. S., Srikumar, V., Atkins, D. C., & Imel, Z. E. (2019) · Journal of Medical Internet Research · doi:10.2196/12529 · Cited by 100
“This new technology allows for the possibility of a computer giving immediate feedback that would not be possible with human raters [25,26].”
Self-citationParish, M. B., Fazio, S., Chan, S., & Yellowlees, P. M. (2017) · Current Psychiatry Reports · doi:10.1007/s11920-017-0839-x · Cited by 50
Berkel, C., Gallo, C. G., Sandler, I. N., Mauricio, A. M., Smith, J. D., & Brown, C. H. (2019) · The Journal of Primary Prevention · doi:10.1007/s10935-018-00534-z · Cited by 27
Metadata
- Authors
- Xiao, B., Imel, Z. E., Georgiou, P. G., Atkins, D. C., & Narayanan, S. S.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-12-02
- Volume
- 10
- Issue
- 12
- Pages
- e0143055
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2181445435
- Cited by
- 111
- References
- 13
- FWCI
- 10.08
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.