Computational psychotherapy research: scaling up the evaluation of patient–provider interactions
Open accessThe DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Abstract
In psychotherapy, the patient-provider interaction contains the treatment's active ingredients. However, the technology for analyzing the content of this interaction has not fundamentally changed in decades, limiting both the scale and specificity of psychotherapy research. New methods are required to "scale up" to larger evaluation tasks and "drill down" into the raw linguistic data of patient-therapist interactions. In the current article, we demonstrate the utility of statistical text analysis models called topic models for discovering the underlying linguistic structure in psychotherapy. Topic models identify semantic themes (or topics) in a collection of documents (here, transcripts). We used topic models to summarize and visualize 1,553 psychotherapy and drug therapy (i.e., medication management) transcripts. Results showed that topic models identified clinically relevant content, including affective, relational, and intervention related topics. In addition, topic models learned to identify specific types of therapist statements associated with treatment-related codes (e.g., different treatment approaches, patient-therapist discussions about the therapeutic relationship). Visualizations of semantic similarity across sessions indicate that topic models identify content that discriminates between broad classes of therapy (e.g., cognitive-behavioral therapy vs. psychodynamic therapy). Finally, predictive modeling demonstrated that topic model-derived features can classify therapy type with a high degree of accuracy. Computational psychotherapy research has the potential to scale up the study of psychotherapy to thousands of sessions at a time. We conclude by discussing the implications of computational methods such as topic models for the future of psychotherapy research and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“If a session were recorded, the labor intensiveness of human review would ensure most sessions would never be analyzed (8).”
From Key Considerations for Incorporating Conversational AI in Psychotherapy (2019) · cites this paper as [8]
In the citing paper
“Additionally, conversational AI is providing clinicians with feedback on their psychotherapy (8)”
From Key Considerations for Incorporating Conversational AI in Psychotherapy (2019) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“As is typical with large-scale psychotherapy clinical trials, there have already been published comments (Clark, 2013) and rejoinders (Leichsenring & Salzer, 2013) on possible explanations for the findings wherein Clark raised questions about the implementation of the CBT and Leichsenring reported…”
Background(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Griffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049
The necessary and sufficient conditions of therapeutic personality change.
Rogers, C. R. (1957) · Journal of Consulting Psychology · doi:10.1037/h0045357 · Cited by 5,599
Bergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
Feature-rich part-of-speech tagging with a cyclic dependency network
Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073478 · Cited by 2,844
Motivational Interviewing: Preparing People for Change, 2nd ed.
Miller, W., & Rollnick, S. (2003) · Journal For Healthcare Quality · doi:10.1097/01445442-200305000-00013 · Cited by 2,634
Research works printed but not linked: 1
- Wampold BE. The great psychotherapy debate: Models, methods, and findings. Erlbaum; Mahwah, NJ: 2001. [Google Scholar]
Cited by
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
“Thus, empowering peer supporters with automated, actionable, just-in-time feedback and training, for example, through human–AI collaboration systems, can help them express higher levels of empathy and, as a result, improve the overall effectiveness of these platforms 29,45–47 .”
BackgroundSelf-citationA Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support
Sharma, A., Miner, A., Atkins, D., & Althoff, T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.425 · Cited by 268
“Our approach is aimed towards developing empathy-based feedback and training systems for peer supporters (consistent with calls to action for improved treatment access and training (Miner et al., 2019; Imel et al., 2015; Kazdin and Rabbitt, 2013)).”
BackgroundSelf-citationAn Introduction to Artificial Intelligence in Behavioral and Mental Health Care
Luxton, D. D. (2016) · Artificial Intelligence in Behavioral and Mental Health Care · doi:10.1016/b978-0-12-420248-1.00001-5 · Cited by 206
Malgaroli, M., Hull, T. D., Zech, J. M., & Althoff, T. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02592-2 · Cited by 201
Advances in the treatment of anorexia nervosa: a review of established and emerging interventions
Brockmeyer, T., Friederich, H.-C., & Schmidt, U. (2018) · Psychological Medicine · doi:10.1017/s0033291717002604 · Cited by 183
“With the availability of novel tools for computational psychotherapy research it is now possible to process much larger amounts of data (Imel et al. 2015; Owen & Imel, 2016), using text mining and machine learning approaches to investigate the underlying linguistic structures and semantic themes of psychotherapy sessi…”
MethodA Scoping Review of Machine Learning in Psychotherapy Research
Aafjes-van Doorn, K., Kamsteeg, C., Bate, J., & Aafjes, M. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1808729 · Cited by 179
Key Considerations for Incorporating Conversational AI in Psychotherapy
Miner, A. S., Shah, N., Bullock, K. D., Arnow, B. A., Bailenson, J., & Hancock, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00746 · Cited by 166
“If a session were recorded, the labor intensiveness of human review would ensure most sessions would never be analyzed (8).”
BackgroundSharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3450097 · Cited by 163
“Empowering peer supporters on online support platforms with feedback and training, for example through machine-in-the-loop writing systems [9, 64], has the potential to help supporters express higher levels of empathy and in turn improve the effectiveness of these platforms [26, 44, 59].”
BackgroundSelf-citationBrockmeyer, T., Zimmermann, J., Kulessa, D., Hautzinger, M., Bents, H., Friederich, H.-C., Herzog, W., & Backenstrass, M. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01564 · Cited by 151
“Finally, future studies on language and psychopathology may not only use quantitative text analysis as applied by the LIWC but also other methods like n-grams and topic models that could help to reveal linguistic structures associated with depression and anxiety (Imel et al., 2015).”
MethodGoldberg, S. B., Flemotomos, N., Martinez, V. R., Tanana, M. J., Kuo, P. B., Pace, B. T., Villatte, J. L., Georgiou, P. G., Van Epps, J., Imel, Z. E., et al. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000382 · Cited by 141
“…in the literature, including using NLP and ML to reliably code motivational interviewing treatment fidelity (Atkins, Steyvers, Imel, & Smyth, 2014; Imel et al., in press), to differentiate classes of psychotherapy (e.g., cognitive behavioral therapy and psychodynamic psychotherapy; Imel, Steyvers, & Atkins, 2015),…”
MethodSelf-citationTanana, M., Hallgren, K. A., Imel, Z. E., Atkins, D. C., & Srikumar, V. (2016) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2016.01.006 · Cited by 137
“Moreover, they are fundamentally different models relative to topic models or maximum entropy models tested in previous research (Atkins et al., 2014; Can et al., in press; Imel et al., 2015).”
MethodSelf-citationIn pursuit of truth: A critical examination of meta-analyses of cognitive behavior therapy
Wampold, B. E., Flückiger, C., Del Re, A. C., Yulish, N. E., Frost, N. D., Pace, B. T., Goldberg, S. B., Miller, S. D., Baardseth, T. P., Laska, K. M., et al. (2017) · Psychotherapy Research · doi:10.1080/10503307.2016.1249433 · Cited by 131
A multivariate meta-analysis of motivational interviewing process and outcome.
Pace, B. T., Dembe, A., Soma, C. S., Baldwin, S. A., Atkins, D. C., & Imel, Z. E. (2017) · Psychology of Addictive Behaviors · doi:10.1037/adb0000280 · Cited by 116
Self-citationXiao, B., Imel, Z. E., Georgiou, P. G., Atkins, D. C., & Narayanan, S. S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0143055 · Cited by 111
“First, a large collection of 1,200 psychotherapy transcripts were used to help define the typical vocabulary and language use patterns of psychotherapy sessions (see [15] for more details on the corpus as well as the technical supplement (S1 File) and below for an overview of the speech recognition process).”
BackgroundResultSelf-citationAutomated evaluation of psychotherapy skills using speech and language technologies
Flemotomos, N., Martinez, V. R., Chen, Z., Singla, K., Ardulov, V., Peri, R., Caperton, D. D., Gibson, J., Tanana, M. J., Georgiou, P., et al. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01623-4 · Cited by 98
Self-citationImel, Z. E., Pace, B. T., Soma, C. S., Tanana, M., Hirsch, T., Gibson, J., Georgiou, P., Narayanan, S., & Atkins, D. C. (2019) · Psychotherapy · doi:10.1037/pst0000221 · Cited by 97
Self-citationHow do you feel? Using natural language processing to automatically rate emotion in psychotherapy
Tanana, M. J., Soma, C. S., Kuo, P. B., Bertagnolli, N. M., Dembe, A., Pace, B. T., Srikumar, V., Atkins, D. C., & Imel, Z. E. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01531-z · Cited by 91
Self-citationTechnology-enhanced human interaction in psychotherapy.
Imel, Z. E., Caperton, D. D., Tanana, M., & Atkins, D. C. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000213 · Cited by 84
Self-citationAssessing the accuracy of automatic speech recognition for psychotherapy
Miner, A. S., Haque, A., Fries, J. A., Fleming, S. L., Wilfley, D. E., Terence Wilson, G., Milstein, A., Jurafsky, D., Arnow, B. A., Stewart Agras, W., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0285-8 · Cited by 81
Moutoussis, M., Shahar, N., Hauser, T. U., & Dolan, R. J. (2018) · Computational Psychiatry · doi:10.1162/cpsy_a_00014 · Cited by 80
Routine outcome monitoring: Coming of age—With the usual developmental challenges.
Wampold, B. E. (2015) · Psychotherapy · doi:10.1037/pst0000037 · Cited by 76
Linguistic analysis of patients with mood and anxiety disorders during cognitive behavioral therapy
Sonnenschein, A. R., Hofmann, S. G., Ziegelmayer, T., & Lutz, W. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1419505 · Cited by 68
Zhang, M., Scandiffio, J., Younus, S., Jeyakumar, T., Karsan, I., Charow, R., Salhia, M., & Wiljer, D. (2023) · JMIR Formative Research · doi:10.2196/47847 · Cited by 67
“For example, natural language processing can be used to monitor psychotherapy sessions by analyzing the semantic content of each session’s transcripts [9].”
BackgroundHow to Identify Hot Topics in Psychology Using Topic Modeling
Bittermann, A., & Fischer, A. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000318 · Cited by 69
Stiles, W. B., Hill, C., & Elliott, R. (2015) · Psychotherapy Research · doi:10.1080/10503307.2014.981681 · Cited by 62
Xiao, B., Huang, C., Imel, Z. E., Atkins, D. C., Georgiou, P., & Narayanan, S. S. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.59 · Cited by 50
Self-citationLanguage style matching in psychotherapy: An implicit aspect of alliance.
Doorn, K. A., Porcerelli, J. H., & Müller-Frommeyer, L. C. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000433 · Cited by 64
Creed, T. A., Salama, L., Slevin, R., Tanana, M., Imel, Z. E., Narayanan, S. S., & Atkins, D. C. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08519-9 · Cited by 59
Self-citationHow therapist self-disclosure relates to alliance and outcomes: A naturalistic study
Levitt, H. M., Minami, T., Greenspan, S. B., Puckett, J. A., Henretty, J. R., Reich, C. M., & Berman, J. S. (2015) · Counselling Psychology Quarterly · doi:10.1080/09515070.2015.1090396 · Cited by 61
Detecting changes in attitudes toward depression on Chinese social media: A text analysis
Yu, L., Jiang, W., Ren, Z., Xu, S. H., Zhang, L., & Hu, X. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.11.040 · Cited by 53
Xezonaki, D., Paraskevopoulos, G., Potamianos, A., & Narayanan, S. S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2819 · Cited by 73
“Moreover, for the General Psychotherapy Corpus 1 , Malandrakis et al. [11] have explored differences in language between therapist and client using psycholinguistic norms and Imel et al. [12] have identified semantic topics discussed in therapy sessions.”
Greene, A. L., Watts, A. L., Forbes, M. K., Kotov, R., Krueger, R. F., & Eaton, N. R. (2022) · Psychological Methods · doi:10.1037/met0000465 · Cited by 49
Content Coding of Psychotherapy Transcripts Using Labeled Topic Models
Gaut, G., Steyvers, M., Imel, Z. E., Atkins, D. C., & Smyth, P. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2503985 · Cited by 47
“As in past research [12], [13], for the General Psychotherapy Corpus, we define documents to be individual talk-turns (although other definitions are possible as well).”
BackgroundSelf-citationHirsch, T., Soma, C., Merced, K., Kuo, P., Dembe, A., Caperton, D. D., Atkins, D. C., & Imel, Z. E. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196776 · Cited by 42
“Unfortunately, this approach is time consuming, expensive, and does not scale up to real-world settings [1, 23].”
BackgroundSelf-citationLiu, S., Zhang, R., & Kishimoto, T. (2020) · Psychology Health & Medicine · doi:10.1080/13548506.2020.1738019 · Cited by 42
“In these scenarios, large amounts of linguistic data are generated and often cannot bemanually categorized (D. C. Atkins et al., 2012; Carron-Arthur et al., 2016; Howes et al., 2013; Imel et al., 2015).”
BackgroundBehavioral Coding of Therapist Language in Addiction Counseling Using Recurrent Neural Networks
Xiao, B., Can, D., Gibson, J., Imel, Z. E., Atkins, D. C., Georgiou, P., & Narayanan, S. S. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1560 · Cited by 47
“5M words) is added to the word-embedding training [30].”
BackgroundSelf-citationThe Structure of Competence: Evaluating the Factor Structure of the Cognitive Therapy Rating Scale
Goldberg, S. B., Baldwin, S. A., Merced, K., Caperton, D. D., Imel, Z. E., Atkins, D. C., & Creed, T. A. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.05.008 · Cited by 48
“The potential availability of technologies capable of automating session coding would support this possibility (Imel et al., 2015).”
BackgroundSelf-citationDimensions of Religion and Spirituality: A Longitudinal Topic Modeling Approach
Kim, S., Lee, N., & King, P. E. (2020) · Journal for the Scientific Study of Religion · doi:10.1111/jssr.12639 · Cited by 56
Measuring Alliance and Symptom Severity in Psychotherapy Transcripts Using Bert Topic Modeling
Lalk, C., Steinbrenner, T., Kania, W., Popko, A., Wester, R. A., Schaffrath, J., Eberhardt, S. T., Schwartz, B., Lutz, W., & Rubel, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01356-4 · Cited by 40
“Imel et al. (2015) showed that topic modeling creates highly relevant topics in individual therapies.”
BackgroundPublic Attitudes Toward Anxiety Disorder on Sina Weibo: Content Analysis
Zhu, J., Li, Z., Zhang, X., Zhang, Z., & Hu, B. (2023) · Journal of Medical Internet Research · doi:10.2196/45777 · Cited by 30
“This is because a questionnaire is an effective way of collecting information in research; other potentially richer measures of attitude, such as qualitative analysis, are often not feasible in large-scale studies due to labor costs [18].”
Friederich, H., Brockmeyer, T., Wild, B., Resmark, G., de Zwaan, M., Dinkel, A., Herpertz, S., Burgmer, M., Löwe, B., Tagay, S., et al. (2017) · Psychotherapy and Psychosomatics · doi:10.1159/000453582 · Cited by 22
Naar, S., MacDonell, K. K., Chapman, J. E., Todd, L., Gurung, S., Cain, D. N., Dilones, R. E., & Parsons, J. T. (2019) · JMIR Research Protocols · doi:10.2196/11200 · Cited by 22
“There is a growing body of literature supporting the educational use of standardized patients in teaching and learning [47,48], including teaching MI skills and practice [49,50].”
Imel, Z. E., Tanana, M., Soma, C. S., Hull, T. D., Pace, B. T., Stanco, S. C., Creed, T. A., Moyers, T. B., & Atkins, D. C. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.52590 · Cited by 35
Self-citationCunningham, P. B., Gilmore, J., Naar, S., Preston, S. D., Eubanks, C. F., Hubig, N., McClendon, J. L., Ghosh, S., & Ryan-Pettes, S. R. (2023) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-023-00451-6 · Cited by 26
“The research by Imel et al. (2015) over the past five years represents a major leap in this domain, illustrating the power of NLP and machine learning to supersede conventional dictionary-based methods, like linguistic inquiry and word count.”
BackgroundMachine-Learning-Based Prediction of Client Distress From Session Recordings
Kuo, P. B., Tanana, M., Goldberg, S. B., Caperton, D. D., Narayanan, S. S., Atkins, D. C., & Imel, Z. E. (2023) · Clinical Psychological Science · doi:10.1177/21677026231172694 · Cited by 27
“For example, topic models have been used to automatically assess motivational-interviewing fidelity (Atkins et al., 2014) and classify types of psychotherapy treatment used by providers (Imel, Steyvers, & Atkins, 2015).”
MethodSelf-citationUnderstanding Therapeutic Change Process Research Through Multilevel Modeling and Text Mining
Smink, W. A. C., Fox, J., Sang, E. T. K., Sools, A. M., Westerhof, G. J., & Veldkamp, B. P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01186 · Cited by 24
“Questions pertaining to the change processes that are beneficial to clients necessitate investigation of the “active ingredients” of therapy, of which many are linguistic (Muntigl and Horvath, 2005; Imel et al., 2015).”
BackgroundMethodBroadbent, M., Grespan, M. M., Axford, K., Zhang, X., Srikumar, V., Kious, B. M., & Imel, Z. E. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1110527 · Cited by 36
“Natural language processing (NLP) is one tool that holds significant promise for developing scalable methods for evaluating the content of crisis counseling (25).”
MethodSelf-citationLeveraging Novel Technologies and Artificial Intelligence to Advance Practice-Oriented Research
Atzil–Slonim, D., Penedo, J. M. G., & Lutz, W. (2023) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01309-3 · Cited by 29
“The technology used to analyze the client-therapist interactions, the active component of psychotherapy, has not undergone significant change in decades, thereby restricting the scale and specificity of process-outcome research (Imel et al., 2015 ).”
Individual-Specific Animated Profiles of Mental Health
Zilcha‐Mano, S. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231226308 · Cited by 19
“Computational psychotherapy is a new approach that to date has been used only sporadically to assess particular aspects of therapy, such as the linguistic one (Imel et al., 2015).”
Scoping review on natural language processing applications in counselling and psychotherapy
Laricheva, M., Liu, Y., Shi, E., & Wu, A. D. (2024) · British Journal of Psychology · doi:10.1111/bjop.12721 · Cited by 26
“Multiclass coding has been conducted to distinguish among therapy stages (Nitti et al., 2010), therapy approaches (Imel et al., 2015), emotions (Tanana et al., 2021) and sentiment expressions (Syzdek, 2020).”
InfluentialTherapy language analysis using automatically generated psycholinguistic norms
Malandrakis, N., & Narayanan, S. S. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-430 · Cited by 17
“As noted in other studies [18], this corpus has some unfavorable characteristics.”
ResultLeveraging the Power of Nondisruptive Technologies to Optimize Mental Health Treatment: Case Study
Sadeh‐Sharvit, S., & Hollon, S. D. (2020) · JMIR Mental Health · doi:10.2196/20646 · Cited by 18
“These capabilities enable timelier identification of trends in patients’ symptoms and the issues troubling them, as well as changes in the therapeutic alliance and the therapeutic relationship [23].”
BackgroundTopics in the Journal of Counseling Psychology, 1963–2015.
Oh, J., Stewart, A. E., & Phelps, R. E. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000218 · Cited by 20
A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity
Ahmadi, A., Noetel, M., Schellekens, M., Parker, P. D., Antczak, D., Beauchamp, M. R., Dicke, T., Diezmann, C. M., Maeder, A., Ntoumanis, N., et al. (2021) · Psychosocial Intervention · doi:10.5093/pi2021a4 · Cited by 20
“…Long Short Term Memory (Bi-LSTM) 3 Chen et al., 2019; Gibson et al., 2019; Suresh et al., 2019 Labelled Topic Model 2 Atkins et al., 2014; Imel et al., 2015 Bayesian Network 2 Blanchard et al., 2016a; Blanchard et al., 2016b Gared Recurrent Unit (GRU) 2 Cao et al., 2019; Park et al.,…”
BackgroundMethodInfluentialMEANING OF AWE IN JAPANESE (CON)TEXT: BEYOND FEAR AND RESPECT
Nakayama, M., & Uchida, Y. (2020) · PSYCHOLOGIA · doi:10.2117/psysoc.2020-b004 · Cited by 19
“Each context was submitted to morphological analysis using the RMeCab package (Ishida, 2017) in R, and words were parsed into their basic forms.”
MethodSmink, W. A. C., Sools, A. M., van der Zwaan, J. M., Wiegersma, S., Veldkamp, B. P., & Westerhof, G. J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225703 · Cited by 18
Weusthoff, S., Gaut, G. R. J., Mark Steyvers, B., Atkins, D. C., Hahlweg, K., Hogan, J. N., Zimmermann, T., Fischer, M. S., Baucom, D. H., Georgiou, P., et al. (2018) · The European Journal of Counselling Psychology · doi:10.5964/ejcop.v7i1.82 · Cited by 15
Self-citationYordanova, K., Demiray, B., Mehl, M. R., & Martin, M. (2019) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom.2019.8767403 · Cited by 17
“For example, in [9] the authors use topic models to analyse psychotherapy and medication therapy transcripts.”
Nelson, J., Kaplan, J., Simerly, G., Nutter, N., Edson-Heussi, A., Woodham, B., & Broman‐Fulks, J. J. (2025) · Current Psychology · doi:10.1007/s12144-025-07320-1 · Cited by 24
“For example, Imel et al. (2015) demonstrated that machine learning algorithms could accurately analyze therapy session transcripts to predict therapeutic alliance strength, a key predictor of treatment outcomes.”
The Influence of Cognitive Biases and Financial Factors on Forecast Accuracy of Analysts
Nardi, P. C. C., Ribeiro, E. M. S., Bueno, J. L. O., & Aggarwal, I. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.773894 · Cited by 23
Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues
Park, S., Kim, D., & Oh, A. H. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1148 · Cited by 13
“…approaches to analyzing the dynamic patterns of relationship between the counselor and the client by focusing on the language of counselors (Imel et al., 2015; Althoff et al., 2016), clustering topics of client issues (Dinakar et al., 2014), and looking at therapy outcomes (Howes et al.,…”
BackgroundMiner, A. S., Fleming, S. L., Haque, A., Fries, J., Althoff, T., Wilfley, D. E., Agras, W. S., Milstein, A. S., Hancock, J. T., Asch, S. M., et al. (2022) · npj Mental Health Research · doi:10.1038/s44184-022-00020-9 · Cited by 28
“Computational approaches using natural language processing offer the potential to move past human limits of attention and reproducibility 19,27 – 32 .”
BackgroundMethodComputational Methods in Psychotherapy: A Scoping Review
Cioffi, V., Mosca, L. L., Moretto, E., Ragozzino, O., Stanzione, R., Bottone, M., Maldonato, M. N., Muzii, B., & Sperandeo, R. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191912358 · Cited by 19
“…shown themselves capable of emulating natural processes and have given a great and definitive impulse to the field of artificial intelligence research [35,36] and are opening the way to research fields such as computational psychology [37,38] and more recently, computational psychotherapy [39–41].”
Zhang, X., Tanana, M., Weitzman, L. M., Narayanan, S., Atkins, D. C., & Imel, Z. E. (2022) · Psychotherapy · doi:10.1037/pst0000460 · Cited by 18
Self-citationMylläri, S., Saarni, S. E., Ritola, V., Joffe, G., Stenberg, J., Solbakken, O. A., Czajkowski, N. O., & Rosenström, T. H. (2022) · Journal of Medical Internet Research · doi:10.2196/38911 · Cited by 13
“LDA has been successful in detecting meaningful topics that occur in face-to-face psychotherapy transcripts [12-14].”
Supervised Speaker Diarization Using Random Forests: A Tool for Psychotherapy Process Research
Fürer, L., Schenk, N., Röth, V., Steppan, M., Schmeck, K., & Zimmermann, R. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01726 · Cited by 14
“, 2020), random forests can be considered a rather understandable machine learning algorithm that has already found its way into psychotherapy research (Imel et al., 2015; Masías et al., 2015; Husain et al., 2016; Sun et al., 2017; Wallert et al., 2018; ZilchaMano, 2019; Rubel et al., 2020; Zimmermann et al., 2020).”
BackgroundTanana, M., Dembe, A., Soma, C. S., Imel, Z. E., Atkins, D. C., & Srikumar, V. (2016) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w16-0304 · Cited by 13
Self-citationTelepsychology: Applications, advantages, and challenges
Yuchang, J. I. N., Zhang, Z., Zheng, P., & Junxiu, A. N. (2021) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2022.00141 · Cited by 12
Chen, Z., Singla, K., Gibson, J., Can, D., Imel, Z. E., Atkins, D. C., Georgiou, P., & Narayanan, S. S. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682885 · Cited by 10
“The embedding layer is pre-trained by the training utterances plus the general psychotherapy corpus [15].”
Self-citationPeavy, K. M., Klipsch, A., Soma, C. S., Pace, B. T., Imel, Z. E., Tanana, M., Soth, S., Ricardo-Bulis, E., & Atkins, D. C. (2024) · Addiction Science & Clinical Practice · doi:10.1186/s13722-024-00435-z · Cited by 18
“…specifically designed for and trained on behavioral health conversations, to both generate a transcription, as well using speech features as inputs to predict the full suite of MI fidelity codes based on the Motivational Interviewing Skills Code system (MISC; [27]; see [23, 23, 41, 42, 44, 45]).”
MethodSelf-citationChanges in verbalizations during sandplay: an empirical study
Punnett, A. F., & Canfield, M. L. (2020) · Journal of Analytical Psychology · doi:10.1111/1468-5922.12597 · Cited by 11
Automated rating of patient and physician emotion in primary care visits
Park, J., Jindal, A., Kuo, P. B., Tanana, M., Lafata, J. E., Tai‐Seale, M., Atkins, D. C., Imel, Z. E., & Smyth, P. (2021) · Patient Education and Counseling · doi:10.1016/j.pec.2021.01.004 · Cited by 13
Self-citationHasan, M., Carcone, A. I., Naar, S., Eggly, S. S., Alexander, G., Brogan, K. E., & Kotov, A. (2018) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-018-0037-6 · Cited by 8
Kulkarni, S. (2022) · Wiley eBooks · doi:10.1002/9781119792376.ch6 · Cited by 10
Zhang, H., Qiao, Z., Wang, H., Duan, B., & Yin, J. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01467-w · Cited by 11
“This is difficult to accept in the ethical discussions of psychological intervention and even the entire medical intervention.”
Xezonaki, D., Paraskevopoulos, G., Potamianos, A., & Narayanan, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08336 · Cited by 7
“Moreover, for the General Psychotherapy Corpus 1 , Malandrakis et al. [11] have explored differences in language between therapist and client using psycholinguistic norms and Imel et al. [12] have identified semantic topics discussed in therapy sessions.”
Nikzad, A. H., Lysaker, P. H., Minor, K. S., Leonhardt, B. L., Liberman, M., Vohs, J. L., Wiesepape, C. N., & Tang, S. X. (2024) · Early Intervention in Psychiatry · doi:10.1111/eip.13628 · Cited by 10
Zainal, N. H., Eckhardt, R., Rackoff, G. N., Fitzsimmons‐Craft, E. E., Rojas-Ashe, E. E., Taylor, C. B., Funk, B., Eisenberg, D., Wilfley, D. E., & Newman, M. G. (2025) · Psychological Medicine · doi:10.1017/s0033291725000340 · Cited by 12
“…Integrity (MITI), comprised more abstract codes (e.g. advising, confrontation, and emphasizing autonomy; Atkins, Steyvers, Imel, & Smyth, 2014; Imel, Steyvers, & Atkins, 2015; Tanana et al., 2016; refer to Supplementary Tables S4 and S5 for more details on these alternative psychotherapy…”
Chen, Z., Flemotomos, N., Imel, Z. E., Atkins, D. C., & Narayanan, S. S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.425 · Cited by 5
Self-citationIn pursuit of truth: A critical examination of meta-analyses of cognitive behavior therapy
Wampold, B. E., Flückiger, C., Re, A. C. D., Yulish, N. E., Frost, N. D., Pace, B. T., Goldberg, S. B., Miller, S. D., Baardseth, T. P., Laska, K. M., et al. (2019) · PERSON · doi:10.24989/person.v23i1.2428 · Cited by 6
Three Flavorings for a Soup to Cure what Ails Mental Health Services
Brown, C. H. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01060-z · Cited by 6
Flemotomos, N., Martínez, V., Chen, Z., Singla, K., Ardulov, V., Peri, R., Caperton, D. D., Gibson, J., Tanana, M., Georgiou, P., et al. (2021) · arXiv (Cornell University) · Cited by 6
Self-citationConstructing Change in Psychotherapy: The Innovative Moments Model
Gonçalves, M. M., & Oliveira, J. T. (2026) · Journal of Constructivist Psychology · doi:10.1080/10720537.2026.2618712 · Cited by 5
Berkel, C., Knox, D., Flemotomos, N., Martínez, V. R., Atkins, D. C., Narayanan, S. S., Rodriguez, L. A., Gallo, C. G., & Smith, J. D. (2023) · Implementation Research and Practice · doi:10.1177/26334895231187906 · Cited by 9
Self-citationA Computer Program for Tracking the Evolution of a Psychotherapy Treatment
Maskit, B., Bucci, W., & Murphy, S. M. (2015) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.3115/v1/w15-1216 · Cited by 5
Coleman, J. J., Owen, J. J., Wright, J. H., Eells, T. D., Antle, B. F., McCoy, M., & Soma, C. S. (2024) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.70023 · Cited by 8
Visualising emotion in support of patient-physician communication: an empirical study
Ma, H., Sun, X. S., Lawson, G., Wang, Q., & Zhang, Y. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2097954 · Cited by 6
Bar, M., Saad, A., Weiss‐Klayman, N., & Mendlovic, S. (2025) · Psychotherapy Research · doi:10.1080/10503307.2025.2455466 · Cited by 7
“Paradoxi-cally, while the data in the area are predominantly linguistic and semantic, the tools available for studying psychotherapy are not (Imel et al., 2015).”
Steinbrenner, T., Lalk, C., Targan, K., Schaffrath, J., Eberhardt, S. T., Haberkamp, A., Lutz, W., & Rubel, J. A. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104857 · Cited by 8
Artificial Intelligence to Support Human-Provided Mental Health Treatment
Soma, C. S., Kuo, P. B., Mehta, M., Srikumar, V., Imel, Z. E., & Atkins, D. C. (2026) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-061724-075336 · Cited by 9
Self-citationСоколова, А., Лобанова, П., & Kuzminov, I. (2023) · foresight · doi:10.1108/fs-02-2023-0026 · Cited by 5
Evaluating Automatic Speech Recognition Quality and Its Impact on Counselor Utterance Coding
Min, D. J., Pérez‐Rosas, V., & Mihalcea, R. F. (2021) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/2021.clpsych-1.18 · Cited by 4
“While most of previous work was conducted on manual transcriptions, there are only a few cases where automatically generated transcripts have been used, limiting the use of computational methods in psychiatry (Imel et al., 2015).”
MethodThe role of general change mechanisms in sudden gains in the treatment of anorexia nervosa
Brockmeyer, T., Titzmann, M., Zipfel, S., Wild, B., Resmark, G., Teufel, M., Giel, K. E., de Zwaan, M., Dinkel, A., Herpertz, S., et al. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104285 · Cited by 6
Leveraging large language models to identify microcounseling skills in psychotherapy transcripts
Hammerfald, K., Schmidt, F., Vlassov, V., Jahren, H. H., & Solbakken, O. A. (2025) · Psychotherapy Research · doi:10.1080/10503307.2025.2539405 · Cited by 8
“…linguistic features (Althoff et al., 2016), indicators of cognitive behavioral therapy (Hilbert et al., 2020), session topics (Dinakar et al., 2015; Imel et al., 2015), emotional expressions (Tanana et al., 2021), detecting suicidal ideation (Bejan et al., 2022), extracting schemas from thought…”
Die Erforschung der Psychotherapie: Aktueller Stand und Themen für die Zukunft
Einsle, F., & Härtling, S. (2015) · Psychotherapie-Wissenschaft · doi:10.23668/psycharchives.4015 · Cited by 6
Förderung persönlicher Kompetenzen in der psychotherapeutischen Aus- und Weiterbildung
Bennemann, B., Caspar, F., Evers, O., Laireiter, A., Lutz, W., Poster, K., Rief, W., Strauss, B. M., & Taubner, S. (2021) · Elsevier eBooks · doi:10.1016/b978-3-437-22601-4.00002-4 · Cited by 6
Miner, A. S., Fleming, S. L., Haque, A., Fries, J., Althoff, T., Wilfley, D. E., Agras, W. S., Milstein, A. S., Hancock, J. T., Ash, S. M., et al. (2022) · medRxiv · doi:10.1101/2022.04.24.22274227 · Cited by 3
Artificial Intelligence-generated Feedback as a Training Tool for Enhancing Psychotherapy Skill
Young, J. N., Schruff, M. A., Mudryk, E. P., & Bentley, J. P. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00603-9 · Cited by 2
Metadata
- Authors
- Imel, Z. E., Steyvers, M., & Atkins, D. C.
- Affiliations
- University of Utah (US)
- University of California, Irvine (US)
- University of Washington (US)
- Corresponding author
- Imel, Zac E.
- Venue
- Psychotherapy
- Published
- 2015
- Volume
- 52
- Issue
- 1
- Pages
- 19-30
- ISSN
- 0033-3204 (ISSN-L)
- 1939-1536
- PMC
- PMC4245387
- OpenAlex
- W2314271279
- Cited by
- 146
- References
- 65
- Citations by year
- 2026: 24 · 2025: 14 · 2024: 9 · 2023: 12 · 2022: 16 · 2021: 12 · 2020: 16 · 2019: 11 · 2018: 6 · 2017: 6 · 2016: 10 · 2015: 9
- FWCI
- 53.99
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Psychotherapy (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institutes of Health (R34/DA034860)
- National Institute on Alcohol Abuse and Alcoholism (R01/AA018673)
- University of Utah
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- topic modeling, patient-therapist interaction, psychotherapy transcripts, cognitive behavioral therapy, psychodynamic psychotherapy, therapeutic relationship, medication management, semantic similarity
- MeSH terms
- Humans, Linguistics, Professional-Patient Relations, Psychotherapy, Research, Semantics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.