A Scoping Review of Machine Learning in Psychotherapy Research

Aafjes-van Doorn, Katie; Kamsteeg, Céline; Bate, Jordan; Aafjes, Marc

doi:10.1080/10503307.2020.1808729

Cited by 179Psychotherapy Techniques and ApplicationsDigital Mental Health InterventionsArtificial Intelligence in Healthcare

A Scoping Review of Machine Learning in Psychotherapy Research

Aafjes-van Doorn, Katie, Kamsteeg, Céline, Bate, Jordan, Aafjes, Marc·2021·Psychotherapy Research

DOI 10.1080/10503307.2020.1808729

Open this paper in CoChat — free account

Abstract

Machine learning (ML) offers robust statistical and probabilistic techniques that can help to make sense of large amounts of data. This scoping review paper aims to broadly explore the nature of research activity using ML in the context of psychological talk therapies, highlighting the scope of current methods and considerations for clinical practice and directions for future research. Using a systematic search methodology, fifty-one studies were identified. A narrative synthesis indicates two types of studies, those who developed and tested an ML model (k=44), and those who reported on the feasibility of a particular treatment tool that uses an ML algorithm (k=7). Most model development studies used supervised learning techniques to classify or predict labeled treatment process or outcome data, whereas others used unsupervised techniques to identify clusters in the unlabeled patient or treatment data. Overall, the current applications of ML in psychotherapy research demonstrated a range of possible benefits for indications of treatment process, adherence, therapist skills and treatment response prediction, as well as ways to accelerate research through automated behavioral or linguistic process coding. Given the novelty and potential of this research field, these proof-of-concept studies are encouraging, however, do not necessarily translate to improved clinical practice (yet).

Integrity signals

Registry

Crossref · 10.1080/10503307.2020.1808729

Cites

111 of 111 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Evaluative studies in applied settings are limited and primarily focus on predicting outcomes in care, not on improving provider-client fit from the start of care.
“Although predictive models show promise for improving healthcare access and quality,23 evaluative studies in applied settings are limited and primarily focus on predicting outcomes in care, not on improving provider-client fit from the start of care.24-26”
From Using Machine Learning to Match Clients and Therapy Providers: Evaluating Clinical Quality and Cost of Care (2025) · cites this paper as [24]
AI has begun to be used in psychology to retrospectively evaluate which factors predict a better response to psychological treatment once it has finished.
“AI has begun to be used in psychology, for example, to retrospectively evaluate which factors predict a better response to psychological treatment once it has finished [72,73].”
From Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review (2022) · cites this paper as [72]

Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · see citing works
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106 · see citing works
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506 · see citing works
A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Statistical Power to Detect the Correct Number of Classes in Latent Profile Analysis
Tein, J., Coxe, S. J., & Cham, H. (2013) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2013.824781 · Cited by 2,042 · see citing works
Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning
Yarkoni, T., & Westfall, J. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617693393 · Cited by 1,930 · see citing works
Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes
Riley, R. D., Snell, K. I., Ensor, J., Burke, D. L., Harrell Jr, F. E., Moons, K. G., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7992 · Cited by 1,409 · see citing works
Small sample size effects in statistical pattern recognition: Recommendations for practitioners
Raudys, S. J., & Jain, A. K. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.75512 · Cited by 1,384 · see citing works
An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study (Preprint)
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063 · see citing works
Machine learning in mental health: a scoping review of methods and applications
Shatte, A. B. R., Hutchinson, D. M., & Teague, S. J. (2019) · Psychological Medicine · doi:10.1017/s0033291719000151 · Cited by 1,047 · see citing works
Artificial intelligence for mental health and mental illnesses: an overview
Graham, S., Depp, C., Lee, E. E., Nebeker, C., Tu, X., Kim, H.-C., & Jeste, D. V. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1094-0 · Cited by 970 · see citing works
Machine Learning Approaches for Clinical Psychology and Psychiatry
Dwyer, D. B., Falkai, P., & Koutsouleris, N. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-045037 · Cited by 966 · see citing works
Encyclopedia of Machine Learning and Data Mining
Webb, G. I., & Sammut, C. (2017) · Springer eBooks · doi:10.1007/978-1-4899-7687-1 · Cited by 926 · see citing works
Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy
Fiske, A., Henningsen, P., & Buyx, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13216 · Cited by 857 · see citing works
Harnessing context sensing to develop a mobile intervention for depression
Burns, M. N., Begale, M., Duffecy, J., Gergle, D., Karr, C. J., Giangrande, E., & Mohr, D. C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1838 · Cited by 646 · see citing works
What is Machine Learning? A Primer for the Epidemiologist
Bi, Q., Goodman, K. E., Kaminsky, J., & Lessler, J. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz189 · Cited by 522 · see citing works
Treatment selection in depression.
Cohen, Z. D., & DeRubeis, R. J. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050817-084746 · Cited by 435 · see citing works
Peering Into the Black Box of Artificial Intelligence: Evaluation Metrics of Machine Learning Methods
Handelman, G. S., Kok, H. K., Chandra, R. V., Razavi, A. H., Huang, S., Brooks, M., Lee, M. J., & Asadi, H. (2018) · American Journal of Roentgenology · doi:10.2214/ajr.18.20224 · Cited by 410 · see citing works
A Mathematical Approach to Medical Diagnosis
Warner, H. R. (1961) · JAMA · doi:10.1001/jama.1961.03040290005002 · Cited by 374 · see citing works
Minimum sample size for developing a multivariable prediction model: Part I – Continuous outcomes
Riley, R. D., Snell, K. I. E., Ensor, J., Burke, D. L., Harrell, F. E., Moons, K. G. M., & Collins, P. G. S. (2018) · Statistics in Medicine · doi:10.1002/sim.7993 · Cited by 372 · see citing works
Sample size tables for logistic regression
Hsieh, F. Y. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080704 · Cited by 367 · see citing works
Mobile Sensing and Support for People With Depression: A Pilot Trial in the Wild
Wahle, F., Kowatsch, T., Fleisch, E., Rufer, M., & Weidt, S. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.5960 · Cited by 333 · see citing works
Large-scale analysis of counseling conversations: an application of natural language processing to mental health
Althoff, T., Clark, K., & Leskovec, J. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00111 · Cited by 322 · see citing works
Sample-size determination methodologies for machine learning in medical imaging research: a systematic review
Balki, I., Amirabadi, A., Levman, J., Martel, A. L., Emersic, Z., Meden, B., Garcia-Pedrero, A., Ramirez, S. C., Kong, D., Moody, A. R., et al. (2019) · Canadian Association of Radiologists Journal · doi:10.1016/j.carj.2019.06.002 · Cited by 299 · see citing works
Prediction of Individual Response to Electroconvulsive Therapy via Machine Learning on Structural Magnetic Resonance Imaging Data
Redlich, R., Opel, N., Grotegerd, D., Dohm, K., Zaremba, D., Bürger, C., Münker, S., Mühlmann, L., Wahl, P., Heindel, W. L., et al. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.0316 · Cited by 267 · see citing works
Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach
Koutsouleris, N., Kahn, R. S., Chekroud, A. M., Leucht, S., Falkai, P., Wobrock, T., Derks, E. M., Fleischhacker, W., & Hasan, A. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)30171-7 · Cited by 217 · see citing works
Applications of text mining within systematic reviews
Thomas, J., McNaught, J., & Ananiadou, S. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.27 · Cited by 212 · see citing works
Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information
Berman, J. J. (2013) · Cited by 210 · see citing works
Dynamic Multimodal Measurement of Depression Severity Using Deep Autoencoding
Dibeklioğlu, H., Hammal, Z., & Cohn, J. F. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2676878 · Cited by 196 · see citing works
Towards integrating personalized feedback research into clinical practice: Development of the Trier Treatment Navigator (TTN)
Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103438 · Cited by 196 · see citing works
A unified view of performance metrics: translating threshold choice into expected classification loss
Hernández‐Orallo, J., Flach, P. A., & Ferri, C. (2012) · Bristol Research (University of Bristol) · Cited by 181 · see citing works
Machine learning and big data: Implications for disease modeling and therapeutic discovery in psychiatry
Tai, A. M. Y., Albuquerque, A., Carmona, N. E., Subramanieapillai, M., Cha, D. S., Sheko, M., Lee, Y. L., Mansur, R. B., & McIntyre, R. S. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101704 · Cited by 168 · see citing works
Key Considerations for Incorporating Conversational AI in Psychotherapy
Miner, A. S., Shah, N., Bullock, K. D., Arnow, B. A., Bailenson, J., & Hancock, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00746 · Cited by 166 · see citing works
Peer review of statistics in medical research: the other problem
Bacchetti, P. (2002) · BMJ · doi:10.1136/bmj.324.7348.1271 · Cited by 157 · see citing works
Predicting long-term outcome of Internet-delivered cognitive behavior therapy for social anxiety disorder using fMRI and support vector machine learning.
Månsson, K. N. T., Frick, A., Boraxbekk, C.-J., Marquand, A. F., Williams, S. C. R., Carlbring, P., Andersson, G., & Furmark, T. (2015) · Translational Psychiatry · doi:10.1038/tp.2015.22 · Cited by 154 · see citing works
A practical guide to big data research in psychology.
Chen, E. E., & Wojcik, S. P. (2016) · Psychological Methods · doi:10.1037/met0000111 · Cited by 154 · see citing works
Multivariate resting-state functional connectivity predicts response to cognitive behavioral therapy in obsessive-compulsive disorder
Reggente, N., Moody, T. D., Morfini, F., Sheen, C., Rissman, J., O’Neill, J., & Feusner, J. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1716686115 · Cited by 154 · see citing works
Computational psychotherapy research: scaling up the evaluation of patient–provider interactions
Imel, Z. E., Steyvers, M., & Atkins, D. C. (2015) · Psychotherapy · doi:10.1037/a0036841 · Cited by 146 · see citing works
A Comparison of Natural Language Processing Methods for Automated Coding of Motivational Interviewing
Tanana, M., Hallgren, K. A., Imel, Z. E., Atkins, D. C., & Srikumar, V. (2016) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2016.01.006 · Cited by 141 · see citing works
Multi-Level Comparison of Machine Learning Classifiers and Their Performance Metrics
Rácz, A., Bajusz, D., & Héberger, K. (2019) · Molecules · doi:10.3390/molecules24152811 · Cited by 141 · see citing works
Measuring Use of Evidence Based Psychotherapy for Posttraumatic Stress Disorder
Shiner, B. R., DʼAvolio, L., Nguyen, T. M., Zayed, M. H., Young‐Xu, Y., Desai, R. A., Schnurr, P. P., Fiore, L. D., & Watts, B. V. (2012) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-012-0421-0 · Cited by 140 · see citing works
Predicting Treatment Response to Cognitive Behavioral Therapy in Panic Disorder With Agoraphobia by Integrating Local Neural Information
Hahn, T., Kircher, T. T. J., Straube, B., Wïttchen, H., Konrad, C., Ströhle, A., Wittmann, A., Pfleiderer, B., Reif, A., Arolt, V., et al. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1741 · Cited by 130 · see citing works
Scaling up the evaluation of psychotherapy: evaluating motivational interviewing fidelity via statistical text classification
Atkins, D. C., Mark Steyvers, B., Imel, Z. E., & Smyth, P. (2014) · Implementation Science · doi:10.1186/1748-5908-9-49 · Cited by 130 · see citing works
Recommendations and future directions for supervised machine learning in psychiatry
Cearns, M., Hahn, T., & Baune, B. T. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0607-2 · Cited by 127 · see citing works
Implementation of Evidence-Based Psychotherapies for Posttraumatic Stress Disorder in VA Specialty Clinics
Watts, B. V., Shiner, B. R., Zubkoff, L., Carpenter-Song, E. A., Ronconi, J. M., & Coldwell, C. M. (2014) · Psychiatric Services · doi:10.1176/appi.ps.201300176 · Cited by 125 · see citing works
Evaluating black-boxes as medical decision aids: issues arising from a study of neural networks
Hart, A., & Wyatt, J. C. (1990) · Medical Informatics · doi:10.3109/14639239009025270 · Cited by 124 · see citing works
Zero-One Loss
Sammut, C., & Webb, G. I. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_884 · Cited by 124 · see citing works
A demonstration of a multi-method variable selection approach for treatment selection: Recommending cognitive–behavioral versus psychodynamic therapy for mild to moderate adult depression
Cohen, Z. D., Kim, T. T., Van, H. L., Dekker, J., & Driessen, E. (2019) · Psychotherapy Research · doi:10.1080/10503307.2018.1563312 · Cited by 122 · see citing works
Single-Subject Anxiety Treatment Outcome Prediction using Functional Neuroimaging
Ball, T. M., Stein, M. B., Ramsawh, H. J., Campbell‐Sills, L., & Paulus, M. P. (2013) · Neuropsychopharmacology · doi:10.1038/npp.2013.328 · Cited by 116 · see citing works
Improving the yield of psychotherapy research
Silberschatz, G. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1076202 · Cited by 113 · see citing works
"Rate My Therapist": Automated Detection of Empathy in Drug and Alcohol Counseling via Speech and Language Processing
Xiao, B., Imel, Z. E., Georgiou, P., Atkins, D. C., & Narayanan, S. S. (2015) · PLoS ONE · doi:10.1371/journal.pone.0143055 · Cited by 111 · see citing works
Quantifying the association between psychotherapy content and clinical outcomes using deep learning
Ewbank, M. P., Cummins, R., Tablan, V., Bateup, S., Catarino, A., Martin, A. J., & Blackwell, A. D. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.2664 · Cited by 108 · see citing works
What big data can do for treatment in psychiatry
Gillan, C. M., & Whelan, R. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.003 · Cited by 106 · see citing works
Targeted prescription of cognitive–behavioral therapy versus person-centered counseling for depression using a machine learning approach.
Delgadillo, J., & Duhne, P. G. S. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000476 · Cited by 101 · see citing works
Development and Evaluation of ClientBot: Patient-Like Conversational Agent to Train Basic Counseling Skills
Tanana, M. J., Soma, C. S., Srikumar, V., Atkins, D. C., & Imel, Z. E. (2019) · Journal of Medical Internet Research · doi:10.2196/12529 · Cited by 100 · see citing works
A Virtual Conversational Agent for Teens with Autism Spectrum Disorder
Ali, M. R., Razavi, S. Z., Langevin, R., Mamun, A. A., Kane, B., Rawassizadeh, R., Schubert, L. K., & Hoque, M. E. (2020) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3383652.3423900 · Cited by 98 · see citing works
Design feasibility of an automated, machine-learning based feedback system for motivational interviewing.
Imel, Z. E., Pace, B. T., Soma, C. S., Tanana, M., Hirsch, T., Gibson, J., Georgiou, P., Narayanan, S. S., & Atkins, D. C. (2019) · Psychotherapy · doi:10.1037/pst0000221 · Cited by 96 · see citing works
Personalized treatment selection in routine care: Integrating machine learning and statistical algorithms to recommend cognitive behavioral or psychodynamic therapy
Schwartz, B., Cohen, Z. D., Rubel, J. A., Zimmermann, D., Wittmann, W. W., & Lutz, W. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1769219 · Cited by 93 · see citing works
Technology-enhanced human interaction in psychotherapy.
Imel, Z. E., Caperton, D. D., Tanana, M., & Atkins, D. C. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000213 · Cited by 88 · see citing works
Resonating minds: A school-independent theoretical conception and its empirical application to psychotherapeutic processes
Mergenthaler, E. (2008) · Psychotherapy Research · doi:10.1080/10503300701883741 · Cited by 84 · see citing works
Applying deep neural networks to unstructured text notes in electronic medical records for phenotyping youth depression
Geraci, J., Wilansky, P., Luca, V. D., Roy, A., Kennedy, J. L., & Strauss, J. (2017) · Evidence-Based Mental Health · doi:10.1136/eb-2017-102688 · Cited by 72 · see citing works
Topic models: A novel method for modeling couple and family text data.
Atkins, D. C., Rubin, T. N., Steyvers, M., Doeden, M. A., Baucom, B., & Christensen, A. (2012) · Journal of Family Psychology · doi:10.1037/a0029607 · Cited by 71 · see citing works
Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up
Tajika, A., Ogawa, Y., Takeshima, N., Hayasaka, Y., & Furukawa, T. A. (2015) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.113.143701 · Cited by 61 · see citing works
“It sounds like...”: A natural language processing approach to detecting counselor reflections in motivational interviewing.
Can, D., Marín, R. A., Georgiou, P., Imel, Z. E., Atkins, D. C., & Narayanan, S. S. (2016) · Journal of Counseling Psychology · doi:10.1037/cou0000111 · Cited by 61 · see citing works
Predicting Social Anxiety Treatment Outcome Based on Therapeutic Email Conversations
Hoogendoorn, M., Berger, T., Schulz, A., Stolz, T., & Peter Szolovits, B. (2016) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2601123 · Cited by 60 · see citing works
A Deep Learning Approach to Modeling Empathy in Addiction Counseling
Gibson, J., Can, D., Xiao, B., Imel, Z. E., Atkins, D. C., Georgiou, P., & Narayanan, S. S. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-554 · Cited by 59 · see citing works
Predicting personalized process-outcome associations in psychotherapy using machine learning approaches-A demonstration.
Rubel, J. A., Zilcha-Mano, S., Giesemann, J., Prinz, J., & Lutz, W. (2020) · Psychotherapy Research · doi:10.1080/10503307.2019.1597994 · Cited by 56 · see citing works
Multimodal assessment of depression from behavioral signals
Cohn, J. F., Cummins, N., Epps, J., Goecke, R., Joshi, J., & Scherer, S. J. (2018) · ACM eBooks · doi:10.1145/3107990.3108004 · Cited by 55 · see citing works
Randomized controlled trial of an online machine learning-driven risk assessment and intervention platform for increasing the use of crisis services.
Jaroszewski, A. C., Morris, R. R., & Nock, M. K. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000389 · Cited by 53 · see citing works
Advances in the Application of Machine Learning Techniques in Drug Discovery, Design and Development
Barrett, S. J., & Langdon, W. B. (2006) · Advances in intelligent and soft computing · doi:10.1007/978-3-540-36266-1_10 · Cited by 52 · see citing works
A technology prototype system for rating therapist empathy from audio recordings in addiction counseling
Xiao, B., Huang, C.-W., Imel, Z. E., Atkins, D. C., Georgiou, P., & Narayanan, S. S. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.59 · Cited by 52 · see citing works
Content Coding of Psychotherapy Transcripts Using Labeled Topic Models
Gaut, G. R. J., Steyvers, M., Imel, Z. E., Atkins, D. C., & Smyth, P. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2503985 · Cited by 47 · see citing works
Head Motion Modeling for Human Behavior Analysis in Dyadic Interaction
Xiao, B., Georgiou, P., Baucom, B., & Narayanan, S. S. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2432671 · Cited by 46 · see citing works
Analyzing psychotherapy process as intersubjective sensemaking: An approach based on discourse analysis and neural networks
Nitti, M., Ciavolino, E., Salvatore, S., & Gennaro, A. (2010) · Psychotherapy Research · doi:10.1080/10503301003641886 · Cited by 46 · see citing works
Individualized treatment response prediction of dialectical behavior therapy for borderline personality disorder using multimodal magnetic resonance imaging
Schmitgen, M. M., Niedtfeld, I., Schmitt, R., Mancke, F., Winter, D., Schmahl, C. G., & Herpertz, S. C. (2019) · Brain and Behavior · doi:10.1002/brb3.1384 · Cited by 44 · see citing works
Individualized psychiatric imaging based on inter-subject neural synchronization in movie watching
Yang, Z., Wu, J., Xu, L., Deng, Z.-Z., Tang, Y., Gao, J., Hu, Y., Zhang, Y., Qin, S., Li, C., et al. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116227 · Cited by 44 · see citing works
A Machine Learning Approach to Identifying Placebo Responders in Late-Life Depression Trials
Zilcha‐Mano, S., Roose, S. P., Brown, P. J., & Rutherford, B. R. (2018) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2018.01.001 · Cited by 42 · see citing works
"It's hard to argue with a computer"
Hirsch, T., Soma, C. S., Merced, K., Kuo, P. B., Dembe, A., Caperton, D. D., Atkins, D. C., & Imel, Z. E. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196776 · Cited by 42 · see citing works
Psychotherapy Bulletin, 2012
(2012) · PsycEXTRA Dataset · doi:10.1037/e616722012-001 · Cited by 41 · see citing works
Classifier performance estimation under the constraint of a finite sample size: Resampling schemes applied to neural network classifiers
Sahiner, B., Chan, H., & Hadjiiski, L. M. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.12.012 · Cited by 39 · see citing works
Machine learning vs addiction therapists: A pilot study predicting alcohol dependence treatment outcome from patient data in behavior therapy with adjunctive medication
Symons, M., Feeney, G. F. X., Gallagher, M. R., Young, R. M., & Connor, J. P. (2019) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2019.01.020 · Cited by 38 · see citing works
Detecting conversation topics in primary care office visits from transcripts of patient-provider interactions
Park, J., Kotzias, D., Kuo, P. B., Logan, R. L., Merced, K., Singh, S., Tanana, M., Taniskidou, E. K., Lafata, J. E., Atkins, D. C., et al. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz140 · Cited by 35 · see citing works
The influence of relative sample size in training artificial neural networks
BLAMIRE, P. A. (1996) · International Journal of Remote Sensing · doi:10.1080/01431169608949000 · Cited by 32 · see citing works
Predicting couple therapy outcomes based on speech acoustic features
Nasir, M., Baucom, B., Georgiou, P., & Narayanan, S. S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0185123 · Cited by 31 · see citing works
Predicting short term mood developments among depressed patients using adherence and ecological momentary assessment data
Mikus, A., Hoogendoorn, M., Rocha, A., Gama, J., Ruwaard, J., & Riper, H. (2018) · Internet Interventions · doi:10.1016/j.invent.2017.10.001 · Cited by 31 · see citing works
Investigation of psycho-physiological interactions between patient and therapist during a psychodynamic therapy and their relation to speech using in terms of entropy analysis using a neural network approach
Villmann, T., Liebers, C., Bergmann, B., Gumz, A., & Geyer, M. A. (2007) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2007.07.010 · Cited by 29 · see citing works
Developing Machine Learning Models for Behavioral Coding
Carcone, A. I., Hasan, M., Alexander, G., Dong, M., Eggly, S. S., Hartlieb, K. B., Naar, S., MacDonell, K. K., & Kotov, A. (2019) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsy113 · Cited by 27 · see citing works
Classifier Performance Estimation Under the Constraint of a Finite Sample Size: Resampling Schemes Applied to Neural Network Classifiers
Sahiner, B., Chan, H., & Hadjiiski, L. M. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4371224 · Cited by 26 · see citing works
Recursive Neural Networks for Coding Therapist and Patient Behavior in Motivational Interviewing
Tanana, M., Hallgren, K. A., Imel, Z. E., Atkins, D. C., Smyth, P., & Srikumar, V. (2015) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.3115/v1/w15-1209 · Cited by 26 · see citing works
The Precision Medicine Initiative and Pervasive Health Research
INTILLE, S. S. (2016) · IEEE Pervasive Computing · doi:10.1109/mprv.2016.2 · Cited by 23 · see citing works
Are there any robust predictors of “sudden gainers,” and how is sustained improvement in treatment outcome achieved following a gain?
Zilcha‐Mano, S., Errázuriz, P., Yaffe-Herbst, L., German, R. E., & DeRubeis, R. J. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000401 · Cited by 23 · see citing works
Computer judgment of main color in a drawing for art psychotherapy assessment
Kim, S.-I. (2008) · The Arts in Psychotherapy · doi:10.1016/j.aip.2008.01.002 · Cited by 22 · see citing works
Neural network based on adaptive resonance theory as compared to experts in suggesting treatment for schizophrenic and unipolar depressed in-patients
Modai, I., Israel, A., Mendel, S., Hines, E. L., & Weizman, R. (1996) · Journal of Medical Systems · doi:10.1007/bf02257284 · Cited by 21 · see citing works
Using fMRI and machine learning to predict symptom improvement following cognitive behavioural therapy for psychosis
Tolmeijer, E., Kumari, V., PETERS, E. R., Williams, S., & Mason, L. (2018) · NeuroImage Clinical · doi:10.1016/j.nicl.2018.10.011 · Cited by 20 · see citing works
A New Measure Based in the Rough Set Theory to Estimate the Training Set Quality
Mota, Y. C., Bello, R., Taboada-Crispí, A., Nowé, A., García, M., & Casas, G. (2006) · International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc.2006.6 · Cited by 17 · see citing works
Predicting treatment outcome by combining different assessment tools: Toward an integrative model of decision support in psychotherapy.
Gori, A., Grotto, R. L., Giannini, M., & Schuldberg, D. (2010) · Journal of Psychotherapy Integration · doi:10.1037/a0019768 · Cited by 16 · see citing works
Approaching Human Performance in Behavior Estimation in Couples Therapy Using Deep Sentence Embeddings
Tseng, S.-Y., Baucom, B., & Georgiou, P. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1621 · Cited by 16 · see citing works
Using Prosodic and Lexical Information for Learning Utterance-level Behaviors in Psychotherapy
Singla, K., Chen, Z., Flemotomos, N., Gibson, J., Can, D., Atkins, D. C., & Narayanan, S. S. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-2551 · Cited by 15 · see citing works
Machine learning for the recognition of emotion in the speech of couples in psychotherapy using the Stanford Suppes Brain Lab Psychotherapy DatasetPreprint
Crangle, C., Wang, R., Guimaraes, M. P., Nguyen, M., Nguyen, D. T., & Suppes, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.04110 · Cited by 15 · see citing works
Feasibility and Acceptability of an Intervention Providing Computer-Generated Tailored Feedback to Target Alcohol Consumption and Depressive Symptoms in Proactively Recruited Health Care Patients and Reactively Recruited Media Volunteers: Results of a Pilot Study
Krause, K., Guertler, D., Moehring, A., Batra, A., Eck, S., Rumpf, H.-J., Bischof, G., Lucht, M., Freyer-Adam, J., Ulbricht, S., et al. (2019) · European Addiction Research · doi:10.1159/000499040 · Cited by 15 · see citing works
Linguistically Aided Speaker Diarization Using Speaker Role Information
Flemotomos, N., Georgiou, P., & Narayanan, S. (2020) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2020-17 · Cited by 14 · see citing works
The language of interpersonal interaction: An interdisciplinary approach to assessing and processing vocal and speech data
Weusthoff, S., Gaut, G. R. J., Steyvers, M., Atkins, D. C., Hahlweg, K., Hogan, J. N., Zimmermann, T., Fischer, M. S., Baucom, D. H., Georgiou, P., et al. (2018) · The European Journal of Counselling Psychology · doi:10.5964/ejcop.v7i1.82 · Cited by 13 · see citing works
Methods for Predictor Analysis of Repeated Measurements: Application to Psychiatric Data
Eisenacher, M., Riesbeck, M., Gaebel, W., Köpcke, W., Seuchter, S. A., & and other members of the A.N.I. Study Group (2004) · Methods of Information in Medicine · doi:10.1055/s-0038-1633857 · Cited by 10 · see citing works
Personality and Brain Disorders
Garcia, D., Archer, T., & Kostrzewa, R. M. (2019) · Contemporary clinical neuroscience · doi:10.1007/978-3-319-90065-0 · Cited by 10 · see citing works
Identifying cognitive distortion by convolutional neural network based text classification
Xing, Z., Zhao, X., & Miao, C. (2017) · DR-NTU (Nanyang Technological University) · Cited by 9 · see citing works
Detecting and tracking ongoing topics in psychotherapeutic conversations.
Chaoua, I., Recupero, D. R., Consoli, S., Härmä, A., & Helaoui, R. (2018) · Research Publications (Maastricht University) · Cited by 3 · see citing works
Language Aided Speaker Diarization Using Speaker Role InformationPreprint
Flemotomos, N., Georgiou, P., & Narayanan, S. S. (2019) · arXiv (Cornell University) · Cited by 1 · see citing works
Machine Learning Methods For The Analysis Of Clinical Conversation
Hasan, M. (2019) · Human Biology · Cited by 1 · see citing works

Flagged first, then by citations

Sorted by citations

The promise of machine learning in predicting treatment outcomes in psychiatry
Chekroud, A. M., Bondar, J., Delgadillo, J., Doherty, G., Wasil, A., Fokkema, M., Cohen, Z., Belgrave, D., DeRubeis, R., Iniesta, R., et al. (2021) · World Psychiatry · doi:10.1002/wps.20882 · Cited by 531 · see citing works
Modern views of machine learning for precision psychiatry
Chen, Z. S., Kulkarni, P., Galatzer‐Levy, I. R., Bigio, B., Nasca, C., & Zhang, Y. (2022) · Patterns · doi:10.1016/j.patter.2022.100602 · Cited by 225 · see citing works
Natural language processing for mental health interventions: a systematic review and research framework
Malgaroli, M., Hull, T. D., Zech, J. M., & Althoff, T. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02592-2 · Cited by 217 · see citing works
Identifying Predictors of Psychological Distress During COVID-19: A Machine Learning Approach
Prout, T. A., Zilcha‐Mano, S., Doorn, K. A., Békés, V., Christman-Cohen, I., Whistler, K., Kui, T., & Giuseppe, M. D. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.586202 · Cited by 154 · see citing works
Report of the ACBS Task Force on the strategies and tactics of contextual behavioral science research
Hayes, S. C., Merwin, R. M., McHugh, L., Sandoz, E. K., A-Tjak, J. G. L., Ruiz, F. J., Barnes-Holmes, D., Bricker, J. B., Ciarrochi, J., Dixon, M. R., et al. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.03.007 · Cited by 136 · see citing works
Psychotherapists’ Challenges With Online Therapy During COVID-19: Concerns About Connectedness Predict Therapists’ Negative View of Online Therapy and Its Perceived Efficacy Over Time
Békés, V., Aafjes-van Doorn, K., Luo, X., Prout, T. A., & Hoffman, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.705699 · Cited by 134 · see citing works
Future Medical Artificial Intelligence Application Requirements and Expectations of Physicians in German University Hospitals: Web-Based Survey
Maassen, O., Fritsch, S., Palm, J., Deffge, S., Kunze, J., Marx, G., Riedel, M., Schuppert, A., & Bickenbach, J. (2021) · Journal of Medical Internet Research · doi:10.2196/26646 · Cited by 125 · see citing works
Ethics and Law in Research on Algorithmic and Data-Driven Technology in Mental Health Care: Scoping Review
Gooding, P., & Kariotis, T. (2021) · JMIR Mental Health · doi:10.2196/24668 · Cited by 100 · see citing works
Estimation of minimal data sets sizes for machine learning predictions in digital mental health interventions
Zantvoort, K., Nacke, B., Görlich, D., Hornstein, S., Jacobi, C., & Funk, B. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01360-w · Cited by 96 · see citing works
Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review
Gual-Montolio, P., Jaén, I., Martínez-Borba, V., Castilla, D., & Suso-Ribera, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19137737 · Cited by 93 · see citing works
Machine Learning Methods for Predicting Postpartum Depression: Scoping Review
Saqib, K., Khan, A. F., & Butt, Z. A. (2021) · JMIR Mental Health · doi:10.2196/29838 · Cited by 89 · see citing works
Pragmatic Precision Psychiatry—A New Direction for Optimizing Treatment Selection
Kessler, R. C., & Luedtke, A. R. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.2500 · Cited by 74 · see citing works
Transparency of Artificial Intelligence in Healthcare: Insights from Professionals in Computing and Healthcare Worldwide
Bernal, J., & Mazo, C. (2022) · Applied Sciences · doi:10.3390/app122010228 · Cited by 70 · see citing works
Personalized Medicine and Cognitive Behavioral Therapies for Depression: Small Effects, Big Problems, and Bigger Data
Lorenzo‐Luaces, L., Peipert, A., Jesús-Romero, R. D., Rutter, L. A., & Rodríguez‐Quintana, N. (2020) · International Journal of Cognitive Therapy · doi:10.1007/s41811-020-00094-3 · Cited by 62 · see citing works
Factors associated with treatment uptake, completion, and subsequent symptom improvement in a national digital mental health service
Cross, S. P., Karin, E., Staples, L. G., Bisby, M. A., Ryan, K., Duke, G., Nielssen, O., Kayrouz, R., Fisher, A., Dear, B. F., et al. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100506 · Cited by 57 · see citing works
Accelerating the impact of artificial intelligence in mental healthcare through implementation science
Nilsén, P., Svedberg, P., Nygren, J., Frideros, M., Johansson, J., & Schueller, S. M. (2022) · Implementation Research and Practice · doi:10.1177/26334895221112033 · Cited by 62 · see citing works
Attitudes Toward the Adoption of 2 Artificial Intelligence–Enabled Mental Health Tools Among Prospective Psychotherapists: Cross-sectional Study
Kleine, A.-K., Kokje, E., Lermer, E., & Gaube, S. (2023) · JMIR Human Factors · doi:10.2196/46859 · Cited by 49 · see citing works
Enhancing the quality of cognitive behavioral therapy in community mental health through artificial intelligence generated fidelity feedback (Project AFFECT): a study protocol
Creed, T. A., Salama, L., Slevin, R., Tanana, M., Imel, Z. E., Narayanan, S. S., & Atkins, D. C. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08519-9 · Cited by 49 · see citing works
Psychological insights into the research and practice of embodied conversational agents, chatbots and social assistive robots: a systematic meta-review
Kiuchi, K., Otsu, K., & Hayashi, Y. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2286528 · Cited by 50 · see citing works
Machine learning predictive models to guide prevention and intervention allocation for anxiety and depressive disorders among college students
Zhai, Y., Zhang, Y., Chu, Z., Geng, B., Almaawali, M. S., Fulmer, R., Lin, Y.-W. D., Xu, Z., Daniels, A. D., Liu, Y., et al. (2024) · Journal of Counseling & Development · doi:10.1002/jcad.12543 · Cited by 53 · see citing works
A scoping review of deliberate practice in the acquisition of therapeutic skills and practices
Mahon, D. (2022) · Counselling and Psychotherapy Research · doi:10.1002/capr.12601 · Cited by 45 · see citing works
Psychotherapists' acceptance of telepsychotherapy during the COVID‐19 pandemic: A machine learning approach
Békés, V., Doorn, K. A., Zilcha‐Mano, S., Prout, T. A., & Hoffman, L. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2682 · Cited by 40 · see citing works
Artificial Intelligence: An Interprofessional Perspective on Implications for Geriatric Mental Health Research and Care
Renn, B. N., Schurr, M. S., Zaslavsky, O., & Pratap, A. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.734909 · Cited by 31 · see citing works
Predicting dropout from psychological treatment using different machine learning algorithms, resampling methods, and sample sizes
Giesemann, J., Delgadillo, J., Schwartz, B., Bennemann, B., & Lutz, W. (2023) · Psychotherapy Research · doi:10.1080/10503307.2022.2161432 · Cited by 26 · see citing works
Measuring Alliance and Symptom Severity in Psychotherapy Transcripts Using Bert Topic Modeling
Lalk, C., Steinbrenner, T., Kania, W., Popko, A., Wester, R. A., Schaffrath, J., Eberhardt, S. T., Schwartz, B., Lutz, W., & Rubel, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01356-4 · Cited by 29 · see citing works
Computational Scientific Discovery in Psychology
Bartlett, L., Pirrone, A., Javed, N., & Gobet, F. R. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221091833 · Cited by 32 · see citing works
Innovations to improve outcomes and uptake of psychotherapies for mental disorders: a state‐of‐the‐art review
Cuijpers, P. J., Harrer, M., & Furukawa, T. A. (2026) · World Psychiatry · doi:10.1002/wps.70002 · Cited by 31 · see citing works
Personalized Prediction of Response to Smartphone-Delivered Meditation Training: Randomized Controlled Trial
Webb, C. A., Hirshberg, M. J., Davidson, R. J., & Goldberg, S. B. (2022) · Journal of Medical Internet Research · doi:10.2196/41566 · Cited by 27 · see citing works
Machine Learning Approaches to Predict Symptoms in People With Cancer: Systematic Review
Zeinali, N., Youn, N., Albashayreh, A., Fan, W., & Gilbertson White, S. (2024) · JMIR Cancer · doi:10.2196/52322 · Cited by 26 · see citing works
What the future holds: Machine learning to predict success in psychotherapy
Taubitz, F.-S., Büdenbender, B., & Alpers, G. W. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104116 · Cited by 25 · see citing works
Investigating psychotherapists’ attitudes towards artificial intelligence in psychotherapy
Wagner, J., & Schwind, A.-S. (2025) · BMC Psychology · doi:10.1186/s40359-025-03071-7 · Cited by 29 · see citing works
Experience in psychological counseling supported by artificial intelligence technology
Ping, Y. (2024) · Technology and Health Care · doi:10.3233/thc-230809 · Cited by 22 · see citing works
Mental health practitioners’ perceptions and adoption intentions of AI-enabled technologies: an international mixed-methods study
Cecil, J., Kleine, A., Lermer, E., & Gaube, S. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12715-8 · Cited by 31 · see citing works
Artificial intelligence and psychotherapy: A counterpoint
Richards, D. (2024) · Counselling and Psychotherapy Research · doi:10.1002/capr.12758 · Cited by 25 · see citing works
Opening the Black Box of Family-Based Treatments: An Artificial Intelligence Framework to Examine Therapeutic Alliance and Therapist Empathy
Cunningham, P. B., Gilmore, J., Naar, S., Preston, S. D., Eubanks, C. F., Hubig, N., McClendon, J. L., Ghosh, S., & Ryan-Pettes, S. R. (2023) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-023-00451-6 · Cited by 21 · see citing works
Predicting treatment response using machine learning: A registered report
Jankowsky, K., Krakau, L., Schroeders, U., Zwerenz, R., & Beutel, M. E. (2023) · British Journal of Clinical Psychology · doi:10.1111/bjc.12452 · Cited by 20 · see citing works
Effect of Mixed Variation of Magnesium Carbonate and Bottom Ash for Stabilization of Subgrade Soils
Firdaus, N. A. A., Ismanti, S., & Rifa'i, A. (2024) · The Open Civil Engineering Journal · doi:10.2174/0118749445304699240416074458 · Cited by 20 · see citing works
Methodological choices and clinical usefulness for machine learning predictions of outcome in Internet-based cognitive behavioural therapy
Isacsson, N. H., Abdesslem, F. B., Forsell, E., Boman, M., & Kaldo, V. (2024) · Communications Medicine · doi:10.1038/s43856-024-00626-4 · Cited by 18 · see citing works
A machine learning approach to identifying pregnant women's risk for persistent post-traumatic stress following childbirth
Rousseau, S., Polachek, I. S., & Frenkel, T. I. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.09.014 · Cited by 19 · see citing works
Use of Machine Learning Algorithms Based on Text, Audio, and Video Data in the Prediction of Anxiety and Posttraumatic Stress in General and Clinical Populations: A Systematic Review
Čihařová, M., Amarti, K., van Breda, W., Peng, X., Lorente-Català, R., Funk, B., Hoogendoorn, M., Koutsouleris, N., Fusar‐Poli, P., Karyotaki, E., et al. (2024) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.06.002 · Cited by 19 · see citing works
Leveraging Novel Technologies and Artificial Intelligence to Advance Practice-Oriented Research
Atzil–Slonim, D., Penedo, J. M. G., & Lutz, W. (2023) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01309-3 · Cited by 20 · see citing works
Using clinical patient characteristics to predict treatment outcome of cognitive behavior therapies for individuals with medically unexplained symptoms: a systematic review and meta-analysis
Sarter, L., Heider, J., Witthöft, M., Rief, W., & Kleinstäuber, M. (2022) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2022.03.001 · Cited by 17 · see citing works
Dataset size versus homogeneity: A machine learning study on pooling intervention data in e-mental health dropout predictions
Zantvoort, K., Isacsson, N. H., Funk, B., & Kaldo, V. (2024) · Digital Health · doi:10.1177/20552076241248920 · Cited by 17 · see citing works
AI-enabled clinical decision support tools for mental healthcare: A product review
Kleine, A., Kokje, E., Hummelsberger, P., Lermer, E., Schaffernak, I., & Gaube, S. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.103052 · Cited by 19 · see citing works
Evaluation of a therapist-guided virtual psychological pain management program when provided as routine care: a prospective pragmatic cohort study
Dear, B. F., Walker, J., Karin, E., Asrianti, L., England, J., Feliciano, I., Bisby, M. A., Nielssen, O. B., Kayrouz, R., Cross, S. P. M., et al. (2023) · Pain Medicine · doi:10.1093/pm/pnad102 · Cited by 17 · see citing works
Advancing mental health care with AI-enabled precision psychiatry tools: A patent review
Kleine, A., Lermer, E., Cecil, J., Heinrich, A., & Gaube, S. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100322 · Cited by 17 · see citing works
Which client with generalized anxiety disorder benefits from a mindfulness ecological momentary intervention versus a self-monitoring app? Developing a multivariable machine learning predictive model
Zainal, N. H., & Newman, M. G. (2024) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2024.102825 · Cited by 17 · see citing works
Scoping review on natural language processing applications in counselling and psychotherapy
Laricheva, M., Liu, Y., Shi, E., & Wu, A. D. (2024) · British Journal of Psychology · doi:10.1111/bjop.12721 · Cited by 19 · see citing works
Matching research and practice: Prediction of individual patient progress and dropout risk for basic routine outcome monitoring
Mütze, K., Witthöft, M., Lutz, W., & Bräscher, A. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1930244 · Cited by 17 · see citing works
A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity
Ahmadi, A., Noetel, M., Schellekens, M., Parker, P. D., Antczak, D., Beauchamp, M. R., Dicke, T., Diezmann, C. M., Maeder, A., Ntoumanis, N., et al. (2021) · Psychosocial Intervention · doi:10.5093/pi2021a4 · Cited by 17 · see citing works
Machine learning applications in studying mental health among immigrants and racial and ethnic minorities: an exploratory scoping review
Park, K. K., Saleem, M., Al-Garadi, M. A., & Ahmed, A. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02663-4 · Cited by 16 · see citing works
Sociodemographic Predictors and Moderators of Treatment Outcomes of Psychotherapeutic Interventions for Young People with Mental Disorders: A Systematic Review
Gergov, V., Prevendar, T., Vousoura, E., Ulberg, R., Dahl, H., Feller, C., Jacobsen, C. F., Karain, A., Milic, B., Poznyak, E., et al. (2023) · Adolescent Research Review · doi:10.1007/s40894-023-00228-6 · Cited by 15 · see citing works
Expanding the scope, reach, and impact of evidence-based psychological treatments
Kazdin, A. E. (2022) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2022.101744 · Cited by 14 · see citing works
Machine Learning Model to Predict Assignment of Therapy Homework in Behavioral Treatments: Algorithm Development and Validation
Peretz, G., Taylor, C. B., Ruzek, J. I., Jefroykin, S., & Sadeh‐Sharvit, S. (2023) · JMIR Formative Research · doi:10.2196/45156 · Cited by 14 · see citing works
Using machine learning algorithms to predict the effects of change processes in psychotherapy: Toward process-level treatment personalization.
Penedo, J. M. G., Rubel, J. A., Meglio, M., Bornhauser, L. P., Krieger, T., Babl, A., Muiños, R., Roussos, A., Delgadillo, J., Flückiger, C., et al. (2023) · Psychotherapy · doi:10.1037/pst0000507 · Cited by 13 · see citing works
A critical review of digital value engineering in building design towards automated construction
Khan, A. M., Alaloul, W. S., & Musarat, M. A. (2024) · Environment Development and Sustainability · doi:10.1007/s10668-024-05595-1 · Cited by 15 · see citing works
Artificial intelligence, machine learning and mental health
Delgadillo, J., & Atzil–Slonim, D. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-91497-0.00177-6 · Cited by 14 · see citing works
Applying dynamic systems theory and complexity theory methods in psychotherapy research: A systematic literature review
Klocek, A., Premus, J., & Řiháček, T. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2252169 · Cited by 12 · see citing works
Machine learning and experiments identifies SPINK1 as a candidate diagnostic and prognostic biomarker for hepatocellular carcinoma
Yi, S., Zhang, C., Li, M., Qu, T., & Wang, J. (2023) · Discover Oncology · doi:10.1007/s12672-023-00849-2 · Cited by 13 · see citing works
Using machine learning to increase access to and engagement with trauma‐focused interventions for posttraumatic stress disorder
Lenton‐Brym, A. P., Collins, A., Lane, J. E. M., Busso, C., Ouyang, J., Fitzpatrick, S., Kuo, J. R., & Monson, C. M. (2024) · British Journal of Clinical Psychology · doi:10.1111/bjc.12468 · Cited by 14 · see citing works
Feasibility of artificial intelligence‐based measurement in psychotherapy practice: Patients' and clinicians' perspectives
Doorn, K. A. (2024) · Counselling and Psychotherapy Research · doi:10.1002/capr.12800 · Cited by 12 · see citing works
Depression Symptoms are Associated with Frequency of Cognitive Distortions in Psychotherapy Transcripts
Lalk, C., Steinbrenner, T., Loray, J. S. P., Kania, W., Schaffrath, J., Eberhardt, S. T., Schwartz, B., Lutz, W., & Rubel, J. A. (2024) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10542-5 · Cited by 12 · see citing works
Prediction of individual patient outcomes to psychotherapy vs medication for major depression
LoParo, D., Dunlop, B. W., Nemeroff, C. B., Mayberg, H. S., & Craighead, W. E. (2025) · npj Mental Health Research · doi:10.1038/s44184-025-00119-9 · Cited by 12 · see citing works
A machine learning approach for text pattern diagnosis in mental health consultations
Juanita, S., Daeli, A. H., Syafrullah, M., Anggraeni, W., & Purnomo, M. H. (2025) · Decision Analytics Journal · doi:10.1016/j.dajour.2025.100572 · Cited by 14 · see citing works
Towards standardization of Prakriti Evaluation: A scoping review of modern assessment tools and their psychometric properties in Ayurvedic medicine
Gupta, A., Singh, V., Chandra, S., & Garg, R. (2025) · Journal of Ayurveda and Integrative Medicine · doi:10.1016/j.jaim.2025.101157 · Cited by 12 · see citing works
Using rich data to inform student mental health practice and policy
Broglia, E., Bewick, B. M., & Barkham, M. (2021) · Counselling and Psychotherapy Research · doi:10.1002/capr.12470 · Cited by 12 · see citing works
A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consistency
Miner, A. S., Fleming, S. L., Haque, A., Fries, J., Althoff, T., Wilfley, D. E., Agras, W. S., Milstein, A. S., Hancock, J. T., Asch, S. M., et al. (2022) · npj Mental Health Research · doi:10.1038/s44184-022-00020-9 · Cited by 17 · see citing works
Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology
(2022) · doi:10.18653/v1/2022.clpsych-1 · Cited by 12 · see citing works
Predictors of functional impairment at assessment and functional improvement after treatment at a national digital mental health service
Cross, S. P. M., Karin, E., Asrianti, L., Walker, J., Staples, L., Bisby, M. A., Nielssen, O. B., Kayrouz, R., Fisher, A. C., Dear, B. F., et al. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100603 · Cited by 12 · see citing works
Real-time internet of medical things framework for early detection of Covid-19
Yıldırım, E., Ci̇ci̇oğlu, M., & Çalhan, A. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07582-x · Cited by 11 · see citing works
How to make the most of routine outcome monitoring (ROM): A multitude of clinical decisions and nuances to consider
Doorn, K. A., & de Jong, K. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23438 · Cited by 11 · see citing works
Moving toward precision PTSD treatment: predicting veterans' intensive PTSD treatment response using continuously updating machine learning models
Smith, D. L., & Held, P. (2022) · Psychological Medicine · doi:10.1017/s0033291722002689 · Cited by 11 · see citing works
You never know what you are going to get: Large-scale assessment of therapists’ supportive counseling skill use.
Zhang, X., Tanana, M., Weitzman, L. M., Narayanan, S., Atkins, D. C., & Imel, Z. E. (2022) · Psychotherapy · doi:10.1037/pst0000460 · Cited by 11 · see citing works
Developing an Automated Assessment of In-session Patient Activation for Psychological Therapy: Codevelopment Approach
Malins, S., Figueredo, G. P., Jilani, T. A., Long, Y., Andrews, J. A., Rawsthorne, M., Manolescu, C., Clos, J., Higton, F., Waldram, D., et al. (2022) · JMIR Medical Informatics · doi:10.2196/38168 · Cited by 10 · see citing works
A Bibliometric Review of Natural Language Processing Applications in Psychology from 1991 to 2023
Chen, G., Tan, B., Laham, N., Tracey, T. J. G., Lapinski, S., & Liu, Y. (2024) · Basic and Applied Social Psychology · doi:10.1080/01973533.2024.2433720 · Cited by 11 · see citing works
Predicting working alliance in psychotherapy: A multi-modal machine learning approach
Doorn, K. A., Cicconet, M., Cohn, J. F., & Aafjes, M. (2025) · Psychotherapy Research · doi:10.1080/10503307.2024.2428702 · Cited by 11 · see citing works
Self-citation
Detecting defense mechanisms from Adult Attachment Interview (AAI) transcripts using machine learning
Tasca, A. N., Carlucci, S., Wiley, J. C., Holden, M., El-Roby, A., & Tasca, G. A. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2156306 · Cited by 10 · see citing works
Unsupervised Machine Learning Driven Analysis of Verbatims of Treatment-Resistant Schizophrenia Patients Having Followed Avatar Therapy
Hudon, A., Beaudoin, M., Phraxayavong, K., Potvin, S., & Dumais, A. (2023) · Journal of Personalized Medicine · doi:10.3390/jpm13050801 · Cited by 10 · see citing works
Improving the quality of counseling and clinical supervision in opioid treatment programs: how can technology help?
Peavy, K. M., Klipsch, A., Soma, C. S., Pace, B. T., Imel, Z. E., Tanana, M., Soth, S., Ricardo-Bulis, E., & Atkins, D. C. (2024) · Addiction Science & Clinical Practice · doi:10.1186/s13722-024-00435-z · Cited by 9 · see citing works
Challenges and opportunities in using interpretable AI to develop relationship interventions
Puhlman, D. J., & Chen, C. (2025) · Family Relations · doi:10.1111/fare.13172 · Cited by 10 · see citing works
Investigating predictors of treatment response in Dialectical Behavior Therapy for borderline personality disorder using LASSO regression
Yin, Q., Stern, M., Kleiman, E. M., & Rizvi, S. L. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2138790 · Cited by 9 · see citing works
Using Machine Learning to Match Clients and Therapy Providers: Evaluating Clinical Quality and Cost of Care
Lee, J. L., Billovits, C., Chen, S.-Y., Wickham, R. E., Kocher, B., Chen, C. E., & Lungu, A. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.002 · Cited by 8 · see citing works
Computational and ethical considerations for using large language models in psychotherapy
Zhang, R.-W., Meng, H., Neubronner, M., & Lee, Y.-C. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00874-x · Cited by 10 · see citing works
Editorial: Assessments and measures in psychotherapy research: going beyond self-report data
Schwartz, B., Uhl, J., & Atzil–Slonim, D. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1276222 · Cited by 8 · see citing works
Machine learning prediction will be part of future treatment of depression
Lennon, M. J., & Harmer, C. J. (2023) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/00048674231158267 · Cited by 7 · see citing works
Systematic review of machine learning utilization within outpatient psychodynamic psychotherapy research
Rollmann, I., Gebhardt, N., Stahl-Toyota, S., Simon, J. J., Sutcliffe, M., Friederich, H., & Nikendei, C. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1055868 · Cited by 7 · see citing works
Automating the assessment of multicultural orientation through machine learning and natural language processing.
Goldberg, S. B., Tanana, M., Haywood, S., Williams, C. Y., Soma, C. S., Atkins, D. C., Imel, Z. E., & Owen, J. J. (2024) · Psychotherapy · doi:10.1037/pst0000519 · Cited by 7 · see citing works
AI and mental health: evaluating supervised machine learning models trained on diagnostic classifications
van Oosterzee, A. (2024) · AI & Society · doi:10.1007/s00146-024-02012-z · Cited by 6 · see citing works
User Preferences on AI Psychotherapy Based on Moderating Effects of Individual Personality Traits: Employing a Clustering Analysis
Lee, J., You, J. S., & Lee, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2307695 · Cited by 7 · see citing works
Identification of Social Anxiety in High School: A Machine Learning Approaches to Real-Time Analysis of Student Characteristics
Abinaya, M., & Vadivu, G. (2024) · IEEE Access · doi:10.1109/access.2024.3407885 · Cited by 6 · see citing works
Protocol for a randomized controlled trial: exercise-priming of CBT for depression (the CBT+ trial)
Meyer, J. D., Kelly, S. J. E., Gidley, J. M., Lansing, J. E., Smith, S. L., Churchill, S. L., Thomas, E. B. K., Goldberg, S. B., Abercrombie, H. C., Murray, T. A., et al. (2024) · Trials · doi:10.1186/s13063-024-08495-x · Cited by 6 · see citing works
Within‐patient perceptions of alliance and attunement: Associations with progress in psychotherapy
Doorn, K. A., Bar‐Sella, A., Zilcha‐Mano, S., Luo, X., Silberschatz, G., Kealy, D., McCollum, J. K., & Snyder, J. F. (2022) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2737 · Cited by 6 · see citing works
Prediction of Chinese clients’ satisfaction with psychotherapy by machine learning
Yao, L., Wang, Z., Gu, H., Zhao, X., Chen, Y., & Liu, L. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.947081 · Cited by 6 · see citing works
Screening for Psychological Distress in Healthcare Workers Using Machine Learning: A Proof of Concept
Geoffrion, S., Morse, C., Dufour, M., Bergeron, N., Guay, S., & Lanovaz, M. J. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-02011-5 · Cited by 6 · see citing works
Predicting Psychopathology in Jewish Ultra-Orthodox IPV Survivors: A Machine Learning Approach
Lipinsky, A. S., Goldner, L., & Hadar, D. (2024) · Journal of Loss and Trauma · doi:10.1080/15325024.2023.2280613 · Cited by 6 · see citing works
Designing Computational Tools for Behavioral and Clinical Science
Salah, A. A. (2021) · ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) · doi:10.1145/3459926.3464906 · Cited by 6 · see citing works
Advanced data analysis and prediction model for student mental health risk assessment
Shi, X., Pan, J., Yuan, D., Li, M., & Pan, Y. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1682083 · Cited by 5 · see citing works
Machine Learning–Based Evaluation of Suicide Risk Assessment in Crisis Counseling Calls
Imel, Z. E., Pace, B. T., Pendergraft, B., Pruett, J., Tanana, M., Soma, C. S., Comtois, K. A., & Atkins, D. C. (2024) · Psychiatric Services · doi:10.1176/appi.ps.20230648 · Cited by 5 · see citing works
Using machine learning methods to identify trajectories of change and predict responders and non-responders to short-term dynamic therapy
Yonatan‐Leus, R., Gwertzman, G., & Tishby, O. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2420725 · Cited by 5 · see citing works
Predictors for improvement in personality functioning during outpatient psychotherapy: A machine learning approach within a psychodynamic psychotherapy sample
Dönnhoff, I., Kindermann, D., Stahl-Toyota, S., Nowak, J. A., Orth, M., Friederich, H., & Nikendei, C. (2024) · European Psychiatry · doi:10.1192/j.eurpsy.2024.1780 · Cited by 5 · see citing works

What the future holds: Machine learning to predict success in psychotherapy
Taubitz, F.-S., Büdenbender, B., & Alpers, G. W. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104116 · Cited by 25 · see citing works
Machine learning: A primer for psychotherapy researchers
Delgadillo, J. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1859638 · Cited by 30 · see citing works
Clinical psychoinformatics
Yamamoto, T., Yoshimoto, J., Alcaraz-Silva, J., Murillo‐Rodríguez, E., Imperatori, C., Machado, S. E. D. C., & Budde, H. (2021) · Elsevier eBooks · doi:10.1016/b978-0-323-85235-7.00013-2 · Cited by 1 · see citing works
Machine Learning Applications in Psychotherapy Research
Schröder‐Pfeifer, P. (2022) · heiDOK (Heidelberg University) · doi:10.11588/heidok.00031214
Machine learning and natural language processing in psychotherapy research: alliance as example use case.
Goldberg, S. B., Flemotomos, N., Martinez, V. R., Tanana, M. J., Kuo, P. B., Pace, B. T., Villatte, J. L., Georgiou, P. G., Van Epps, J., Imel, Z. E., et al. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000382 · Cited by 141 · see citing works

Authors: Aafjes-van Doorn, K., Kamsteeg, C., Bate, J., & Aafjes, M.
Affiliations: Ferkauf Graduate School of Psychology, Yeshiva University, Bronx, NY, USA
Deliberate.ai, New York, NY, USA
Yeshiva University (US)
ORCID: Aafjes-van Doorn, Katie
Bate, Jordan
Aafjes, Marc
Venue: Psychotherapy Research
Publisher: Informa UK Limited
Published: 2020-08-29
Volume: 31
Issue: 1
Pages: 92-116
ISSN: 1050-3307 (ISSN-L)
1468-4381
DOI: 10.1080/10503307.2020.1808729
OpenAlex: W3082929259
Cited by: 179
References: 111
Citations by year: 2026: 16 · 2025: 35 · 2024: 43 · 2023: 33 · 2022: 32 · 2021: 18 · 2020: 2
FWCI: 8.49
Type: journal article
Language: en
Open access: closed
Locations: Psychotherapy Research (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords: machine learning, supervised learning, unsupervised learning, treatment outcome prediction, medication adherence, therapist skills, automated coding, clinical implementation
MeSH terms: Machine Learning, Humans, Psychotherapy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Scoping Review of Machine Learning in Psychotherapy Research

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks