Towards integrating personalized feedback research into clinical practice: Development of the Trier Treatment Navigator (TTN)
Abstract
In this study, a computer-based feedback, decision and clinical problem-solving system for clinical practice will be described - the Trier Treatment Navigator (TTN). The paper deals with the underlying research concepts related to personalized pre-treatment recommendations for drop-out risk and optimal treatment strategy selection as well as personalized adaptive recommendations during treatment. The development sample consisted of 1234 patients treated with cognitive behavioral therapy (CBT). Modern statistical machine learning techniques were used to develop personalized recommendations. Drop-out analyses resulted in seven significant predictors explaining 12.0% of variance. The prediction of optimal treatment strategies resulted in differential prediction models substantially improving effect sizes and reliable improvement rates. The dynamic failure boundary reliably identified patients with a higher risk for no improvement or deterioration and indicated the usage of clinical problem-solving tools in risk areas. The probability to be reliably improved for patients identified as at risk for treatment failure was about half of the probability for other patients (35% vs. 62.15%; χ 2df=1 = 82.77, p < .001). Results related to the computer-based feedback system are discussed with regard to the implication for clinical applications as well as clinical training and future research possibilities.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two of the before-after studies suffered from potential bias in their loss to follow-up.
“Additionally, two of the studies suffered from potential bias in their loss to follow-up [82,83].”
From Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review (2022) · cites this paper as [82]
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
ggplot2: Elegant Graphics for Data Analysis
Ginestet, C. E. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 14,484 · see citing works
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447 · see citing works
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770 · see citing works
Bergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383 · see citing works
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506 · see citing works
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019247 · Cited by 657 · see citing works
Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress.
Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996) · American Psychologist · doi:10.1037/0003-066x.51.10.1059 · Cited by 573 · see citing works
Self-citationA lab test and algorithms for identifying clients at risk for treatment failure.
Hannan, C. R., Lambert, M., Harmon, C., Nielsen, S. L., Smart, D., Shimokawa, K., & Sutton, S. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20108 · Cited by 550 · see citing works
DeRubeis, R. J., Cohen, Z. D., Forand, N. R., Fournier, J. C., Gelfand, L. A., & Lorenzo-Luaces, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0083875 · Cited by 509 · see citing works
Lambert, M. (2007) · Psychotherapy Research · doi:10.1080/10503300601032506 · Cited by 461 · see citing works
Living with a depressed person.
Coyne, J. C., Kessler, R. C., Tal, M., Turnbull, J., Wortman, C. B., & Greden, J. F. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.3.347 · Cited by 457 · see citing works
Treatment selection in depression.
Cohen, Z. D., & DeRubeis, R. J. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050817-084746 · Cited by 435 · see citing works
Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring.
Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018) · Psychotherapy · doi:10.1037/pst0000167 · Cited by 408 · see citing works
Exploring the idiographic dynamics of mood and anxiety via network analysis
Fisher, A. J., Reeves, J. W., Lawyer, G., Medaglia, J. D., & Rubel, J. A. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000311 · Cited by 385 · see citing works
Self-citationWhipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003) · Journal of Counseling Psychology · doi:10.1037/0022-0167.50.1.59 · Cited by 374 · see citing works
Annual Research Review: Building a science of personalized intervention for youth mental health
Ng, M. Y., & Weisz, J. R. (2016) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12470 · Cited by 291 · see citing works
Baldwin, S. A., Berkeljon, A., Atkins, D. C., Olsen, J. A., & Nielsen, S. L. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0015235 · Cited by 274 · see citing works
Kendrick, T., El-Gohary, M., Stuart, B. L., Gilbody, S. M., Churchill, R. C., Aiken, L., Bhattacharya, A., Gimson, A., Brütt, A. L., de Jong, K., et al. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011119.pub2 · Cited by 235 · see citing works
Huibers, M. J. H., Cohen, Z. D., Lemmens, L. H. J. M., Arntz, A., Peeters, F. P. M. L., Cuijpers, P., & DeRubeis, R. J. (2015) · PLOS ONE · doi:10.1371/journal.pone.0140771 · Cited by 215 · see citing works
Enhancing the Personalization of Psychotherapy With Dynamic Assessment and Modeling
Fisher, A. J., & Boswell, J. F. (2016) · Assessment · doi:10.1177/1073191116638735 · Cited by 205 · see citing works
The Affective Style Questionnaire: Development and Psychometric Properties
Hofmann, S. G., & Kashdan, T. B. (2010) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-009-9142-4 · Cited by 203 · see citing works
Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder
Kessler, R. C., van Loo, H. M., Wardenaar, K. J., Bossarte, R. M., Brenner, L. A., Ebert, D. D., de Jonge, P., Nierenberg, A. A., Rosellini, A. J., Sampson, N. A., et al. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000020 · Cited by 193 · see citing works
Delgadillo, J., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Gilbody, S., Ali, S., Aguirre, E., Appleton, M., Nevin, J., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30162-7 · Cited by 152 · see citing works
Self-citationOutcome Questionnaire: Is It Sensitive to Changes in Counseling Center Clients?
Vermeersch, D. A., Whipple, J. L., Lambert, M. J., Hawkins, E. J., Burchfield, C. M., & Okiishi, J. C. (2004) · Journal of Counseling Psychology · doi:10.1037/0022-0167.51.1.38 · Cited by 145 · see citing works
Finch, A. E., Lambert, M. J., & Schaalje, B. G. (2001) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.286 · Cited by 143 · see citing works
Predicting change for individual psychotherapy clients on the basis of their nearest neighbors.
Lutz, W., Leach, C., Barkham, M., Lucock, M. P., Stiles, W. B., Evans, C., Noble, R., & Iveson, S. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.904 · Cited by 128 · see citing works
Self-citationPersonalized Psychotherapy for Adult Depression: A Meta-Analytic Review
Cuijpers, P., Ebert, D. D., Acarturk, C., Andersson, G., & Cristea, I. A. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.04.007 · Cited by 128 · see citing works
Lutz, W., Schwartz, B., Hofmann, S., Fisher, A., Husen, K., & Rubel, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-25953-0 · Cited by 124 · see citing works
Self-citationWhat to Do When Manuals Fail? The Dual Model of Psychotherapy
Schulte, D., & Eifert, G. H. (2002) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.9.3.312 · Cited by 119 · see citing works
Therapist Effects on and Predictors of Non‐Consensual Dropout in Psychotherapy
Zimmermann, D., Rubel, J. A., Page, A. C., & Lutz, W. (2016) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2022 · Cited by 114 · see citing works
Self-citationPatient-focused and feedback research in psychotherapy: Where are we and where do we want to go?
Lutz, W., de Jong, K., & Rubel, J. A. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1079661 · Cited by 113 · see citing works
Self-citationDose–effect relationship in routine outpatient psychotherapy: Does treatment duration matter?
Stulz, N., Lutz, W., Kopta, S. M., Minami, T., & Saunders, S. M. (2013) · Journal of Counseling Psychology · doi:10.1037/a0033589 · Cited by 112 · see citing works
Self-citationFurukawa, T. A., Efthimiou, O., Weitz, E. S., Cipriani, A., Keller, M. B., Kocsis, J. H., Klein, D. N., Michalak, J., Salanti, G., Cuijpers, P., et al. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000489227 · Cited by 109 · see citing works
Case complexity as a guide for psychological treatment selection.
Delgadillo, J., Huey, D., Bennett, H., & McMillan, D. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000231 · Cited by 105 · see citing works
Lutz, W., Tholen, S., Schürch, E., & Berking, M. (2006) · Diagnostica · doi:10.1026/0012-1924.52.1.11 · Cited by 98 · see citing works
Self-citationPersonalized feedback on symptom dynamics of psychopathology: A proof-of-principle study
Kroeze, R., van der Veen, D. C., Servaas, M. N., Bastiaansen, J. A., Voshaar, R. C. O. V., Borsboom, D., Ruhe, H. G., Schoevers, R. A., & Riese, H. (2017) · Journal for Person-Oriented Research · doi:10.17505/jpor.2017.01 · Cited by 93 · see citing works
Rubel, J. A., Fisher, A. J., Husen, K., & Lutz, W. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000487769 · Cited by 86 · see citing works
Self-citationIndividual treatment selection for patients with posttraumatic stress disorder
Deisenhofer, A., Delgadillo, J., Rubel, J. A., Boehnke, J. R., Zimmermann, D., Schwartz, B., & Lutz, W. (2018) · Depression and Anxiety · doi:10.1002/da.22755 · Cited by 85 · see citing works
Self-citationEllsworth, J. R., Lambert, M. J., & Johnson, J. E. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.503 · Cited by 82 · see citing works
Abandoning personalization to get to precision in the pharmacotherapy of depression
Perlis, R. H. (2016) · World Psychiatry · doi:10.1002/wps.20345 · Cited by 78 · see citing works
Lutz, W., Saunders, S. M., Leon, S. C., Martinovich, Z., Kosfelder, J., Schulte, D., Grawe, K., & Tholen, S. (2006) · Psychological Assessment · doi:10.1037/1040-3590.18.2.133 · Cited by 68 · see citing works
Self-citationHusen, K., Rafaeli, E., Rubel, J. A., Bar‐Kalifa, E., & Lutz, W. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.08.025 · Cited by 55 · see citing works
Self-citationGuo, P., Zeng, F., Hu, X.-M., Zhang, D., Zhu, S., Deng, Y., & Yuantao, H. (2015) · PLoS ONE · doi:10.1371/journal.pone.0134151 · Cited by 53 · see citing works
Rubel, J., Lutz, W., Kopta, S., Köck, K., Minami, T., Zimmermann, D., & Saunders, S. (2015) · Psychological Assessment · doi:10.1037/pas0000060 · Cited by 46 · see citing works
Self-citationDer „Affective Style Questionnaire (ASQ)”: Deutsche Adaption und Validitäten
Graser, J., Bohn, C., Kelava, A., Schreiber, F., Hofmann, S. G., & Stangier, U. (2012) · Diagnostica · doi:10.1026/0012-1924/a000056 · Cited by 39 · see citing works
Lutz, W., Zimmermann, D., Müller, V., Deisenhofer, A., & Rubel, J. A. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1464-2 · Cited by 38 · see citing works
Self-citationUsing the Assessment for Signal Clients as a feedback tool for reducing treatment failure
White, M. M., Lambert, M. J., Ogles, B. M., Mclaughlin, S. B., Bailey, R. J., & Tingey, K. M. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1009862 · Cited by 36 · see citing works
Probst, T., Lambert, M. J., Dahlbender, R. W., Loew, T. H., & Tritt, K. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.03.010 · Cited by 32 · see citing works
Five types of clinical difference to monitor in practice.
Langkaas, T. F., Wampold, B. E., & Hoffart, A. (2018) · Psychotherapy · doi:10.1037/pst0000194 · Cited by 30 · see citing works
Therapist–client agreement in assessments of clients’ functioning.
Bar‐Kalifa, E., Atzil–Slonim, D., Rafaeli, E., Peri, T., Rubel, J. A., & Lutz, W. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000157 · Cited by 25 · see citing works
Self-citationUsing Item and Test Information to Optimize Targeted Assessments of Psychological Distress
Boehnke, J. R., & Lutz, W. (2014) · Assessment · doi:10.1177/1073191114529152 · Cited by 15 · see citing works
Self-citationHooke, G. R., Sng, A., Cunningham, N. K., & Page, A. C. (2017) · Cognitive Therapy and Research · doi:10.1007/s10608-017-9851-z · Cited by 10 · see citing works
Flagged first, then by citations
1 of the 54 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The promise of machine learning in predicting treatment outcomes in psychiatry
Chekroud, A. M., Bondar, J., Delgadillo, J., Doherty, G., Wasil, A., Fokkema, M., Cohen, Z., Belgrave, D., DeRubeis, R., Iniesta, R., et al. (2021) · World Psychiatry · doi:10.1002/wps.20882 · Cited by 531 · see citing works
Psychopathological networks: Theory, methods and practice
Bringmann, L. F., Albers, C. J., Bockting, C., Borsboom, D., Ceulemans, E., Cramer, A. O. J., Epskamp, S., Eronen, M. I., Hamaker, E. L., Kuppens, P., et al. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.104011 · Cited by 297 · see citing works
Self-citationBickman, L. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01065-8 · Cited by 184 · see citing works
A Scoping Review of Machine Learning in Psychotherapy Research
Aafjes-van Doorn, K., Kamsteeg, C., Bate, J., & Aafjes, M. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1808729 · Cited by 179 · see citing works
Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations
Barkham, M., de Jong, K., Delgadillo, J., & Lutz, W. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2181114 · Cited by 173 · see citing works
Self-citationMeasurement-Based and Data-Informed Psychological Therapy
Lutz, W., Schwartz, B., & Delgadillo, J. (2021) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-071720-014821 · Cited by 129 · see citing works
Self-citationA Development Pathway Towards Precision Mental Health Care
Delgadillo, J., & Lutz, W. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1048 · Cited by 108 · see citing works
Self-citationDeisenhofer, A., Barkham, M., Beierl, E. T., Schwartz, B., Doorn, K. A., Beevers, C. G., Berwian, I. M., Blackwell, S. E., Bockting, C., Brakemeier, E., et al. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104443 · Cited by 115 · see citing works
Self-citationGual-Montolio, P., Jaén, I., Martínez-Borba, V., Castilla, D., & Suso-Ribera, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19137737 · Cited by 93 · see citing works
Schwartz, B., Cohen, Z. D., Rubel, J. A., Zimmermann, D., Wittmann, W. W., & Lutz, W. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1769219 · Cited by 93 · see citing works
Self-citationMachine Learning Methods for Predicting Postpartum Depression: Scoping Review
Saqib, K., Khan, A. F., & Butt, Z. A. (2021) · JMIR Mental Health · doi:10.2196/29838 · Cited by 89 · see citing works
Distinct roles of state-like and trait-like patient–therapist alliance in psychotherapy
Zilcha‐Mano, S., & Fisher, H. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00029-z · Cited by 89 · see citing works
Bone, C., Simmonds-Buckley, M., Thwaites, R., Sandford, D., Merzhvynska, M., Rubel, J., Deisenhofer, A.-K., Lutz, W., & Delgadillo, J. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00018-2 · Cited by 79 · see citing works
Self-citationSchuster, R., Topooco, N., Keller, A., Radvogin, E., & Laireiter, A.-R. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100326 · Cited by 77 · see citing works
Heterogeneity of treatment effects in trials on psychotherapy of depression.
Kaiser, T., Volkmann, C., Volkmann, A., Karyotaki, E., Cuijpers, P. J., & Brakemeier, E. (2022) · Clinical Psychology Science and Practice · doi:10.1037/cps0000079 · Cited by 77 · see citing works
Bos, F. M., von Klipstein, L., Emerencia, A. C., Veermans, E., Verhage, T., Snippe, E., Doornbos, B., Hadders-Prins, G., Wichers, M., & Riese, H. (2022) · JMIR Mental Health · doi:10.2196/36430 · Cited by 62 · see citing works
Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review
de Jong, K., Douglas, S. R., Wolpert, M., Delgadillo, J., Aas, B., Bovendeerd, B., Carlier, I. V. E., Compare, A., Edbrooke-Childs, J. H., Janse, P. D., et al. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01381-3 · Cited by 60 · see citing works
Self-citationESMvis: a tool for visualizing individual Experience Sampling Method (ESM) data
Bringmann, L. F., van der Veen, D. C., Wichers, M., Riese, H., & Stulp, G. (2020) · Quality of Life Research · doi:10.1007/s11136-020-02701-4 · Cited by 51 · see citing works
McAleavey, A. A., de Jong, K., Nissen‐Lie, H. A., Boswell, J. F., Moltu, C., & Lutz, W. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01351-9 · Cited by 50 · see citing works
Self-citationLåver, J., McAleavey, A. A., Valaker, I., Castonguay, L. G., & Moltu, C. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2222896 · Cited by 44 · see citing works
Prinz, J. N., Boyle, K. S., Ramseyer, F., Kabus, W., Bar‐Kalifa, E., & Lutz, W. (2020) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2498 · Cited by 42 · see citing works
Self-citationMarchant, N. L., Barnhofer, T., Coueron, R., Wirth, M., Lutz, A., Arenaza-Urquijo, E. M., Collette, F., Poisnel, G., Demnitz-King, H., Schild, A.-K., et al. (2021) · Psychotherapy and Psychosomatics · doi:10.1159/000515669 · Cited by 40 · see citing works
Treatment Personalization and Precision Mental Health Care: Where are we and where do we want to go?
Moggia, D., Lutz, W., Brakemeier, E., & Bickman, L. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01407-w · Cited by 40 · see citing works
Self-citationBrakemeier, E., & Herpertz, S. C. (2019) · Der Nervenarzt · doi:10.1007/s00115-019-00808-9 · Cited by 36 · see citing works
Towards personalized allocation of patients to therapists.
Delgadillo, J., Rubel, J. A., & Barkham, M. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000507 · Cited by 37 · see citing works
Self-citationDecoding emotions: Exploring the validity of sentiment analysis in psychotherapy
Eberhardt, S. T., Schaffrath, J., Danilo Moggia, N., Schwartz, B., Jaehde, M., Rubel, J. A., Baur, T., André, E., & Lutz, W. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2322522 · Cited by 36 · see citing works
Self-citationHehlmann, M. I., Schwartz, B., Lutz, T., Penedo, J. M. G., Rubel, J. A., & Lutz, W. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.613085 · Cited by 38 · see citing works
Self-citationZimmermann, D., Wampold, B. E., Rubel, J. A., Schwartz, B., Poster, K., Schilling, V. N. L. S., Deisenhofer, A., Hehlmann, M. I., Penedo, J. M. G., & Lutz, W. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1847344 · Cited by 34 · see citing works
Self-citationGiesemann, J., Delgadillo, J., Schwartz, B., Bennemann, B., & Lutz, W. (2023) · Psychotherapy Research · doi:10.1080/10503307.2022.2161432 · Cited by 26 · see citing works
Self-citationEvaluating and Enhancing Artificial Intelligence Models for Predicting Student Learning Outcomes
Farhood, H., Joudah, I. N., Beheshti, A., & Müller, S. (2024) · Informatics · doi:10.3390/informatics11030046 · Cited by 29 · see citing works
Evidence-Based Strategies for Treatment Personalization: A Review
Stumpp, N. E., & Sauer-Zavala, S. (2022) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2021.10.004 · Cited by 29 · see citing works
Boyle, K. S., Deisenhofer, A., Rubel, J. A., Bennemann, B., Weinmann‐Lutz, B., & Lutz, W. (2019) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2019.1625945 · Cited by 30 · see citing works
Self-citationCuijpers, P. J., Harrer, M., & Furukawa, T. A. (2026) · World Psychiatry · doi:10.1002/wps.70002 · Cited by 31 · see citing works
Terhürne, P., Schwartz, B., Baur, T., Schiller, D., Eberhardt, S. T., André, E., & Lutz, W. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.1026015 · Cited by 28 · see citing works
Self-citationPenedo, J. M. G., Schwartz, B., Giesemann, J., Rubel, J. A., Deisenhofer, A., & Lutz, W. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1930242 · Cited by 27 · see citing works
Self-citationWucherpfennig, F., Schwartz, B., & Rubel, J. A. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2206051 · Cited by 26 · see citing works
Self-citationWhat the future holds: Machine learning to predict success in psychotherapy
Taubitz, F.-S., Büdenbender, B., & Alpers, G. W. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104116 · Cited by 25 · see citing works
Schilling, V. N. L. S., Zimmermann, D., Rubel, J. A., Boyle, K. S., & Lutz, W. (2020) · Quality of Life Research · doi:10.1007/s11136-020-02664-6 · Cited by 25 · see citing works
Self-citationEspel‐Huynh, H., Zhang, F., Thomas, J. G., Boswell, J. F., Thompson‐Brenner, H., Juarascio, A. S., & Lowe, M. R. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23510 · Cited by 24 · see citing works
Investigating psychotherapists’ attitudes towards artificial intelligence in psychotherapy
Wagner, J., & Schwind, A.-S. (2025) · BMC Psychology · doi:10.1186/s40359-025-03071-7 · Cited by 29 · see citing works
Personalization of Treatment for Patients with Childhood-Abuse-Related Posttraumatic Stress Disorder
Hoeboer, C. M., Oprel, D. A. C., de Kleine, R. A., Schwartz, B., Deisenhofer, A., Schoorl, M., van der Does, W. A. J., van Minnen, A., & Lutz, W. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10194522 · Cited by 22 · see citing works
Self-citationSchuster, R., Schreyer, M. L., Kaiser, T., Berger, T., Klein, J. P., Moritz, S., Laireiter, A.-R., & Trutschnig, W. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100313 · Cited by 21 · see citing works
Schaffrath, J., Weinmann‐Lutz, B., & Lutz, W. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23362 · Cited by 21 · see citing works
Self-citationMoggia, D., Bennemann, B., Schwartz, B., Hehlmann, M., Driver, C. C., & Lutz, W. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2222892 · Cited by 22 · see citing works
Self-citationSchuster, R., Kaiser, T., Terhorst, Y., Meßner, E.-M., Strohmeier, L.-M., & Laireiter, A. (2021) · Psychological Medicine · doi:10.1017/s003329172100129x · Cited by 21 · see citing works
Working Towards the Development and Implementation of Precision Mental Healthcare: An Example
Lutz, W., Schwartz, B., Penedo, J. M. G., Boyle, K. S., & Deisenhofer, A. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01053-y · Cited by 19 · see citing works
Self-citationSchwartz, B., Rubel, J. A., Deisenhofer, A., & Lutz, W. (2022) · Digital Health · doi:10.1177/20552076221129098 · Cited by 18 · see citing works
Self-citationLeveraging Novel Technologies and Artificial Intelligence to Advance Practice-Oriented Research
Atzil–Slonim, D., Penedo, J. M. G., & Lutz, W. (2023) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01309-3 · Cited by 20 · see citing works
Self-citationZilcha‐Mano, S., & Ramseyer, F. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000426 · Cited by 18 · see citing works
Precision Mental Health and Data-Informed Decision Support in Psychological Therapy: An Example
Lutz, W., Schaffrath, J., Eberhardt, S. T., Hehlmann, M. I., Schwartz, B., Deisenhofer, A., Vehlen, A., Schürmann, S., Uhl, J., & Danilo Moggia, N. (2023) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01330-6 · Cited by 17 · see citing works
Self-citationIndividual-Specific Animated Profiles of Mental Health
Zilcha‐Mano, S. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231226308 · Cited by 18 · see citing works
Predicting the outcome of psychotherapy for chronic depression by person‐specific symptom networks
Schumacher, L., Klein, J. P., Hautzinger, M., Härter, M. C., Schramm, E., & Kriston, L. (2024) · World Psychiatry · doi:10.1002/wps.21241 · Cited by 17 · see citing works
McAleavey, A. A., Castonguay, L. G., Hayes, J. A., & Locke, B. D. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000529 · Cited by 17 · see citing works
Mütze, K., Witthöft, M., Lutz, W., & Bräscher, A. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1930244 · Cited by 17 · see citing works
Self-citationSarter, L., Heider, J., Witthöft, M., Rief, W., & Kleinstäuber, M. (2022) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2022.03.001 · Cited by 17 · see citing works
Demir, S., Schwarz, F., & Kaiser, T. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23408 · Cited by 16 · see citing works
Defining and operationalizing personalized psychological treatment – a systematic literature review
Harnas, S. J., Knoop, H., Sprangers, M., & Braamse, A. (2024) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2024.2333345 · Cited by 17 · see citing works
Hall, M., Lappenbusch, L. M., Wiegmann, E., & Rubel, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01333-3 · Cited by 14 · see citing works
Self-citationScholten, S., Schemer, L., Herzog, P., Haas, J. W., Heider, J., Winter, D., Reis, D., & Glombiewski, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01363-5 · Cited by 18 · see citing works
Gonçalves, M. M., Lutz, W., Schwartz, B., Oliveira, J. T., Saarni, S. E., Tishby, O., Rubel, J. A., Boehnke, J. R., Montesano, A., Paiva, D., et al. (2024) · Clinical Psychology in Europe · doi:10.32872/cpe.13827 · Cited by 14 · see citing works
Self-citationEinsatz internetbasierter Verlaufsmessung in der Psychotherapieforschung
Wilhelm, M., Feldhege, J., Bauer, S., & Moessner, M. (2020) · Psychotherapeut · doi:10.1007/s00278-020-00461-7 · Cited by 14 · see citing works
Finnish Psychotherapy Quality Register: rationale, development, and baseline results
Saarni, S. E., Rosenström, T. H., Stenberg, J., Plattonen, A., Holi, M. M., Ekelund, J., Granö, N., Komsi, N., & Saarni, S. I. (2022) · Nordic Journal of Psychiatry · doi:10.1080/08039488.2022.2150788 · Cited by 14 · see citing works
What if we wait? Using synthetic waiting lists to estimate treatment effects in routine outcome data
Kaiser, T., Brakemeier, E., & Herzog, P. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2182241 · Cited by 14 · see citing works
Penedo, J. M. G., Rubel, J. A., Meglio, M., Bornhauser, L. P., Krieger, T., Babl, A., Muiños, R., Roussos, A., Delgadillo, J., Flückiger, C., et al. (2023) · Psychotherapy · doi:10.1037/pst0000507 · Cited by 13 · see citing works
Self-citationQualitätsmerkmale und Versorgungsrelevanz psychodynamischer Ausbildungsambulanzen: Das QVA-Projekt
Benecke, C., & Volz, M. (2023) · Zeitschrift für psychosomatische Medizin und Psychotherapie · doi:10.13109/zptm.2023.69.oa4 · Cited by 13 · see citing works
Valdiviezo‐Oña, J., Montesano, A., Evans, C., & Paz, C. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-071875 · Cited by 13 · see citing works
Lutz, W., Edelbluth, S., Deisenhofer, A., Delgadillo, J., Danilo Moggia, N., Prinz, J. N., & Schwartz, B. (2021) · Psychotherapy and Psychosomatics · doi:10.1159/000515543 · Cited by 13 · see citing works
Self-citationWester, R. A., Rubel, J. A., & Mayer, A. (2022) · Clinical Psychological Science · doi:10.1177/21677026211071043 · Cited by 13 · see citing works
Self-citationAre some therapists better at facilitating and consolidating sudden gains than others?
Deisenhofer, A., Rubel, J. A., Bennemann, B., Aderka, I. M., & Lutz, W. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1921302 · Cited by 12 · see citing works
Self-citationPenedo, J. M. G., Babl, A., Dyresen, A., Fernández‐Álvarez, J., Flückiger, C., & Holtforth, M. G. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104343 · Cited by 12 · see citing works
Schwartz, B., Hehlmann, M. I., Deisenhofer, A., Rubel, J. A., Fischer, L., Lutz, W., & Schöttke, H. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104689 · Cited by 12 · see citing works
Self-citationTrans‐theoretical clinical models and the implementation of precision mental health care
Lutz, W., & Schwartz, B. (2021) · World Psychiatry · doi:10.1002/wps.20888 · Cited by 12 · see citing works
Self-citationVousoura, E., Gergov, V., Tulbure, B. T., Camilleri, N., Saliba, A., Garcia-Lopez, L., Podina, I. R., Prevendar, T., Löffler-Stastka, H., Chiarenza, G. A., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01788-1 · Cited by 11 · see citing works
“In more recent years, advances in computational science have enabled psychotherapy researchers to develop and test methods of optimizing treatment packages for the individual patient [34, 35], although there are inconsistent findings regarding the extent to which personalized treatment has in fact led to improved outc…”
BackgroundPenedo, J. M. G., Rubel, J. A., Krieger, T., Alalú, N., Babl, A., Roussos, A., & Holtforth, M. G. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000528 · Cited by 11 · see citing works
Self-citationKooiman, B., Robberegt, S. J., Albers, C., Bockting, C., Stikkelbroek, Y. A. J., & Nauta, M. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1229713 · Cited by 9 · see citing works
Senger, K., Schröder, A., Kleinstäuber, M., Rubel, J. A., Rief, W., & Heider, J. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1916120 · Cited by 11 · see citing works
Self-citationNOVA: A Tool for Explanatory Multimodal Behavior Analysis and Its Application to Psychotherapy
Baur, T., Clausen, S. A., Heimerl, A., Lingenfelser, F., Lutz, W., & André, E. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-37734-2_47 · Cited by 11 · see citing works
Self-citationTherapist Interventions and Skills as Predictors of Dropout in Outpatient Psychotherapy
Poster, K., Bennemann, B., Hofmann, S. G., & Lutz, W. (2021) · Behavior Therapy · doi:10.1016/j.beth.2021.05.001 · Cited by 10 · see citing works
Self-citationUsing feedback tools to enhance the quality and experience of care
Boehnke, J. R., & Rutherford, C. (2021) · Quality of Life Research · doi:10.1007/s11136-021-03008-8 · Cited by 10 · see citing works
A scoping review of clinical decision support systems for child and adolescent mental health
Bertie, L., McDermott, E. A., Quiroz, J. C., & Hudson, J. L. (2025) · Current Psychology · doi:10.1007/s12144-025-07365-2 · Cited by 9 · see citing works
Barkham, M., de Jong, K., Delgadillo, J., & Lutz, W. (2023) · doi:10.1093/oso/9780197611012.003.0015 · Cited by 9 · see citing works
Self-citationJonášová, K., Čevelíček, M., Doležal, P., & Řiháček, T. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01348-4 · Cited by 9 · see citing works
Danilo Moggia, N., Schwartz, B., Rubel, J. A., Zimmermann, D., Kästel, B., & Lutz, W. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2126334 · Cited by 9 · see citing works
Self-citationDynamic learning of individual-level suicidal ideation trajectories to enhance mental health care
Varidel, M., Hickie, I. B., Prodan, A., Skinner, A., Marchant, R., Cripps, S., Oliveria, R., Chong, M. K., Scott, E. M., Scott, J. L., et al. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00071-0 · Cited by 8 · see citing works
Penedo, J. M. G., Schwartz, B., Deisenhofer, A., Rubel, J. A., Babl, A., & Lutz, W. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.10.043 · Cited by 8 · see citing works
Self-citationEdelbluth, S., Schwartz, B., & Lutz, W. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01361-7 · Cited by 7 · see citing works
Self-citationde Beurs, E., Bruinsma, C., & Warmerdam, L. (2020) · The European Journal of Psychiatry · doi:10.1016/j.ejpsy.2019.12.005 · Cited by 7 · see citing works
Penedo, J. M. G., Areas, M., Manubens, R. T., Babl, A., Challú, L., Juan, S., Muiños, R., Fernández-Álvarez, J., Alalú, N., Roussos, A., et al. (2021) · Revista Evaluar · doi:10.35670/1667-4545.v21.n2.34393 · Cited by 7 · see citing works
Self-citationRoutine outcome monitoring: The need for case examples
de Jong, K., & Doorn, K. A. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23441 · Cited by 7 · see citing works
Danilo Moggia, N., Lutz, W., Kazantzis, N., Schwartz, B., & Bakker, D. (2023) · Behavior Therapy · doi:10.1016/j.beth.2023.05.014 · Cited by 7 · see citing works
Self-citationTechnological Interventions for Emotion Regulation
Fernández‐Álvarez, J., Díaz‐García, A., Colombo, D., Botella, C., Cipresso, P., & Riva, G. (2020) · Comprehensive Clinical Psychology · doi:10.1016/b978-0-12-818697-8.00033-9 · Cited by 8 · see citing works
Scholten, S., Herzog, P., Glombiewski, J. A., & Kaiser, T. (2024) · Pain · doi:10.1097/j.pain.0000000000003363 · Cited by 6 · see citing works
Deres-Cohen, K., Lipsitz‐Odess, I., Fisher, H., Ramseyer, F., Lutz, W., & Zilcha‐Mano, S. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1966542 · Cited by 6 · see citing works
Self-citationHilhorst, L., van der Stappen, J., Lokkerbol, J., Hiligsmann, M., Risseeuw, A. H., & Tiemens, B. G. (2022) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-022-01194-2 · Cited by 6 · see citing works
Prediction of Chinese clients’ satisfaction with psychotherapy by machine learning
Yao, L., Wang, Z., Gu, H., Zhao, X., Chen, Y., & Liu, L. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.947081 · Cited by 6 · see citing works
Hall, M., Wagner, A. A., Scherner, P. V., Michael, K. L., Lawyer, G., Lutz, W., & Rubel, J. A. (2022) · doi:10.31234/osf.io/8deyj · Cited by 6 · see citing works
Self-citationCognitive-behavioural therapy and personalized treatment: an introduction to the special issue
Spinhoven, P., Cuijpers, P., & Hollon, S. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103595 · Cited by 5 · see citing works
Feedback-Informed Psychological Therapy in Practice: The Trier Treatment Navigator (TTN)
Lutz, W., Danilo Moggia, N., Bommer, J., Schaffrath, J., Deisenhofer, A., Eberhardt, S. T., Vehlen, A., Schwartz, B., & Rubel, J. A. (2025) · CBT: science into practice · doi:10.1007/978-3-031-90428-8_2 · Cited by 5 · see citing works
Self-citationWester, R. A., Schwartz, B., Lutz, W., Hall, M., Hoos, T., & Rubel, J. A. (2024) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000866 · Cited by 5 · see citing works
Self-citationResponding to key process markers as a focus of psychotherapy training and practice
Boswell, J. F., Constantino, M. J., Gaines, A. N., & Smith, A. E. (2024) · Clinical Psychology in Europe · doi:10.32872/cpe.11967 · Cited by 5 · see citing works
- Authors
- Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K.
- Affiliations
- Universität Trier (DE)
- Corresponding author
- Lutz, Wolfgang
- Venue
- Behaviour Research and Therapy
- Publisher
- Elsevier BV
- Published
- 2019-09
- Volume
- 120
- Pages
- 103438
- ISSN
- 0005-7967 (ISSN-L)
- OpenAlex
- W2956805281
- Cited by
- 196
- References
- 54
- Citations by year
- 2026: 18 · 2025: 26 · 2024: 39 · 2023: 25 · 2022: 22 · 2021: 34 · 2020: 25 · 2019: 7
- FWCI
- 24.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Behaviour Research and Therapy (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (LU 660/8-1, LU 660/10-1)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Keywords
- personalized feedback, cognitive behavioral therapy, treatment strategy selection, clinical problem solving, computer-based decision support, treatment failure prediction, clinical teaching, machine learning prediction models
- MeSH terms
- Clinical Decision-Making, Machine Learning, Adjustment Disorders, Adolescent, Adult, Aged, Anxiety Disorders, Mental Disorders, Feedback, Female, Humans, Male, Middle Aged, Patient Dropouts, Personality Disorders, Problem Solving, Surveys and Questionnaires, Stress Disorders, Post-Traumatic, Cognitive Behavioral Therapy, Mood Disorders, Decision Support Systems, Clinical, Young Adult, Precision Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review