The DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index.
Towards integrating personalized feedback research into clinical practice: Development of the Trier Treatment Navigator (TTN)
Abstract
In this study, a computer-based feedback, decision and clinical problem-solving system for clinical practice will be described - the Trier Treatment Navigator (TTN). The paper deals with the underlying research concepts related to personalized pre-treatment recommendations for drop-out risk and optimal treatment strategy selection as well as personalized adaptive recommendations during treatment. The development sample consisted of 1234 patients treated with cognitive behavioral therapy (CBT). Modern statistical machine learning techniques were used to develop personalized recommendations. Drop-out analyses resulted in seven significant predictors explaining 12.0% of variance. The prediction of optimal treatment strategies resulted in differential prediction models substantially improving effect sizes and reliable improvement rates. The dynamic failure boundary reliably identified patients with a higher risk for no improvement or deterioration and indicated the usage of clinical problem-solving tools in risk areas. The probability to be reliably improved for patients identified as at risk for treatment failure was about half of the probability for other patients (35% vs. 62.15%; χ 2df=1 = 82.77, p < .001). Results related to the computer-based feedback system are discussed with regard to the implication for clinical applications as well as clinical training and future research possibilities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First, based on the patient's initial characteristics and a dataset of already treated patients, the TTN predicts the optimal treatment strategy (i.e., problem-solving, motivation-oriented or mixed) for the individual patient.”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Lutz et al., 2019
In the citing paper
“When the alert of one of these areas is activated, the therapist can access the CSTs that guide how to address that particular area (e.g., a clinical guideline on how to work on motivation to change).”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Lutz et al., 2019
In the citing paper
“The TTN offers personalized clinical recommendations, helps monitor the process and provides clinical support tools (CSTs) for patients who are not improving as expected.”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Lutz et al., 2019
In the citing paper
“Second, during treatment, the feedback system comprises an expected treatment response (i.e., predicted trajectory of progress for the patient) and a dynamic failure boundary informed by specific patient's pre-treatment features and the experienced change progress.”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Lutz et al., 2019
In the citing paper
“Additionally, the TTN estimates the patient dropout probability and offers the results of the psychometrical initial assessment.”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Lutz et al., 2019
In the citing paper
“When a patient is not progressing as expected, the TTN generates an alert signal, and CSTs are activated according to the assessment of critical areas that might hinder the change process (i.e., suicidality risk, motivation to change and therapy goals, therapeutic alliance, social support and critical life events, emotionand self-regulation).”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Lutz et al., 2019
In the citing paper
“Additionally, two of the studies suffered from potential bias in their loss to follow-up [82,83].”
From Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review (2022) · cites this paper as [82]
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
ggplot2: Elegant Graphics for Data Analysis
Ginestet, C. E. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 14,511
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Bergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019247 · Cited by 657
Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress.
Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996) · American Psychologist · doi:10.1037/0003-066x.51.10.1059 · Cited by 573
Self-citationA lab test and algorithms for identifying clients at risk for treatment failure.
Hannan, C. R., Lambert, M., Harmon, C., Nielsen, S. L., Smart, D., Shimokawa, K., & Sutton, S. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20108 · Cited by 550
DeRubeis, R. J., Cohen, Z. D., Forand, N. R., Fournier, J. C., Gelfand, L. A., & Lorenzo-Luaces, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0083875 · Cited by 509
Lambert, M. (2007) · Psychotherapy Research · doi:10.1080/10503300601032506 · Cited by 461
Living with a depressed person.
Coyne, J. C., Kessler, R. C., Tal, M., Turnbull, J., Wortman, C. B., & Greden, J. F. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.3.347 · Cited by 457
Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring.
Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018) · Psychotherapy · doi:10.1037/pst0000167 · Cited by 440
Treatment selection in depression.
Cohen, Z. D., & DeRubeis, R. J. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050817-084746 · Cited by 435
Exploring the idiographic dynamics of mood and anxiety via network analysis
Fisher, A. J., Reeves, J. W., Lawyer, G., Medaglia, J. D., & Rubel, J. A. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000311 · Cited by 385
Self-citationWhipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003) · Journal of Counseling Psychology · doi:10.1037/0022-0167.50.1.59 · Cited by 374
Annual Research Review: Building a science of personalized intervention for youth mental health
Ng, M. Y., & Weisz, J. R. (2016) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12470 · Cited by 291
Baldwin, S. A., Berkeljon, A., Atkins, D. C., Olsen, J. A., & Nielsen, S. L. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0015235 · Cited by 274
Kendrick, T., El-Gohary, M., Stuart, B. L., Gilbody, S. M., Churchill, R. C., Aiken, L., Bhattacharya, A., Gimson, A., Brütt, A. L., de Jong, K., et al. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011119.pub2 · Cited by 252
Enhancing the Personalization of Psychotherapy With Dynamic Assessment and Modeling
Fisher, A. J., & Boswell, J. F. (2016) · Assessment · doi:10.1177/1073191116638735 · Cited by 219
Huibers, M. J. H., Cohen, Z. D., Lemmens, L. H. J. M., Arntz, A., Peeters, F. P. M. L., Cuijpers, P., & DeRubeis, R. J. (2015) · PLOS ONE · doi:10.1371/journal.pone.0140771 · Cited by 215
The Affective Style Questionnaire: Development and Psychometric Properties
Hofmann, S. G., & Kashdan, T. B. (2010) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-009-9142-4 · Cited by 203
Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder
Kessler, R. C., van Loo, H. M., Wardenaar, K. J., Bossarte, R. M., Brenner, L. A., Ebert, D. D., de Jonge, P., Nierenberg, A. A., Rosellini, A. J., Sampson, N. A., et al. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000020 · Cited by 193
Delgadillo, J., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Gilbody, S., Ali, S., Aguirre, E., Appleton, M., Nevin, J., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30162-7 · Cited by 152
Self-citationOutcome Questionnaire: Is It Sensitive to Changes in Counseling Center Clients?
Vermeersch, D. A., Whipple, J. L., Lambert, M. J., Hawkins, E. J., Burchfield, C. M., & Okiishi, J. C. (2004) · Journal of Counseling Psychology · doi:10.1037/0022-0167.51.1.38 · Cited by 145
Lutz, W., Schwartz, B., Hofmann, S. G., Fisher, A. J., Husen, K., & Rubel, J. A. (2018) · Scientific Reports · doi:10.1038/s41598-018-25953-0 · Cited by 145
Self-citationFinch, A. E., Lambert, M. J., & Schaalje, B. G. (2001) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.286 · Cited by 143
Predicting change for individual psychotherapy clients on the basis of their nearest neighbors.
Lutz, W., Leach, C., Barkham, M., Lucock, M. P., Stiles, W. B., Evans, C., Noble, R., & Iveson, S. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.904 · Cited by 136
Self-citationDose–effect relationship in routine outpatient psychotherapy: Does treatment duration matter?
Stulz, N., Lutz, W., Kopta, S. M., Minami, T., & Saunders, S. M. (2013) · Journal of Counseling Psychology · doi:10.1037/a0033589 · Cited by 130
Self-citationTherapist Effects on and Predictors of Non‐Consensual Dropout in Psychotherapy
Zimmermann, D., Rubel, J. A., Page, A. C., & Lutz, W. (2016) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2022 · Cited by 129
Self-citationPersonalized Psychotherapy for Adult Depression: A Meta-Analytic Review
Cuijpers, P., Ebert, D. D., Acarturk, C., Andersson, G., & Cristea, I. A. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.04.007 · Cited by 128
Patient-focused and feedback research in psychotherapy: Where are we and where do we want to go?
Lutz, W., de Jong, K., & Rubel, J. A. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1079661 · Cited by 127
Self-citationWhat to Do When Manuals Fail? The Dual Model of Psychotherapy
Schulte, D., & Eifert, G. H. (2002) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.9.3.312 · Cited by 119
Case complexity as a guide for psychological treatment selection.
Delgadillo, J., Huey, D., Bennett, H., & McMillan, D. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000231 · Cited by 111
Furukawa, T. A., Efthimiou, O., Weitz, E. S., Cipriani, A., Keller, M. B., Kocsis, J. H., Klein, D. N., Michalak, J., Salanti, G., Cuijpers, P., et al. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000489227 · Cited by 109
Lutz, W., Tholen, S., Schürch, E., & Berking, M. (2006) · Diagnostica · doi:10.1026/0012-1924.52.1.11 · Cited by 107
Self-citationPersonalized feedback on symptom dynamics of psychopathology: A proof-of-principle study
Kroeze, R., van der Veen, D. C., Servaas, M. N., Bastiaansen, J. A., Voshaar, R. C. O. V., Borsboom, D., Ruhe, H. G., Schoevers, R. A., & Riese, H. (2017) · Journal for Person-Oriented Research · doi:10.17505/jpor.2017.01 · Cited by 93
Individual treatment selection for patients with posttraumatic stress disorder
Deisenhofer, A., Delgadillo, J., Rubel, J. A., Boehnke, J. R., Zimmermann, D., Schwartz, B., & Lutz, W. (2018) · Depression and Anxiety · doi:10.1002/da.22755 · Cited by 93
Self-citationRubel, J. A., Fisher, A. J., Husen, K., & Lutz, W. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000487769 · Cited by 86
Self-citationEllsworth, J. R., Lambert, M. J., & Johnson, J. E. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.503 · Cited by 82
Abandoning personalization to get to precision in the pharmacotherapy of depression
Perlis, R. H. (2016) · World Psychiatry · doi:10.1002/wps.20345 · Cited by 82
Lutz, W., Saunders, S. M., Leon, S. C., Martinovich, Z., Kosfelder, J., Schulte, D., Grawe, K., & Tholen, S. (2006) · Psychological Assessment · doi:10.1037/1040-3590.18.2.133 · Cited by 75
Self-citationHusen, K., Rafaeli, E., Rubel, J. A., Bar‐Kalifa, E., & Lutz, W. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.08.025 · Cited by 55
Self-citationGuo, P., Zeng, F., Hu, X.-M., Zhang, D., Zhu, S., Deng, Y., & Yuantao, H. (2015) · PLoS ONE · doi:10.1371/journal.pone.0134151 · Cited by 53
Der „Affective Style Questionnaire (ASQ)”: Deutsche Adaption und Validitäten
Graser, J., Bohn, C., Kelava, A., Schreiber, F., Hofmann, S. G., & Stangier, U. (2012) · Diagnostica · doi:10.1026/0012-1924/a000056 · Cited by 47
Rubel, J., Lutz, W., Kopta, S., Köck, K., Minami, T., Zimmermann, D., & Saunders, S. (2015) · Psychological Assessment · doi:10.1037/pas0000060 · Cited by 46
Self-citationLutz, W., Zimmermann, D., Müller, V., Deisenhofer, A., & Rubel, J. A. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1464-2 · Cited by 41
Self-citationUsing the Assessment for Signal Clients as a feedback tool for reducing treatment failure
White, M. M., Lambert, M. J., Ogles, B. M., Mclaughlin, S. B., Bailey, R. J., & Tingey, K. M. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1009862 · Cited by 36
Probst, T., Lambert, M. J., Dahlbender, R. W., Loew, T. H., & Tritt, K. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.03.010 · Cited by 32
Therapist–client agreement in assessments of clients’ functioning.
Bar‐Kalifa, E., Atzil–Slonim, D., Rafaeli, E., Peri, T., Rubel, J. A., & Lutz, W. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000157 · Cited by 30
Self-citationFive types of clinical difference to monitor in practice.
Langkaas, T. F., Wampold, B. E., & Hoffart, A. (2018) · Psychotherapy · doi:10.1037/pst0000194 · Cited by 30
Using Item and Test Information to Optimize Targeted Assessments of Psychological Distress
Boehnke, J. R., & Lutz, W. (2014) · Assessment · doi:10.1177/1073191114529152 · Cited by 15
Self-citationHooke, G. R., Sng, A., Cunningham, N. K., & Page, A. C. (2017) · Cognitive Therapy and Research · doi:10.1007/s10608-017-9851-z · Cited by 10
Flagged first, then by citations
1 of the 54 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The promise of machine learning in predicting treatment outcomes in psychiatry
Chekroud, A. M., Bondar, J., Delgadillo, J., Doherty, G., Wasil, A., Fokkema, M., Cohen, Z., Belgrave, D., DeRubeis, R., Iniesta, R., et al. (2021) · World Psychiatry · doi:10.1002/wps.20882 · Cited by 531
Psychopathological networks: Theory, methods and practice
Bringmann, L. F., Albers, C. J., Bockting, C., Borsboom, D., Ceulemans, E., Cramer, A. O. J., Epskamp, S., Eronen, M. I., Hamaker, E. L., Kuppens, P., et al. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.104011 · Cited by 325
Self-citationBickman, L. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01065-8 · Cited by 184
A Scoping Review of Machine Learning in Psychotherapy Research
Aafjes-van Doorn, K., Kamsteeg, C., Bate, J., & Aafjes, M. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1808729 · Cited by 179
Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations
Barkham, M., de Jong, K., Delgadillo, J., & Lutz, W. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2181114 · Cited by 210
Self-citation
Metadata
- Authors
- Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K.
- Affiliations
- Universität Trier (DE)
- Corresponding author
- Lutz, Wolfgang
- Venue
- Behaviour Research and Therapy
- Publisher
- Elsevier BV
- Published
- 2019-09
- Volume
- 120
- Pages
- 103438
- ISSN
- 0005-7967 (ISSN-L)
- OpenAlex
- W2956805281
- Cited by
- 196
- References
- 54
- Citations by year
- 2026: 18 · 2025: 26 · 2024: 39 · 2023: 25 · 2022: 22 · 2021: 34 · 2020: 25 · 2019: 7
- FWCI
- 24.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Behaviour Research and Therapy (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (LU 660/8-1, LU 660/10-1)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Keywords
- personalized feedback, cognitive behavioral therapy, treatment strategy selection, clinical problem solving, computer-based decision support, treatment failure prediction, clinical teaching, machine learning prediction models
- MeSH terms
- Clinical Decision-Making, Machine Learning, Adjustment Disorders, Adolescent, Adult, Aged, Anxiety Disorders, Mental Disorders, Feedback, Female, Humans, Male, Middle Aged, Patient Dropouts, Personality Disorders, Problem Solving, Surveys and Questionnaires, Stress Disorders, Post-Traumatic, Cognitive Behavioral Therapy, Mood Disorders, Decision Support Systems, Clinical, Young Adult, Precision Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
