The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Feedback-informed treatment versus usual psychological treatment for depression and anxiety: a multisite, open-label, cluster randomised controlled trial
Open accessAbstract
Background Previous research suggests that the use of outcome feedback technology can enable psychological therapists to identify and resolve obstacles to clinical improvement. We aimed to assess the effectiveness of an outcome feedback quality assurance system applied in stepped care psychological services.Methods This multisite, open-label, cluster randomised controlled trial was done at eight National Health Service (NHS) Trusts in England, involving therapists who were qualified to deliver evidence-based low-intensity or high-intensity psychological interventions. Adult patients (18 years or older) who accessed individual therapy with participating therapists were eligible for inclusion, except patients who accessed group therapies and those who attended less than two individual therapy sessions. Therapists were randomly assigned (1:1) to an outcome feedback intervention group or a treatment-as-usual control group by use of a computer-generated randomisation algorithm. The allocation of patients to therapists was quasi-random, whereby patients on waiting lists were allocated sequentially on the basis of therapist availability. All patients received low-intensity (less than eight sessions) or high-intensity (up to 20 sessions) psychological therapies for the duration of the 1-year study period. An automated computer algorithm alerted therapists in the outcome feedback group to patients who were not on track, and primed them to review these patients in clinical supervision. The primary outcome was symptom severity on validated depression (Patient Health Questionnaire-9 [PHQ-9]) and anxiety (Generalised Anxiety Disorder-7 [GAD-7]) measures after treatment of varying durations, which were compared between groups with multilevel modelling, controlling for cluster (therapist) effects. We used an intention-to-treat approach. This trial was prospectively registered with ISRCTN, number ISRCTN12459454.Findings In total, 79 therapists were recruited to the study between Jan 8, 2016, and July 15, 2016, but two did not participate. Of these participants, 39 (51%) were randomly assigned to the outcome feedback group and 38 (49%) to the control group. Overall, 2233 patients were included in the trial (1176 [53%] were treated by therapists in the outcome feedback group, and 1057 [47%] by therapists in the control group). Patients classified as not on track had less severe symptoms after treatment if they were allocated to the outcome feedback group than those in the control group (PHQ-9 d=0·23, B=-1·03 [95% CI -1·84 to -0·23], p=0·012; GAD-7 d=0·19, B=-0·85 [-1·56 to -0·14], p=0·019).Interpretation Supplementing psychological therapy with low-cost feedback technology can reduce symptom severity in patients at risk of poor response to treatment. This evidence supports the implementation of outcome feedback in stepped care psychological services.Funding English NHS and Department of Health Sciences, University of York, York, UK.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Beyond CF, it has been shown that practising data-informed psychotherapy with technological augmentation tools (i.e., CDSS, ROM and feedback systems, CSTs) contributes to better treatment outcomes (Cohen et al., 2021;de Jong et al., 2021;Delgadillo et al., 2018).”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Delgadillo et al., 2018
Experimental — accuracy not yet measuredIn the citing paper
“Because providing continuous feedback to patients has been shown to be a valuable therapeutic procedure [29], we also included studies that adopted EMA-based feedback as a therapeutic tool for clinically depressed patients.”
From Current state and future directions of technology-based ecological momentary assessment and intervention for major depressive disorder: a systematic review (2019) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection
Kroenke, K., Spitzer, R. L., Williams, J. B. W., Monahan, P. O., & Löwe, B. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-5-200703060-00004 · Cited by 4,741
Bergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
The Work and Social Adjustment Scale: a simple measure of impairment in functioning
Mundt, J. C., Marks, I. M., Shear, M. K., & Greist, J. M. (2002) · British Journal of Psychiatry · doi:10.1192/bjp.180.5.461 · Cited by 2,515
Multiple Imputation for Multivariate Missing-Data Problems: A Data Analyst's Perspective
Schafer, J. L., & Olsen, M. K. (1998) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3304_5 · Cited by 1,518
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783
Clark, D. M. (2011) · International Review of Psychiatry · doi:10.3109/09540261.2011.606803 · Cited by 699
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019247 · Cited by 657
Routine outcome monitoring and feedback on physical or mental health status: evidence and theory
Carlier, I. V. E., Meuldijk, D., van Vliet, I. M., van Fenema, E. M., van der Wee, N. J. A., & Zitman, F. G. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01543.x · Cited by 288
Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis
Knaup, C., Koesters, M., Schöfer, D., Becker, T. M., & Puschner, B. (2009) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.108.053967 · Cited by 281
Cited by
Sorted by citations
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102002 · Cited by 417
Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103438 · Cited by 196
Self-citationColombo, D., Fernández-Álvarez, J., Patané, A., Semonella, M., Kwiatkowska, M., García-Palacios, A., Cipresso, P., Riva, G., & Botella, C. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8040465 · Cited by 194
“Because providing continuous feedback to patients has been shown to be a valuable therapeutic procedure [29], we also included studies that adopted EMA-based feedback as a therapeutic tool for clinically depressed patients.”
MethodThieme, A., Hanratty, M., Lyons, M., Palacios, J. E., Marques, R. F., Morrison, C. P., & Doherty, G. J. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3564752 · Cited by 197
Colombo, D., Fernández‐Álvarez, J., Suso‐Ribera, C., Cipresso, P., Valev, H., Leufkens, T., Sas, C., García‐Palacios, A., Riva, G., & Botella, C. (2020) · Emotion · doi:10.1037/emo0000671 · Cited by 178
“EMA could indeed change the way we assess ER in psychopathology by helping us to understand the dynamics of this process in different emotional disorders, as well as the impact of emotion dysregulation feedback to patients, which has been shown to be a valuable practice (Delgadillo et al., 2018).”
BackgroundJuul, S., Gluud, C., Simonsen, S., Frandsen, F. W., Kirsch, I., & Jakobsen, J. C. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111407 · Cited by 135
Trajectories of depression and anxiety symptom change during psychological therapy
Saunders, R., Buckman, J. E. J., Cape, J., Fearon, P., Leibowitz, J., & Pilling, S. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.02.043 · Cited by 99
Webb, C. A., Cohen, Z. D., Beard, C., Forgeard, M. J. C., Peckham, A., & Björgvinsson, Þ. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000451 · Cited by 96
“…monitoring (see Carlier et al., 2012; Knaup, Koesters, Schoefer, Becker, & Puschner, 2009; Shimokawa, Lambert, & Smart, 2010; Kendrick et al., 2016; Delgadillo et al., 2018), (3) highlight which specific baseline characteristics are contributing to an individual’s predicted poor prognosis, which…”
BackgroundInfluentialEnrique, A., Palacios, J. E., Ryan, H., & Richards, D. (2019) · Journal of Medical Internet Research · doi:10.2196/12775 · Cited by 81
“the intervention period, these patients had significantly less severe symptoms after treatment compared with similar patients assigned to therapists who were not receiving this feedback [41].”
Bone, C., Simmonds-Buckley, M., Thwaites, R., Sandford, D., Merzhvynska, M., Rubel, J., Deisenhofer, A.-K., Lutz, W., & Delgadillo, J. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00018-2 · Cited by 79
Self-citationRozental, A., Andersson, G., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00589 · Cited by 74
Hornstein, S., Forman‐Hoffman, V. L., Nazander, A., Ranta, K., & Hilbert, K. (2021) · Digital Health · doi:10.1177/20552076211060659 · Cited by 87
“First studies suggested that monitoring patients, for whom low outcomes are predicted, can help to improve their outcomes.(42) This appears as a promising use case also for a DMH setting, as the monitoring would be easy to implement and not too cost-intensive.”
BackgroundJensen‐Doss, A., Douglas, S. R., Phillips, D. A., Gencdur, O., Zalman, A., & Gomez, N. E. (2020) · Evidence-Based Practice in Child and Adolescent Mental Health · doi:10.1080/23794925.2020.1784062 · Cited by 71
“…a recent trial demonstrated that a MBC feedback condition that included flags that clients were NOT and guidance about how to discuss this feedback in supervision was associated with better treatment outcomes than a MBC conditions without these risk signals and supports (Delgadillo et al., 2018).”
BackgroundWade, T. D., Shafran, R., & Cooper, Z. (2023) · International Journal of Eating Disorders · doi:10.1002/eat.24008 · Cited by 57
The future of psychological treatments: The Marburg Declaration
Rief, W., Asmundson, G. J. G., Bryant, R. A., Clark, D. M., Ehlers, A., Holmes, E. A., McNally, R. J., Neufeld, C. B., Wilhelm, S., Jaroszewski, A. C., et al. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102417 · Cited by 54
Nickels, S., Edwards, M. D., Poole, S. F., Winter, D., Gronsbell, J. L., Rozenkrants, B., Miller, D. P., Fleck, M. S., McLean, A., Peterson, B. E., et al. (2021) · JMIR Mental Health · doi:10.2196/27589 · Cited by 50
“The location example (Figure 3) shows one area in which a clinician could gain insights about a patient’s routine that could augment clinical decision-making and strengthen feedback-informed care [25].”
BackgroundUsing Progress Feedback to Enhance Treatment Outcomes: A Narrative Review
de Jong, K., Douglas, S. R., Wolpert, M., Delgadillo, J., Aas, B., Bovendeerd, B., Carlier, I. V. E., Compare, A., Edbrooke-Childs, J. H., Janse, P. D., et al. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01381-3 · Cited by 66
Self-citationMcAleavey, A. A., de Jong, K., Nissen‐Lie, H. A., Boswell, J. F., Moltu, C., & Lutz, W. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01351-9 · Cited by 78
Self-citationLatent variable mixture modelling and individual treatment prediction
Saunders, R., Buckman, J. E. J., & Pilling, S. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103505 · Cited by 53
Moving from tradition-based to competence-based psychotherapy
Rief, W. (2021) · Evidence-Based Mental Health · doi:10.1136/ebmental-2020-300219 · Cited by 46
O’Dea, B., Subotic-Kerry, M., KING, C. A., Mackinnon, A. J., Achilles, M. R., Anderson, M., Parker, B., Werner‐Seidler, A., Torok, M. H., Cockayne, N. L., et al. (2021) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2021.100178 · Cited by 46
Börjesson, S., & Boström, P. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1630780 · Cited by 41
“When therapists receive feedback from clients with poor initial treatment response, the percentage of negative outcomes appears to decrease (Delgadillo et al., 2018; Lambert, Harmon, Slade, Whipple, & Hawkins, 2005).”
BackgroundGual-Montolio, P., Martínez-Borba, V., Bretón-López, J. M., Osma, J., & Suso-Ribera, C. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17093170 · Cited by 38
“While the presented results are, overall, encouraging, especially for not-on-track patients, more research is required in this field, especially RCTs comparing technology-supported MBC with traditional MBC without technology.”
BackgroundMethodInfluentialDavey, E., Allen, K. L., Bennett, S. D., Bryant‐Waugh, R., Clarke, T., Cooper, Z., Dixon‐Ward, K., Dudley, J., Eisler, I., Griffiths, J., et al. (2023) · European Eating Disorders Review · doi:10.1002/erv.2981 · Cited by 39
“…extent in practice, it is suggested that (a) such processes are as automated as possible; and (b) clinicians are trained in the importance of PROMs as a technique to improve outcome (e.g., Delgadillo et al., 2018), as well as its importance ethically with regard to clinical decision making.”
BackgroundCost-effectiveness of feedback-informed psychological treatment: Evidence from the IAPT-FIT trial
Delgadillo, J., McMillan, D., Gilbody, S., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Aguirre, E., & Ali, S. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.103873 · Cited by 41
Self-citationHovland, R. T., Ytrehus, S., Mellor‐Clark, J., & Moltu, C. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.22992 · Cited by 37
“…have demonstrated that it has positive effects on therapy outcomes compared to treatment as usual (Brattland et al., 2018; Carlier et al., 2012; Delgadillo et al., 2018; Gondek, Edbrooke‐Childs, Fink, Deighton, & Wolpert, 2016; Knaup, Koesters, Schoefer, Becker, & Puschner, 2009; Lambert,…”
MethodPredicting and preventing dropout in research, assessment and treatment with refugees
Semmlinger, V., & Ehring, T. W. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2672 · Cited by 34
Kendrick, T., Dowrick, C. F., Lewis, G., Moore, M. V., Leydon, G., Geraghty, A. W. A., Griffiths, G. O., Zhu, S., Yao, G., May, C. R., et al. (2023) · Health Technology Assessment · doi:10.3310/plrq4216 · Cited by 37
Skelton, M., Carr, E., Buckman, J. E. J., Davies, M. R., Goldsmith, K., Hirsch, C. R., Peel, A. J., Rayner, C., Rimes, K. A., Saunders, R., et al. (2022) · Psychological Medicine · doi:10.1017/s0033291722003403 · Cited by 36
Measuring Alliance and Symptom Severity in Psychotherapy Transcripts Using Bert Topic Modeling
Lalk, C., Steinbrenner, T., Kania, W., Popko, A., Wester, R. A., Schaffrath, J., Eberhardt, S. T., Schwartz, B., Lutz, W., & Rubel, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01356-4 · Cited by 36
Self-citationMoshe, I., Terhorst, Y., Paganini, S., Schlicker, S., Pulkki-Råback, L., Baumeister, H., Sander, L. B., & Ebert, D. D. (2022) · Journal of Medical Internet Research · doi:10.2196/38261 · Cited by 28
“…feedback technology that identifies individuals at risk of deterioration during treatment has been shown to improve eventual treatment outcomes [43,44], dropout warning systems could be used to alert the supporting clinician, guide care pathways (eg, in stepped-care models), or personalize the…”
Benito, K., Machan, J. T., Freeman, J. B., Garcia, A. M., Walther, M. R., Frank, H. E., Wellen, B. C. M., Stewart, E., Edmunds, J., Sapyta, J. J., et al. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.07.004 · Cited by 27
The Future of Enhanced Psychotherapy: Towards Precision Psychotherapy
Zipfel, S., Lutz, W., Schneider, S., Schramm, E., Delgadillo, J., & Giel, K. E. (2024) · Psychotherapy and Psychosomatics · doi:10.1159/000539022 · Cited by 25
Self-citationSchilling, V. N. L. S., Zimmermann, D., Rubel, J. A., Boyle, K. S., & Lutz, W. (2020) · Quality of Life Research · doi:10.1007/s11136-020-02664-6 · Cited by 27
Self-citationEspel‐Huynh, H., Zhang, F., Thomas, J. G., Boswell, J. F., Thompson‐Brenner, H., Juarascio, A. S., & Lowe, M. R. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23510 · Cited by 24
“If applied in clinical practice and integrated into a decision support tool similar to a “treatment navigator” (Delgadillo et al., 2018; Lutz et al., 2019), this model could yield feedback early enough during a treatment episode to allow clinicians to tailor treatment based upon the patient’s…”
MethodFaija, C., Bee, P., Lovell, K., Lidbetter, N., Gellatly, J. L., Ardern, K., Rushton, K., Brooks, H. L., McMillan, D., Armitage, C. J., et al. (2022) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12400 · Cited by 22
“Empirical and clinical value of session- by- session monitoring using feedback is well documented in the literature (e.g. de Jong et al., 2021; Delgadillo et al., 2018; Lambert et al., 2018; Solstad et al., 2019), including tracking responses at an item level (Cross et al., 2015) and thereby…”
MethodSelf-citationBeyond the Therapist’s Office: Merging Measurement-Based Care and Digital Medicine in the Real World
Chiauzzi, E. J., & Wicks, P. J. (2021) · Digital Biomarkers · doi:10.1159/000517748 · Cited by 23
Prasad, N., Chien, I., Regan, T. W., Enrique, Á., Palacios, J. E., Keegan, D., Munir, U., Tanno, R., Richardson, H., Nori, A., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0272685 · Cited by 25
Working Towards the Development and Implementation of Precision Mental Healthcare: An Example
Lutz, W., Schwartz, B., Penedo, J. M. G., Boyle, K. S., & Deisenhofer, A. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01053-y · Cited by 19
Self-citationPeterson, K., Anderson, J., & Bourne, D. S. (2018) · Europe PMC (PubMed Central) · Cited by 20
Eberhardt, S. T., Vehlen, A., Schaffrath, J., Schwartz, B., Baur, T., Schiller, D., Hallmen, T., André, E., & Lutz, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-14923-y · Cited by 23
Self-citationBear, H. A., Dalzell, K., Edbrooke-Childs, J. H., & Wolpert, M. (2021) · British Journal of Clinical Psychology · doi:10.1111/bjc.12322 · Cited by 18
“…as well as for elevating the efficiency and quality of mental health care is well-documented in the research literature (Bickman, Kelley, Breda, de Andrade, et al., 2011; Connors et al., 2020; Delgadillo et al., 2018; Jensen-Doss et al., 2020; Lambert & Harmon, 2018; Shimokawa et al., 2010).”
Do PROMS improve outcomes in patients with depression in primary care?
Kendrick, T., & Maund, E. (2020) · British medical journal · doi:10.1136/bmj.m3313 · Cited by 15
Kwan, B., Rickwood, D., & Brown, P. M. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1829738 · Cited by 18
“The addition of expected change trajectories and risk signals have been shown to enhance the treatment effects already seen in MFS (Delgadillo et al., 2018).”
BackgroundAndrews, L. A., Hayes, A. M., Abel, A., & Kuyken, W. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000467 · Cited by 20
Mütze, K., Witthöft, M., Lutz, W., & Bräscher, A. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1930244 · Cited by 19
“As mentioned, prior experimental research has shown that ROM in combination with feedback to therapists significantly reduces deterioration in psychological treatment (Delgadillo et al., 2018; Lambert et al., 2018).”
MethodSelf-citationHovland, R. T., & Moltu, C. (2019) · International Journal of Mental Health Systems · doi:10.1186/s13033-019-0324-5 · Cited by 16
Weisz, J. R., Fitzpatrick, O. M., Venturo‐Conerly, K. E., Sternberg, A., Steinberg, J. S., & Ng, M. Y. (2023) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13820 · Cited by 20
Cheron, D. M., Chiu, A. W., Stanick, C., Stern, H. G., Donaldson, A. R., Daleiden, E. L., & Chorpita, B. F. (2019) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-019-00922-5 · Cited by 13
A guide for self-help guides: best practice implementation
Shafran, R., Egan, S. J., de Valle, M. K., Davey, E., Carlbring, P., Creswell, C., & Wade, T. D. (2024) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2024.2369637 · Cited by 12
“This is because of the strong data showing that such progress monitoring improves retention and outcomes in mental health interventions (de Jong et al., 2021; Delgadillo et al., 2018).”
BackgroundHutson, J., Hooke, G. R., & Page, A. C. (2020) · Psychotherapy Research · doi:10.1080/10503307.2019.1710305 · Cited by 12
“A number of studies have provided evidence of the benefits of providing progress feedback to therapists (Delgadillo et al., 2018; Gondek, Edbrooke-Childs, Fink, Deighton, & Wolpert, 2016; Lambert, 2010; Shimokawa, Lambert, & Smart, 2010) and patients (de Jong et al., 2014; Hawkins, Lambert,…”
BackgroundResultInfluentialGates, V., Hsiao, M., Zieve, G. G., Courry, R., & Persons, J. B. (2021) · PsyArXiv (OSF Preprints) · doi:10.1016/j.brat.2021.103874 · Cited by 12
Cawthorne, T., Käll, A., Bennett, S. D., Baker, E., Andersson, G., & Shafran, R. (2023) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465823000231 · Cited by 12
“Goal-based outcomes have been shown to improve treatment retention, clinical outcomes and client progress (Delgadillo et al ., 2018; Tryon et al ., 2018).”
BackgroundAdam, A., Jain, A., Pletnikova, A., Bagga, R., Vita, A., Richey, L. N., Gould, N. F., Munshaw, S., Misrilall, K., & Peters, M. E. (2020) · JMIR Formative Research · doi:10.2196/17722 · Cited by 11
“In recent years, feedback-informed care (FIC) models have been used to improve patient compliance and psychotherapy outcomes [9,10].”
BackgroundFlygare, O., Chen, L. L., de la Cruz, L. F., Rück, C., Andersson, E., Enander, J., & Mataix‐Cols, D. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.10.006 · Cited by 11
Praxisorientierte Forschung in der Psychotherapie
Herzog, P., Kaiser, T., & Brakemeier, E. (2022) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000665 · Cited by 10
Technology-Based Ecological Momentary Assessment in Clinical Psychology
Colombo, D., Suso‐Ribera, C., Fernández‐Álvarez, J., Cipresso, P., García‐Palacios, A., Riva, G., & Botella, C. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00003-0 · Cited by 9
Barkham, M., & Broglia, E. (2023) · Counselling and Psychotherapy Research · doi:10.1002/capr.12694 · Cited by 11
Johnson, C., Egan, S. J., Carlbring, P., Shafran, R., & Wade, T. D. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100795 · Cited by 9
“In addition, a self-scoring version of the CPQ that could chart their progress weekly was supplied to each participant, given the evidence that feedback-informed interventions improve retention and outcomes (Delgadillo et al., 2018).”
Barkham, M., de Jong, K., Delgadillo, J., & Lutz, W. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197611012.003.0015 · Cited by 9
Self-citationAltimir, C., Mantilla, C., & Serralta, F. B. (2022) · Studies in Psychology Estudios de Psicología · doi:10.1080/02109395.2022.2139998 · Cited by 10
Hooke, G. R., Savani, P., Stewart, B., Araujo, S., & Page, A. C. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23426 · Cited by 8
“ROM benefits clinical practice by identifying patients who are “not on track” and open up a conversation enabling staff and patient to work collaboratively towards a positive outcome (de Jong et al., 2021; Delgadillo et al., 2018).”
BackgroundDepression follow-up monitoring with the PHQ-9: an open cluster-randomised controlled trial
Kendrick, T., Dowrick, C. F., Lewis, G., Moore, M. V., Leydon, G., Geraghty, A. W. A., Griffiths, G. O., Zhu, S., Yao, G., May, C. R., et al. (2024) · British Journal of General Practice · doi:10.3399/bjgp.2023.0539 · Cited by 14
The challenges of making clinical feedback in psychotherapy benefit all users: A qualitative study
Hovland, R. T., & Moltu, C. (2019) · Nordic Psychology · doi:10.1080/19012276.2019.1684348 · Cited by 8
Delli Colli, C., Viglione, A., Poggini, S., Cirulli, F., Chiarotti, F., Giuliani, A., & Branchi, I. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03246-1 · Cited by 7
Hegland, P. A., Aasprang, A., Kolotkin, R. L., Moltu, C., Tell, G. S., & Andersen, J. R. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-037685 · Cited by 6
Varela, C., Oda-Montecinos, C., Andrés, A., & Saldaña, C. (2021) · Journal of Eating Disorders · doi:10.1186/s40337-021-00432-6 · Cited by 10
“However, traditionally, the intensity of feedback was studied based on its frequency and it was demonstrated that personalized feedback is a relevant variable to consider in webbased interventions [12, 28].”
BackgroundJiang, H., & Yin, X. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.918274 · Cited by 7
“Current research does not specifically study the establishment of user portraits, but for different business scenarios, user portraits are modeled as features of the recommendation system (Jaime et al., 2018).”
BackgroundPredicting Undesired Treatment Outcomes With Machine Learning in Mental Health Care: Multisite Study
van Mens, K., Lokkerbol, J., Wijnen, B. F. M., Janssen, R. T. J. M., de Lange, R. P. J., & Tiemens, B. G. (2023) · JMIR Medical Informatics · doi:10.2196/44322 · Cited by 7
“Inspired by this knowledge, the study aims to use machine learning on ROM data as a feedback device to signal which patients have an elevated risk of a poor response to treatment [16].”
Progress towards clinically informative data-driven decision support tools in psychotherapy
Webb, C. A., & Cohen, Z. D. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00042-x · Cited by 7
Wade, T. D., & Waller, G. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104826 · Cited by 7
Flückiger, C., Barkham, M., & Penedo, J. M. G. (2026) · Psychotherapy Research · doi:10.1080/10503307.2026.2653992 · Cited by 6
Delamain, H., Buckman, J. E. J., O’Driscoll, C., Suh, J. W., Stott, J. C. H., Singh, S., Naqvi, S. A., Leibowitz, J., Pilling, S., & Saunders, R. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.115910 · Cited by 9
“This process has been found to be successful with other data-informed treatment predictions in similar settings (Delgadillo et al., 2018).”
ResultThe Effect of Patient Progress Feedback on Psychotherapy Outcome
Schöttke, H., Unrath, M., & Uhlmann, C. (2019) · Verhaltenstherapie · doi:10.1159/000503765 · Cited by 6
“Progress monitoring occurred once per quarter in the present study, whereas other studies conducted weekly assessments [Delgadillo et al., 2018].”
BackgroundMethodResultInfluentialGuided Self-help Teletherapy for Behavioural Difficulties in Children with Epilepsy
Bennett, S. D., Heyman, I., Varadkar, S., Coughtrey, A. E., Walji, F., & Shafran, R. (2021) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-021-09768-2 · Cited by 6
Bongers, I. L., Buitenweg, D. C., van Kuijk, R. E. F. M., & van Nieuwenhuizen, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191710834 · Cited by 7
Validation of the intersession alliance measure: Individual, couple, and family versions
Anderson, S. R., Johnson, L. N., Witting, A. B., Miller, R. B., Bradford, A. B., Hunt, Q. A., & Bean, R. A. (2024) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12702 · Cited by 8
Diagnostik in der Psychotherapie
Stieglitz, R., & Spitzer, C. (2018) · Psychotherapeut · doi:10.1007/s00278-018-0308-y · Cited by 6
eXplainable AI for routine outcome monitoring and clinical feedback
Westbye, H. J., Moltu, C., & McAleavey, A. A. (2024) · Counselling and Psychotherapy Research · doi:10.1002/capr.12764 · Cited by 5
Chong, M. K., Hickie, I. B., Ekambareshwar, M., Poulsen, A., McKenna, S., Hutcheon, A., Hockey, S. J., Scott, E. M., & Iorfino, F. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12889-1 · Cited by 5
Delgadillo, J., Nissen‐Lie, H. A., de Jong, K., Schroder, T. A., & Barkham, M. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2310635 · Cited by 9
“In one study, therapists with low job satisfaction and high occupational burnout (specifically an attitude of disengagement with work) were found to be less effective than those who reported lower levels of burnout (Delgadillo et al., 2018a).”
Self-citationLotfaliany, M., Agustini, B., Walker, A. J., Turner, A., Wrobel, A. L., Williams, L. J., Dean, O. M., Miles, S., Rossell, S. L., Berk, M., et al. (2024) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/00048674241253452 · Cited by 5
Rapee, R. M., Kuhnert, R., Bowsher, I., Burns, J. R., Coen, J., Dixon, J. E., Kotselas, P., Lourey, C., McLellan, L. F., Mihalopoulos, C., et al. (2025) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2025.01.032 · Cited by 5
Malins, S., Owen, R., Wright, I., Borrill, H., Limond, J., Gibson, F., Grundy, R. G., Bailey, S., Clifford, S. C., Lowis, S. P., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051091 · Cited by 6
Senger, K., Rubel, J. A., Kleinstäuber, M., Schröder, A., Köck, K., Lambert, M. J., Lutz, W., & Heider, J. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1993376 · Cited by 5
“…as symptom characteristics such as intensity (e.g., Cella et al., 2011; Crawley et al., 2013), duration (e.g., Burger et al., 2016; Crawley et al., 2013) or comorbidity (e.g., Flo & Chalder, 2014; Gilpin et al., 2017; Heider & Zaby, 2010) have been repeatedly investigated with conflicting reports.”
BackgroundSelf-citationMeasurement-Based Care Implementation Challenges and Opportunities in Real-World Practice
Ridout, K. K., Vanderlip, E. R., Ridout, S. J., & Alter, C. L. (2023) · Psychiatric Annals · doi:10.3928/00485713-20230818-02 · Cited by 5
Perspektiven einer evidenzbasierten und personalisierten Psychotherapie
Lutz, W., Clausen, S. A., & Deisenhofer, A. (2019) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000518 · Cited by 6
Self-citationMalins, S., Moghaddam, N. G., Morriss, R. K., & Schroder, T. A. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1657250 · Cited by 6
“Current evidence from Randomized Controlled Trials (RCTs) and subsequent meta-analyses suggest that ROM can have beneficial effects for both treat-ment efficacy and efficiency (Delgadillo et al., 2018; Lambert, Whipple, & Kleinstäuber, 2018; Shimokawa, Lambert, & Smart, 2010).”
BackgroundLindegaard, T., Persson, L., Thorängen, M., & Rozental, A. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2420717 · Cited by 4
“…of negative effects of psychotherapy indicate that around 5–10% of all patients with common mental disorders deteriorate during treatment (Cuijpers et al., 2018; Delgadillo et al., 2018), defined as a change in a negative direction that exceeds measurement error (i.e., the Reliable Change Index).”
James, K., Saxon, D. H., & Barkham, M. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01403-0 · Cited by 4
Flygare, O., Wallert, J., Chen, L. L., de la Cruz, L. F., Lundström, L., Mataix‐Cols, D., Rück, C., & Andersson, E. (2022) · Behavior Therapy · doi:10.1016/j.beth.2022.06.009 · Cited by 4
Cawthorne, T., Käll, A., Bennett, S. D., Andersson, G., & Shafran, R. (2022) · PLoS ONE · doi:10.1371/journal.pone.0278746 · Cited by 5
“Goal-based outcomes have been shown to improve treatment retention, clinical outcomes and client progress [62, 63].”
BackgroundHuang, S., Wahlquist, A. E., & Dahne, J. (2024) · Substance Use & Misuse · doi:10.1080/10826084.2024.2369155 · Cited by 5
“Another effective approach involves closely monitoring the depressive symptoms of users with lower education levels (e.g., by their primary care providers) throughout treatment (Delgadillo et al., 2018).”
Data-Informed Case Formulation with the Trier Treatment Navigator
Moggia, D., Schaffrath, J., Bommer, J., Weinmann-Lutz, B., & Lutz, W. (2022) · Revista de Psicoterapia · doi:10.33898/rdp.v33i123.35971 · Cited by 4
“, CDSS, ROM and feedback systems, CSTs) contributes to better treatment outcomes (Cohen et al., 2021; de Jong et al., 2021; Delgadillo et al., 2018).”
BackgroundSelf-citationBasnet, S., & Chaiton, M. (2023) · JMIR Research Protocols · doi:10.2196/48703 · Cited by 4
“13 for changes in the GAD-7 score (based on the expected effects of the dashboard) [15], an expected dropout rate of no more than 35% [16], and population mean and SD estimates of PHQ-9 and GAD-7 scores [17,18], we estimate having a power of 0.90 and a type I error rate of 0.05 for detecting…”
InfluentialSong, J., Freedman, G., Li, L., & Persons, J. B. (2024) · British Journal of Clinical Psychology · doi:10.1111/bjc.12470 · Cited by 3
Mahon, D., Minami, T., & Brown, J. (2024) · Counselling and Psychotherapy Research · doi:10.1002/capr.12827 · Cited by 4
Using Client and Supervision Feedback to Improve Supervision in Health Care
Mahon, D. (2022) · Counseling and Family Therapy Scholarship Review · doi:10.53309/2576-926x.1044 · Cited by 2
Krause, K. R., Calderón, A., Pino, V. G., Edbrooke-Childs, J. H., Moltrecht, B., & Wolpert, M. (2023) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-023-02140-9 · Cited by 3
Blackwell, S. E. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01340-4 · Cited by 3
Metadata
- Authors
- Delgadillo, J., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Gilbody, S., Ali, S., Aguirre, E., Appleton, M., Nevin, J., O'Hayon, H., Patel, U., Sainty, A., Spencer, P., & McMillan, D.
- Affiliations
- University of Sheffield (GB)
- University of Huddersfield (GB)
- Universität Trier (DE)
- University of York (GB)
- National Health Service (GB)
- Pennine Care NHS Foundation Trust (GB)
- Cheshire and Wirral Partnership NHS Foundation Trust (GB)
- Whittington Health NHS Trust (GB)
- Cambridgeshire and Peterborough NHS Foundation Trust (GB)
- Humber Teaching NHS Foundation Trust (GB)
- South West Yorkshire Partnership NHS Foundation Trust (GB)
- Corresponding author
- Delgadillo, Jaime
- ORCID
- Venue
- The Lancet Psychiatry
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 5
- Issue
- 7
- Pages
- 564-572
- ISSN
- 2215-0366 (ISSN-L)
- OpenAlex
- W2809000673
- Cited by
- 152
- References
- 31
- Citations by year
- 2026: 13 · 2025: 15 · 2024: 28 · 2023: 13 · 2022: 19 · 2021: 21 · 2020: 19 · 2019: 20 · 2018: 3
- FWCI
- 18.76
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- The Lancet Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Huddersfield Research Portal (University of Huddersfield) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- outcome feedback, feedback-informed treatment, depression, anxiety, stepped care, psychotherapy, clinical supervision, low-intensity psychological interventions, symptom severity, treatment progress monitoring, automated alerts, individual psychotherapy, waiting lists, NHS mental health services, intention-to-treat analysis
- MeSH terms
- Adult, Anxiety Disorders, Cost-Benefit Analysis, Depressive Disorder, England, Feedback, Female, Humans, Male, Middle Aged, Psychotherapy, Quality of Life, State Medicine, Treatment Outcome, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
