The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Feedback-informed treatment versus usual psychological treatment for depression and anxiety: a multisite, open-label, cluster randomised controlled trial
Open accessAbstract
Background Previous research suggests that the use of outcome feedback technology can enable psychological therapists to identify and resolve obstacles to clinical improvement. We aimed to assess the effectiveness of an outcome feedback quality assurance system applied in stepped care psychological services.Methods This multisite, open-label, cluster randomised controlled trial was done at eight National Health Service (NHS) Trusts in England, involving therapists who were qualified to deliver evidence-based low-intensity or high-intensity psychological interventions. Adult patients (18 years or older) who accessed individual therapy with participating therapists were eligible for inclusion, except patients who accessed group therapies and those who attended less than two individual therapy sessions. Therapists were randomly assigned (1:1) to an outcome feedback intervention group or a treatment-as-usual control group by use of a computer-generated randomisation algorithm. The allocation of patients to therapists was quasi-random, whereby patients on waiting lists were allocated sequentially on the basis of therapist availability. All patients received low-intensity (less than eight sessions) or high-intensity (up to 20 sessions) psychological therapies for the duration of the 1-year study period. An automated computer algorithm alerted therapists in the outcome feedback group to patients who were not on track, and primed them to review these patients in clinical supervision. The primary outcome was symptom severity on validated depression (Patient Health Questionnaire-9 [PHQ-9]) and anxiety (Generalised Anxiety Disorder-7 [GAD-7]) measures after treatment of varying durations, which were compared between groups with multilevel modelling, controlling for cluster (therapist) effects. We used an intention-to-treat approach. This trial was prospectively registered with ISRCTN, number ISRCTN12459454.Findings In total, 79 therapists were recruited to the study between Jan 8, 2016, and July 15, 2016, but two did not participate. Of these participants, 39 (51%) were randomly assigned to the outcome feedback group and 38 (49%) to the control group. Overall, 2233 patients were included in the trial (1176 [53%] were treated by therapists in the outcome feedback group, and 1057 [47%] by therapists in the control group). Patients classified as not on track had less severe symptoms after treatment if they were allocated to the outcome feedback group than those in the control group (PHQ-9 d=0·23, B=-1·03 [95% CI -1·84 to -0·23], p=0·012; GAD-7 d=0·19, B=-0·85 [-1·56 to -0·14], p=0·019).Interpretation Supplementing psychological therapy with low-cost feedback technology can reduce symptom severity in patients at risk of poor response to treatment. This evidence supports the implementation of outcome feedback in stepped care psychological services.Funding English NHS and Department of Health Sciences, University of York, York, UK.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Beyond CF, it has been shown that practising data-informed psychotherapy with technological augmentation tools (i.e., CDSS, ROM and feedback systems, CSTs) contributes to better treatment outcomes (Cohen et al., 2021;de Jong et al., 2021;Delgadillo et al., 2018).”
From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as Delgadillo et al., 2018
Experimental — accuracy not yet measuredIn the citing paper
“Because providing continuous feedback to patients has been shown to be a valuable therapeutic procedure [29], we also included studies that adopted EMA-based feedback as a therapeutic tool for clinically depressed patients.”
From Current state and future directions of technology-based ecological momentary assessment and intervention for major depressive disorder: a systematic review (2019) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection
Kroenke, K., Spitzer, R. L., Williams, J. B. W., Monahan, P. O., & Löwe, B. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-5-200703060-00004 · Cited by 4,741
Bergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
The Work and Social Adjustment Scale: a simple measure of impairment in functioning
Mundt, J. C., Marks, I. M., Shear, M. K., & Greist, J. M. (2002) · British Journal of Psychiatry · doi:10.1192/bjp.180.5.461 · Cited by 2,515
Multiple Imputation for Multivariate Missing-Data Problems: A Data Analyst's Perspective
Schafer, J. L., & Olsen, M. K. (1998) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3304_5 · Cited by 1,518
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783
Clark, D. M. (2011) · International Review of Psychiatry · doi:10.3109/09540261.2011.606803 · Cited by 699
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019247 · Cited by 657
Routine outcome monitoring and feedback on physical or mental health status: evidence and theory
Carlier, I. V. E., Meuldijk, D., van Vliet, I. M., van Fenema, E. M., van der Wee, N. J. A., & Zitman, F. G. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01543.x · Cited by 288
Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis
Knaup, C., Koesters, M., Schöfer, D., Becker, T. M., & Puschner, B. (2009) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.108.053967 · Cited by 281
de Jong, K., van Sluis, P., Nugter, A., Heiser, W. J., & Spinhoven, P. (2012) · Psychotherapy Research · doi:10.1080/10503307.2012.673023 · Cited by 267
Kendrick, T., El-Gohary, M., Stuart, B. L., Gilbody, S. M., Churchill, R. C., Aiken, L., Bhattacharya, A., Gimson, A., Brütt, A. L., de Jong, K., et al. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011119.pub2 · Cited by 252
Self-citationApplied analysis of variance in behavioral science
Edwards, L. K. (1993) · M. Dekker eBooks · Cited by 233
Common mental health disorders : identification and pathways to care
Excellence, C. (2011) · Medical Entomology and Zoology · Cited by 229
Richards, D. A., & Borglin, G. (2011) · Journal of Affective Disorders · doi:10.1016/j.jad.2011.03.024 · Cited by 217
Delgadillo, J., Moreea, O., & Lutz, W. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.02.003 · Cited by 153
Self-citationFinch, A. E., Lambert, M. J., & Schaalje, B. G. (2001) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.286 · Cited by 143
Patient-focused and feedback research in psychotherapy: Where are we and where do we want to go?
Lutz, W., de Jong, K., & Rubel, J. A. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1079661 · Cited by 127
Self-citationLutz, W., Martinovich, Z., & Howard, K. I. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.67.4.571 · Cited by 124
Self-citationLutz, W., Martinovich, Z., & Howard, K. I. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.4.571 · Cited by 97
Self-citationImproving the efficiency of psychological treatment using outcome feedback technology
Delgadillo, J., Overend, K., Lucock, M. P., Groom, M., Kirby, N., McMillan, D., Gilbody, S. M., Lutz, W., Rubel, J. A., & de Jong, K. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.09.011 · Cited by 89
Self-citationSchiefele, A.-K., Lutz, W., Barkham, M., Rubel, J. A., Boehnke, J. R., Delgadillo, J., Kopta, S. M., Schulte, D., Saxon, D. H., Nielsen, S. L., et al. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-016-0736-3 · Cited by 86
Self-citationImproving the efficiency of cognitive-behavioural therapy by using formal client feedback
Janse, P. D., de Jong, K., van Dijk, M. K., Hutschemaekers, G. J. M., & Verbraak, M. J. P. M. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1152408 · Cited by 70
Amble, I., Gude, T., Stubdal, S., Andersen, B. J., & Wampold, B. E. (2014) · Psychotherapy Research · doi:10.1080/10503307.2014.928756 · Cited by 57
Lutz, W., Martinovich, Z., Howard, K. I., & Leon, S. C. (2002) · Journal of Clinical Psychology · doi:10.1002/jclp.10070 · Cited by 47
Self-citationLucock, M. P., Halstead, J. E., Leach, C., Barkham, M., Tucker, S., Randal, C., Middleton, J., Khan, W., Catlow, H., Waters, E. L., et al. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1051163 · Cited by 46
Self-citationde Jong, K., Segaar, J. A., Ingenhoven, T., Busschbach, J. J. V., & Timman, R. (2017) · Journal of Personality Disorders · doi:10.1521/pedi_2017_31_297 · Cited by 42
Lutz, W., Zimmermann, D., Müller, V., Deisenhofer, A., & Rubel, J. A. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1464-2 · Cited by 41
Self-citationde Jong, K., & de Goede, M. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1076198 · Cited by 38
Using outcome feedback in psychological therapy: A guideline for IAPT practitioners
Delgadillo, J., Lucock, M. P., & de Jong, K. (2015) · University of Huddersfield Repository (University of Huddersfield) · Cited by 2
Self-citation
Cited by
Sorted by citations
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102002 · Cited by 417
Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103438 · Cited by 196
Self-citationColombo, D., Fernández-Álvarez, J., Patané, A., Semonella, M., Kwiatkowska, M., García-Palacios, A., Cipresso, P., Riva, G., & Botella, C. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8040465 · Cited by 194
“Because providing continuous feedback to patients has been shown to be a valuable therapeutic procedure [29], we also included studies that adopted EMA-based feedback as a therapeutic tool for clinically depressed patients.”
MethodThieme, A., Hanratty, M., Lyons, M., Palacios, J. E., Marques, R. F., Morrison, C. P., & Doherty, G. J. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3564752 · Cited by 197
Colombo, D., Fernández‐Álvarez, J., Suso‐Ribera, C., Cipresso, P., Valev, H., Leufkens, T., Sas, C., García‐Palacios, A., Riva, G., & Botella, C. (2020) · Emotion · doi:10.1037/emo0000671 · Cited by 178
“EMA could indeed change the way we assess ER in psychopathology by helping us to understand the dynamics of this process in different emotional disorders, as well as the impact of emotion dysregulation feedback to patients, which has been shown to be a valuable practice (Delgadillo et al., 2018).”
Background
Metadata
- Authors
- Delgadillo, J., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Gilbody, S., Ali, S., Aguirre, E., Appleton, M., Nevin, J., O'Hayon, H., Patel, U., Sainty, A., Spencer, P., & McMillan, D.
- Affiliations
- University of Sheffield (GB)
- University of Huddersfield (GB)
- Universität Trier (DE)
- University of York (GB)
- National Health Service (GB)
- Pennine Care NHS Foundation Trust (GB)
- Cheshire and Wirral Partnership NHS Foundation Trust (GB)
- Whittington Health NHS Trust (GB)
- Cambridgeshire and Peterborough NHS Foundation Trust (GB)
- Humber Teaching NHS Foundation Trust (GB)
- South West Yorkshire Partnership NHS Foundation Trust (GB)
- Corresponding author
- Delgadillo, Jaime
- ORCID
- Venue
- The Lancet Psychiatry
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 5
- Issue
- 7
- Pages
- 564-572
- ISSN
- 2215-0366 (ISSN-L)
- OpenAlex
- W2809000673
- Cited by
- 152
- References
- 31
- Citations by year
- 2026: 13 · 2025: 15 · 2024: 28 · 2023: 13 · 2022: 19 · 2021: 21 · 2020: 19 · 2019: 20 · 2018: 3
- FWCI
- 18.76
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- The Lancet Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Huddersfield Research Portal (University of Huddersfield) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- outcome feedback, feedback-informed treatment, depression, anxiety, stepped care, psychotherapy, clinical supervision, low-intensity psychological interventions, symptom severity, treatment progress monitoring, automated alerts, individual psychotherapy, waiting lists, NHS mental health services, intention-to-treat analysis
- MeSH terms
- Adult, Anxiety Disorders, Cost-Benefit Analysis, Depressive Disorder, England, Feedback, Female, Humans, Male, Middle Aged, Psychotherapy, Quality of Life, State Medicine, Treatment Outcome, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
