CoChat
Sign Up
Cited by 417Psychotherapy Techniques and ApplicationsAnxiety, Depression, Psychometrics, Treatment, Cognitive ProcessesHealth Policy Implementation Science

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.

Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis

Open access
de Jong, Kim, Conijn, Judith M., Gallagher, Roisin A.V., Reshetnikova, Alexandra S., Heij, Marya, Lutz, Miranda C.·2021·Clinical Psychology Review
DOI 10.1016/j.cpr.2021.102002
Open this paper in CoChat — free accountOpen PDF

Abstract

Progress feedback is an intervention aimed at enhancing patient outcomes in routine clinical practice. This study reports a comprehensive multilevel meta-analysis on the effectiveness of progress feedback in psychological treatments in curative care. The short- and long-term effects of feedback on symptom reduction were investigated using 58 (randomized and non-randomized) studies, analyzing 110 effect sizes in a total of 21,699 patients. Effects of feedback on dropout rate, percentage of deteriorated cases, and treatment duration were also examined. Moderation analyses were conducted for study and feedback characteristics. A small significant effect of progress feedback on symptom reduction (d = 0.15, 95% CI: [0.10, 0.20]) was found, compared to control groups. This was also true for not-on-track cases (d = 0.17, 95% CI: [0.11, 0.22]). In addition, feedback had a small favorable effect on dropout rates (OR = 1.19, 95% CI: [1.03, 1.38]). The moderation analyses identified several potentially interesting variables for further research, including feedback instrument, outcome instrument, type of feedback, feedback frequency, treatment intensity, and country in which the study was conducted. Future studies should report on these variables more consistently so that we can obtain a better understanding of when and why feedback improves outcomes.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.cpr.2021.102002

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Clinicians' behaviour and attitudes, perceived usefulness, and commitment to outcome measurement mediate the impact on the effectiveness of such tools in clinical practice.

    “Clinicians' behaviour and attitudes, perceived usefulness, and commitment to outcome measurement mediate the impact on the effectiveness of such tools in clinical practice (de Jong et al., 2021;Lutz et al., 2021b).”

    From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as de Jong et al., 2021

    The paper’s “Mechanisms and moderators” section reports that feedback was effective only among therapists who actively used the reports. Its “Limitations and suggestions for further research” section also identifies therapist expectations as a potential moderator, but says the meta-analysis could not assess it. Our copy does not establish perceived usefulness or commitment to outcome measurement as mediators, so the sentence is more specific and certain than this source supports; page numbers were not available. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Practising data-informed psychotherapy with technological augmentation tools (i.e., CDSS, ROM and feedback systems, CSTs) contributes to better treatment outcomes.

    “Beyond CF, it has been shown that practising data-informed psychotherapy with technological augmentation tools (i.e., CDSS, ROM and feedback systems, CSTs) contributes to better treatment outcomes (Cohen et al., 2021;de Jong et al., 2021;Delgadillo et al., 2018).”

    From Data-Informed Case Formulation with the Trier Treatment Navigator (2022) · cites this paper as de Jong et al., 2021

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 67

  1. Amble, 2015, The effect of implementing the Outcome Questionnaire-45.2 feedback system in Norway: A multisite randomized clinical trial in a naturalistic setting, Psychotherapy Research, 25, 669, 10.1080/10503307.2014.928756 · DOI
  2. Amble, 2016, How and when feedback works in psychotherapy: Is it the signal?, Psychotherapy Research, 26, 545, 10.1080/10503307.2015.1053552 · DOI
  3. Anker, 2009, Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting, Journal of Consulting and Clinical Psychology, 77, 693, 10.1037/a0016062 · DOI
  4. Bergman, 2018, Client feedback in psychological therapy for children and adolescents with mental health problems, Cochrane Database of Systematic Reviews, 8
  5. Berking, 2006, How effective is systematic feedback of treatment progress to the therapist? An empirical study in a cognitive-behavioural-oriented inpatient setting, Zeitschrift fur Klinische Psychologie und Psychotherapie: Forschung und Praxis, 35, 21, 10.1026/1616-3443.35.1.21 · DOI
  6. Bickman, 2008, A measurement feedback system (MFS) is necessary to improve mental health outcomes, Journal of the American Academy of Child and Adolescent Psychiatry, 47, 1114
  7. Bickman, 2016, Implementing a measurement feedback system: A tale of two sites, Administration and Policy in Mental Health and Mental Health Services Research, 43, 410, 10.1007/s10488-015-0647-8 · DOI
  8. Brattland, 2018, The effects of routine outcome monitoring (ROM) on therapy outcomes in the course of an implementation process: A randomized clinical trial, Journal of Counseling Psychology, 65, 641, 10.1037/cou0000286 · DOI
  9. Brosan, 2008, Self-evaluation of cognitive therapy performance: do therapists know how competent they are?, Behavioural and Cognitive Psychotherapy, 36, 581, 10.1017/S1352465808004438 · DOI
  10. Carlier, 2012, Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory, Journal of Evaluation in Clinical Practice, 18, 104, 10.1111/j.1365-2753.2010.01543.x · DOI
  11. Connolly Gibbons, 2015, The effectiveness of clinician feedback in the treatment of depression in the community mental health system, Journal of Consulting and Clinical Psychology, 83, 748, 10.1037/a0039302 · DOI
  12. Crits-Christoph, 2012, A preliminary study of the effects of individual patient-level feedback in outpatient substance abuse treatment programs, Journal of Substance Abuse Treatment, 42, 301, 10.1016/j.jsat.2011.09.003 · DOI
  13. Davidsen, 2017, Feedback in group psychotherapy for eating disorders: A randomized clinical trial, Journal of Consulting and Clinical Psychology, 85, 484, 10.1037/ccp0000173 · DOI
  14. Davidson, 2014, Would continuous feedback of patient's clinical outcomes to practitioners improve NHS psychological therapy services? Critical analysis and assessment of quality of existing studies, Psychology and Psychotherapy: Theory, Research and Practice, 88, 21, 10.1111/papt.12032 · DOI
  15. De Jong, 2015, Why do some therapists not deal with outcome monitoring feedback? A feasibility study on the effect of regulatory focus and person–organization fit on attitude and outcome, Psychotherapy Research, 25, 661, 10.1080/10503307.2015.1076198 · DOI · Paper page
  16. De Jong, 2017, Adverse effects of outcome monitoring feedback in patients with personality disorders: A randomized controlled trial in day treatment and inpatient settings, Journal of Personality Disorders, 1
  17. De Jong, 2014, The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial, Psychotherapy Research, 24, 629, 10.1080/10503307.2013.871079 · DOI
  18. De Jong, 2012, Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial, Psychotherapy Research, 22, 464, 10.1080/10503307.2012.673023 · DOI
  19. Delgadillo, 2017, Improving the efficiency of psychological treatment using outcome feedback technology, Behaviour Research and Therapy, 99, 89, 10.1016/j.brat.2017.09.011 · DOI
  20. DerSimonian, 1986, Meta-analysis in clinical trials, Controlled Clinical Trials, 7, 177, 10.1016/0197-2456(86)90046-2 · DOI · Paper page
  21. Egger, 1997, Bias in meta-analysis detected by a simple, graphical test, BMJ, 315, 629, 10.1136/bmj.315.7109.629 · DOI · Paper page
  22. Errázuriz, 2018, In psychotherapy with severe patients discouraging news may be worse than no news: The impact of providing feedback to therapists on psychotherapy outcome, session attendance, and the alliance, Journal of Consulting and Clinical Psychology, 86, 125, 10.1037/ccp0000277 · DOI
  23. Greene, 2004, Distinguishing between heterogeneity and inefficiency: Stochastic frontier analysis of the World Health Organization’s panel data on national health care systems, Health Economics, 13, 959, 10.1002/hec.938 · DOI
  24. Grizzell, 2016, The use of feedback in group counseling in a state vocational rehabilitation setting, Journal of Applied Rehabilitation Counseling, 47, 10, 10.1891/0047-2220.47.4.10 · DOI
  25. Hannan, 2005, A lab test and algorithms for identifying clients at risk for treatment failure, Journal of Clinical Psychology, 61, 155, 10.1002/jclp.20108 · DOI
  26. Hansen, 2015, Impact of client feedback on clinical outcomes for young people using public mental health services: A pilot study, Psychiatry Research, 229, 617, 10.1016/j.psychres.2015.05.007 · DOI
  27. Hansson, 2013, Intervention with feedback using outcome questionnaire 45 (OQ-45) in a Swedish psychiatric outpatient population. A randomized controlled trial, Nordic Journal of Psychiatry, 67, 274, 10.3109/08039488.2012.736534 · DOI
  28. Harmon, 2005, Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients, Journal of Clinical Psychology, 61, 175, 10.1002/jclp.20109 · DOI
  29. Harmon, 2007, Enhancing outcome for potential treatment failures: Therapist-client feedback and clinical support tools, Psychotherapy Research, 17, 379, 10.1080/10503300600702331 · DOI
  30. Hatfield, 2010, Do we know when our clients get worse? An investigation of therapists’ ability to detect negative client change, Clinical Psychology & Psychotherapy: An International Journal of Theory & Practice, 17, 25, 10.1002/cpp.656 · DOI · Paper page
  31. Janse, 2017, Improving the efficiency of cognitive-behavioural therapy by using formal client feedback, Psychotherapy Research, 27, 525, 10.1080/10503307.2016.1152408 · DOI
  32. Kazdin, 2008, Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care, American Psychologist, 63, 146, 10.1037/0003-066X.63.3.146 · DOI · Paper page
  33. Kendrick, 2016, Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults, Cochrane Database of Systematic Reviews, 7
  34. Knaup, 2009, Effect of feedback of treatment outcome in specialist mental healthcare: Meta-analysis, The British Journal of Psychiatry, 195, 15, 10.1192/bjp.bp.108.053967 · DOI
  35. Koementas-de Vos, 2018, Does progress feedback enhance the outcome of group psychotherapy?, Psychotherapy, 55, 151, 10.1037/pst0000164 · DOI
  36. Krägeloh, 2015, Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology, Psychiatric Services, 66, 224, 10.1176/appi.ps.201400141 · DOI
  37. Lambert, 2018, Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring, Psychotherapy, 55, 520, 10.1037/pst0000167 · DOI
  38. Lambert, 2001, The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced?, Psychotherapy Research, 11, 49, 10.1080/713663852 · DOI
  39. Langkaas, 2018, Five types of clinical difference to monitor in practice, Psychotherapy, 55, 241, 10.1037/pst0000194 · DOI
  40. Lutz, 2011, Diagnostik und psychometrische Verlaufsrückmeldungen im Rahmen eines Modellprojektes zur Qualitätssicherung in der ambulanten Psychotherapie, Zeitschrift für Klinische Psychologie und Psychotherapie, 40, 283, 10.1026/1616-3443/a000125 · DOI
  41. Lutz, 2015, Patient-focused and feedback research in psychotherapy: Where are we and where do we want to go?, Psychotherapy Research, 25, 625, 10.1080/10503307.2015.1079661 · DOI
  42. McClintock, 2017, Enhancing psychotherapy process with common factors feedback: A randomized, clinical trial, Journal of Counseling Psychology, 64, 247, 10.1037/cou0000188 · DOI
  43. Miller, 2005, The partners for change outcome management system, Journal of Clinical Psychology, 61, 199, 10.1002/jclp.20111 · DOI
  44. Murphy, 2012, The relationship between progress feedback and therapeutic outcome in student counselling: A randomized controlled trial, Counselling Psychology Quarterly, 25, 1, 10.1080/09515070.2012.662349 · DOI
  45. Newnham, 2010, Progress monitoring and feedback in psychiatric care reduces depressive symptoms, Journal of Affective Disorders, 127, 139, 10.1016/j.jad.2010.05.003 · DOI
  46. Østergård, 2018, The effect of using the Partners for Change Outcome Management System as feedback tool in psychotherapy—A systematic review and meta-analysis, Psychotherapy Research, 1
  47. Probst, 2014, Providing patient progress feedback and clinical support tools to therapists: Is the therapeutic process of patients on-track to recovery enhanced in psychosomatic in-patient therapy under the conditions of routine practice?, Journal of Psychosomatic Research, 76, 477, 10.1016/j.jpsychores.2014.03.010 · DOI
  48. Probst, 2013, Feedback on patient progress and clinical support tools for therapists: Improved outcome for patients at risk of treatment failure in psychosomatic in-patient therapy under the conditions of routine practice, Journal of Psychosomatic Research, 75, 255, 10.1016/j.jpsychores.2013.07.003 · DOI
  49. Puschner, 2009, Outcome management in in-patient psychiatric care, Acta Psychiatrica Scandinavica, 120, 308, 10.1111/j.1600-0447.2009.01397.x · DOI
  50. Reese, 2009, Does a continuous feedback system improve psychotherapy outcome?, Psychotherapy: Theory, Research, Practice, Training, 46, 418, 10.1037/a0017901 · DOI
  51. Reese, 2010, Effect of client feedback on couple psychotherapy outcomes, Psychotherapy: Theory, Research, Practice, Training, 47, 616, 10.1037/a0021182 · DOI
  52. Rise, 2012, The short-term effect on alliance and satisfaction of using patient feedback scales in mental health out-patient treatment. A randomised controlled trial, BMC Health Services Research, 12, 348, 10.1186/1472-6963-12-348 · DOI
  53. Schmidt, 2006, Does personalized feedback improve the outcome of cognitive-behavioural guided self-care in bulimia nervosa? A preliminary randomized controlled trial, British Journal of Clinical Psychology, 45, 111, 10.1348/014466505X29143 · DOI
  54. Schöttke, 2019, The Effect of Patient Progress Feedback on Psychotherapy Outcome, Verhaltenstherapie, 1
  55. Schuman, 2015, Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment, Psychotherapy Research, 25, 396, 10.1080/10503307.2014.900875 · DOI
  56. She, 2018, Client feedback in China: A randomized clinical trial in a college counseling center, Journal of Counseling Psychology, 65, 727, 10.1037/cou0000300 · DOI
  57. Simon, 2013, Effects of providing patient progress feedback and clinical support tools to psychotherapists in an inpatient eating disorders treatment program: A randomized controlled study, Psychotherapy Research, 23, 287, 10.1080/10503307.2013.787497 · DOI
  58. Simon, 2012, Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure, Psychotherapy Research, 22, 638, 10.1080/10503307.2012.698918 · DOI · Paper page
  59. Slade, 2008, Improving psychotherapy outcome: The use of immediate electronic feedback and revised clinical support tools, Clinical Psychology & Psychotherapy, 15, 287, 10.1002/cpp.594 · DOI
  60. Slone, 2015, Evaluating the efficacy of client feedback in group psychotherapy, Group Dynamics: Theory, Research, and Practice, 19, 122, 10.1037/gdn0000026 · DOI
  61. Tam, 2017, The application of a feedback-informed approach in psychological service with youth: Systematic review and meta-analysis, Clinical Psychology Review, 55, 41, 10.1016/j.cpr.2017.04.005 · DOI
  62. Tilden, 2019, Feedback in couple and family therapy: A randomized clinical trial, Family Process
  63. Tzur Bitan, 2019, The effect of process and outcome feedback in highly distressed outpatients: A randomized controlled trial, Psychotherapy Research, 1
  64. Van Oenen, 2016, Feedback-informed treatment in emergency psychiatry; a randomised controlled trial, BMC Psychiatry, 16, 110, 10.1186/s12888-016-0811-z · DOI
  65. Wampold, 2015, Routine outcome monitoring: Coming of age—With the usual developmental challenges, Psychotherapy, 52, 458, 10.1037/pst0000037 · DOI
  66. Whipple, 2003, Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice, Journal of Counseling Psychology, 50, 59, 10.1037/0022-0167.50.1.59 · DOI
  67. Winkelhorst, 2013, Verhoogt routine process monitoring (RPM) de effectiviteit van behandeling?, Een vergelijkende pilotstudy. Tijdschrift voor Psychotherapie, 39, 146, 10.1007/s12485-013-0028-2 · DOI

Cited by

Sorted by citations

Metadata

Authors
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C.
Affiliations
  • Leiden University (NL)
  • Vrije Universiteit Amsterdam (NL)
Corresponding author
de Jong, Kim
ORCID
Venue
Clinical Psychology Review
Publisher
Elsevier BV
Published
2021-04
Volume
85
Pages
102002
ISSN
0272-7358 (ISSN-L)
DOI
10.1016/j.cpr.2021.102002
OpenAlex
W3134663541
Cited by
417
References
23
Citations by year
2026: 90 · 2025: 78 · 2024: 121 · 2023: 66 · 2022: 50 · 2021: 12
FWCI
7.18
Type
journal article
Language
en
Open access
Licence
Locations
  • Clinical Psychology Review (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • VU Research Portal (publishedVersion, other-oa)
  • Leiden Repository (Leiden University) (submittedVersion, cc-by)
  • UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics
  • Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Keywords
progress feedback, psychological intervention, symptom relief, treatment discontinuation, treatment deterioration, treatment duration, feedback frequency, feedback instruments, treatment intensity, curative treatment
MeSH terms
Duration of Therapy, Feedback, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review