The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring.
Abstract
This systematic review and meta-analysis examines the impact of measuring, monitoring, and feeding back information on client progress to clinicians while they deliver psychotherapy. It considers the effects of the 2 most frequently studied routine outcome monitoring (ROM) practices: The Partners for Change Outcome Management System and the Outcome Questionnaire System. Like other ROM practices, they typify attempts to enhance routine care by assisting psychotherapists in recognizing problematic treatment response and increasing collaboration between therapist and client to overcome poor treatment response. A total of 24 studies were identified and considered suitable for analysis. Two-thirds of the studies found that ROM-assisted psychotherapy was superior to treatment-as-usual offered by the same practitioners. Mean standardized effect sizes indicated that the effects ranged from small to moderate. Feedback practices reduced deterioration rates and nearly doubled clinically significant/reliable change rates in clients who were predicted to have a poor outcome. Clinical examples, diversity considerations, and therapeutic advances are provided. (PsycINFO Database Record (c) 2018 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783
Self-citationShimokawa, K., Lambert, M. J., & Smart, D. W. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019247 · Cited by 657
Self-citationIs it time for clinicians to routinely track patient outcome? A meta-analysis
Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpg025 · Cited by 608
Self-citationPatient-focused research: Using patient outcome data to enhance treatment effects.
Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.69.2.159 · Cited by 603
Self-citationA Tipping Point for Measurement-Based Care
Fortney, J. C., Unützer, J., Wrenn, G., Pyne, J. M., Smith, G. R., Schoenbaum, M., & Harbin, H. T. (2017) · Psychiatric Services · doi:10.1176/appi.ps.201500439 · Cited by 558
A lab test and algorithms for identifying clients at risk for treatment failure
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20108 · Cited by 542
Self-citationLambert, M. J., & Shimokawa, K. (2011) · Psychotherapy · doi:10.1037/a0022238 · Cited by 452
Self-citationMiller, S. D., Duncan, B. L., & Brown, J. (2003) · Cited by 437
Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001) · Psychotherapy Research · doi:10.1080/713663852 · Cited by 432
Self-citationAn investigation of self-assessment bias in mental health providers
Walfish, S., McAlister, B., O'Donnell, P., & Lambert, M. J. (2012) · Psychological Reports · doi:10.2466/02.07.17.pr0.110.2.639-644 · Cited by 394
Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003) · Journal of Counseling Psychology · doi:10.1037//0022-0167.50.1.59 · Cited by 374
Self-citationAnker, M. G., Duncan, B. L., & Sparks, J. A. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0016062 · Cited by 347
Hatfield, D., McCullough, L., Frantz, S. H. B., & Krieger, K. (2010) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.656 · Cited by 321
Enhancing psychotherapy outcomes via providing feedback on client progress: a replication
Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., & Goates, M. (2002) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.324 · Cited by 306
Self-citationRoutine outcome monitoring and feedback on physical or mental health status: evidence and theory
Carlier, I. V. E., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, N. J. A., & Zitman, F. G. (2012) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01543.x · Cited by 275
Harmon, S. C., Lambert, M. J., Smart, D. M., Hawkins, E., Nielsen, S. L., Slade, K., & Lutz, W. (2007) · Psychotherapy Research · doi:10.1080/10503300600702331 · Cited by 268
Self-citationEffect of feedback of treatment outcome in specialist mental healthcare: meta-analysis
Knaup, C., Koesters, M., Schoefer, D., Becker, T., & Puschner, B. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.053967 · Cited by 265
The partners for change outcome management system
Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20111 · Cited by 264
Feedback to clinicians: Theory, research, and practice
Sapyta, J., Riemer, M., & Bickman, L. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20107 · Cited by 253
Okiishi, J. C., Lambert, M. J., Eggett, D., Nielsen, L., Dayton, D. D., & Vermeersch, D. A. (2006) · Journal of Clinical Psychology · doi:10.1002/jclp.20272 · Cited by 253
Self-citationde Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012) · Psychotherapy Research · doi:10.1080/10503307.2012.673023 · Cited by 247
Outcome Questionnaire: Item Sensitivity to Change
Vermeersch, D. A., Lambert, M. J., & Burlingame, G. M. (2000) · Journal of Personality Assessment · doi:10.1207/s15327752jpa7402_6 · Cited by 247
Self-citationKendrick, T., El-Gohary, M., Stuart, B., Gilbody, S., Churchill, R., Aiken, L., Bhattacharya, A., Gimson, A., Brütt, A. L., de Jong, K., et al. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011119.pub2 · Cited by 237
The therapeutic effects of providing patient progress information to therapists and patients
Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., & Tuttle, K. C. (2004) · Psychotherapy Research · doi:10.1093/ptr/kph027 · Cited by 230
Self-citationBoyce, M. B., & Browne, J. P. (2013) · Quality of Life Research · doi:10.1007/s11136-013-0390-0 · Cited by 228
Does a continuous feedback system improve psychotherapy outcome?
Reese, R. J., Norsworthy, L. A., & Rowlands, S. R. (2009) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/a0017901 · Cited by 218
The Outcome Questionnaire (OQ‐45) in a Dutch population: A cross‐cultural validation
de Jong, K., Nugter, M. A., Polak, M. G., Wagenborg, J. E. A., Spinhoven, P., & Heiser, W. J. (2007) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.529 · Cited by 203
Gondek, D., Edbrooke-Childs, J. H., Fink, E., Deighton, J., & Wolpert, M. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0710-5 · Cited by 169
Effect of client feedback on couple psychotherapy outcomes.
Reese, R. J., Toland, M. D., Slone, N. C., & Norsworthy, L. A. (2010) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/a0021182 · Cited by 167
Slade, K., Lambert, M. J., Harmon, S. C., Smart, D. W., & Bailey, R. (2008) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.594 · Cited by 159
Self-citationKrägeloh, C. U., Czuba, K., Billington, D. R., Kersten, P., & Siegert, R. J. (2014) · Psychiatric Services · doi:10.1176/appi.ps.201400141 · Cited by 154
Outcome Questionnaire: Is It Sensitive to Changes in Counseling Center Clients?
Vermeersch, D. A., Whipple, J. L., Lambert, M. J., Hawkins, E. J., Burchfield, C. M., & Okiishi, J. C. (2004) · Journal of Counseling Psychology · doi:10.1037/0022-0167.51.1.38 · Cited by 151
Self-citationFinch, A. E., Lambert, M. J., & Schaalje, B. G. (2001) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.286 · Cited by 143
Self-citationSimon, W., Lambert, M. J., Harris, M. W., Busath, G., & Vazquez, A. (2012) · Psychotherapy Research · doi:10.1080/10503307.2012.698918 · Cited by 137
Self-citationLambert, M. J., Whipple, J. L., Bishop, M. J., Vermeersch, D. A., Gray, G. V., & Finch, A. E. (2002) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.333 · Cited by 128
Self-citationde Jong, K., Timman, R., Roijen, L. H., Vermeulen, P., Kooiman, K., Passchier, J., & Busschbach, J. J. V. (2014) · Psychotherapy Research · doi:10.1080/10503307.2013.871079 · Cited by 110
Progress monitoring and feedback in psychiatric care reduces depressive symptoms
Newnham, E. A., Hooke, G. R., & Page, A. C. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2010.05.003 · Cited by 104
Lutz, W., Lambert, M. J., Harmon, S. C., Tschitsaz, A., Schürch, E., & Stulz, N. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.496 · Cited by 103
Self-citationSafran, J. D., Muran, J. C., Samstag, L. W., & Winston, A. (2005) · Psychotherapy · doi:10.1037/0033-3204.42.4.512 · Cited by 102
Duncan, B. L., & Reese, R. J. (2015) · Psychotherapy · doi:10.1037/pst0000026 · Cited by 91
Therapist effects, effective therapists, and the law of variability.
Barkham, M., Lutz, W., Lambert, M. J., & Saxon, D. H. (2017) · American Psychological Association eBooks · doi:10.1037/0000034-002 · Cited by 89
Self-citationSchuman, D. L., Slone, N. C., Reese, R. J., & Duncan, B. L. (2014) · Psychotherapy Research · doi:10.1080/10503307.2014.900875 · Cited by 87
Ellsworth, J. R., Lambert, M. J., & Johnson, J. E. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.503 · Cited by 82
Self-citationCrits-Christoph, P. F., Ring‐Kurtz, S., Hamilton, J. L., Lambert, M. J., Gallop, R., McClure, B., Kulaga, A., & Rotrosen, J. (2011) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2011.09.003 · Cited by 81
Self-citationThe factorial structure of the outcome questionnaire‐45: a study with an Italian sample
Coco, G. L., Chiappelli, M., Bensi, L., Gullo, S., Prestano, C., & Lambert, M. J. (2008) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.601 · Cited by 76
Self-citationEvaluating the efficacy of client feedback in group psychotherapy.
Slone, N. C., Reese, R. J., Mathews-Duvall, S., & Kodet, J. (2015) · Group Dynamics Theory Research and Practice · doi:10.1037/gdn0000026 · Cited by 75
Improving the efficiency of cognitive-behavioural therapy by using formal client feedback
Janse, P. D., de Jong, K., van Dijk, M. K., Hutschemaekers, G. J. M., & Verbraak, M. J. P. M. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1152408 · Cited by 71
Amble, I., Gude, T., Stubdal, S., Andersen, B. J., & Wampold, B. E. (2014) · Psychotherapy Research · doi:10.1080/10503307.2014.928756 · Cited by 63
Simon, W., Lambert, M. J., Busath, G. L., Vazquez, A., Berkeljon, A., Hyer, K., Granley, M., & Berrett, M. E. (2013) · Psychotherapy Research · doi:10.1080/10503307.2013.787497 · Cited by 61
Self-citationProbst, T., Lambert, M. J., Loew, T. H., Dahlbender, R. W., Göllner, R., & Tritt, K. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2013.07.003 · Cited by 60
Self-citationDavidson, K. M., Perry, A., & Bell, L. (2014) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12032 · Cited by 58
The bilevel structure of the Outcome Questionnaire–45.
Bludworth, J., Tracey, T. J. G., & Glidden-Tracey, C. E. (2010) · Psychological Assessment · doi:10.1037/a0019187 · Cited by 56
A comparison of rational versus empirical methods in the prediction of psychotherapy outcome
Spielmans, G. I., Masters, K. S., & Lambert, M. J. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.491 · Cited by 49
Self-citationFeedback in group psychotherapy for eating disorders: A randomized clinical trial.
Davidsen, A. H., Poulsen, S., Lindschou, J., Winkel, P., Tróndarson, M., Waaddegaard, M., & Lau, M. E. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000173 · Cited by 46
Probst, T., Lambert, M. J., Dahlbender, R. W., Loew, T. H., & Tritt, K. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.03.010 · Cited by 32
Self-citationMurphy, K. P., Rashleigh, C. M., & Timulák, L. (2012) · Counselling Psychology Quarterly · doi:10.1080/09515070.2012.662349 · Cited by 30
Hansson, H., Rundberg, J., Österling, A., Öjehagen, A., & Berglund, M. (2012) · Nordic Journal of Psychiatry · doi:10.3109/08039488.2012.736534 · Cited by 28
Trudeau, L. (2000) · Iowa State University Digital Repository (Iowa State University) · doi:10.31274/rtd-180813-13636 · Cited by 12
The Use of Feedback in Group Counseling in a State Vocational Rehabilitation Setting
Grizzell, S. T., Smart, J. F., Lambert, M. J., & Fargo, J. D. (2016) · Journal of Applied Rehabilitation Counseling · doi:10.1891/0047-2220.47.4.10 · Cited by 5
Self-citation
Cited by
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102002 · Cited by 417
Lam, R. W., Kennedy, S. H., Adams, C., Bahji, A., Beaulieu, S., Bhat, V., Blier, P., Blumberger, D. M., Brietzke, E., Chakrabarty, T., et al. (2024) · The Canadian Journal of Psychiatry · doi:10.1177/07067437241245384 · Cited by 369
Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103438 · Cited by 196
Bickman, L. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01065-8 · Cited by 184
Flückiger, C., Rubel, J., Del Re, A. C., Horvath, A. O., Wampold, B. E., Crits-Christoph, P., Atzil-Slonim, D., Compare, A., Falkenström, F., Ekeblad, A., et al. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000594 · Cited by 180
Metadata
- Authors
- Lambert, M. J., Whipple, J. L., & Kleinstäuber, M.
- Venue
- Psychotherapy
- Published
- 2018-12
- Volume
- 55
- Issue
- 4
- Pages
- 520-537
- ISSN
- 0033-3204 (ISSN-L)
- 1939-1536
- OpenAlex
- W2896510587
- Cited by
- 392
- References
- 64
- FWCI
- 7.21
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.