Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system.
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Outcome research has documented worsening among a minority of the patient population (5% to 10%). In this study, we conducted a meta-analytic and mega-analytic review of a psychotherapy quality assurance system intended to enhance outcomes in patients at risk of treatment failure. METHOD: Original data from six major studies conducted at a large university counseling center and a hospital outpatient setting (N = 6,151, mean age = 23.3 years, female = 63.2%, Caucasian = 85%) were reanalyzed to examine the effects of progress feedback on patient outcome. In this quality assurance system, the Outcome Questionnaire-45 was routinely administered to patients to monitor their therapeutic progress and was utilized as part of an early alert system to identify patients at risk of treatment failure. Patient progress feedback based on this alert system was provided to clinicians so that they could intervene before treatment failure occurred. Meta-analytic and mega-analytic approaches were applied in intent-to-treat and efficacy analyses of the effects of feedback interventions. RESULTS: Three forms of feedback interventions-integral elements of this quality assurance system-were effective in enhancing treatment outcome, especially for signal alarm patients. Two of the three feedback interventions were also effective in preventing treatment failure (clinical support tools and the provision of patient progress feedback to therapists). CONCLUSIONS: The current state of evidence appears to support the efficacy and effectiveness of feedback interventions in enhancing treatment outcome.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“therapist supervision and feedback can improve patient outcomes, 52,53”
From Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder (2016) · cites this paper as [52]
In the citing paper
“Using MBC results in more clinically effective,17 cost-effective,18 and efficient treatment.19”
From Using Machine Learning to Match Clients and Therapy Providers: Evaluating Clinical Quality and Cost of Care (2025) · cites this paper as [19]
Citations
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 1×
“Following recommendations by Overton (1998) and Hedges and Vevea (1998), we used a random-effects model, given that the research on providing feedback for the purpose of enhancing treatment outcome is relatively new in psychotherapy outcome literature, that studies included in this meta-analysis contained slight varia…”
MethodBergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
Self-citationFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783 · cited 1×
“Similarly, an argument can be made against causal statements based on the data of studies not directly compared in the same randomized trials, such as in the case of comparing the CST Fb group and the pooled TAU group in the studies of Harmon et al. (2007) and Slade et al.”
BackgroundSelf-citationÆgisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., et al. (2006) · The Counseling Psychologist · doi:10.1177/0011000005285875 · Cited by 737 · cited 2×
“As has been repeatedly demonstrated in clinical research, any prediction relying on statistical or actuarial methods tends to fare better than clinical judgment alone (Ægisdóttir et al., 2006; Grove, 2005).”
Background
Cited by
Items 301–400 of 660
Do clinicians really use feedback-monitoring systems? A qualitative analysis of 16 group leaders.
Whitcomb, K. E., Woodland, S. C., & Burlingame, G. M. (2018) · Psychotherapy · doi:10.1037/pst0000141 · Cited by 11
MONITORIZACIÓN SISTEMÁTICA Y FEEDBACK EN PSICOTERAPIA
Gimeno-Peón, A., Barrio-Nespereira, A., Prado-Abril, J., Práctica privada, G., de Asturias, S. D. S. D. P., & de Salud-Osasunbidea, S. N. (2018) · Papeles del Psicólogo - Psychologist Papers · doi:10.23923/pap.psicol2018.2872 · Cited by 20
Hegland, P. A., Aasprang, A., Øygard, S. H., Nordberg, S. S., Kolotkin, R. L., Moltu, C., Tell, G. S., & Andersen, J. R. (2018) · Clinical Obesity · doi:10.1111/cob.12277 · Cited by 13
“However, if patients are responding poorly to psychiatric treatment, the use of PRO/CFS has been shown to yield clinically important effects on treatment outcome (1, 5, 44, 45).”
BackgroundA short scale to evaluate supervision and supervisor competence—The SE‐SC8
Gonsalvez, C. J. (2020) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2510 · Cited by 11
Implementing routine outcome monitoring in a psychodynamic training clinic: it’s complicated
Doorn, K. A., & Meisel, J. (2022) · Counselling Psychology Quarterly · doi:10.1080/09515070.2022.2110451 · Cited by 13
Cross, S., Mellor‐Clark, J., & Macdonald, J. B. (2014) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1929 · Cited by 11
Koffmann, A. (2017) · Psychotherapy Research · doi:10.1080/10503307.2017.1294771 · Cited by 13
“in their therapist feedback system, where cases are flagged for special attention when there is evidence of a failure to improve as expected, rather than because of frank deterioration (Shimokawa et al., 2010).”
BackgroundResultEffectiveness of Psychotherapeutic Treatment
Lambert, M. J. (2015) · Cited by 30
Self-citationLindsey, S., Hiskey, S., Irons, C., & Andrews, L. (2022) · OBM Integrative and Complementary Medicine · doi:10.21926/obm.icm.2203039 · Cited by 10
Cordony, I., Mills, L., Mammen, K., & Lintzeris, N. (2023) · Drug and Alcohol Review · doi:10.1111/dar.13742 · Cited by 15
Martinez, R. G., Weiner, B. J., Meza, R. D., Dorsey, S., Palazzo, L. G., Matson, A., Bain, C. E., Mettert, K. D., Pullmann, M. D., & Lewis, C. C. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00526-z · Cited by 9
“Especially in youth, MBC increases the rate of symptom improvement [1], detects clients who would otherwise deteriorate [6], and alerts clinicians to non-responders [6, 7].”
Posttraumatic Stress Disorder and Substance Use Disorder Screening, Assessment, and Treatment
Patton, S. C., Watkins, L. E., Killeen, T. K., & Hien, D. A. (2024) · Current Psychiatry Reports · doi:10.1007/s11920-024-01547-8 · Cited by 13
“Bene-fits of measurement-based care include improved outcomes and increased patient engagement [ 108 , 109].”
An item reduction analysis of the Group Questionnaire.
Jensen, J., & Burlingame, G. M. (2018) · Psychotherapy · doi:10.1037/pst0000145 · Cited by 11
How to Use Research to Become More Effective Therapists
Wampold, B. E. (2017) · Springer eBooks · doi:10.1007/978-3-319-50675-3_14 · Cited by 11
Praxisorientierte Forschung in der Psychotherapie: Entwicklung, Gegenwart und Zukunft
Herzog, P., Kaiser, T., & Brakemeier, E. (2022) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000665 · Cited by 11
la Rosa-Cáceres, A. D., Lozano, O. M., Sánchez‐García, M., Fernández‐Calderón, F., Rossi, G. M. P., & Díaz, C. (2023) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-022-10008-6 · Cited by 14
Hooke, G. R., Sng, A., Cunningham, N. K., & Page, A. C. (2017) · Cognitive Therapy and Research · doi:10.1007/s10608-017-9851-z · Cited by 12
“Hence, the provision of feedback that a person is on track to a positive outcome will not change therapeutic practice or outcomes and probably this difference potentially explains the apparent difference between the conclusions of the two literature reviews (Kendrick et al. 2016; Shimokawa et al. 2010).”
Sawchuk, C. N., Mulholland, H., Trane, S. T., Lebow, J. R., Puspitasari, A. J., & Lombardi, N. (2020) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2020.05.003 · Cited by 11
Different lens or different picture? Comparing methods of defining dramatic change in psychotherapy
Erekson, D. M., Horner, J. R., & Lambert, M. J. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1247217 · Cited by 12
Self-citationHow Do I Know Whether My Efforts Are Helpful for the Client? Implementing Feedback in Norway
Tilden, T. (2017) · Springer eBooks · doi:10.1007/978-3-319-50675-3_1 · Cited by 10
Beneciuk, J. M., Ballengee, L. A., & George, S. Z. (2019) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2019.03.004 · Cited by 14
Can we predict failure in couple therapy early enough to enhance outcome?
Pepping, C., Halford, W. K., & Doss, B. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.12.015 · Cited by 9
Jonášová, K., Čevelíček, M., Doležal, P., & Řiháček, T. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01348-4 · Cited by 17
“ROM was found to be especially useful in preventing negative treatment outcomes by reducing the number of deterio-rating cases (Lambert et al., 2018; Shimokawa et al., 2010), probably because a stark difference between the expected course of treatment and the client’s progress provides more…”
Gorban, C., Chong, M. K., Poulsen, A., Turner, A., LaMonica, H. M., McKenna, S., Scott, E. M., Hickie, I. B., & Iorfino, F. (2025) · JMIR Mental Health · doi:10.2196/70154 · Cited by 14
“Such DMHTs are able to detect at-risk clients [9,10], support client-clinician communication [11], and improve treatment engagement and outcomes [12-14].”
Styła, R. (2014) · Psychiatria Polska · doi:10.12740/pp/19494 · Cited by 14
Barkham, M., de Jong, K., Delgadillo, J., & Lutz, W. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197611012.003.0015 · Cited by 9
Hickie, I. B., Iorfino, F., Rohleder, C., Song, Y. J. C., Nichles, A., Zmicerevska, N., Capon, W., Guastella, A. J., Leweke, F. M., Scott, J. L., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-072082 · Cited by 10
“Despite good evidence for its effectiveness and its customary use in physical disease management, 31 32 it remains largely absent from youth mental health care.”
BackgroundEgeter, J., Hüfner, K., Sztankay, M. J., Holzner, B., & Sperner‐Unterweger, B. (2017) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2017.12.009 · Cited by 10
“weight, pharmacological serum levels, blood pressure) for continuous observation of patients health status [21], provides data that can be of use for all HCPs and improve treatment outcomes by feedback to patients in daily clinical routine [22].”
BackgroundMoltu, C., McAleavey, A. A., Helleseth, M. M., Møller, G. H., & Nordberg, S. S. (2021) · International Journal of Mental Health Systems · doi:10.1186/s13033-021-00465-z · Cited by 9
“Several meta-analyses have concluded that ROM/CFSs can produce beneficial effects [1, 3], but a recent independent review found significant variability in the effects on patient outcomes produced by ROM/CFSs [8], concluding that evidence is still insufficient to advocate their general use.”
BackgroundStatistical significance testing and clinical effectiveness studies.
Wise, E. A. (2011) · Psychotherapy · doi:10.1037/a0022701 · Cited by 11
“In contrast, Shimokawa, Lambert, and Smart (2010) remind us that 5% - 10% of adults treated in RCTs and effectiveness studies will likely deteriorate with treatment, irrespective of what treatment is delivered, under what conditions (RCTs vs. routine practice), or patient characteristics (Hansen,…”
BackgroundUsing client feedback in psychotherapy from an interpersonal process perspective.
Reese, R. J., Slone, N. C., & Miserocchi, K. M. (2013) · Psychotherapy · doi:10.1037/a0032522 · Cited by 9
Davidson, K. M., Rankin, M., Begley, A., Lloyd, S. M., Barry, S., McSkimming, P., Bell, L., ALLAN, C. A., Osborne, M., Ralston, G. E., et al. (2017) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465817000029 · Cited by 10
“…improvement in clinical outcomes in counselling settings, particularly for clients at risk of treatment failure, when feedback is given to therapists during therapy (Lambert et al., 2001; Lambert et al., 2002; Whipple et al., 2003; Harmon et al., 2007; Slade et al., 2008; Shimokawa et al., 2010).”
BackgroundAllen, J. G., Fowler, J. C., Madan, A., Ellis, T. E., Oldham, J. M., & Frueh, B. C. (2017) · Bulletin of the Menninger Clinic · doi:10.1521/bumc.2017.81.1.1 · Cited by 11
Causal Knowledge and Reasoning in Decision Making
Hagmayer, Y., & Witteman, C. L. M. (2017) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2017.03.004 · Cited by 11
Garb, H. N., Wood, J. M., & Baker, M. T. (2018) · Psychological Assessment · doi:10.1037/pas0000542 · Cited by 9
Burr, S., Fowler, J. C., Allen, J. G., Wiltgen, A., & Madan, A. (2017) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000250 · Cited by 10
“3%) and (2) the value of the reports in directly influencing treatment decisions (n = 7, 23.”
BackgroundOutcome Research: Methods for Improving Outcome in Routine Care
Lambert, M. J. (2014) · Springer eBooks · doi:10.1007/978-3-7091-1382-0_29 · Cited by 13
Self-citationSingle case designs in clinical practice: A contemporary CBS perspective on why and how to
Holman, G. I., & Koerner, K. (2014) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2014.04.006 · Cited by 18
Use of outcome measurements in clinical practice: How specific should one be?
Nugter, A., Hermens, M. L., Robbers, S. C. C., van Son, G. E., Theunissen, J., & Engelsbel, F. (2017) · Psychotherapy Research · doi:10.1080/10503307.2017.1408975 · Cited by 10
“…patients who do not improve well, the so-called Not On Track (NOT) cases (Carlier et al., 2010; Kendrick et al., 2016; Knaup et al., 2009; Shimokawa et al., 2010), in short-term therapies (De Jong et al., 2014; Knaup et al., 2009), and when feedback is given more than once (Shimokawa et al., 2010).”
BackgroundGrip, K. K. (2012) · Gothenburg University Publications Electronic Archive (Gothenburg University) · Cited by 20
Lee, J. L., Billovits, C., Chen, S.-Y., Wickham, R. E., Kocher, B., Chen, C. E., & Lungu, A. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.002 · Cited by 8
Met kennis oogsten: monitoring en doorontwikkeling zorg voor jeugd
van Yperen, T. (2013) · Kind en adolescent · doi:10.1007/s12453-013-0015-y · Cited by 13
Confer, J. R., WHITE, M., Groat, M., Madan, A., Allen, J. G., Fowler, J. C., & Kahn, D. A. (2015) · Journal of Psychiatric Practice · doi:10.1097/01.pra.0000460624.18052.1d · Cited by 8
Using Routine Outcome Measures to Provide Feedback at the Service Agency Level
Roe, D., Pickman, L. L., Baloush-Kleinman, V., Garber-Epstein, P., Gornemann, M. I., & Gelkopf, M. (2016) · Community Mental Health Journal · doi:10.1007/s10597-016-0039-x · Cited by 8
Development of the clinically adaptive multidimensional outcome survey
Sanders, P. W., Richards, P. S., & McBride, J. A. (2017) · Psychotherapy Research · doi:10.1080/10503307.2016.1277039 · Cited by 14
Qualitätssicherung in der Psychotherapie
Rubel, J. A., Zimmermann, D., Müller, V., & Lutz, W. (2017) · PPmP - Psychotherapie · Psychosomatik · Medizinische Psychologie · doi:10.1055/s-0043-103272 · Cited by 9
“[36] Shimokawa K, Lambert MJ, Smart DW.”
BackgroundInfluentialGmeinwieser, S., Kuhlencord, M., Ruhl, U. E., Hagmayer, Y., & Probst, T. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1609709 · Cited by 7
Does feedback improve psychotherapy outcomes compared to treatment-as-usual for adults and youth?
Dyason, K. M., Shanley, D. C., O’Donovan, A., & Low‐Choy, S. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1620367 · Cited by 12
“These results are consistent with previous research that feedback is effective at improving outcomes for adults in psychotherapy (Duncan & Reese, 2015; Lambert et al., 2018; Lutz et al., 2015; Shimokawa et al., 2010) and provides much needed findings in relation to youth psychotherapy.”
BackgroundMethodResultInfluentialScheitern in der Psychotherapie – der aktuelle Wissensstand
Strauss, B. M. (2021) · Psychotherapeut · doi:10.1007/s00278-021-00510-9 · Cited by 10
Liu, J., Munson, S. A., Chang, A., Voss, C., & Graham, A. K. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23700 · Cited by 7
Session-by-session outcome monitoring in CAMHS: clinicians’ beliefs
James, K., Elgie, S., Adams, J., Henderson, T., & Salkovskis, P. M. (2015) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x15000653 · Cited by 10
Transporting a Manualized Treatment for Children’s Disruptive Behavior to a Community Clinic
Shapiro, J. P., Youngstrom, J. K., Youngstrom, E. A., & Marcinick, H. F. (2012) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-012-9206-8 · Cited by 9
Cohen, D. (2015) · Journal of Technology in Human Services · doi:10.1080/15228835.2015.1039687 · Cited by 9
“There have been a handful of success stories documented about the use of EDSS in mental health settings (Dewan et al., 2002; Hodges & Wotring, 2012; Shimokawa et al., 2010; Slade et al., 2008; Unu¨tzer, Choi, Cook, & Oishi, 2002).”
BackgroundInfluentialReligious Attendance Serves as a Protective Variable Against Suicidal Ideation During Treatment
Price, S. D., & Callahan, J. L. (2016) · Pastoral Psychology · doi:10.1007/s11089-016-0725-1 · Cited by 12
Spengler, P. M., & Lee, N. A. (2017) · Counselling Psychology Quarterly · doi:10.1080/09515070.2017.1305948 · Cited by 9
The challenges of making clinical feedback in psychotherapy benefit all users: A qualitative study
Hovland, R. T., & Moltu, C. (2019) · Nordic Psychology · doi:10.1080/19012276.2019.1684348 · Cited by 8
Is There a Future for Therapists?
King, R., & Bickman, L. (2017) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-017-0814-1 · Cited by 8
Williams, N. J., Aarons, G. A., Ehrhart, M. G., Esp, S. M., Vega, N. R., Sklar, M., Carandang, K. M., Brookman-Frazee, L. I., & Marcus, S. C. (2025) · Psychiatric Services · doi:10.1176/appi.ps.20240302 · Cited by 9
Rober, P., & Tricht, K. V. (2023) · Family Process · doi:10.1111/famp.12936 · Cited by 7
Smits, D., Claes, L., Stinckens, N., & Smits, D. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-014-0597-6 · Cited by 10
“Systematic client feedback has proven to increase treatment outcome, especially in clients predicted to have poor outcomes (Knaup et al. 2009; Shimokawa et al. 2010).”
BackgroundFeedback to semi-professional counselors in treating child aggression
Shechtman, Z., & Tutian, R. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1095368 · Cited by 7
Hatchett, G. T. (2016) · Journal of Mental Health Counseling · doi:10.17744/mehc.39.2.02 · Cited by 9
Areas, M., Roussos, A., Hirsch, H., Hirsch, P., Becerra, P., & Gómez Penedo, J. M. (2018) · REVISTA ARGENTINA DE CLINICA PSICOLOGICA · doi:10.24205/03276716.2018.1064 · Cited by 7
Brown, C. L., Bosley, H. G., Kenyon, A. D., Chen, K., & LEVENSON, R. W. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.03.014 · Cited by 7
Redefining Therapeutic Outcomes of Depression Treatment
Slofstra, C., Booij, S. H., Hoenders, R. H. J., & Castelein, S. (2019) · Journal for Person-Oriented Research · doi:10.17505/jpor.2019.10 · Cited by 10
“Eventually, treatment selection models and feedback systems may be integrated in the routine outcome monitoring systems to inform clinicians and patients on the best course of treatment (Kendrick et al., 2016; Shimokawa et al., 2010).”
BackgroundDrug and alcohol practitioners’ attitudes toward the use of standardized assessment
Revill, A. S., Anderson, L. E., Kidd, C., & Gullo, M. J. (2021) · Addictive Behaviors · doi:10.1016/j.addbeh.2021.107231 · Cited by 8
Client feedback in psychological therapy for children and adolescents with mental health problems
Kornør, H., Bergman, H., Hanssen‐Bauer, K., Soares‐Weiser, K., Tollefsen, T. K., & Bjørndal, A. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011729 · Cited by 7
Early trajectories of symptom change and working alliance as predictors of treatment outcome
Lin, T., Anderson, T., Austin, M., & Mischkowski, D. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2080028 · Cited by 9
“Whereas some patients showed early improvements (Lutz et al., 2009), others failed to benefit (Cuijpers et al., 2018; Hansen et al., 2002; Lin & Farber, 2021; Shimokawa et al., 2010).”
BackgroundArora, P. G., Awad, M. N., Parr, K. M., & Connors, E. H. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01419-6 · Cited by 7
“MBC has been associated with improved client outcomes in a variety of treatment settings and presenting concerns among adults (Guo et al., 2015; Lambert et al., 2018; Shimokawa et al., 2010) and youth (Bickman et al., 2011; Parikh et al., 2020).”
McComb, J. L., Diamond, R. M., Breunlin, D. C., Chambers, A. L., & Murray, K. S. F. (2018) · Journal of Family Therapy · doi:10.1111/1467-6427.12222 · Cited by 9
An Exploration of Intervention Augmentation in a Single Case
Boswell, J. F., & Schwartzman, C. M. (2018) · Behavior Modification · doi:10.1177/0145445518796202 · Cited by 7
“…to therapists and/or clients, specifically for clients who are at risk for a negative outcome, seems effective for increasing the rate at which clients improve, enhancing outcomes, and reducing treatment failures in short-term therapies (De Jong et al., 2014; Shimokawa, Lambert, & Smart, 2010).”
BackgroundYoung, A. S., Cohen, A. N., Hamilton, A. B., Hellemann, G. S., Reist, C., & Whelan, F. (2018) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-018-9641-8 · Cited by 7
“Outcome feedback has been most effective when delivered to both clinician and patient, providing an opportunity to influence treatment.(22) It has not been clear how to broadly implement patient-reported outcomes and evidence-based practices for schizophrenia.”
BackgroundMethodBriscoe, H., Ashworth, S., & Shelton, L. (2019) · Advances in Mental Health and Intellectual Disabilities · doi:10.1108/amhid-02-2019-0005 · Cited by 8
Kelly, P. J., Beck, A. K., Baker, A., Deane, F. P., Hides, L. M., Manning, V., Shakeshaft, A. P., Larance, B., Neale, J., Kelly, J. F., et al. (2020) · JMIR Research Protocols · doi:10.2196/15113 · Cited by 8
“Evidence also supports clinician use of ROM and tailored feedback to enhance outcomes and/or prevent further deterioration for those clients identified as not on track early in addiction and/or mental health treatment [7,8].”
BackgroundImproving Measurement in Couple and Family Therapy: An Item Response Theory Primer
Anderson, S. R., & Miller, R. B. (2020) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12448 · Cited by 8
Rubel, J. A., & Lutz, W. (2017) · Springer eBooks · doi:10.1007/978-3-319-50675-3_13 · Cited by 9
Courtney, D., Cheung, A., Henderson, J. L., BENNETT, K. J., Battaglia, M., Strauss, J., Mitchell, R. H. B., Wang, K., & Szatmári, P. (2019) · PubMed · Cited by 8
The Use of Feedback in Mental Health Services: Expanding Horizons on Reach and Implementation
Connors, E. H., Janse, P. D., de Jong, K., & Bickman, L. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01426-7 · Cited by 8
Nordberg, S. S., Moltu, C., & Råbu, M. (2015) · Nordic Psychology · doi:10.1080/19012276.2015.1071204 · Cited by 6
“Similar feedback systems have demonstrated an ability to improve treatment outcomes for patients who are getting worse during the treatment (Shimokawa et al., 2010), perhaps because patients are able communicate to the therapist they need to change their approach.”
BackgroundSimon, W., S̀liwka, P., Sobański, J. A., Klasa, K., Sala, P., Żak, W., Busath, G. L., & Lambert, M. J. (2015) · Psychiatria Polska · doi:10.12740/pp/59064 · Cited by 7
“Meta/mega-analytic review of the OQ [8] confirms that the OQ based progress feedback with alert signals has a statistically and clinically significant effect on outcome in reducing deterioration rates in patients predicted to be treatment failures, i.”
Self-citationSchout, G., & de Jong, G. (2016) · Child & Family Social Work · doi:10.1111/cfs.12336 · Cited by 6
“There are significant improvements in the outcomes of treatment when therapists have access to formal, real-time feedback from clients on the process and outcomes of care (Duncan et al. 2004; Lambert & Shimokawa 2011; Reese et al. 2009; Shimokawa et al. 2010; Whipple & Lambert 2011).”
BackgroundThe Effect of Patient Progress Feedback on Psychotherapy Outcome
Schöttke, H., Unrath, M., & Uhlmann, C. (2019) · Verhaltenstherapie · doi:10.1159/000503765 · Cited by 8
“Several studies have already shown increased effectiveness with the use of “clinical tools” [Lambert, 2013; Shimokawa et al., 2010].”
BackgroundUckelstam, C.-J. (2021) · Linköping studies in behavioural science · doi:10.3384/diss.diva-172754 · Cited by 6
Lui, J. H. L., Brookman-Frazee, L. I., Smith, A. M., Lind, T., Terrones, L., Rodríguez, A., Motamedi, M., Villodas, M. T., & Lau, A. S. (2021) · Psychological Services · doi:10.1037/ser0000456 · Cited by 6
Burchett, D. L., Schneider, K. G., Bezdjian, S., & Baker, M. T. (2015) · Military Psychology · doi:10.1037/mil0000102 · Cited by 7
Child and Adolescent Psychotherapy
Kazdin, A. E. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-397045-9.00245-7 · Cited by 6
Peterson, A. P. (2015) · ResearchWorks at the University of Washington (University of Washington) · Cited by 6
Chronic pain management group psychotherapy for psychiatric inpatients: A pilot study
Osborne, P. J., Jimenez-Torres, G. J., Landa, Y., Mahoney, J., & Madan, A. (2017) · Bulletin of the Menninger Clinic · doi:10.1521/bumc.2017.81.2.123 · Cited by 5
Assessment and Treatment of Bipolar Disorder in the Community
Youngstrom, E. A. (2026) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-061724-084055 · Cited by 8
Chong, M. K., Hickie, I. B., Ekambareshwar, M., Poulsen, A., McKenna, S., Hutcheon, A., Hockey, S. J., Scott, E. M., & Iorfino, F. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12889-1 · Cited by 8
“In particular, it was especially effective for individuals who were “not on track”, as the regular monitoring and feedback loop allowed clinicians to identify the stagnant process early in treatment and rapidly adapt interventions accordingly [14, 15].”
Adventure Therapy Treatment for Young Adult Males Struggling with Addictions
Russell, K. C., Gillis, H. L. “., & Hayes, M. W. (2020) · Journal of Health Service Psychology · doi:10.1007/s42843-020-00003-4 · Cited by 5
Guérin, E., Garber, B. G., & Lee, J. E. C. (2021) · Journal of Military Veteran and Family Health · doi:10.3138/jmvfh-2020-0016 · Cited by 6
Hautmann, C., Rausch, J., Geldermann, N., Oswald, F., Gehlen, D., Hellmich, M., Rosenberger, K. D., Samel, C., Woitecki, K., Walter, D., et al. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03502-w · Cited by 5
“28 (or higher depending on the subsample) [21],”
BackgroundHogue, A., MacLean, A., Bobek, M., Dunnsue, S., Porter, N., Jensen‐Doss, A., & Henderson, C. E. (2023) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2023.2222405 · Cited by 6
“…focuses on client outcome data has led to gains in treatment outcomes and reduced treatment dropout across diverse samples (de Jong et al., 2021; Shimokawa et al., 2010); feedback attuned to service delivery characteristics could prompt similar gains in EBI use among cases being monitored…”
Data visualization for psychotherapy progress tracking
Koerner, K., Dailey, D., Lipp, M., Connor, H., & Sharma, R. (2012) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/2379057.2379099 · Cited by 5
“…how our multi-disciplinary team successfully combined agile methodology with participatory usercentered design to understand and address the needs of users who vary widely in their ability to use data visualizations and who make decisions under tight time constraints and high cognitiveloads.”
BackgroundThe Use of Feedback in Group Counseling in a State Vocational Rehabilitation Setting
Grizzell, S. T., Smart, J. F., Lambert, M. J., & Fargo, J. D. (2016) · Journal of Applied Rehabilitation Counseling · doi:10.1891/0047-2220.47.4.10 · Cited by 5
Self-citationSensitivity to Change of the Counseling Center Assessment of Psychological Symptoms-34
Youn, S. J., Castonguay, L. G., McAleavey, A. A., Nordberg, S. S., Hayes, J. A., & Locke, B. D. (2019) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2019.1691459 · Cited by 5
Tilden, T. (2020) · European Family Therapy Association series · doi:10.1007/978-3-030-36560-8_21 · Cited by 7
Implementing Routine Outcome Monitoring (ROM) in Clinical Practice
Lambert, M. J. (2017) · Journal of Health Service Psychology · doi:10.1007/bf03544650 · Cited by 5
Self-citationAlldredge, C. T., Burlingame, G. M., Olsen, J. A., & Epps, J. V. (2020) · Group Dynamics Theory Research and Practice · doi:10.1037/gdn0000121 · Cited by 5
Metadata
- Authors
- Shimokawa, K., Lambert, M. J., & Smart, D. W.
- Affiliations
- Brigham Young University (US)
- ORCID
- Lambert, Michael J.
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2010
- Volume
- 78
- Issue
- 3
- Pages
- 298-311
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2143726303
- Semantic Scholar
- 94c5e3fb3707d960610a79e604c2b070280c0014
- Cited by
- 657
- References
- 38
- Citations by year
- 2026: 18 · 2025: 23 · 2024: 64 · 2023: 24 · 2022: 37 · 2021: 57 · 2020: 47 · 2019: 47 · 2018: 59 · 2017: 54 · 2016: 45 · 2015: 61 · 2014: 44 · 2013: 31 · 2012: 30
- FWCI
- 26.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Brigham Young University
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Clinical Psychology › Counseling Practices and Supervision
- Social Sciences › Psychology › Clinical Psychology › Counseling, Therapy, and Family Dynamics
- Keywords
- treatment failure, early alert systems, progress feedback, patient outcomes, Outcome Questionnaire-45, clinical support tools, therapist feedback, counseling centers
- MeSH terms
- Adult, Mental Disorders, Feedback, Female, Humans, Male, Psychology, Clinical, Psychotherapy, Quality Assurance, Health Care, Surveys and Questionnaires, Referral and Consultation, Risk, Outcome Assessment, Health Care, Treatment Failure, Disease Progression, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.