Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system.
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Outcome research has documented worsening among a minority of the patient population (5% to 10%). In this study, we conducted a meta-analytic and mega-analytic review of a psychotherapy quality assurance system intended to enhance outcomes in patients at risk of treatment failure. METHOD: Original data from six major studies conducted at a large university counseling center and a hospital outpatient setting (N = 6,151, mean age = 23.3 years, female = 63.2%, Caucasian = 85%) were reanalyzed to examine the effects of progress feedback on patient outcome. In this quality assurance system, the Outcome Questionnaire-45 was routinely administered to patients to monitor their therapeutic progress and was utilized as part of an early alert system to identify patients at risk of treatment failure. Patient progress feedback based on this alert system was provided to clinicians so that they could intervene before treatment failure occurred. Meta-analytic and mega-analytic approaches were applied in intent-to-treat and efficacy analyses of the effects of feedback interventions. RESULTS: Three forms of feedback interventions-integral elements of this quality assurance system-were effective in enhancing treatment outcome, especially for signal alarm patients. Two of the three feedback interventions were also effective in preventing treatment failure (clinical support tools and the provision of patient progress feedback to therapists). CONCLUSIONS: The current state of evidence appears to support the efficacy and effectiveness of feedback interventions in enhancing treatment outcome.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“therapist supervision and feedback can improve patient outcomes, 52,53”
From Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder (2016) · cites this paper as [52]
In the citing paper
“Using MBC results in more clinically effective,17 cost-effective,18 and efficient treatment.19”
From Using Machine Learning to Match Clients and Therapy Providers: Evaluating Clinical Quality and Cost of Care (2025) · cites this paper as [19]
Citations
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 1×
“Following recommendations by Overton (1998) and Hedges and Vevea (1998), we used a random-effects model, given that the research on providing feedback for the purpose of enhancing treatment outcome is relatively new in psychotherapy outcome literature, that studies included in this meta-analysis contained slight varia…”
MethodBergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
Self-citationFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783 · cited 1×
“Similarly, an argument can be made against causal statements based on the data of studies not directly compared in the same randomized trials, such as in the case of comparing the CST Fb group and the pooled TAU group in the studies of Harmon et al. (2007) and Slade et al.”
BackgroundSelf-citationÆgisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., et al. (2006) · The Counseling Psychologist · doi:10.1177/0011000005285875 · Cited by 737 · cited 2×
“As has been repeatedly demonstrated in clinical research, any prediction relying on statistical or actuarial methods tends to fare better than clinical judgment alone (Ægisdóttir et al., 2006; Grove, 2005).”
Background
Cited by
Items 101–200 of 660
Probst, T., Lambert, M. J., Loew, T. H., Dahlbender, R. W., Göllner, R., & Tritt, K. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2013.07.003 · Cited by 66
“Providing patient progress feedback to therapists has been shown to improve the outcomeof patients at risk in counseling [19,20], outpatient psychotherapy [21], and psychiatric care [22].”
BackgroundSelf-citationWhipple, J. L., & Lambert, M. J. (2011) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-040510-143938 · Cited by 55
Self-citationReese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0036915 · Cited by 55
Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review
de Jong, K., Douglas, S. R., Wolpert, M., Delgadillo, J., Aas, B., Bovendeerd, B., Carlier, I. V. E., Compare, A., Edbrooke-Childs, J. H., Janse, P. D., et al. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01381-3 · Cited by 74
“Effect sizes tend to be stronger in NOT cases ( d = 0.17 to d = 0.53; see Table 1), and when CSTs are added for NOT cases ( d = 0.35 to 0.49; De Jong et al., 2021 ; Lambert et al., 2018 ).”
InfluentialTarescavage, A. M., & Ben-Porath, Y. S. (2014) · Journal of Clinical Psychology · doi:10.1002/jclp.22080 · Cited by 53
Douglas, S., Button, S., & Casey, S. E. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-014-0607-8 · Cited by 38
“MFSs have demonstrated effectiveness in improving outcomes in mental health care with adults (Amble et al. 2014; Harmon et al. 2007; Lambert et al. 2001/2005; Shimokawa et al. 2010), couples (Morten et al.”
Toward a paradigm shift in treatment and research of mental disorders
Leichsenring, F., Steinert, C., & Ioannidis, J. P. A. (2019) · Psychological Medicine · doi:10.1017/s0033291719002265 · Cited by 53
Sundet, R. (2012) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0027776 · Cited by 63
Davidson, K. M., Perry, A., & Bell, L. (2014) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12032 · Cited by 65
“Y ,o n ly if re su lt s st at is ti ca lly si gn ifi ca n t; N ,n o ;N R ,n o t re p o rt e d ;N A ,n o t ap p lic ab le ;N O T ,n o t o n tr ac k . feedback andCSTs the average reported effect size is .70 (Shimokawa et al., 2010).”
BackgroundMethodInfluentialStrauss, B. M., Lutz, W., Steffanowski, A., Wittmann, W. W., Boehnke, J. R., Rubel, J., Scheidt, C. E., Caspar, F., Vogel, H., Altmann, U., et al. (2015) · Psychotherapy Research · doi:10.1080/10503307.2013.856046 · Cited by 51
“In former studies, feedback was used to inform therapeutic interventions within the psychotherapeutic process in order to improve outcome (Lambert, Whipple, Smart, Vermeersch, & Nielsen, 2001; Shimokawa et al., 2010; Whipple et al. 2003).”
MethodMadson, M. B., Mohn, R. S., Zuckoff, A., Schumacher, J. A., Kogan, J. N., Hutchison, S. L., Magee, E., & Stein, B. D. (2012) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2012.08.015 · Cited by 71
Informant Reports in Clinical Assessment
Reyes, A. D. L., & Makol, B. A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00113-8 · Cited by 57
Measuring Psychiatric Symptoms Remotely: a Systematic Review of Remote Measurement-Based Care
Goldberg, S. B., Buck, B., Raphaely, S., & Fortney, J. C. (2018) · Current Psychiatry Reports · doi:10.1007/s11920-018-0958-z · Cited by 59
Bearman, S. K., & Weisz, J. R. (2015) · Child and Adolescent Mental Health · doi:10.1111/camh.12092 · Cited by 48
“In ameta-analysis of OQ studies, Shimokawa, Lambert, and Smart (2010) analyzed the combined data of six OQ feedback studies and demonstrated that the average at-risk client whose clinician received feedback was better off than approximately 70% of clients in the treatment as usual (TAU) condition…”
MethodToward Making Progress Feedback an Effective Common Factor in Couple Therapy
Halford, W. K., Hayes, S., Christensen, A., Lambert, M. J., Baucom, D. H., & Atkins, D. C. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.03.005 · Cited by 52
Self-citationSchiepek, G. K., Stöger‐Schmidinger, B., Kronberger, H., Aichhorn, W., Kratzer, L., Heinz, P., Viol, K., Lichtwarck‐Aschoff, A., & Schöller, H. J. (2019) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2384 · Cited by 59
Assessment issues in child and adolescent psychotherapy.
Reyes, A. D. L., Augenstein, T. M., & Aldao, A. (2018) · Cited by 45
Rubel, J., Lutz, W., Kopta, S. M., Köck, K., Minami, T., Zimmermann, D., & Saunders, S. M. (2015) · Psychological Assessment · doi:10.1037/pas0000060 · Cited by 45
Wittich, L., Tsatsaronis, C., Kuklinski, D., Schöner, L., Steinbeck, V., Busse, R., & Rombey, T. (2024) · Value in Health · doi:10.1016/j.jval.2024.05.013 · Cited by 54
Shernoff, E. S., Bearman, S. K., & Kratochwill, T. R. (2017) · School Psychology Review · doi:10.17105/spr-2015-0118.v46-2 · Cited by 47
Towards an evidence-based unified psychodynamic protocol for emotional disorders
Leichsenring, F., & Steinert, C. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.11.036 · Cited by 47
“…are (1) focusing on mentalization (Lemma et al., 2010), (2) focusing on patients at risk for non-response by giving feedback on patient progress (Lambert et al., 2002; Shimokawa et al., 2010), (3) maintenance treatment and (4) treatment options for non-responders (Leichsenring et al., 2015a).”
BackgroundMetz, M. J., Veerbeek, M., Twisk, J. W. R., Feltz-Cornelis, C. M. V. D., de Beurs, E., & Beekman, A. T. F. (2018) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-018-1589-8 · Cited by 43
Rubel, J., Lutz, W., & Schulte, D. (2015) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1868 · Cited by 42
“These clinical support tools have shown to be an effective means in reducing the number of treatment failures and show the practical importance of this kind of research (Lambert, 2007; Lambert & Shimokawa, 2011; Shimokawa et al., 2010; Whipple et al., 2003).”
MethodSmith, J. D., Eichler, W. C., Norman, K. R., & Smith, S. R. (2014) · Journal of Personality Assessment · doi:10.1080/00223891.2014.955917 · Cited by 54
The science of training III: Supervision, competency, and internship training.
Callahan, J. L., & Watkins, C. E., Jr. (2018) · Training and Education in Professional Psychology · doi:10.1037/tep0000208 · Cited by 48
Låver, J., McAleavey, A. A., Valaker, I., Castonguay, L. G., & Moltu, C. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2222896 · Cited by 51
“While some quantitative meta-analyses show positive effects (Lambert et al., 2018; Shimokawa et al., 2010), others are inconclusive (Kendrick et al., 2016).”
BackgroundDouglas, S. R., Jonghyuk, B., de Andrade, A. R. V., Tomlinson, M. M., Hargraves, R., & Bickman, L. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1059966 · Cited by 43
“…†Note: Susan Douglas has been previously published as Susan Douglas Kelley. supports the benefits of routine outcome monitoring and feedback (e.g., Shimokawa et al., 2010; Youn, Kraus, & Castonguay, 2012), albeit with some caveats because the challenges in implementation may be substantial.”
Backgroundde Jong, K., Segaar, J., Ingenhoven, T., van Busschbach, J., & Timman, R. (2018) · Journal of Personality Disorders · doi:10.1521/pedi_2017_31_297 · Cited by 42
Williams, N. J., Marcus, S. C., Ehrhart, M. G., Sklar, M., Esp, S. M., Carandang, K. M., Vega, N. R., Gomes, A. E., Brookman-Frazee, L. I., & Aarons, G. A. (2023) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2023.11.010 · Cited by 49
Börjesson, S., & Boström, P. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1630780 · Cited by 48
“, 2015), and to enhance treatment effectiveness by providing the clinician with information about individual clients’ treatment progress (Shimokawa et al., 2010).”
BackgroundInfluentialDiscussing measurement-based care with patients: An analysis of clinician-patient dyads
Holliday, S. B., Hepner, K. A., Farmer, C. M., Mahmud, A., Kimerling, R. E., Smith, B. N., & Rosen, C. S. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1776413 · Cited by 44
“, 2008) or detection of potential treatment failure early in psychotherapy (Shimokawa et al., 2010).”
BackgroundInfluentialMeasuring what matters to patients: Using goal content to inform measure choice and development
Jacob, J., Edbrooke-Childs, J. H., Law, D., & Wolpert, M. (2015) · Clinical Child Psychology and Psychiatry · doi:10.1177/1359104515615642 · Cited by 42
Emotional intelligence and psychotherapy outcomes in the training clinic.
Rieck, T. P., & Callahan, J. L. (2013) · Training and Education in Professional Psychology · doi:10.1037/a0031659 · Cited by 55
Brand, B., & Loewenstein, R. J. (2014) · Journal of Trauma & Dissociation · doi:10.1080/15299732.2013.828150 · Cited by 40
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39
The Expanding Relevance of Routinely Collected Outcome Data for Mental Health Care Decision Making
Boswell, J. F., Constantino, M. J., Kraus, D. R., Bugatti, M., & Oswald, J. M. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0649-6 · Cited by 43
Gual-Montolio, P., Martínez-Borba, V., Bretón-López, J. M., Osma, J., & Suso-Ribera, C. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17093170 · Cited by 38
“Overall, medium effect sizes have been reported when using MBC [1,14].”
BackgroundResultInfluentialBenchmarking Routine Psychological Services: A Discussion of Challenges and Methods
Delgadillo, J., McMillan, D., Leach, C., Lucock, M. P., Gilbody, S. M., & Wood, N. (2012) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s135246581200080x · Cited by 48
Timman, R., de Jong, K., & de Neve-Enthoven, N. G. M. (2015) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1979 · Cited by 43
“Feedback based on the OQ-45 supplemented with the use of clinical support tools (CSTs) has been shown to be especially effective in improving treatment outcomes (Shimokawa et al., 2010).”
MethodLutz, W., Zimmermann, D., Müller, V. N. L. S., Deisenhofer, A.-K., & Rubel, J. A. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1464-2 · Cited by 38
“More detailed information about the size of the effects derive from several meta-analyses [17, 18, 20].”
BackgroundCost-effectiveness of feedback-informed psychological treatment: Evidence from the IAPT-FIT trial
Delgadillo, J., McMillan, D., Gilbody, S., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Aguirre, E., & Ali, S. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.103873 · Cited by 41
de Jong, K., & de Goede, M. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1076198 · Cited by 38
de Jong, K. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-014-0589-6 · Cited by 47
Burlingame, G. M., Whitcomb, K. E., Woodland, S. C., Olsen, J. A., Beecher, M. E., & Gleave, R. (2018) · Psychotherapy · doi:10.1037/pst0000133 · Cited by 44
Spengler, P. M., & Pilipis, L. A. (2015) · Journal of Counseling Psychology · doi:10.1037/cou0000065 · Cited by 56
“In a metaanalysis of research from 1970 to 1996 of judgments made by 4,607 participants from 74 studies, Spengler, White, Ægisdóttir, Maugherman, Anderson, et al. (2009) found an experience-accuracy fixed effect of d .”
BackgroundHonkalampi, K. M., Urhonen, H.-R., & Virtanen, M. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2301972 · Cited by 44
“As reported by Shimokawa et al. (2010) in their meta-analysis of previous meta-analyses, feedback gives information on the client ’ s progress to both the therapist and the client.”
Pinsof, W. M., Goldsmith, J. Z., & Latta, T. A. (2012) · Couple and Family Psychology Research and Practice · doi:10.1037/a0031023 · Cited by 44
Görg, N., Priebe, K., Boehnke, J. R., Steil, R., Dyer, A., & Kleindienst, N. (2017) · Borderline Personality Disorder and Emotion Dysregulation · doi:10.1186/s40479-017-0065-5 · Cited by 54
“These data can identify patients at risk of treatment failure [69].”
BackgroundErrázuriz, P., & Zilcha-Mano, S. (2018) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000277 · Cited by 35
“And fourth, patients in our sample had lower psychological functioning at baseline as measured with the OQ than most previous studies (Shimokawa et al., 2010).”
BackgroundMethodResultInfluentialReyes, A. D. L., & Langer, D. A. (2018) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2018.1458314 · Cited by 39
Towards personalized allocation of patients to therapists.
Delgadillo, J., Rubel, J., & Barkham, M. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000507 · Cited by 36
Using the Assessment for Signal Clients as a feedback tool for reducing treatment failure
White, M. M., Lambert, M. J., Ogles, B. M., Mclaughlin, S. B., Bailey, R. J., & Tingey, K. M. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1009862 · Cited by 40
“Use of this intervention in controlled trials further reduced deterioration and bolstered positive treatment response (Shimokawa et al., 2010).”
MethodSelf-citationProbst, T., Lambert, M. J., Loew, T. H., Dahlbender, R. W., & Tritt, K. (2015) · Psychotherapy Research · doi:10.1080/10503307.2014.981682 · Cited by 35
Self-citationAchenbach, T. M. (2017) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2016.1220315 · Cited by 35
Arnon-Ribenfeld, N., Bloom, R., Atzil-Slonim, D., Peri, T., de Jong, S., & Hasson-Ohayon, I. (2018) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20180037 · Cited by 35
Tasca, G. A., Angus, L. E., Bonli, R., Drapeau, M., Fitzpatrick, M. R., Hunsley, J. D., & Knoll, M. L. (2019) · Canadian Psychology/Psychologie canadienne · doi:10.1037/cap0000181 · Cited by 40
Fornells‐Ambrojo, M., Johns, L., Onwumere, J., Garety, P. A., Milosh, C., Iredale, C., PETERS, E. R., Webster, A., & Jolley, S. (2017) · British Journal of Clinical Psychology · doi:10.1111/bjc.12136 · Cited by 36
The Art Therapy Working Alliance Inventory: the development of a measure
Or, M. B., & Zilcha‐Mano, S. (2018) · International Journal of Art Therapy · doi:10.1080/17454832.2018.1518989 · Cited by 34
From distal to proximal: routine educational data monitoring in school-based mental health
Lyon, A. R., Borntrager, C., Nakamura, B., & Higa-McMillan, C. (2013) · Advances in School Mental Health Promotion · doi:10.1080/1754730x.2013.832008 · Cited by 34
Kendrick, T., Dowrick, C. F., Lewis, G., Moore, M. V., Leydon, G., Geraghty, A. W. A., Griffiths, G. O., Zhu, S., Yao, G., May, C. R., et al. (2023) · Health Technology Assessment · doi:10.3310/plrq4216 · Cited by 39
Enhancing psychotherapy process with common factors feedback: A randomized, clinical trial.
McClintock, A. S., Perlman, M. R., McCarrick, S. M., Anderson, T., & Himawan, L. K. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000188 · Cited by 48
Restifo, E., Kashyap, S., Hooke, G. R., & Page, A. C. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1006707 · Cited by 34
Kendrick, T., Moore, M. V., Gilbody, S. M., Churchill, R. C., Stuart, B. L., & El-Gohary, M. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011119 · Cited by 35
Williams, N. J., Ehrhart, M. G., Aarons, G. A., Esp, S. M., Sklar, M., Carandang, K. M., Vega, N. R., Brookman-Frazee, L. I., & Marcus, S. C. (2024) · Implementation Science · doi:10.1186/s13012-024-01356-w · Cited by 41
Errors in treatment outcome monitoring: Implications for real-world psychotherapy.
McAleavey, A. A., Nordberg, S. S., Kraus, D. R., & Castonguay, L. G. (2012) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0027833 · Cited by 44
Challenges of using progress monitoring measures: Insights from practicing clinicians.
Ionita, G., Fitzpatrick, M. R., Tomaro, J., Chen, V. V., & Overington, L. (2015) · Journal of Counseling Psychology · doi:10.1037/cou0000122 · Cited by 37
A Tipping Point for Measurement-Based Care
Fortney, J. C., Unützer, J., Wrenn, G. L., Pyne, J. M., Smith, G. R., Schoenbaum, M., & Harbin, H. T. (2018) · FOCUS The Journal of Lifelong Learning in Psychiatry · doi:10.1176/appi.focus.16303 · Cited by 34
Boswell, J. F. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1686192 · Cited by 35
Janse, P. D., de Jong, K., Veerkamp, C., van Dijk, M. K., Hutschemaekers, G. J. M., & Verbraak, M. J. P. M. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000549 · Cited by 42
Probst, T., Lambert, M. J., Dahlbender, R. W., Loew, T. H., & Tritt, K. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.03.010 · Cited by 37
“While our non-significant feedback effects on the outcome of OT patients are in line with two previous RCTs [10,16] and a quasi-experimental study [17], they are in contrast to a quasi-experimental study [11] and a mega-analysis [18], that showed a significantly improved outcome for OT patients when therapists receive…”
MethodResultInfluentialSelf-citationA collaborative outcome resource network (ACORN): Tools for increasing the value of psychotherapy.
Brown, G. S. (., Simon, A., Cameron, J., & Minami, T. (2015) · Psychotherapy · doi:10.1037/pst0000033 · Cited by 34
Evidence-Based Practice: The Components, History, and Process
Yates, C. M. (2013) · Counseling Outcome Research and Evaluation · doi:10.1177/2150137812472193 · Cited by 45
Rye, M., Rognmo, K., Aarons, G. A., & Skre, I. (2019) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-019-00968-5 · Cited by 40
Youth Psychotherapy Change Trajectories and Early Warning System Accuracy in a Managed Care Setting
Nelson, P. L., Warren, J. S., Gleave, R. L., & Burlingame, G. M. (2013) · Journal of Clinical Psychology · doi:10.1002/jclp.21963 · Cited by 38
Russell, K. C., & Gillis, H. L. “. (2017) · Journal of Experiential Education · doi:10.1177/1053825917690541 · Cited by 38
“ROM is also an evidence-based approach, regardless of treatment type or model (see Shimokawa et al., 2010).”
BackgroundDecision making and the use of evidence-based practice: Is the three-legged stool balanced?
Stewart, R. E., Chambless, D. L., & Stirman, S. W. (2018) · Practice Innovations · doi:10.1037/pri0000063 · Cited by 41
Kivlighan, D. M., Hooley, I. W., Bruno, M. G., Ethington, L. L., Keeton, P. M., & Schreier, B. A. (2018) · Journal of Counseling Psychology · doi:10.1037/cou0000316 · Cited by 32
Sabbioni, D., Feehan, S., Nicholls, C., Soong, W., Rigoli, D., Follett, D., Carastathis, G., Gomes, A., Griffiths, J., Curtis, K., et al. (2018) · Early Intervention in Psychiatry · doi:10.1111/eip.12563 · Cited by 33
“Another advantage of these measures is that the SRS provides a quantitative measure of the relational bond which exists between client and practitioner (Bickman et al., 2011; Hall et al., 2013; Hansen, Howe, Sutton, & Ronan, 2015; Kwan & Rickwood, 2015; Shimokawa et al., 2010).”
BackgroundWert, M. J. V., Malik, M. A., Memel, B., Moore, R., Buccino, D. L., Hackerman, F., Kumari, S., Everett, A. S., & Narrow, W. E. (2020) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13457 · Cited by 33
Callahan, J. L., Gustafson, S. A., Misner, J. B., Paprocki, C. M., Sauer, E. M., Saules, K. K., Schwartz, J. L., Swift, J. K., Whiteside, D. M., Wierda, K., et al. (2014) · Training and Education in Professional Psychology · doi:10.1037/tep0000047 · Cited by 31
Moshe, I., Terhorst, Y., Paganini, S., Schlicker, S., Pulkki-Råback, L., Baumeister, H., Sander, L. B., & Ebert, D. D. (2022) · Journal of Medical Internet Research · doi:10.2196/38261 · Cited by 28
“…feedback technology that identifies individuals at risk of deterioration during treatment has been shown to improve eventual treatment outcomes [43,44], dropout warning systems could be used to alert the supporting clinician, guide care pathways (eg, in stepped-care models), or personalize the…”
Whitmyre, E. D., Esposito-Smythers, C. L., López, R., Goldberg, D. G., Liu, F. F., & Defayette, A. B. (2024) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-024-00498-z · Cited by 36
Murphy, K. P., Rashleigh, C. M., & Timulák, L. (2012) · Counselling Psychology Quarterly · doi:10.1080/09515070.2012.662349 · Cited by 35
“All of the above-mentioned studies were meta-analysed by Shimokawa, Lambert, and Smart (2010), who summarised that feedback improved outcomes for ‘noton-track’ clients when compared to no-feedback groups.”
ResultSteidtmann, D. K., Manber, R., Blasey, C. M., Markowitz, J. C., Klein, D. N., ROTHBAUM, B. O., Thase, M. E., Kocsis, J. H., & Arnow, B. A. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033250 · Cited by 29
Hansson, H., Rundberg, J., Österling, A., Öjehagen, A., & Berglund, M. (2012) · Nordic Journal of Psychiatry · doi:10.3109/08039488.2012.736534 · Cited by 35
“The key finding was a tendency to a larger improvement in the feedback group than the control group, indicating that the method is effective and basically supports the finding of Shimokawa et al. (2010) [8].”
BackgroundResultInfluentialDyer, K., Hooke, G. R., & Page, A. C. (2014) · Psychotherapy Research · doi:10.1080/10503307.2014.983207 · Cited by 32
“This finding is in line, though of smaller size than previous findings of medium-large effect sizes for feedback using composite indices (Shimokawa et al., 2010).”
ResultDevelopment and validation of MyLifeTracker: a routine outcome measure for youth mental health
Kwan, B., Rickwood, D. J., & Telford, N. R. (2018) · Psychology Research and Behavior Management · doi:10.2147/prbm.s152342 · Cited by 27
“11 ,23–25 This suggests that ROM feedback could be particularly beneficial for population groups who are difficult to engage in mental health services.”
BackgroundBecker, E. M., Becker, K. D., & Ginsburg, G. S. (2012) · School Mental Health · doi:10.1007/s12310-012-9080-2 · Cited by 35
Melchior, H., Schulz, H., Kriston, L., Hergert, A., Hofreuter-Gätgens, K., Bergelt, C., Morfeld, M., Koch, U., & Watzke, B. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.02.046 · Cited by 31
“The monitoring of and direct feedback about patients’ changes enables the therapists to react immediately and to modify treatment processes in the early phase of treatment, supports adaptive treatment planning during the course of treatment, and, as a result, enhances the likelihood of positive treatment outcomes (Lam…”
BackgroundRonk, F. R., Hooke, G. R., & Page, A. C. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0895-5 · Cited by 36
“Methods employed in the feedback literature [53–56] could then be used to predict which patients are “at-risk” of deteriorating, allowing clinicians to intervene during treatment.”
MethodThe science of training II: Prepracticum and practicum training.
Callahan, J. L., & Watkins, C. E., Jr. (2018) · Training and Education in Professional Psychology · doi:10.1037/tep0000209 · Cited by 33
Connors, E. H., Lyon, A. R., Garcia, K. A., Sichel, C. E., Hoover, S., Weist, M. D., & Tebes, J. K. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00319-w · Cited by 31
“70 are attributable to MBC with feedback, particularly feedback provided to both the patient and providers, or when clinical support tools are provided [21, 22].”
BackgroundThe Profession of Couple, Marital, and Family Therapy (CMFT): Defining Ourselves and Moving Forward
Wampler, K. S., Blow, A. J., McWey, L. M., Miller, R. B., & Wampler, R. S. (2017) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12294 · Cited by 29
van Sonsbeek, M. A. M. S., Hutschemaekers, G. J. M., Veerman, J. W., Vermulst, A. A., Kleinjan, M., & Tiemens, B. G. (2020) · Child & Youth Care Forum · doi:10.1007/s10566-020-09574-1 · Cited by 27
Schilling, V. N. L. S., Zimmermann, D., Rubel, J. A., Boyle, K. S., & Lutz, W. (2020) · Quality of Life Research · doi:10.1007/s11136-020-02664-6 · Cited by 27
“To further enhance these effects, CSTs are added to feedback [3, 14].”
BackgroundInfluentialTherapist–client agreement in assessments of clients’ functioning.
Bar‐Kalifa, E., Atzil–Slonim, D., Rafaeli, E., Peri, T., Rubel, J. A., & Lutz, W. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000157 · Cited by 31
Can Machine Learning Approaches Lead Toward Personalized Cognitive Training?
Shani, R., Tal, S., Zilcha‐Mano, S., & Okon‐Singer, H. (2019) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2019.00064 · Cited by 34
“Shimokawa et al. (2010) found that although some patients continue improving and benefitting from therapy (on-track patients—OT), others seem to deviate from this positive trajectory (not-on-track patients—”
BackgroundBruns, E. J., Hyde, K. L., Sather, A. K., Hook, A. N., & Lyon, A. R. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0658-5 · Cited by 26
“Research on MFS shows they can enhance communication and transparency (Carlier et al. 2010), strengthen client engagement (Shimokawa et al. 2010), and improve adult and youth outcomes independent of the specific treatment approach (Bickman et al. 2011; Lambert et al. 2003).”
BackgroundSembill, A., Vocks, S., Kosfelder, J., & Schöttke, H. (2019) · Psychotherapy Research · doi:10.1080/10503307.2017.1405170 · Cited by 24
Practical, Evidence-Based Clinical Decision Making: Introduction to the Special Series
Jensen‐Doss, A. (2014) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2014.08.001 · Cited by 27
Metadata
- Authors
- Shimokawa, K., Lambert, M. J., & Smart, D. W.
- Affiliations
- Brigham Young University (US)
- ORCID
- Lambert, Michael J.
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2010
- Volume
- 78
- Issue
- 3
- Pages
- 298-311
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2143726303
- Semantic Scholar
- 94c5e3fb3707d960610a79e604c2b070280c0014
- Cited by
- 657
- References
- 38
- Citations by year
- 2026: 18 · 2025: 23 · 2024: 64 · 2023: 24 · 2022: 37 · 2021: 57 · 2020: 47 · 2019: 47 · 2018: 59 · 2017: 54 · 2016: 45 · 2015: 61 · 2014: 44 · 2013: 31 · 2012: 30
- FWCI
- 26.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Brigham Young University
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Clinical Psychology › Counseling Practices and Supervision
- Social Sciences › Psychology › Clinical Psychology › Counseling, Therapy, and Family Dynamics
- Keywords
- treatment failure, early alert systems, progress feedback, patient outcomes, Outcome Questionnaire-45, clinical support tools, therapist feedback, counseling centers
- MeSH terms
- Adult, Mental Disorders, Feedback, Female, Humans, Male, Psychology, Clinical, Psychotherapy, Quality Assurance, Health Care, Surveys and Questionnaires, Referral and Consultation, Risk, Outcome Assessment, Health Care, Treatment Failure, Disease Progression, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.