Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system.
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Outcome research has documented worsening among a minority of the patient population (5% to 10%). In this study, we conducted a meta-analytic and mega-analytic review of a psychotherapy quality assurance system intended to enhance outcomes in patients at risk of treatment failure. METHOD: Original data from six major studies conducted at a large university counseling center and a hospital outpatient setting (N = 6,151, mean age = 23.3 years, female = 63.2%, Caucasian = 85%) were reanalyzed to examine the effects of progress feedback on patient outcome. In this quality assurance system, the Outcome Questionnaire-45 was routinely administered to patients to monitor their therapeutic progress and was utilized as part of an early alert system to identify patients at risk of treatment failure. Patient progress feedback based on this alert system was provided to clinicians so that they could intervene before treatment failure occurred. Meta-analytic and mega-analytic approaches were applied in intent-to-treat and efficacy analyses of the effects of feedback interventions. RESULTS: Three forms of feedback interventions-integral elements of this quality assurance system-were effective in enhancing treatment outcome, especially for signal alarm patients. Two of the three feedback interventions were also effective in preventing treatment failure (clinical support tools and the provision of patient progress feedback to therapists). CONCLUSIONS: The current state of evidence appears to support the efficacy and effectiveness of feedback interventions in enhancing treatment outcome.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“therapist supervision and feedback can improve patient outcomes, 52,53”
From Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder (2016) · cites this paper as [52]
In the citing paper
“Using MBC results in more clinically effective,17 cost-effective,18 and efficient treatment.19”
From Using Machine Learning to Match Clients and Therapy Providers: Evaluating Clinical Quality and Cost of Care (2025) · cites this paper as [19]
Citations
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 1×
“Following recommendations by Overton (1998) and Hedges and Vevea (1998), we used a random-effects model, given that the research on providing feedback for the purpose of enhancing treatment outcome is relatively new in psychotherapy outcome literature, that studies included in this meta-analysis contained slight varia…”
MethodBergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
Self-citationFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783 · cited 1×
“Similarly, an argument can be made against causal statements based on the data of studies not directly compared in the same randomized trials, such as in the case of comparing the CST Fb group and the pooled TAU group in the studies of Harmon et al. (2007) and Slade et al.”
BackgroundSelf-citationÆgisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., et al. (2006) · The Counseling Psychologist · doi:10.1177/0011000005285875 · Cited by 737 · cited 2×
“As has been repeatedly demonstrated in clinical research, any prediction relying on statistical or actuarial methods tends to fare better than clinical judgment alone (Ægisdóttir et al., 2006; Grove, 2005).”
Background
Cited by
Items 201–300 of 660
Schaffrath, J., Weinmann‐Lutz, B., & Lutz, W. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23362 · Cited by 24
Quick, A., Boehnke, J. R., Wright, J., & Pickett, K. E. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2016-7 · Cited by 24
“In these settings repeated assessments during the course of care are obtained from the individual patients and generally these assessments, if fed back to clinicians, have been shown to improve treatment outcomes especially for patients that were on a trajectory of stagnation or even deterioration [9, 10].”
BackgroundWarren, J. S., Nelson, P. L., Burlingame, G. M., & Mondragon, S. A. (2011) · Journal of Clinical Psychology · doi:10.1002/jclp.20831 · Cited by 32
Schueller, S. M., Kwasny, M. J., Dear, B. F., Titov, N., & Mohr, D. C. (2015) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.05.009 · Cited by 22
“In psychotherapy, when clinicians receive feedback corresponding to symptoms early during treatment, it can decrease the proportion of patients who reach the end of treatment without benefiting [10,11].”
BackgroundMackelprang, J. L., Hoffman, J. M., Garbaccio, C., & Bombardier, C. H. (2016) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2016.03.002 · Cited by 25
The predictive value of early response in patients with depressive disorders
Schlagert, H. S., & Hiller, W. (2016) · Psychotherapy Research · doi:10.1080/10503307.2015.1119329 · Cited by 26
“, Stiles et al., 2003). The present study provides strong evidence that early response is a good predictor for later treatment outcome regarding depressive symptoms, as well as overall psychological distress among patients with major depression or dysthymic disorder under naturalistic CBT. These results are in line wi…”
ResultTherapist–client relationships in a psychological therapy trial for psychosis and substance misuse
Berry, K., Gregg, L., Hartwell, R., Haddock, G., Fitzsimmons, M., & Barrowclough, C. (2015) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2015.04.006 · Cited by 26
“The nding that therapists do not accurately perceive client symptom hange is problematic for both therapist performance and client utcomes, potentially highlighting the merits of asking clients hemselves for regular feedback about the progress of psychotherpy in improving outcomes (Shimokawa et al., 2010).”
BackgroundKaiser, T., & Laireiter, A. (2017) · JMIR Medical Informatics · doi:10.2196/medinform.6808 · Cited by 24
“Figure 2 shows a graphical depiction of the model.”
InfluentialTrajectories of depression in psychotherapy: How client characteristics predict clinical improvement
Lin, T., & Farber, B. A. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23119 · Cited by 23
Organization-level variation in therapists’ attitudes toward and use of measurement-based care
Williams, N. J., Vega, N. R., Esp, S. M., Watts, A., & Marcus, S. C. (2022) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-022-01206-1 · Cited by 23
“Given that higher fidelity to MBC predicts improved clinical benefit from services (Fortney et al., 2017; Lambert, 2015; Lewis et al., 2019; Shimokawa et al., 2010), this finding also has implications for behavioral healthcare participants: the 1 3 937 Administration and Policy in Mental Health and…”
BackgroundInfluentialKazantzis, N., Cronin, T. J., Norton, P. J., Lai, J., & Hofmann, S. G. (2015) · Journal of Clinical Psychology · doi:10.1002/jclp.22178 · Cited by 21
Holliday, S. B., Hepner, K. A., Farmer, C. M., Ivany, C., Iyiewuare, P. O., McGee-Vincent, P. A., McCaslin, S. E., & Rosen, C. S. (2019) · Psychological Services · doi:10.1037/ser0000390 · Cited by 22
The need for evidence-based intake, progress, and outcomes assessment
Achenbach, T. M. (2016) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-016-0926-2 · Cited by 24
Miller, S. D., Hubble, M. A., & Chow, D. L. (2017) · Wiley eBooks · doi:10.1002/9781119165590.ch2 · Cited by 23
Jochems, E. C., Feltz-Cornelis, C. M. V. D., van Dam, A., Duivenvoorden, H. J., & Mulder, C. L. (2015) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s95190 · Cited by 21
The first session is the one that counts: An exploratory study of therapeutic alliance
Olvera, F. J. D. R., Rodríguez-Mora, Á., Senín‐Calderón, C., & Testal, J. F. R. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1016963 · Cited by 33
“Standardized tools can help professionals identify when the patients are not progressing in the therapy, and have been related with better results when the patient does not respond without them (Shimokawa et al., 2010).”
BackgroundMachine-Learning-Based Prediction of Client Distress From Session Recordings
Kuo, P. B., Tanana, M., Goldberg, S. B., Caperton, D. D., Narayanan, S. S., Atkins, D. C., & Imel, Z. E. (2023) · Clinical Psychological Science · doi:10.1177/21677026231172694 · Cited by 27
“In measurement-based care (MBC), therapists use client ratings to guide discussion regarding treat-ment goals and approach (Fortney et al., 2017; Goldberg et al., 2018; Lewis et al., 2019; Scott & Lewis, 2015; Shimokawa et al., 2010).”
MethodChallenges in the Implementation of Measurement Feedback Systems
de Jong, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0697-y · Cited by 22
Bryan, C. J., Blount, T. H., Kanzler, K. E., Morrow, C. E., Corso, K. A., Corso, M. L., & Ray-Sannerud, B. N. (2014) · Families Systems & Health · doi:10.1037/fsh0000014 · Cited by 24
Rober, P., Tricht, K. V., & Sundet, R. (2020) · Journal of Family Therapy · doi:10.1111/1467-6427.12292 · Cited by 21
Making the case for using personalised outcome measures to track progress in psychotherapy
Green, D. (2016) · European Journal of Psychotherapy & Counselling · doi:10.1080/13642537.2015.1130075 · Cited by 27
“…of meta-analyses and narrative literature reviews (Boswell, Kraus, Miller, & Lambert, 2013; Green & Latchford, 2012; Lambert & Shimokawa, 2011; Shimokawa et al., 2010) have presented convincing evidence that the regular collection of patient-recorded outcome measures (PrOMs) that are swiftly…”
BackgroundRussell, K. C., Gillis, H. L. “., Law, L. A., & Couillard, J. (2018) · Child & Youth Care Forum · doi:10.1007/s10566-018-9437-2 · Cited by 25
Connors, E. H., Arora, P. G., Resnick, S. G., & McKay, M. M. (2022) · Psychological Services · doi:10.1037/ser0000617 · Cited by 22
Prasad, N., Chien, I., Regan, T. W., Enrique, Á., Palacios, J. E., Keegan, D., Munir, U., Tanno, R., Richardson, H., Nori, A., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0272685 · Cited by 31
Wright, M., Galante, J., Corneille, J. S., Grabovac, A. D., Ingram, D. M., & Sacchet, M. D. (2024) · Mindfulness · doi:10.1007/s12671-024-02356-z · Cited by 29
“Yet, early identification of potentially challenging effects from EP is important for the mitigation of spiritual crisis (Shimokawa et al., 2010) and for the successful cognitive interpretation and integration of the EP (Parker, 2018).”
Early identification of treatment non-response utilizing the Patient Health Questionnaire (PHQ-9)
Fowler, J. C., Patriquin, M. A., Madan, A., Allen, J. G., Frueh, B. C., & Oldham, J. M. (2015) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2015.06.018 · Cited by 21
“…demonstrate improved outcomes for psychiatric patients (Howard et al., 1996; Whipple et al., 2003; Newnham et al., 2010; CritsCristoph et al., 2012; Shimokawa et al., 2010) patient-profiling allows Menninger practitioners to compare individual patient trajectories with the average treatment…”
BackgroundPractice‐Friendly Research Review: Collaboration in Routine Care
Lambert, M. J., & Cattani, K. (2012) · Journal of Clinical Psychology · doi:10.1002/jclp.21835 · Cited by 20
Self-citationEmotion suppression, not reappraisal, predicts psychotherapy outcome
Scherer, A., Boecker, M., Pawelzik, M. R., Gauggel, S., & Forkmann, T. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1080875 · Cited by 23
Attitudes of Austrian Psychotherapists Towards Process and Outcome Monitoring
Kaiser, T., Schmutzhart, L., & Laireiter, A. (2018) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-018-0862-1 · Cited by 22
“A lot of studies demonstrated that these monitoring approaches enhance treatment outcomes and prevent negative outcomes (Shimokawa et al. 2010).”
BackgroundPeterson, K., Anderson, J., & Bourne, D. S. (2018) · Europe PMC (PubMed Central) · Cited by 35
An exploration of faculty supervisor variables and their impact on client outcomes.
Wrape, E. R., Callahan, J. L., Ruggero, C. J., & Watkins, C. E., Jr. (2014) · Training and Education in Professional Psychology · doi:10.1037/tep0000014 · Cited by 22
Delgadillo, J., & Groom, M. (2017) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465816000643 · Cited by 22
“Future studies could combine role inductions plus troubleshooting strategies such as outcome tracking and feedback methods (Lambert et al., 2003; Shimokawa et al., 2010), or systematic clinical supervision of cases with risk-signals (e.”
BackgroundTherapeutic strategies and techniques in early cognitive-behavioral therapy.
King, B. R., & Boswell, J. F. (2019) · Psychotherapy · doi:10.1037/pst0000202 · Cited by 18
Connors, E., Lawson, G. M., Wheatley-Rowe, D., & Hoover, S. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01082-7 · Cited by 21
Bear, H. A., Dalzell, K., Edbrooke-Childs, J. H., & Wolpert, M. (2021) · British Journal of Clinical Psychology · doi:10.1111/bjc.12322 · Cited by 25
“…as well as for elevating the efficiency and quality of mental health care is well-documented in the research literature (Bickman, Kelley, Breda, de Andrade, et al., 2011; Connors et al., 2020; Delgadillo et al., 2018; Jensen-Doss et al., 2020; Lambert & Harmon, 2018; Shimokawa et al., 2010).”
InfluentialClients’ Hope for Counseling as a Predictor of Outcome in Psychotherapy
Bartholomew, T. T., Joy, E. E., & Gundel, B. E. (2021) · The Counseling Psychologist · doi:10.1177/00110000211033255 · Cited by 21
“Monitoring clients’ symptom distress contributes to therapeutic outcomes (i.e., symptom decrease; Lambert et al., 2002; Shimokawa et al., 2010), and trainees benefit from outcome monitoring as well (Minieri et al., 2015).”
BackgroundErekson, D. M., Bailey, R. J., Cattani, K., Fox, S. T., & Goates-Jones, M. K. (2020) · Counselling Psychology Quarterly · doi:10.1080/09515070.2020.1807914 · Cited by 19
“Shimokawa, Lambert, and Smart (2010), in a combined meta- and mega-analysis, found that clinicians who received feedback from routine outcome monitoring were likely to decrease the rate of clinically significant deterioration from approximately 20% to approximately 11%.”
BackgroundDo PROMS improve outcomes in patients with depression in primary care?
Kendrick, T., & Maund, E. (2020) · British medical journal · doi:10.1136/bmj.m3313 · Cited by 15
Margraf, J. (2009) · Springer eBooks · doi:10.1007/978-3-540-79541-4_17 · Cited by 20
Bradley, S. C., & Drapeau, M. (2014) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0036453 · Cited by 21
Does progress feedback enhance the outcome of group psychotherapy?
Vos, M. M. W. K.-D., Nugter, A., Engelsbel, F., & de Jong, K. (2018) · Psychotherapy · doi:10.1037/pst0000164 · Cited by 21
Kwan, B., Rickwood, D., & Brown, P. M. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1829738 · Cited by 23
“MFS are particularly useful for clients “not on track” (NOT) or who are at risk of treatment failure, with feedback systems significantly reducing deterioration rates and increasing rates of clinically significant improvement (Shimokawa et al., 2010).”
BackgroundMacrynikola, N., Chen, K., Lane, E., Nguyen, N. S., Pinto, J., Yen, S., & Torous, J. B. (2024) · JMIR Mental Health · doi:10.2196/65222 · Cited by 22
“In addition, measurement-based care is a well-established yet underutilized approach to increasing care quality [37-39].”
Mütze, K., Witthöft, M., Lutz, W., & Bräscher, A. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1930244 · Cited by 19
“Individually tailored treatment can prevent stagnation, deterioration, or premature dropout and increases the effectiveness of psychotherapy (Lambert & Shimokawa, 2011; Shimokawa et al., 2010).”
BackgroundSvendsen, T. S., Bjørnestad, J. R., Slyngstad, T. E., McKay, J. R., Skaalevik, A. W., Veseth, M., Moltu, C., & Nesvåg, S. M. (2020) · Substance Abuse Treatment Prevention and Policy · doi:10.1186/s13011-020-0254-x · Cited by 19
“Clinical feedback from patients to clinicians during treatment is shown to enhance collaboration and improve outcomes [16], especially for patients who deteriorate [17, 18] or who are not on track to recovery [19].”
BackgroundAnderson, S. R., Johnson, L. N., Miller, R. B., & Barham, C. C. (2021) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12541 · Cited by 24
Ryu, J., Heisig, S., McLaughlin, C., Katz, M., Mayberg, H. S., & Gu, X. (2023) · iScience · doi:10.1016/j.isci.2023.106860 · Cited by 25
“These drawbacks have limited the development of a scalable, real-time feedback system to treatment pairs that can improve clinical outcomes through the timely identification of negative session-level alliance.(3)”
BackgroundCrits-Christoph, P. F., Markell, H. M., Gallop, R., Gibbons, M. B. C., McClure, B., & Rotrosen, J. (2015) · Psychotherapy Research · doi:10.1080/10503307.2014.994146 · Cited by 16
“…the use of initial level to define normative recovery curves has worked exceptional well, as evidenced by results of feedback studies showing statistically significant and clinically meaningful benefits from feedback based on deviation from such normative recovery curves (Shimokawa et al., 2010).”
BackgroundHovland, R. T., & Moltu, C. (2019) · International Journal of Mental Health Systems · doi:10.1186/s13033-019-0324-5 · Cited by 19
Kelly, P. J., Beck, A. K., Deane, F. P., Larance, B., Baker, A., Hides, L. M., Manning, V., Shakeshaft, A. P., Neale, J., Kelly, J. F., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/25217 · Cited by 19
“A range of clinical benefits have been identified [7-9], particularly for those people identified as not on track early in the course of treatment [10,11].”
Working along sudden gains: Responsiveness on small and subtle early changes and exceptions.
Flückiger, C., Holtforth, M. G., Re, A. C. D., & Lutz, W. (2013) · Psychotherapy · doi:10.1037/a0031940 · Cited by 22
How and when feedback works in psychotherapy: Is it the signal?
Amble, I., Gude, T., Ulvenes, P., Stubdal, S., & Wampold, B. E. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1053552 · Cited by 19
“…the efficacy of FB by eliminating patients in the FB condition who do not have an OQ-45 score post signal, but failing to similarly trim the NFB condition, limits the generalizability of the outcome across all patients and biases the result in favor of the FB condition (Shimokawa et al., 2010).”
BackgroundResultInfluentialUsing Item and Test Information to Optimize Targeted Assessments of Psychological Distress
Boehnke, J. R., & Lutz, W. (2014) · Assessment · doi:10.1177/1073191114529152 · Cited by 16
Reese, R. J., Duncan, B. L., Kodet, J., Brown, H. M., Meiller, C., Farook, M. W., Lengerich, A. J., Vasilj, I., Hong, S.-H., & Bohanske, R. T. (2017) · Psychological Services · doi:10.1037/ser0000163 · Cited by 28
Developing a therapeutic relationship monitoring system for group treatment.
Janis, R. A., Burlingame, G. M., & Olsen, J. A. (2018) · Psychotherapy · doi:10.1037/pst0000139 · Cited by 22
LA BRECHA ENTRE LA INVESTIGACIÓN Y LA PRÁCTICA CLÍNICA: HACIA LA INTEGRACIÓN DE LA PSICOTERAPIA
Fernández‐Álvarez, J., Prado-Abril, J., Servicio Aragonés de Salud, I. A., Sánchez-Reales, S., de Salud, S. M., Molinari, G., III., A. A. C. F. O. Y. N. (. I. S. C., Penedo, J. M. G., Universidad de Buenos Aires. Consejo nacional de investigaciones científicas y técnicas, C., & Youn, S. J. (2020) · Papeles del Psicólogo - Psychologist Papers · doi:10.23923/pap.psicol2020.2932 · Cited by 28
Accurso, E. C., & Waller, G. (2020) · International Journal of Eating Disorders · doi:10.1002/eat.23449 · Cited by 19
“To this aim, ROM may be an important potential avenue to pursue.”
BackgroundInfluentialRoutine outcome measures in Germany
Puschner, B., Becker, T. M., & Bauer, S. (2015) · International Review of Psychiatry · doi:10.3109/09540261.2015.1014025 · Cited by 17
“It may not contribute to better outcome for all patients or for the ‘ average ’ patient, but the results of the study add to the ample evidence (Shimokawa et al., 2010) that feedback of ROM may help to direct the service providers ’ attention to patients who respond poorly and are at risk of…”
Reliable Change and Outcome Trajectories Across Levels of Care in a Mental Health System for Youth
Jackson, D. S., Keir, S. S., Sender, M., & Mueller, C. W. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0690-5 · Cited by 20
The Effect of Client Progress Feedback on Child/Adolescent’s Group-Counseling Outcomes
Shechtman, Z., & Sarig, O. (2016) · The Journal for Specialists in Group Work · doi:10.1080/01933922.2016.1232323 · Cited by 15
Götz, F. M., Stieger, S., & Reips, U. (2018) · Environment and Behavior · doi:10.1177/0013916518754610 · Cited by 19
Children Exposed to Intimate Partner Violence and the Reported Effects of Psychosocial Interventions
Grip, K. K., Almqvist, K., Axberg, U., & Broberg, A. G. (2013) · Violence and Victims · doi:10.1891/0886-6708.vv-d-12-00012 · Cited by 18
Clinical Versus Mechanical Prediction
Spengler, P. M. (2012) · Wiley eBooks · doi:10.1002/9781118133880.hop210002 · Cited by 15
Negative effects during multicomponent group-based treatment: A multisite study
Pourová, M., Řiháček, T., Chvála, Ľ., Vybíral, Z., & Boehnke, J. R. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2095237 · Cited by 15
“Thus, our findings suggest that symptomatic deterioration during treatment, on average, should be considered a detrimental effect that decreases patients’ likelihood of successful outcomes and may lead to demoralization (Kissane & Clark, 2002; Shimokawa et al., 2010; Strauss et al., 2015).”
BackgroundMirabella, M., Muzi, L., Franco, A., Urgese, A., Rugo, M. A., Mazzeschi, C., Speranza, A. M., McWilliams, N., & Lingiardi, V. (2023) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-023-01562-3 · Cited by 20
“Second, these approaches are not intended to serve as reliable assessment tools for regular, quantitative evaluations of symptomatic change throughout treatment (or measures at the initiation or termination of treatment); thus, they do not necessarily support treatment monitoring [32, 33].”
BackgroundUse of an Anecdotal Client Feedback Note in Family Therapy
Haber, R. A., Carlson, R. G., & Braga, C. (2014) · Family Process · doi:10.1111/famp.12070 · Cited by 15
Zilcha‐Mano, S., Muran, J. C., Eubanks, C. F., Safran, J. D., & Winston, A. (2018) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000293 · Cited by 32
Pardes, A. M., Lynch, W., Miclette, M., McGeoch, E., & Daly, B. P. (2021) · Innovations in Digital Health Diagnostics and Biomarkers · doi:10.36401/iddb-21-03 · Cited by 14
Minieri, A. M., Reese, R. J., Miserocchi, K. M., & Pascale-Hague, D. (2015) · Counselling Psychology Quarterly · doi:10.1080/09515070.2015.1055236 · Cited by 26
Nordberg, S. S., McAleavey, A. A., Duszak, E., Locke, B. D., Hayes, J. A., & Castonguay, L. G. (2018) · Counselling Psychology Quarterly · doi:10.1080/09515070.2016.1202809 · Cited by 13
The development of a reliable change index and cutoff score for the SCORE‐15
Miller, R. B., Nebeker-Adams, C. A., Anderson, S. R., Bradford, A. B., & Johnson, L. N. (2022) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12600 · Cited by 15
Sichel, C. E., & Connors, E. H. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00356-5 · Cited by 18
Barber, J., Childs, A. W., Resnick, S. G., & Connors, E. H. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01364-4 · Cited by 13
“Initial MBC meta-analyses suggested effect sizes on patient outcomes between 0.28 and 0.70 (Knaup et al., 2009; Lambert et al., 2018; Shimokawa et al., 2010), but more recent meta-analyses including studies outside of the United States and with a wider array of populations and settings found small…”
InfluentialSmith, R. E., Fagan, C., Wilson, N. L., Chen, J. A., Corona, M., Nguyen, H. V., Racz, S. J., & Shoda, Y. (2011) · Professional Psychology Research and Practice · doi:10.1037/a0025392 · Cited by 19
Kendrick, T., Moore, M. V., Leydon, G., Stuart, B. L., Geraghty, A. W. A., Yao, G., Lewis, G., Griffiths, G. O., May, C. R., Dewar-Haggart, R., et al. (2020) · Trials · doi:10.1186/s13063-020-04344-9 · Cited by 14
Behavioral Intervention Fidelity in Routine Practice: Pragmatism Moves to Head of the Class
Hogue, A. (2022) · School Mental Health · doi:10.1007/s12310-021-09488-w · Cited by 14
Andersson, C. R., Öjehagen, A., Olsson, M. O., Brådvik, L., & Håkansson, A. (2016) · International Journal of Behavioral Medicine · doi:10.1007/s12529-016-9625-0 · Cited by 16
“health and substance use patients with systematic and continuous feedback during treatment is known to result in a positive development over time [30, 31].”
BackgroundOudejans, S. C. C., Oene, G. H. D. W.-V., Spits, M., de Wildt, W., Merkx, M. J. M., Dekker, J., Visch, I., & Goudriaan, A. E. (2019) · European Addiction Research · doi:10.1159/000503625 · Cited by 17
Carr, M. M., Saules, K. K., Koch, E. I., & Waltz, T. J. (2016) · Training and Education in Professional Psychology · doi:10.1037/tep0000135 · Cited by 14
Rognstad, K., Helland, S. S., Neumer, S., Baardstu, S., & Kjøbli, J. (2022) · BMC Psychology · doi:10.1186/s40359-022-00894-6 · Cited by 17
Fitzpatrick, M. R. (2012) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0028051 · Cited by 18
Oanes, C. J., Borg, M., & Karlsson, B. (2015) · Australian and New Zealand Journal of Family Therapy · doi:10.1002/anzf.1110 · Cited by 14
Rubel, J. A., Zimmermann, D., Deisenhofer, A., Müller, V., & Lutz, W. (2017) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000413 · Cited by 13
Schielke, H. J., Brand, B. L., & Marsic, A. (2017) · European Journal of Psychotraumatology · doi:10.1080/20008198.2017.1380471 · Cited by 12
Hogue, A., MacLean, A., Bobek, M., Porter, N., Bruynesteyn, L., Jensen‐Doss, A., & Henderson, C. E. (2022) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2022.2051529 · Cited by 13
“Measurement feedback has led to impressive gains in treatment outcomes across diverse clinical samples (see Shimokawa et al., 2010), a success that has generated enthusiasm about the value of developing complementary procedures for EBI implementation.”
BackgroundVos, M. M. W. K.-D., van Dijk, M., Tiemens, B. G., de Jong, K., Witteman, C. L. M., & Nugter, A. (2022) · International Journal of Group Psychotherapy · doi:10.1080/00207284.2022.2086557 · Cited by 15
“therapist and patient receive feedback, compared to solely informing the therapist, and it increases even more when therapists receive additional treatment suggestions in case of deterioration (Shimokawa et al., 2010).”
BackgroundNew Frontiers in the Treatment of Couples
Halford, W. K., & Doss, B. D. (2016) · International Journal of Cognitive Therapy · doi:10.1521/ijct.2016.9.2.124 · Cited by 12
Patients' perspectives on care management services for complex substance use disorders
Hawkins, E. J., Lott, A. M. K., Malte, C. A., Frank, A. N., Hamilton, B., Sayre, G. G., Painter, J. M., & Baer, J. S. (2017) · Journal of Addictive Diseases · doi:10.1080/10550887.2017.1326654 · Cited by 14
“The next stage of treatment addressed self-management of substance use and psychiatric disorders, encouraged use of evidencedbased alcohol and/or opioid pharmacotherapies (disulfiram, oral and extended release naltrexone, acamprosate, or buprenorphine),(40) and educated patients’ about medications and medication compl…”
Backgroundde Jong, R., Snoek, H., Staal, W., & Klip, H. (2018) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-018-1247-4 · Cited by 13
“…rating bias, we decided to use two outcome measures which (1) differ from the feedback measures collected by the therapist during the treatment (see also Shimokawa 2010 [6]) and (2) are also assessed by other people (in this case the parents) than the therapists (see also Kendrick 2016 [23]).”
MethodHutson, J., Hooke, G. R., & Page, A. C. (2020) · Psychotherapy Research · doi:10.1080/10503307.2019.1710305 · Cited by 13
Espel‐Huynh, H., Thompson‐Brenner, H., Boswell, J. F., Zhang, F., Juarascio, A. S., & Lowe, M. R. (2020) · European Eating Disorders Review · doi:10.1002/erv.2718 · Cited by 15
“Robust evidence supports their utility in improving outcomes in general outpatient psychotherapy settings (Lambert & Harmon, 2018; Shimokawa et al., 2010).”
BackgroundProbst, T., Kleinstäuber, M., Lambert, M. J., Tritt, K., Pieh, C., Loew, T. H., Dahlbender, R. W., & Delgadillo, J. (2020) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2441 · Cited by 13
Self-citationA routine outcome measure for youth mental health: Clinically interpreting MyLifeTracker
Kwan, B., & Rickwood, D. (2020) · Early Intervention in Psychiatry · doi:10.1111/eip.13016 · Cited by 15
Hogue, A., Dauber, S., Bobek, M., Jensen-Doss, A., & Henderson, C. E. (2019) · Implementation Science · doi:10.1186/s13012-019-0874-6 · Cited by 11
“MFS has led to impressive gains in treatment outcomes across diverse clinical samples [32], including substance users (e.”
BackgroundSimple methods for enhancing patient outcome in routine care: Measuring, monitoring, and feedback
Lambert, M. J., & Coco, G. L. (2014) · Research in Psychotherapy Psychopathology Process and Outcome · doi:10.4081/ripppo.2013.147 · Cited by 15
“The meta-analysis (Shimokawa et al., 2010) involved both intent-to-treat (ITT) and efficacy analyses on the effects of various feedback interventions in relation to TAU.”
MethodSelf-citationUtprøving av systematisk tilbakemelding i par- og familieterapi - Barrierer og utfordringer
Tilden, T., Håland, Å. T., Hunnes, K., Fossli, G., & Oanes, C. J. (2015) · Fokus på familien · doi:10.18261/issn0807-7487-2015-04-03 · Cited by 11
Postmodern‐oriented practices and patient‐focused research: possibilities and hazards
Sundet, R. (2012) · Australian and New Zealand Journal of Family Therapy · doi:10.1017/aft.2012.38 · Cited by 15
Perspectives of internship training directors on the use of progress monitoring measures.
Overington, L., Fitzpatrick, M. R., Drapeau, M., & Hunsley, J. D. (2016) · Canadian Psychology/Psychologie canadienne · doi:10.1037/cap0000051 · Cited by 13
Metadata
- Authors
- Shimokawa, K., Lambert, M. J., & Smart, D. W.
- Affiliations
- Brigham Young University (US)
- ORCID
- Lambert, Michael J.
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2010
- Volume
- 78
- Issue
- 3
- Pages
- 298-311
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2143726303
- Semantic Scholar
- 94c5e3fb3707d960610a79e604c2b070280c0014
- Cited by
- 657
- References
- 38
- Citations by year
- 2026: 18 · 2025: 23 · 2024: 64 · 2023: 24 · 2022: 37 · 2021: 57 · 2020: 47 · 2019: 47 · 2018: 59 · 2017: 54 · 2016: 45 · 2015: 61 · 2014: 44 · 2013: 31 · 2012: 30
- FWCI
- 26.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Brigham Young University
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Clinical Psychology › Counseling Practices and Supervision
- Social Sciences › Psychology › Clinical Psychology › Counseling, Therapy, and Family Dynamics
- Keywords
- treatment failure, early alert systems, progress feedback, patient outcomes, Outcome Questionnaire-45, clinical support tools, therapist feedback, counseling centers
- MeSH terms
- Adult, Mental Disorders, Feedback, Female, Humans, Male, Psychology, Clinical, Psychotherapy, Quality Assurance, Health Care, Surveys and Questionnaires, Referral and Consultation, Risk, Outcome Assessment, Health Care, Treatment Failure, Disease Progression, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.