Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system.
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Outcome research has documented worsening among a minority of the patient population (5% to 10%). In this study, we conducted a meta-analytic and mega-analytic review of a psychotherapy quality assurance system intended to enhance outcomes in patients at risk of treatment failure. METHOD: Original data from six major studies conducted at a large university counseling center and a hospital outpatient setting (N = 6,151, mean age = 23.3 years, female = 63.2%, Caucasian = 85%) were reanalyzed to examine the effects of progress feedback on patient outcome. In this quality assurance system, the Outcome Questionnaire-45 was routinely administered to patients to monitor their therapeutic progress and was utilized as part of an early alert system to identify patients at risk of treatment failure. Patient progress feedback based on this alert system was provided to clinicians so that they could intervene before treatment failure occurred. Meta-analytic and mega-analytic approaches were applied in intent-to-treat and efficacy analyses of the effects of feedback interventions. RESULTS: Three forms of feedback interventions-integral elements of this quality assurance system-were effective in enhancing treatment outcome, especially for signal alarm patients. Two of the three feedback interventions were also effective in preventing treatment failure (clinical support tools and the provision of patient progress feedback to therapists). CONCLUSIONS: The current state of evidence appears to support the efficacy and effectiveness of feedback interventions in enhancing treatment outcome.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“therapist supervision and feedback can improve patient outcomes, 52,53”
From Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder (2016) · cites this paper as [52]
In the citing paper
“Using MBC results in more clinically effective,17 cost-effective,18 and efficient treatment.19”
From Using Machine Learning to Match Clients and Therapy Providers: Evaluating Clinical Quality and Cost of Care (2025) · cites this paper as [19]
Citations
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 1×
“Following recommendations by Overton (1998) and Hedges and Vevea (1998), we used a random-effects model, given that the research on providing feedback for the purpose of enhancing treatment outcome is relatively new in psychotherapy outcome literature, that studies included in this meta-analysis contained slight varia…”
MethodBergin and Garfield's Handbook of Psychotherapy and Behavior Change
Lambert, M. J. (2003) · Cited by 3,383
Self-citationFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
The psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783 · cited 1×
“Similarly, an argument can be made against causal statements based on the data of studies not directly compared in the same randomized trials, such as in the case of comparing the CST Fb group and the pooled TAU group in the studies of Harmon et al. (2007) and Slade et al.”
BackgroundSelf-citationÆgisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., et al. (2006) · The Counseling Psychologist · doi:10.1177/0011000005285875 · Cited by 737 · cited 2×
“As has been repeatedly demonstrated in clinical research, any prediction relying on statistical or actuarial methods tends to fare better than clinical judgment alone (Ægisdóttir et al., 2006; Grove, 2005).”
BackgroundStatistical Considerations in the Intent-to-Treat Principle
Lachin, J. M. (2000) · Controlled Clinical Trials · doi:10.1016/s0197-2456(00)00046-5 · Cited by 652 · cited 1×
“The statistical methods used in the previous feedback studies reflect two distinct approaches: effectiveness analysis based on the intent-totreat (ITT) principle and efficacy analysis (Atkins, 2009; Lachin, 2000).”
MethodIs it time for clinicians to routinely track patient outcome? A meta-analysis
Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpg025 · Cited by 608 · cited 2×
“Contrary to the previous meta-analysis (Lambert et al., 2003), this study highlighted effects of feedback interventions on OT patients.”
ResultSelf-citationPatient-focused research: Using patient outcome data to enhance treatment effects.
Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.69.2.159 · Cited by 603
Self-citationEvaluation of psychotherapy: Efficacy, effectiveness, and patient progress.
Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996) · American Psychologist · doi:10.1037/0003-066x.51.10.1059 · Cited by 573
A lab test and algorithms for identifying clients at risk for treatment failure
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20108 · Cited by 542
Self-citationDeRubeis, R. J., Gelfand, L. A., Tang, T. Z., & Simons, A. D. (1999) · American Journal of Psychiatry · doi:10.1176/ajp.156.7.1007 · Cited by 448
Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001) · Psychotherapy Research · doi:10.1080/713663852 · Cited by 432
Self-citationWhipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003) · Journal of Counseling Psychology · doi:10.1037//0022-0167.50.1.59 · Cited by 374
Self-citationOverton, R. C. (1998) · Psychological Methods · doi:10.1037//1082-989x.3.3.354 · Cited by 356
Negative outcome in psychotherapy: A critical review.
Mohr, D. C. (1995) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.1995.tb00022.x · Cited by 330 · cited 3×
“Following recommendations by Overton (1998) and Hedges and Vevea (1998), we used a random-effects model, given that the research on providing feedback for the purpose of enhancing treatment outcome is relatively new in psychotherapy outcome literature, that studies included in this meta-analysis contained slight varia…”
BackgroundMethodHatfield, D., McCullough, L., Frantz, S. H. B., & Krieger, K. (2010) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.656 · Cited by 321
Enhancing psychotherapy outcomes via providing feedback on client progress: a replication
Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., & Goates, M. (2002) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.324 · Cited by 306
Self-citationHarmon, S. C., Lambert, M. J., Smart, D. M., Hawkins, E., Nielsen, S. L., Slade, K., & Lutz, W. (2007) · Psychotherapy Research · doi:10.1080/10503300600702331 · Cited by 268
Self-citationIntroduction and historical overview.
Quick, J. C., Camara, W. J., Hurrell, J. J., Jr., Johnson, J. V., Piotrkowski, C. S., Sauter, S. L., & Spielberger, C. D. (1997) · Journal of Occupational Health Psychology · doi:10.1037//1076-8998.2.1.3 · Cited by 241
The therapeutic effects of providing patient progress information to therapists and patients
Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., & Tuttle, K. C. (2004) · Psychotherapy Research · doi:10.1093/ptr/kph027 · Cited by 230 · cited 3×
“The primary reason for this heterogeneity was the significantly higher mean posttest and pretest scores of patients in Study 4 (Hawkins et al., 2004).”
MethodResultInfluentialSelf-citationWarren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0018544 · Cited by 188
Evidence-based psychotherapy: Where practice and research meet.
(2006) · American Psychological Association eBooks · doi:10.1037/11423-000 · Cited by 182
Slade, K., Lambert, M. J., Harmon, S. C., Smart, D. W., & Bailey, R. (2008) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.594 · Cited by 149
Self-citationOutcome Questionnaire: Is It Sensitive to Changes in Counseling Center Clients?
Vermeersch, D. A., Whipple, J. L., Lambert, M. J., Hawkins, E. J., Burchfield, C. M., & Okiishi, J. C. (2004) · Journal of Counseling Psychology · doi:10.1037/0022-0167.51.1.38 · Cited by 145
Self-citationFinch, A. E., Lambert, M. J., & Schaalje, B. G. (2001) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.286 · Cited by 143
Self-citationLambert, M. J., Whipple, J. L., Bishop, M. J., Vermeersch, D. A., Gray, G. V., & Finch, A. E. (2002) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.333 · Cited by 127
Self-citationA multiperspective, multivariable evaluation of reliable change.
Lunnen, K. M., & Ogles, B. M. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.66.2.400 · Cited by 124
Clinical versus statistical prediction: The contribution of Paul E. Meehl
Grove, W. M. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20179 · Cited by 99
Clinical significance of the Outcome Questionnaire (OQ-45.2).
Beckstead, D. J., Hatch, A. L., Lambert, M. J., Eggett, D. L., Goates, M. K., & Vermeersch, D. A. (2003) · The Behavior Analyst Today · doi:10.1037/h0100015 · Cited by 98
Self-citationA comparison of rational versus empirical methods in the prediction of psychotherapy outcome
Spielmans, G. I., Masters, K. S., & Lambert, M. J. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.491 · Cited by 49
Self-citationReed, G. M., & Eisman, E. J. (2007) · American Psychological Association eBooks · doi:10.1037/11423-001 · Cited by 42
Pooling pharmacogenetic studies on the serotonin transporter: A mega-analysis
Serretti, A., Cusin, C., Rausch, J. L., Bondy, B., & Smeraldi, E. (2006) · Psychiatry Research · doi:10.1016/j.psychres.2005.09.020 · Cited by 40
Clinical trials methodology: randomization, intent-to-treat, and random-effects regression
Atkins, D. C. (2009) · Depression and Anxiety · doi:10.1002/da.20594 · Cited by 26
Cited by
Implementing Measurement-Based Care in Behavioral Health: A Review
Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., et al. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.3329 · Cited by 605
“Measurement-based care can be separated into 4 core components: (1) a routinely administered symptom, outcome, or process measure (ie, PRO), ideally before each clinical encounter; (2) practitioner review of data; (3) patient review of data; and (4) collaborative reevaluation of the treatment plan informed by data.”
Parikh, S. V., Quilty, L. C., Ravitz, P., Rosenbluth, M., Pavlova, B., Grigoriadis, S., Velyvis, V., Kennedy, S. H., Lam, R. W., MacQueen, G. M., et al. (2016) · The Canadian Journal of Psychiatry · doi:10.1177/0706743716659418 · Cited by 562
A Tipping Point for Measurement-Based Care
Fortney, J. C., Unützer, J., Wrenn, G., Pyne, J. M., Smith, G. R., Schoenbaum, M., & Harbin, H. T. (2017) · Psychiatric Services · doi:10.1176/appi.ps.201500439 · Cited by 558
Lindahl, J. R., Fisher, N. E., Cooper, D. J., Rosen, R. K., & Britton, W. B. (2017) · PLOS ONE · doi:10.1371/journal.pone.0176239 · Cited by 476
Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions
Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2015) · Psychotherapy Research · doi:10.1080/10503307.2013.817696 · Cited by 494
“The meta-analysis (Shimokawa et al., 2010) involved both intent-to-treat (ITT) and efficacy analyses on the effects of various feedback interventions in relation to TAU (treatment without feedback) on clients who were predicted to have a negative outcome.”
MethodSelf-citation
Metadata
- Authors
- Shimokawa, K., Lambert, M. J., & Smart, D. W.
- Affiliations
- Brigham Young University (US)
- ORCID
- Lambert, Michael J.
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2010
- Volume
- 78
- Issue
- 3
- Pages
- 298-311
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2143726303
- Semantic Scholar
- 94c5e3fb3707d960610a79e604c2b070280c0014
- Cited by
- 657
- References
- 38
- Citations by year
- 2026: 18 · 2025: 23 · 2024: 64 · 2023: 24 · 2022: 37 · 2021: 57 · 2020: 47 · 2019: 47 · 2018: 59 · 2017: 54 · 2016: 45 · 2015: 61 · 2014: 44 · 2013: 31 · 2012: 30
- FWCI
- 26.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Brigham Young University
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Clinical Psychology › Counseling Practices and Supervision
- Social Sciences › Psychology › Clinical Psychology › Counseling, Therapy, and Family Dynamics
- Keywords
- treatment failure, early alert systems, progress feedback, patient outcomes, Outcome Questionnaire-45, clinical support tools, therapist feedback, counseling centers
- MeSH terms
- Adult, Mental Disorders, Feedback, Female, Humans, Male, Psychology, Clinical, Psychotherapy, Quality Assurance, Health Care, Surveys and Questionnaires, Referral and Consultation, Risk, Outcome Assessment, Health Care, Treatment Failure, Disease Progression, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.