Efficacy versus Effectiveness: A Direct Comparison of the Outcome of Treatment for Mild to Moderate Depression in Randomized Controlled Trials and Daily Practice
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Background: Results from randomized controlled trials (RCTs) are considered to give the most reliable information on treatment outcome (efficacy). Yet, the generalizability of efficacy results to daily practice (effectiveness) might be diminished by the design of RCTs. The STAR*D trial approached daily practice as much as possible, but still has some properties of an RCT. In this study, we compare results from treatment of major depressive disorder (MDD) in routine clinical practice to those of RCTs and STAR*D. Methods: Effectiveness in routine clinical practice was compared with efficacy results from 15 meta-analyses on antidepressant, psychotherapeutic and combination treatment and results from STAR*D. Data on daily practice patients and treatments were derived from a routine outcome monitoring (ROM) system. Treatment outcome was defined as proportion of remitters (MADRS ≤10) and within-group effect size. Results: From ROM, 598 patients suffering from a MDD episode according to the MINI-plus were included. Remission percentages were lower in routine practice than in meta-analyses for all treatment modalities (32 vs.40–74%). Differences were less explicit for antidepressants (21 vs. 34–47%) than for individual psychotherapy (27 vs. 34–58%; effect size 0.85 vs. 1.71) and combination therapy (21 vs. 45–63%), since only 60% of the meta-analyses for antidepressants showed significant differences with ROM, while for psychotherapy and combination treatment almost all meta-analyses showed significant differences. No differences in effectiveness were found between routine practice and STAR*D (antidepressants 27 vs. 28%; individual psychotherapy 27 vs. 25%; combination treatment 21 vs. 23%, respectively). Conclusions: Effectiveness of treatment for mild-to-moderate MDD in daily practice is similar to STAR*D and significantly lower than efficacy results from RCTs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“RCTs provide a highly structured environment, which is considered to possibly have an adherence-fostering effect [21,22].”
Citations
Cites
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
Practical Statistics for Medical Research.
Walter, S. D., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,668
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
Development of a Rating Scale for Primary Depressive Illness
HAMILTON, M. (1967) · British Journal of Social and Clinical Psychology · doi:10.1111/j.2044-8260.1967.tb00530.x · Cited by 8,410
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Trivedi, M. H., Rush, A. J., Wisniewski, S. R., Nierenberg, A. A., Warden, D., Ritz, L., Norquist, G., Howland, R. H., Lebowitz, B., McGrath, P. J., et al. (2006) · American Journal of Psychiatry · doi:10.1176/appi.ajp.163.1.28 · Cited by 3,879
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Tunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724
A COMPREHENSIVE PSYCHOPATHOLOGICAL RATING SCALE
Åsberg, M., Montgomery, S. A., Perris, C., Schalling, D., & Sedvall, G. (1978) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1978.tb02357.x · Cited by 1,326
Synthesizing standardized mean‐change measures
Becker, B. J. (1988) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1988.tb00901.x · Cited by 889
Remission rates during treatment with venlafaxine or selective serotonin reuptake inhibitors
Thase, M. E., Entsuah, A. R., & Rudolph, R. L. (2001) · British Journal of Psychiatry · doi:10.1192/bjp.178.3.234 · Cited by 801
Treatment of major depression with psychotherapy or psychotherapy-pharmacotherapy combinations
Thase, M. E. (1997) · Archives of General Psychiatry · doi:10.1001/archpsyc.1997.01830230043006 · Cited by 515
Wells, K. B. (1999) · American Journal of Psychiatry · doi:10.1176/ajp.156.1.5 · Cited by 390
Efficacy and effectiveness of antidepressants: current status of research
Pigott, H. E., Leventhal, A. M., Alter, G. S., & Boren, J. J. (2010) · Psychotherapy and Psychosomatics · doi:10.1159/000318293 · Cited by 347
Thase, M. E., Friedman, E. S., Biggs, M. M., Wisniewski, S. R., Trivedi, M. H., Luther, J. F., Fava, M., Nierenberg, A. A., McGrath, P. J., Warden, D., et al. (2007) · American Journal of Psychiatry · doi:10.1176/ajp.2007.164.5.739 · Cited by 332
Thase, M. E., Haight, B. R., Richard, N., Rockett, C. B., Mitton, M., Modell, J. G., VanMeter, S., Harriett, A. E., & Wang, Y. (2005) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v66n0803 · Cited by 304
Defining remission by cut off score on the MADRS: selecting the optimal value
Hawley, C. J., Gale, T. M., & Sivakumaran, T. (2002) · Journal of Affective Disorders · doi:10.1016/s0165-0327(01)00451-7 · Cited by 266
Entsuah, A. R., Huang, H., & Thase, M. E. (2001) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v62n1106 · Cited by 245
The Montgomery Äsberg and the Hamilton ratings of depression: A comparison of measures
Carmody, T. J., Rush, A. J., Bernstein, I., Warden, D., Brannan, S., Burnham, D., Woo, A., & Trivedi, M. H. (2006) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2006.04.008 · Cited by 245
Wisniewski, S. R., Rush, A. J., Nierenberg, A. A., Gaynes, B. N., Warden, D., Luther, J. F., McGrath, P. J., Lavori, P. W., Thase, M. E., Fava, M., et al. (2009) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2008.08071027 · Cited by 223
Bandelow, B., Baldwin, D. S., Dolberg, O. T., Andersen, H. F., & Stein, D. J. (2006) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v67n0914 · Cited by 203
de Beurs, E., den Hollander‐Gijsman, M. E., van Rood, Y. R., van der Wee, N. J. A., Giltay, E. J., van Noorden, M. S., van der Lem, R., van Fenema, E., & Zitman, F. G. (2011) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.696 · Cited by 195
Self-citationde Maat, S. M., Dekker, J., Schoevers, R. A., & de Jonghe, F. (2007) · European Psychiatry · doi:10.1016/j.eurpsy.2006.10.008 · Cited by 192
Bech, P., Cialdella, P., Haugh, M., Birkett, M. A., Hours, A. T., Boissel, J., & Tollefson, G. D. (2000) · The British Journal of Psychiatry · doi:10.1192/bjp.176.5.421 · Cited by 157
Comparative efficacy between venlafaxine and SSRIs: a pooled analysis of patients with depression
Stahl, S. M., Entsuah, R., & Rudolph, R. L. (2002) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01425-7 · Cited by 151
Davidson, J. R. T., Meoni, P., Haudiquet, V., Cantillon, M., & Hackett, D. (2002) · Depression and Anxiety · doi:10.1002/da.10045 · Cited by 144
Leonard, K. L. (2007) · Journal of Health Economics · doi:10.1016/j.jhealeco.2007.07.004 · Cited by 131
Nelson, J. C. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(99)00173-0 · Cited by 127
Defining Remission on the Montgomery-Asberg Depression Rating Scale
Zimmerman, M. A., Posternak, M. A., & Chelminski, I. (2004) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v65n0204 · Cited by 120
Benchmarks for psychotherapy efficacy in adult major depression.
Minami, T., Wampold, B. E., Serlin, R. C., Kircher, J. C., & Brown, G. S. (. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.2.232 · Cited by 113
Einarson, T. R., Arikian, S. R., Casciano, J. P., & Doyle, J. (1999) · Clinical Therapeutics · doi:10.1016/s0149-2918(00)88287-9 · Cited by 111
Assessing Clinically Significant Change: Application to the SCL-90–R
Schmitz, N., Hartkamp, N., & Franke, G. H. (2000) · Psychological Reports · doi:10.2466/pr0.2000.86.1.263 · Cited by 98
CRF/NPY interactions: A potential role in sleep dysregulation in depression and anxiety
Ehlers, C. L., Somes, C., Seifritz, E., & Rivier, J. E. (1997) · Depression and Anxiety · doi:10.1002/(sici)1520-6394(1997)6:1<40::aid-da6>3.0.co;2-9 · Cited by 91
New Modalities of Assessment and Treatment Planning in Depression: The Sequential Approach
Fava, G. A., & Tomba, E. (2010) · CNS Drugs · doi:10.2165/11531580-000000000-00000 · Cited by 80
Effects of psychological therapies in randomized trials and practice‐based studies
Barkham, M., Stiles, W. B., Connell, J., Twigg, E., Leach, C., Lucock, M. P., Mellor‐Clark, J., Bower, P., King, M., Shapiro, D. A., et al. (2008) · British Journal of Clinical Psychology · doi:10.1348/014466508x311713 · Cited by 80
Generalisability of results from randomised drug trials: A trial on antimanic treatment
Licht, R. W., Gouliaev, G., Vestergaard, P., & Frydenberg, M. (1997) · British Journal of Psychiatry · doi:10.1192/bjp.170.3.264 · Cited by 78
Short-term efficacy of tricyclic antidepressants revisited: a meta-analytic study
Storosum, J. G., Elferink, A. J. A., van Zwieten, B. J., van den Brink, W., Gersons, B. P. R., STRIK, R. V., & Broekmans, A. W. (2001) · European Neuropsychopharmacology · doi:10.1016/s0924-977x(01)00083-9 · Cited by 73
The Outpatient Treatment of Depression: Implications of Outcome Research for Clinical Practice
WEXLER, B. E., & CICCHETTI, D. V. (1992) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199205000-00001 · Cited by 72
Six-month compliance with antidepressant medication in the treatment of major depressive disorder
Demyttenaere, K., Adelin, A., Patrick, M., Walthère, D., Katrien, D. B., & Michèle, S. (2008) · International Clinical Psychopharmacology · doi:10.1097/yic.0b013e3282f1c1d8 · Cited by 65
The generalizability of antidepressant efficacy trials to routine psychiatric out-patient practice
van der Lem, R., van der Wee, N. J. A., van Veen, T., & Zitman, F. G. (2011) · Psychological Medicine · doi:10.1017/s0033291710002175 · Cited by 63
Self-citationBeasley, C. M., Nilsson, M. E., Koke, S. C., & Gonzales, J. S. (2000) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v61n1003 · Cited by 62
Bech, P. (2010) · Psychotherapy and Psychosomatics · doi:10.1159/000320118 · Cited by 56
Can mildly depressed outpatients with atypical depression benefit from antidepressants?
(1992) · American Journal of Psychiatry · doi:10.1176/ajp.149.5.615 · Cited by 50
Symptom Severity and Exclusion From Antidepressant Efficacy Trials
Zimmerman, M., Posternak, M. A., & Chelminski, I. (2002) · Journal of Clinical Psychopharmacology · doi:10.1097/00004714-200212000-00011 · Cited by 49
IJff, M. A., Huijbregts, K. M., van Marwijk, H. W., Beekman, A. T., Hakkaart-van Roijen, L., Rutten, F. F., Unützer, J., & van der Feltz-Cornelis, C. M. (2007) · BMC Health Services Research · doi:10.1186/1472-6963-7-34 · Cited by 45
Montgomery, S. (2001) · International Clinical Psychopharmacology · doi:10.1097/00004850-200105000-00006 · Cited by 42
Montgomery-Åsberg severity gradations
(1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.9.1320b · Cited by 39
Kasper, S. F., Zivkov, M., Roes, K. C. B., & Pols, A. G. (1997) · European Neuropsychopharmacology · doi:10.1016/s0924-977x(96)00394-x · Cited by 38
Randomized Controlled Trials in Psychiatry. Part II: Their Relationship to Clinical Practice
Mulder, R. T., Frampton, C., Joyce, P. R., & Porter, R. (2003) · Australian & New Zealand Journal of Psychiatry · doi:10.1046/j.1440-1614.2003.01176.x · Cited by 36
Depression rating scales can be related to each other by simple equations
Hawley, C. J., Gale, T. M., Smith, V. R. H., & Sen, P. (1998) · International Journal of Psychiatry in Clinical Practice · doi:10.3109/13651509809115359 · Cited by 21
Clinically significant change in psychotherapy for depressive disorders
Moleiro, C., & Beutler, L. E. (2008) · Journal of Affective Disorders · doi:10.1016/j.jad.2008.09.009 · Cited by 15
Routine Monitoring of Outcome over 11 Years in a Residential Behavioural Psychotherapy Unit
McKenzie, N., & Marks, I. M. (2003) · Psychotherapy and Psychosomatics · doi:10.1159/000070787 · Cited by 14
McKay, R., & McDonald, R. (2008) · Australasian Psychiatry · doi:10.1080/10398560802192270 · Cited by 11
Imaginative death experience in hypochondriasis
Praško, J. P., & Prašková, H. (2007) · European Psychiatry · doi:10.1016/j.eurpsy.2007.01.040 · Cited by 6
Research works printed but not linked: 4
- 10.1002%2F%28SICI%291520-6394%281997%296%3A1%3C10%3A%3AAID-DA2%3E3.0.CO%3B2-9 · DOI · Paper page
- 10.1176%2Fappi.ajp.164.5.739 · DOI
- 10.1037%2F%2F0022-006X.59.1.12 · doi:10.1037//0022-006x.59.1.12 · Paper page
- 10.2466%2FPR0.86.1.263-274 · DOI
Cited by
Pre-post effect sizes should be avoided in meta-analyses
Cuijpers, P., Weitz, E., Cristea, I. A., & Twisk, J. (2017) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000809 · Cited by 432
“For example, in some meta-analyses pre-post SMDs are used to compare improvement found in routine practice with improvement found in patients participating in randomised trials (van der Lem et al. 2012a, b).”
MethodEtzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356
Berking, M., Ebert, D., Cuijpers, P., & Hofmann, S. G. (2013) · Psychotherapy and Psychosomatics · doi:10.1159/000348448 · Cited by 328
More treatment but no less depression: The treatment-prevalence paradox
Ormel, J., Hollon, S. D., Kessler, R. C., Cuijpers, P., & Monroe, S. M. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102111 · Cited by 213
Wergeland, G. J., Riise, E. N., & Öst, L. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101918 · Cited by 149
Metadata
- Authors
- van der Lem, R., van der Wee, N. J. A., van Veen, T., & Zitman, F. G.
- Affiliations
- Leiden University Medical Center (NL)
- ORCID
- Zitman, Frans G.
- Venue
- Psychotherapy and Psychosomatics
- Publisher
- S. Karger AG
- Published
- 2012
- Volume
- 81
- Issue
- 4
- Pages
- 226-234
- ISSN
- 0033-3190 (ISSN-L)
- 1423-0348
- OpenAlex
- W1997227880
- Cited by
- 63
- References
- 55
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 4 · 2023: 1 · 2022: 1 · 2021: 7 · 2020: 5 · 2019: 1 · 2018: 1 · 2017: 6 · 2016: 3 · 2015: 8 · 2014: 8 · 2013: 13 · 2012: 3
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.karger.com/Services/SiteLicenses (vor)
- www.karger.com/Services/SiteLicenses (tdm)
- Locations
- Psychotherapy and Psychosomatics (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Health Sciences › Medicine › Pharmacology (medical) › Treatment of Major Depression
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- major depressive disorder, treatment efficacy, treatment effectiveness, STAR*D trial, routine clinical practice, antidepressants, individual psychotherapy, combination therapy, remission, generalizability of trial results, routine outcome monitoring
- MeSH terms
- Adult, Ambulatory Care, Antidepressive Agents, Combined Modality Therapy, Major Depressive Disorder, Depressive Disorder, Major, Female, Humans, Male, Middle Aged, Netherlands, Patient Compliance, Psychotherapy, Severity of Illness Index, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Treatment Outcome, Outcome Assessment, Health Care, Patient Selection, Evidence-Based Medicine, Guideline Adherence, Lost to Follow-Up
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.