Efficacy versus Effectiveness: A Direct Comparison of the Outcome of Treatment for Mild to Moderate Depression in Randomized Controlled Trials and Daily Practice
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Background: Results from randomized controlled trials (RCTs) are considered to give the most reliable information on treatment outcome (efficacy). Yet, the generalizability of efficacy results to daily practice (effectiveness) might be diminished by the design of RCTs. The STAR*D trial approached daily practice as much as possible, but still has some properties of an RCT. In this study, we compare results from treatment of major depressive disorder (MDD) in routine clinical practice to those of RCTs and STAR*D. Methods: Effectiveness in routine clinical practice was compared with efficacy results from 15 meta-analyses on antidepressant, psychotherapeutic and combination treatment and results from STAR*D. Data on daily practice patients and treatments were derived from a routine outcome monitoring (ROM) system. Treatment outcome was defined as proportion of remitters (MADRS ≤10) and within-group effect size. Results: From ROM, 598 patients suffering from a MDD episode according to the MINI-plus were included. Remission percentages were lower in routine practice than in meta-analyses for all treatment modalities (32 vs.40–74%). Differences were less explicit for antidepressants (21 vs. 34–47%) than for individual psychotherapy (27 vs. 34–58%; effect size 0.85 vs. 1.71) and combination therapy (21 vs. 45–63%), since only 60% of the meta-analyses for antidepressants showed significant differences with ROM, while for psychotherapy and combination treatment almost all meta-analyses showed significant differences. No differences in effectiveness were found between routine practice and STAR*D (antidepressants 27 vs. 28%; individual psychotherapy 27 vs. 25%; combination treatment 21 vs. 23%, respectively). Conclusions: Effectiveness of treatment for mild-to-moderate MDD in daily practice is similar to STAR*D and significantly lower than efficacy results from RCTs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“RCTs provide a highly structured environment, which is considered to possibly have an adherence-fostering effect [21,22].”
Citations
Cites
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
Practical Statistics for Medical Research.
Walter, S. D., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,668
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
Development of a Rating Scale for Primary Depressive Illness
HAMILTON, M. (1967) · British Journal of Social and Clinical Psychology · doi:10.1111/j.2044-8260.1967.tb00530.x · Cited by 8,410
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Trivedi, M. H., Rush, A. J., Wisniewski, S. R., Nierenberg, A. A., Warden, D., Ritz, L., Norquist, G., Howland, R. H., Lebowitz, B., McGrath, P. J., et al. (2006) · American Journal of Psychiatry · doi:10.1176/appi.ajp.163.1.28 · Cited by 3,879
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Tunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724
A COMPREHENSIVE PSYCHOPATHOLOGICAL RATING SCALE
Åsberg, M., Montgomery, S. A., Perris, C., Schalling, D., & Sedvall, G. (1978) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1978.tb02357.x · Cited by 1,326
Research works printed but not linked: 4
- 10.1002%2F%28SICI%291520-6394%281997%296%3A1%3C10%3A%3AAID-DA2%3E3.0.CO%3B2-9 · DOI · Paper page
- 10.1176%2Fappi.ajp.164.5.739 · DOI · Paper page
- 10.1037%2F%2F0022-006X.59.1.12 · doi:10.1037//0022-006x.59.1.12 · Paper page
- 10.2466%2FPR0.86.1.263-274 · DOI
Cited by
Pre-post effect sizes should be avoided in meta-analyses
Cuijpers, P., Weitz, E., Cristea, I. A., & Twisk, J. (2017) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000809 · Cited by 432
“For example, in some meta-analyses pre-post SMDs are used to compare improvement found in routine practice with improvement found in patients participating in randomised trials (van der Lem et al. 2012a, b).”
MethodEtzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356
Berking, M., Ebert, D., Cuijpers, P., & Hofmann, S. G. (2013) · Psychotherapy and Psychosomatics · doi:10.1159/000348448 · Cited by 328
More treatment but no less depression: The treatment-prevalence paradox
Ormel, J., Hollon, S. D., Kessler, R. C., Cuijpers, P., & Monroe, S. M. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102111 · Cited by 213
Wergeland, G. J., Riise, E. N., & Öst, L. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101918 · Cited by 149
McIntyre, R. S., Rodrigues, N. B., Lee, Y., Lipsitz, O., Subramaniapillai, M., Gill, H., Nasri, F., Majeed, A., Lui, L. M. W., Senyk, O., et al. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.05.088 · Cited by 110
Mojtabai, R. (2013) · Psychotherapy and Psychosomatics · doi:10.1159/000345968 · Cited by 101
Ebert, D., Tarnowski, T., Gollwitzer, M., Sieland, B., & Berking, M. (2013) · Psychotherapy and Psychosomatics · doi:10.1159/000345967 · Cited by 93
Prognostic Subgroups for Citalopram Response in the STAR*D Trial
Jakubovski, E., & Bloch, M. H. (2014) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.13m08727 · Cited by 85
Wiersma, J. E., Van Schaik, D. J. F., Hoogendorn, A. W., Dekker, J. J., Van, H. L., Schoevers, R. A., Blom, M. B. J., Maas, K., Smit, J. H., McCullough Jr., J. P., et al. (2014) · Psychotherapy and Psychosomatics · doi:10.1159/000360795 · Cited by 79
Effectiveness of cognitive behavioral group therapy for depression in routine practice
Thimm, J. C., & Antonsen, L. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0292-x · Cited by 78
“The lower effect sizes found in this study are in accordance with previous studies showing significantly lower effect sizes for treatment of depression in routine care settings compared to research trials [24,52].”
Resultvan der Schaft, J., Politiek, K., van den Reek, J. M. P. A., Christoffers, W. A., Kievit, W., de Jong, E. M. G. J., Bruijnzeel-Koomen, C. A. F. M., Schuttelaar, M. L. A., & de Bruin-Weller, M. S. (2015) · British Journal of Dermatology · doi:10.1111/bjd.13730 · Cited by 78
van der Lem, R., de Wever, W. W., van der Wee, N. J., van Veen, T., Cuijpers, P., & Zitman, F. G. (2012) · BMC Psychiatry · doi:10.1186/1471-244x-12-192 · Cited by 37
Self-citationde Gee, E. A., Verdurmen, J. E. E., Bransen, E., de Jonge, J. M., & Schippers, G. M. (2014) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2014.05.001 · Cited by 42
Wiegand, H. F., Sievers, C., Schillinger, M., & Godemann, F. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.09.013 · Cited by 33
Koeser, L., Donisi, V., Goldberg, D. P., & McCrone, P. (2015) · Psychological Medicine · doi:10.1017/s0033291715000951 · Cited by 35
“Yet, this also represents a limitation of this study because the generalizability of RCTs in depression to routine clinical practice has been questioned (van der Lem et al. 2012).”
BackgroundAllain, N., Leven, C., Falissard, B., Allain, J., Batail, J.-M. V., Polard, É., Montastruc, F., Drapier, D., & Naudet, F. (2016) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12672 · Cited by 33
“On the other hand, previous work (28, 29) suggested that RCTs could overestimate the beneficial effects of antidepressants in major depressive disorder.”
BackgroundTully, P. J., Wittert, G., Selkow, T., & Baumeister, H. (2014) · PLoS ONE · doi:10.1371/journal.pone.0085928 · Cited by 35
Hoenders, R. H. J., Bos, E. H., Bartels‐Velthuis, A. A., Vollbehr, N. K., der Ploeg, K. M. V., de Jonge, P., & de Jong, J. T. V. M. (2013) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0511-7 · Cited by 30
“We did this because we observed this practice in studies from leading ROM authors in the Netherlands (Van der Lem et al., 2012; Van Noorden et al., 2012).”
BackgroundResultGeurts, D. E. M., Haegens, N. M., Van Beek, M. H. C. T., Schroevers, M. J., Compen, F. R., & Speckens, A. E. M. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.07.043 · Cited by 26
Psychodynamic interventions in cancer care I: psychometric results of a randomized controlled trial
Ludwig, G., Krenz, S., Zdrojewski, C., Bot, M., Rousselle, I., Stagno, D., Luethi, F., Leyvraz, S., & Stiefel, F. (2013) · Psycho-Oncology · doi:10.1002/pon.3374 · Cited by 25
“This is, to the best of our knowledge, the first research and treatment evaluating psychodynamic interventions in the oncology setting; a strength of the design is that the study aimed to assess effectiveness and not efficacy, approaching as much as possible the everyday clinical reality [75].”
BackgroundThe antidepressant standoff: Why it continues and how to resolve it
Ormel, J., Spinhoven, P., de Vries, Y. A., Cramer, A. O. J., Siegle, G. J., Bockting, C. L. H., & Hollon, S. D. (2020) · Psychological Medicine · doi:10.1017/s0033291719003295 · Cited by 22
Forbes, L. J. L., Forster, A. S., Dodd, R. H., Tucker, L., Laming, R., Sellars, S., Patnick, J., & Ramirez, A. J. (2012) · Journal of Cancer Epidemiology · doi:10.1155/2012/835167 · Cited by 20
“We were surprised at this finding: interventions, whether pharmacological or complex, are often less potent in routine clinical practice than in randomised controlled trials [10–12].”
BackgroundGeurts, D. E. M., Compen, F. R., Beek, M. H. C. T. V., & Speckens, A. E. M. (2020) · BJPsych Open · doi:10.1192/bjo.2020.118 · Cited by 19
More Treatment, but Not Less Anxiety and Mood Disorders: Why? Seven Hypotheses and Their Evaluation
Ormel, J., & Emmelkamp, P. M. G. (2023) · Psychotherapy and Psychosomatics · doi:10.1159/000528544 · Cited by 23
Effectiveness and predictors of outcome in routine out-patient mental health care for older adults
Veerbeek, M., Voshaar, R. C. O., & Pot, A. M. (2014) · International Psychogeriatrics · doi:10.1017/s1041610214000647 · Cited by 21
“Such naturalistic studies in secondary health care and pragmatic clinical trials show less favorable results compared to RCTs (Sinyor et al., 2010; Van der Lem et al., 2012).”
BackgroundResultInfluentialGrunebaum, M. F., Keilp, J. G., Ellis, S. P., Sudol, K., Bauer, N., Burke, A. K., Oquendo, M. A., & Mann, J. J. (2013) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.12m08000 · Cited by 23
Improving study design for antidepressant effectiveness assessment
Naudet, F., Millet, B., Reymann, J. M., & Falissard, B. (2013) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1391 · Cited by 17
“This result has recently been replicated (van der Lem et al., 2012).”
Vliek, L., Overbeek, G. J., & de Castro, B. O. (2013) · European Journal of Developmental Psychology · doi:10.1080/17405629.2013.811405 · Cited by 17
Dyrvig, A., Gerke, O., Kidholm, K., & Vondeling, H. (2015) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x15572202 · Cited by 14
Every effect size has its place: a commentary on the avoidance of pre–post effect sizes
Kösters, M. (2017) · Epidemiology and Psychiatric Sciences · doi:10.1017/s204579601700004x · Cited by 13
“As already mentioned by the authors, pre–post effect sizes can for example be informative in a comparison of efficacy and effectiveness trials (van der Lem et al. 2012).”
BackgroundPan, Y., Liu, S.-K., & Yeh, L. (2013) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2013.03.005 · Cited by 12
Schneider, C., Breilmann, J., Reuter, B., Becker, T. M., & Kösters, M. (2021) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13293 · Cited by 14
“In- and outpatient 27 (11) 13 (7) 14 (19)”
Sison, J., Vega, R. M. R., Da-yi, H., Bader, G., & Brunel, P. (2017) · Current Medical Research and Opinion · doi:10.1080/03007995.2017.1412682 · Cited by 10
Ezawa, I. D., Robinson, N., & Hollon, S. D. (2024) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-080822-040442 · Cited by 11
van der Glas, H. W., & van Grootel, R. J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-123 · Cited by 10
“While this risk might be present for chronic pain patients whose data of evaluation originates in part from manual clinical tests, such a risk is absent when exclusively questionnaires are used of which the data are collected by a person who is neither a clinician nor an assessor, like in Routine Outcome Monitoring of…”
BackgroundUse of outcome measurements in clinical practice: How specific should one be?
Nugter, A., Hermens, M. L., Robbers, S. C. C., van Son, G. E., Theunissen, J., & Engelsbel, F. (2017) · Psychotherapy Research · doi:10.1080/10503307.2017.1408975 · Cited by 10
Audhoe, S. S., Hoving, J. L., Zijlstra, B. J. H., Frings‐Dresen, M. H. W., & Nieuwenhuijsen, K. (2021) · BMC Public Health · doi:10.1186/s12889-021-10704-0 · Cited by 7
“While acknowledging the advantages of our pragmatic trial, with a good applicability of the intervention [41] and high external validity [41, 42] our design also has some disadvantages, including the low protocol adherence by professionals in the real OH setting.”
BackgroundEvidence for the treatment of moderate depression: a systematic review
Aherne, D., Fitzgerald, A., Aherne, C., Fitzgerald, N., Slattery, M., & Whelan, N. (2017) · Irish Journal of Psychological Medicine · doi:10.1017/ipm.2017.10 · Cited by 10
“…review draws attention to the use of classifications of depression such as ‘mild to moderate’ and ‘moderate to severe’within research (e.g. van der Lem et al. 2012) and suggests that such classifications, whilst at times clinically useful perhaps, may not be the most useful to inform research.”
BackgroundAudhoe, S. S., Nieuwenhuijsen, K., Hoving, J. L., Sluiter, J. K., & Frings‐Dresen, M. H. W. (2015) · BMC Public Health · doi:10.1186/s12889-015-1728-z · Cited by 6
“This increases the applicability of the intervention and has the advantage that the results will be more in line with daily occupational health practice, resulting in a high external validity [35-38].”
BackgroundCBT for treatment resistant depression
Otto, M. W., & Wisniewski, S. R. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61844-3 · Cited by 7
Kan, K., Feenstra, T., de Vries, S. O., Visser, E., Schoevers, R. A., & Jörg, F. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.07.010 · Cited by 5
Overcoming barriers in the pharmacological treatment of anxiety disorders
Baldwin, D. S., Rafiq, R., & Talat, B. (2013) · Neuropsychiatry · doi:10.2217/npy.12.73 · Cited by 4
USE OF LARGE DATA SETS AND THE FUTURE OF PERSONALIZED TREATMENT
Perlis, R. H. (2014) · Depression and Anxiety · doi:10.1002/da.22321 · Cited by 3
“[1]) Further, because clinical trials tend to focus on very specific patient populations, it may be challenging to generalize from models built in clinical trials to general clinical populations with greater medical and psychiatric comorbidity, lesser severity, and less consistent follow-up.[2] A further challenge to…”
BackgroundMéthodologie des essais thérapeutiques
Naudet, F., & Falissard, B. (2014) · Time to knit · doi:10.1016/s0246-1072(14)43770-2 · Cited by 4
Psychological treatment of depression in older adults: A network meta-analysis
Cuijpers, P. J., Miguel, C., Karyotaki, E., & Harrer, M. (2026) · International Psychogeriatrics · doi:10.1016/j.inpsyc.2026.100213 · Cited by 2
Etzelmueller, A., Heber, E., Horvath, H., Radkovsky, A., Lehr, D., & Ebert, D. D. (2024) · Journal of Medical Internet Research · doi:10.2196/42976 · Cited by 4
“In addition, RCTs are assumed to have a potential adherence-fostering effect because of their highly structured nature [43,44].”
Understanding the Antidepressant Debate in the Treatment of Major Depressive Disorder
Naudet, F., Boussageon, R., Palpacuer, C., Gallet, L., Reymann, J., & Falissard, B. (2015) · Therapies · doi:10.2515/therapie/2014228 · Cited by 3
Vliek, L. (2015) · Utrecht University Repository (Utrecht University) · Cited by 7
Ormel, J., & Emmelkamp, P. M. G. (2025) · Verhaltenstherapie · doi:10.1159/000542334 · Cited by 1
Laux, G. (2017) · Springer eBooks · doi:10.1007/978-3-662-49295-6_67 · Cited by 3
Effect of Adding Homeopathy to the Routine Treatments of Veterans with Chronic Depression
Mahmoudian, S., Kheirabadi, G. R., & Maracy, M. R. (2015) · DOAJ (DOAJ: Directory of Open Access Journals)
Masennuksen pariterapian vaikuttavuuden erot tutkimusyksiköiden välillä ja niitä välittävät tekijät
Laaksamo, S.-R., & Marjamaa, E. (2013) · Jyväskylä University Digital Archive (University of Jyväskylä)
Botto, A., & Leighton, C. (2021) · Depression and personality · doi:10.1007/978-3-030-70699-9_6
Laux, G. (2016) · Springer eBooks · doi:10.1007/978-3-642-45028-0_67-1 · Cited by 1
Fountoulakis, K. Ν. (2013) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.718163897.793486373
Effects of Topper Training on psychosocial problems, self-esteem, and peer victimisation.
Vliek, L. (2019) · http://isrctn.com/ · doi:10.1186/isrctn14967790
Improving personalized approaches to lifestyle modification for patients with overweight and obesity
Kiro, L. S., Zak, M., & Kushnіrenko, І. V. (2022) · Izdevniecība “Baltija Publishing” eBooks · doi:10.30525/978-9934-26-240-1-6
Biomarkers from bench to bedside: Realizing a future of EEG-based stratified psychiatry
Voetterl, H. (2024) · Research Publications (Maastricht University) · doi:10.26481/dis.20240625hv
Antidepressant Medication and Psychotherapy: Why It Matters
Chiesa, M. (2024) · British Journal of Psychotherapy · doi:10.1111/bjp.12938
Audhoe, S. S., Hoving, J. L., Zijlstra, B. J. H., Frings‐Dresen, M. H. W., & Nieuwenhuijsen, K. (2021) · Figshare · doi:10.6084/m9.figshare.14396674
Audhoe, S. S., Hoving, J. L., Zijlstra, B. J. H., Frings‐Dresen, M. H. W., & Nieuwenhuijsen, K. (2021) · Figshare · doi:10.6084/m9.figshare.14396671
Metadata
- Authors
- van der Lem, R., van der Wee, N. J. A., van Veen, T., & Zitman, F. G.
- Affiliations
- Leiden University Medical Center (NL)
- ORCID
- Zitman, Frans G.
- Venue
- Psychotherapy and Psychosomatics
- Publisher
- S. Karger AG
- Published
- 2012
- Volume
- 81
- Issue
- 4
- Pages
- 226-234
- ISSN
- 0033-3190 (ISSN-L)
- 1423-0348
- OpenAlex
- W1997227880
- Cited by
- 63
- References
- 55
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 4 · 2023: 1 · 2022: 1 · 2021: 7 · 2020: 5 · 2019: 1 · 2018: 1 · 2017: 6 · 2016: 3 · 2015: 8 · 2014: 8 · 2013: 13 · 2012: 3
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.karger.com/Services/SiteLicenses (vor)
- www.karger.com/Services/SiteLicenses (tdm)
- Locations
- Psychotherapy and Psychosomatics (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Health Sciences › Medicine › Pharmacology (medical) › Treatment of Major Depression
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- major depressive disorder, treatment efficacy, treatment effectiveness, STAR*D trial, routine clinical practice, antidepressants, individual psychotherapy, combination therapy, remission, generalizability of trial results, routine outcome monitoring
- MeSH terms
- Adult, Ambulatory Care, Antidepressive Agents, Combined Modality Therapy, Major Depressive Disorder, Depressive Disorder, Major, Female, Humans, Male, Middle Aged, Netherlands, Patient Compliance, Psychotherapy, Severity of Illness Index, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Treatment Outcome, Outcome Assessment, Health Care, Patient Selection, Evidence-Based Medicine, Guideline Adherence, Lost to Follow-Up
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.