Evidence-based psychotherapies I: qualifiers and limitations in what we know
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Psychosocial interventions have advanced remarkably for the treatment of a broad range of psychiatric disorders and related sources of impairment among children, adolescents, and adults. By one count, over 320 interventions have been identified as evidence-based in light of rigorously controlled studies and replication of treatment effects. Yet, how evidence-based psychotherapies have been evaluated and reported raises questions about their impact. This article evaluates both methodological and substantive issues that limit what can be stated about evidence-based psychotherapies and their effects. Among the methodological topics are the control conditions to which evidence-based psychotherapies are compared, selective reporting of measures, and the limited evidence that evidence-based psychotherapies have clinically significant impact. Among the substantive issues are the paucity of research on moderators that would help us better direct patients to treatments from which they are likely to profit and our limited understanding of the mechanisms responsible for therapeutic change. The issues discussed are fundamental to what can be stated about the impact of evidence-based psychotherapies and impact and the bases for their effects. It is not clear at present whether concerted efforts are in place to alter research in ways that would redress the issues. There are, however, novel new directions for research that build on the evidence-based psychotherapies, and these are addressed in a companion article.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., Donaldson, A., & Varley, J. (2010) · Pediatrics · doi:10.1542/peds.2009-0958 · Cited by 2,835
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Empirically supported psychological interventions: Controversies and evidence
Chambless, D. L., & Ollendick, T. H. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.685 · Cited by 2,097
Prevalence and Treatment of Mental Disorders, 1990 to 2003
Kessler, R. C., Demler, O., Frank, R. G., Olfson, M., Pincus, H. A., Walters, E. E., Wang, P., Wells, K. B., & Zaslavsky, A. M. (2005) · New England Journal of Medicine · doi:10.1056/nejmsa043266 · Cited by 1,910
The global burden of mental disorders: An update from the WHO World Mental Health (WMH) surveys
Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., Üstün, T. B., & Wang, P. S. (2009) · Epidemiologia e Psichiatria Sociale · doi:10.1017/s1121189x00001421 · Cited by 1,895
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“We know generally that there is a publication bias in research, namely, that results showing group differences are more likely to be published than results showing no group differences (e.g., Dwan et al., 2008; Joober, Schmitz, Annable, & Boksa, 2012; Spielmans & Kirsch, 2014).”
ResultSen, S., Duman, R., & Sanacora, G. (2008) · Biological Psychiatry · doi:10.1016/j.biopsych.2008.05.005 · Cited by 1,232
Yes, but what’s the mechanism? (don’t expect an easy answer).
Bullock, J. G., Green, D. P., & Ha, S. E. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0018933 · Cited by 1,063
The Descriptive Epidemiology of Commonly Occurring Mental Disorders in the United States
Kessler, R. C., & Wang, P. S. (2008) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.29.020907.090847 · Cited by 888 · cited 1×
“…Health Organization (WHO) assessing the global burden of mental illness found the rates of lifetime psychiatric disorder among 17 participating countries to range from 12.0% to 47.4%, with the highest lifetime prevalence estimate in the United States (Kessler et al., 2009; Kessler & Wang, 2008).”
BackgroundInfluentialJacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.3.300 · Cited by 853
Arbitrary metrics in psychology
Blanton, H., & Jaccard, J. (2006) · American Psychologist · doi:10.1037/0003-066x.61.1.27 · Cited by 681 · cited 1×
“An assessment issue infrequently raised is the notion of arbitrary metric (Blanton & Jaccard, 2006).”
BackgroundBoot, W. R., Simons, D. J., Stothart, C., & Stutts, C. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691613491271 · Cited by 670
Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.671 · Cited by 560
DeRubeis, R. J., Cohen, Z. D., Forand, N. R., Fournier, J. C., Gelfand, L. A., & Lorenzo-Luaces, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0083875 · Cited by 509 · cited 1×
“The question continues to dominate the treatment research agenda (e.g., DeRubeis et al., 2014; Kaplan, 2008; Roth & Fonagy, 2005).”
BackgroundVoluntary exercise produces antidepressant and anxiolytic behavioral effects in mice
Duman, C. H., Schlesinger, L., Russell, D. S., & Duman, R. S. (2008) · Brain Research · doi:10.1016/j.brainres.2007.12.047 · Cited by 465
Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.1176 · Cited by 412 · cited 2×
“Yet, magnitude of effect often is confused with practical or clinical significance (e.g., Rutledge & Loh, 2004; Sun, Pan, & Wang, 2010; Weisz et al., 2013).”
BackgroundMethodAssessment of Adult Psychopathology: Meta-Analyses and Implications of Cross-Informant Correlations.
Achenbach, T. M., Krukowski, R. A., Dumenci, L., & Ivanova, M. Y. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.3.361 · Cited by 406
Altered Placebo and Drug Labeling Changes the Outcome of Episodic Migraine Attacks
Kam-Hansen, S., Jakubowski, M., Kelley, J. M., Kirsch, I., Hoaglin, D. C., Kaptchuk, T. J., & Burstein, R. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3006175 · Cited by 399
Freedland, K. E., Mohr, D. C., Davidson, K. W., & Schwartz, J. E. (2011) · Psychosomatic Medicine · doi:10.1097/psy.0b013e318218e1fb · Cited by 369
Cavanagh, K., Strauss, C., Forder, L., & Jones, F. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.01.001 · Cited by 340
Nelson, J. C., & Devanand, D. P. (2011) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2011.03355.x · Cited by 284
Predicting treatment response in social anxiety disorder from functional magnetic resonance imaging.
Doehrmann, O., Ghosh, S. S., Polli, F. E., Reynolds, G. O., Horn, F., Keshavan, A., Triantafyllou, C., Saygin, Z. M., Whitfield-Gabrieli, S., Hofmann, S. G., et al. (2013) · JAMA Psychiatry · doi:10.1001/2013.jamapsychiatry.5 · Cited by 262
The Effects of Family Therapies for Adolescent Delinquency and Substance Abuse: A Meta‐analysis
Baldwin, S. A., Christian, S., Berkeljon, A., & Shadish, W. R. (2012) · Journal of Marital and Family Therapy · doi:10.1111/j.1752-0606.2011.00248.x · Cited by 244
Speer, D. C., & Greenbaum, P. E. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.1044 · Cited by 216
Reproducible Research: Moving toward Research the Public Can Really Trust
Laine, C., Goodman, S. N., Griswold, M. E., & Sox, H. C. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-6-200703200-00154 · Cited by 213
Conceptualizing changes in behavior in intervention research: the range of possible changes model
De Los Reyes, A., & Kazdin, A. E. (2006) · Psychological Review · doi:10.1037/0033-295x.113.3.554 · Cited by 172 · cited 4×
“The inconsistencies occur with measures that might be delineated as the primary focus (e.g., target symptoms or disorder) as well as among measures that are more ancillary (e.g., other symptoms, functioning in other domains; see, for example, De Los Reyes & Kazdin, 2006).”
BackgroundMethodInfluentialRutledge, T., & Loh, C. (2004) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2702_9 · Cited by 144 · cited 3×
“Indeed, one clinical trial comparing aspirin and placebo was stopped early because the results were so clear in favor of aspirin, but the strength of the relationship was small (r = .03; see Rutledge & Loh, 2004).”
BackgroundWampold, B. E., Budge, S. L., Laska, K. M., Del Re, A. C., Baardseth, T. P., Flűckiger, C., Minami, T., Kivlighan, D. M., & Gunn, W. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.07.012 · Cited by 134 · cited 1×
“…the full range of outcomes in relation to EBPs; sometimes they are more, sometimes less, and sometimes equally effective as the EBPs to which they are compared (Cahill, Barkham, & Stiles, 2010; Freedland, Mohr, Davidson, & Schwartz, 2011; Wampold et al., 2011; Weisz, Jensen-Doss, & Hawley, 2006).”
BackgroundModerators, mediators and mechanisms of change in psychotherapy
Kazdin, A. E. (2014) · Quantitative and Qualitative Methods in Psychotherapy Research · doi:10.4324/9780203386071-6 · Cited by 99 · cited 1×
“Even when these conditions are met, ambiguity can remain about the precise role of the mediator (Bullock, Green, & Ha, 2010; Kazdin, 2014).”
BackgroundSelf-citationSystematic review of practice‐based research on psychological therapies in routine clinic settings
Cahill, J., Barkham, M., & Stiles, W. B. (2010) · British Journal of Clinical Psychology · doi:10.1348/014466509x470789 · Cited by 92
Research works printed but not linked: 56
- 10.1037/0022-006X.71.6.973 · DOI · Paper page
- Benedetti F., 2009, Placebo effects: Understanding the mechanisms in health and disease
- 10.1037/14137-000 · doi:10.1037/14137-000
- Cohen J. S., 1988, Statistical power analysis in the behavioral sciences
- 10.1037/arc0000001 · DOI
- 10.31887/DCNS.2011.13.4/mdavis · DOI · Paper page
- 10.1056/NEJMe058127 · DOI · Paper page
- 10.1111/j.1467-8721.2008.00546.x · DOI · Paper page
- 10.1146/annurev-clinpsy-050212-185617 · DOI · Paper page
- 10.1176/appi.ajp.2013.12070899 · DOI · Paper page
- 10.1038/mp.2008.13 · DOI · Paper page
- 10.1037/12075-000 · DOI · Paper page
- 10.1177/0956797613512465 · DOI · Paper page
- 10.1016/S0140-6736(09)61706-2 · DOI · Paper page
- 10.1017/S0033291710000553 · DOI · Paper page
- 10.1016/j.cpr.2007.05.002 · DOI · Paper page
- 10.1001/2013.jamapsychiatry.60 · DOI · Paper page
- 10.1001/jamapsychiatry.2013.285 · doi:10.1001/jamapsychiatry.2013.285
- 10.1037/0022-006X.64.5.973 · doi:10.1037/0022-006x.64.5.973 · Paper page
- Grissom R. J., 2011, Effect sizes for research: Univariate and multivariate applications
- 10.1177/1745691614521781 · DOI · Paper page
- 10.1037/a0030571 · DOI · Paper page
- 10.1503/jpn.120065 · DOI · Paper page
- Kaplan R. M., 2008, Health Psychology, 27, S99, 10.1037/0278-6133.27.2(Suppl.).S99 · DOI · Paper page
- 10.1037/0022-006X.68.1.166 · doi:10.1037/0022-006x.68.1.166 · Paper page
- Kazdin A. E., 2010, Evidence-based psychotherapies for children and adolescents, 211
- 10.1001/jamapsychiatry.2013.2112 · DOI
- Kazdin A. E., South African Journal of Psychology
- Kazdin A. E., Research design in clinical psychology
- 10.1037/0022-006X.57.1.138 · doi:10.1037/0022-006x.57.1.138 · Paper page
- 10.1037/0022-006X.74.3.455 · DOI · Paper page
- 10.1002/sim.5734 · DOI · Paper page
- Lambert M. J., 2013, Bergin and Garfield’s handbook of psychotherapy and behavior change
- Lambert M. J., 2013, Bergin and Garfield’s handbook of psychotherapy and behavior change, 169
- 10.1080/15374416.2013.769169 · DOI · Paper page
- 10.1080/10503300902948053 · DOI
- Miles M. B., 2014, Qualitative data analysis: A methods sourcebook
- Morris B., 2005, Discovering bits and pieces of me: Research exploring women’s experiences of psychoanalytic psychotherapy
- 10.1007/s00737-009-0101-6 · DOI
- Nathan P. E., Treatments that work
- 10.1080/10503300601139988 · DOI · Paper page
- Ogles B. M., 2013, Bergin and Garfield’s handbook of psychotherapy and behavior change, 134
- Paul G. L., 1966, Insight versus desensitization in psychotherapy: An experiment in anxiety reduction
- 10.1037/h0024436 · DOI · Paper page
- 10.1176/jnp.2009.21.2.114 · DOI · Paper page
- 10.1016/j.neubiorev.2011.09.004 · DOI · Paper page
- 10.1016/j.cpr.2012.02.004 · DOI · Paper page
- 10.1017/S0033291705005064 · DOI · Paper page
- Roth A., 2005, What works for whom: A critical review of psychotherapy research
- 10.1146/annurev-clinpsy-050212-185533 · DOI
- 10.1037/a0019507 · DOI · Paper page
- 10.1001/jamapsychiatry.2013.1960 · DOI · Paper page
- Weisz J. R., 2010, Evidence-based psychotherapies for children and adolescents
- 10.1176/ps.2010.61.12.1248 · DOI · Paper page
- World Federation for Mental Health, 2011, The great push: Investing in mental health
- World Health Organization (WHO), 2008, The global burden of disease: 2004 update
Cited by
The Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624
Response rates for CBT for anxiety disorders: need for standardized criteria
Loerinc, A. G., Meuret, A. E., Twohig, M. P., Rosenfield, D., Bluett, E. J., & Craske, M. G. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.08.004 · Cited by 592
Measuring the active elements of cognitive-behavioral therapies
Cohen, Z. D., Barnes-Horowitz, N. M., Forbes, C. N., & Craske, M. G. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104364 · Cited by 58
Waters, A. M., LeBeau, R. T., & Craske, M. G. (2017) · Psychopathology Review · doi:10.5127/pr.038015 · Cited by 32
Metadata
- Authors
- Kazdin, A. E.
- Venue
- South African Journal of Psychology
- Publisher
- SAGE Publications
- Published
- 2014-05-13
- Volume
- 44
- Issue
- 4
- Pages
- 381-403
- ISSN
- 0081-2463 (ISSN-L)
- OpenAlex
- W2145527202
- Cited by
- 67
- References
- 32
- FWCI
- 6.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.