Empirically Supported Treatment: Recommendations for a New Model
Open accessThe DOI matches its Crossref record; 138 of the 138 references it cites were found in a registry or an index.
Abstract
Over the 20 years since the criteria for empirically supported treatments (ESTs) were published, standards for synthesizing evidence have evolved and more systematic approaches to reviewing the findings from intervention trials have emerged. Currently, the APA is planning the development of treatment guidelines, a process that will likely take many years. As an intermediate step, we recommend a revised set of criteria for ESTs that will utilize existing systematic reviews of all of the available literature, and recommendations that address the methodological quality, outcomes, populations, and treatment settings included in the literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Consistent with recommendations for effectiveness studies, participants were not randomized to treatment, treatment was provided by community clinicians, and participants were not excluded based on co -morbidities or medication use (Tolin et al ., 2015).”
From Effectiveness and predictors of group cognitive behaviour therapy outcome for generalised anxiety disorder in an out-patient hospital setting (2024) · cites this paper as Tolin et al., 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · cited 6×
“The Cochrane Handbook (Higgins & Green, 2008) and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Liberati et al., 2009) are well-respected systems for evaluation; the Task Force will use, at least in its initial efforts, the AMSTAR checklist (Shea, Bouter, et al.,…”
BackgroundMethodInfluentialGRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Diagnostic and Statistical Manual of Mental Disorders, 5th ed.
Carter, M. J. (2014) · Therapeutic Recreation Journal · Cited by 15,169
Hao (2015) · doi:10.1002/14651858 · Cited by 11,321
Crossing the Quality Chasm: A New Health System for the 21st Century
Committee on Quality of Health Care in America (2002) · Journal For Healthcare Quality · doi:10.1111/j.1945-1474.2002.tb00463.x · Cited by 8,483
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., Sanislow, C., & Wang, P. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.09091379 · Cited by 7,067
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005) · Exceptional Children · doi:10.1177/001440290507100203 · Cited by 3,368 · cited 2×
“Assessment of the quality of single-subject designs could employ published quality indicators (Horner et al., 2005; Smith et al., 2007), in a manner that parallels the procedures used to evaluate RCTs. Consistent with current approaches to evidence synthesis, however, we do not recommend that…”
MethodResultClinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Defining empirically supported therapies.
Chambless, D. L., & Hollon, S. D. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.1.7 · Cited by 3,067 · cited 2×
“…12 report on ESTs delineated specific criteria (see Table 1) by which a treatment would be regarded as “probably efficacious” or “well established” (Chambless & Hollon, 1998; Chambless & Ollendick, 2001; Task Force on Promotion and Dissemination of Psychological Procedures, 1993), and these…”
BackgroundMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · cited 1×
“Detailed and stringent guidelines have now been published regarding the execution and reporting of methodologically sound treatment outcome studies (Moher, Schulz, & Altman, 2001), and leading psychology journals such as the Journal of Consulting and Clinical Psychology require that manuscripts…”
BackgroundBellg, A. J., Treatment Fidelity Workgroup of the NIH Behavior Change Consortium, Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., et al. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.443 · Cited by 2,620 · cited 1×
“Bellg et al. (2004) provide a thorough discussion of elements of treatment fidelity and steps that can be taken to enhance treat-ment fidelity in trials of behavior change studies.”
BackgroundWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · cited 1×
“…generally, design weaknesses related to allocation concealment, blinding, and randomization methods may be expected to influ-ence effect estimates, particularly when outcomes are subjective (Savovic et al., 2012), which is the case in most trials of psychological treatments (Wood et al., 2008).”
BackgroundThe Work and Social Adjustment Scale: a simple measure of impairment in functioning
Mundt, J. C., Marks, I. M., Shear, M. K., & Greist, J. M. (2002) · British Journal of Psychiatry · doi:10.1192/bjp.180.5.461 · Cited by 2,515
Hayes, S. C. (2004) · Behavior Therapy · doi:10.1016/s0005-7894(04)80013-3 · Cited by 2,493
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
Remission in Schizophrenia: Proposed Criteria and Rationale for Consensus
Andreasen, N. C., Carpenter, W. T., Kane, J. M., Lasser, R. A., Marder, S. R., & Weinberger, D. R. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.3.441 · Cited by 2,343
A Critical Appraisal of the Quality of Quality-of-Life Measurements
Gill, T. M. (1994) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1994.03520080061045 · Cited by 2,208
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1×
“Financial con-flict of interest, a demonstrated source of publication bias in pharmaceutical studies (Friedman & Richter, 2004; Lexchin et al., 2003; Perlis et al., 2005), may also be considered in rating risk of bias (Bero, 2013; Roseman et al., 2011, 2012), although conflict of interest may be…”
BackgroundEmpirically supported psychological interventions: Controversies and evidence
Chambless, D. L., & Ollendick, T. H. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.685 · Cited by 2,097
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Frank, E. (1991) · Archives of General Psychiatry · doi:10.1001/archpsyc.1991.01810330075011 · Cited by 1,959 · cited 1×
“…of treatment CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE V22 N4, DECEMBER 2015 response and remission (Andreasen et al., 2005; Doyle & Pollack, 2003; Frank et al., 1991; McIntyre, Fallu, & Konarski, 2006; Simpson, Huppert, Petkova, Foa, & Liebowitz, 2006), usually operationalized as a cutoff score…”
BackgroundShea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755 · cited 1×
“…(PRISMA; Liberati et al., 2009) are well-respected systems for evaluation; the Task Force will use, at least in its initial efforts, the AMSTAR checklist (Shea, Bouter, et al., 2007; Shea, Grimshaw, et al., 2007; Shea et al., 2009) as described above and shown in the online supplement.”
MethodIndustry sponsorship and research outcome
Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub3 · Cited by 1,697
Research summary on the therapeutic relationship and psychotherapy outcome.
Lambert, M. J., & Barley, D. E. (2001) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/0033-3204.38.4.357 · Cited by 1,621 · cited 1×
“Indeed, many have suggested that the therapeutic relationship accounts for greater variance in clinical outcomes than do those aspects of the therapy that are described as “techniques” (Blatt & Zuroff, 2005; Henry, 1998; Lambert & Barley, 2001; Norcross, 1999).”
BackgroundThe effectiveness of psychotherapy: the Consumer Reports study
Seligman, M. E. P. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.965 · Cited by 1,608 · cited 1×
“Therefore, a number of scholars have cautioned against the overreliance of symptom-based evaluations of efficacy and have instead urged consideration of wellness, quality of life, well-being, and functionality (Cowen, 1991; Hayes, 2004; Seligman, 1995).”
BackgroundPRACTICE GUIDELINE FOR THE Treatment of Patients With Major Depressive Disorder
Gelenberg, A. J., Freeman, P. B., Markowitz, J. C., Rosenbaum, J. F., Thase, M. E., Trivedi, M. H., & Rhoads, R. S. (2010) · Cited by 1,510
Update on empirically validated therapies, II: (619622010-001)
Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun, K. S., Crits-Christoph, P., Daiuto, A., DeRubeis, R., Detweiler, J., Haaga, D. A. F., et al. (1997) · PsycEXTRA Dataset · doi:10.1037/e619622010-001 · Cited by 1,488 · cited 2×
“The original EST criteria (Chambless et al., 1998) operationalized methodological adequacy as including the use of a treatment manual, a well-characterized sample, and random assignment to treatment and control conditions.”
MethodMoher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426
Culturally adapted mental health intervention: a meta-analytic review
Griner, D., & Smith, T. B. (2006) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/0033-3204.43.4.531 · Cited by 1,236 · cited 1×
“These modifica-tions might include specific cultural adaptations (Ben-ish, Quintana, & Wampold, 2011; Griner & Smith, 2006; van Loon, van Schaik, Dekker, & Beekman, 2013) or the addition (either concurrently or sequentially) of an EST for drinking problems such as behavioral couples therapy…”
BackgroundAtkins, D., Eccles, M., Flottorp, S., Guyatt, G. H., Henry, D., Hill, S., Liberati, A., O'Connell, D., Oxman, A. D., Phillips, B., et al. (2004) · BMC Health Services Research · doi:10.1186/1472-6963-4-38 · Cited by 1,235 · cited 2×
“…and (b) a committee-based appraisal of the evidence, using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system (Atkins et al., 2004; Guyatt et al., 2006, 2008) to assess the quality of relevant evidence and degree to which benefits are established in excess of…”
MethodGrading Strength of Recommendations and Quality of Evidence in Clinical Guidelines
Guyatt, G., Gutterman, D., Baumann, M. H., Addrizzo-Harris, D., Hylek, E. M., Phillips, B., Raskob, G., Lewis, S. Z., & Schünemann, H. (2006) · Chest · doi:10.1378/chest.129.1.174 · Cited by 1,220 · cited 1×
“…appraisal of the evidence, using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system (Atkins et al., 2004; Guyatt et al., 2006, 2008) to assess the quality of relevant evidence and degree to which benefits are established in excess of potential harms.”
MethodWesten, D., Novotny, C. M., & Thompson-Brenner, H. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.4.631 · Cited by 1,212 · cited 1×
“…also likely decrease the EST movement’s reliance on a large number of treatment manuals, a process to which many authors, even those supportive of the broad EST movement, object (e.g., Fonagy, 1999; Goldfried & Eubanks-Carter, 2004; Levant, 2004; Norcross, 1999; Wachtel, 2010; Westen et al., 2004).”
BackgroundHandbook of Psychiatric Measures
SALZMAN, C. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.8.1343 · Cited by 1,174
Effectiveness of psychotherapy
(2011) · Revision MCQs and EMIs for the MRCPsych · doi:10.1201/b13482-76 · Cited by 1,058
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
Finding What Works in Health Care: Standards for Systematic Reviews
Eden, J., Levit, L. A., Berg, A. O., & Morton, S. C. (2013) · Cited by 943
(2001) · Medical Entomology and Zoology · Cited by 915
Lock, J., Le Grange, D., Agras, W. S., Moye, A., Bryson, S. W., & Jo, B. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.128 · Cited by 866
Handbook of Psychiatric Measures
Rush, A. J., First, M. B., & Blacker, D. (2000) · Medical Entomology and Zoology · Cited by 846
Dave, K. A., & White, S. W. (2002) · World Highways/Routes du Monde · Cited by 765
Interpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696
Culturally adapted psychotherapy and the legitimacy of myth: A direct-comparison meta-analysis.
Benish, S. G., Quintana, S., & Wampold, B. E. (2011) · Journal of Counseling Psychology · doi:10.1037/a0023626 · Cited by 661
A Brief Behavioral Activation Treatment for Depression: Treatment Manual
Lejuez, C. W., Hopko, D. R., & Hopko, S. D. (2001) · Behavior Modification · doi:10.1177/0145445501252005 · Cited by 611
Industry sponsorship and research outcome
Lundh, A., Sismondo, S., Lexchin, J., Busuioc, O. A., & Bero, L. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub2 · Cited by 605
The researcher's own therapy allegiances: A "wild card" in comparisons of treatment efficacy.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., Halperin, G., Bishop, M., Berman, J. S., & Schweizer, E. (1999) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.6.1.95 · Cited by 572 · cited 2×
“The internal validity and degree of research bias in clinical trials are not adequately addressed, potentially making the results prone to false-positive results (Luborsky et al., 1999; Wachtel, 2010).”
BackgroundExternal Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569 · cited 1×
“…(PRISMA; Liberati et al., 2009) are well-respected systems for evaluation; the Task Force will use, at least in its initial efforts, the AMSTAR checklist (Shea, Bouter, et al., 2007; Shea, Grimshaw, et al., 2007; Shea et al., 2009) as described above and shown in the online supplement.”
MethodAttention modification program in individuals with generalized anxiety disorder
Amir, N., Beard, C., Burns, M., & Bomyea, J. (2009) · Journal of Abnormal Psychology · doi:10.1037/a0012589 · Cited by 539
Functional impairment in social phobia.
Schneier, F. R., Heckelman, L., Garfinkel, R. S., Campeas, R. B., Fallon, B. A., Gitow, A., Street, L. L., Bene, D. D., & Liebowitz, M. R. (1994) · PubMed · Cited by 534
State of the Science on Psychosocial Interventions for Ethnic Minorities
Miranda, J., Bernal, G., Lau, A., Kohn, L., Hwang, W.-C., & LaFromboise, T. (2005) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.1.102803.143822 · Cited by 534
Principles of Therapeutic Change That Work
(2005) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195156843.001.0001 · Cited by 524
Give them prizes and they will come: Contingency management for treatment of alcohol dependence.
Petry, N. M., Martin, B., Cooney, J. L., & Kranzler, H. R. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.2.250 · Cited by 515
What are quality of life measurements measuring?
Muldoon, M. F., Barger, S. D., Flory, J. D., & Manuck, S. B. (1998) · BMJ · doi:10.1136/bmj.316.7130.542 · Cited by 490
Designing Research Studies on Psychosocial Interventions in Autism
Smith, T., Scahill, L., Dawson, G., Guthrie, D., Lord, C., Odom, S., Rogers, S., & Wagner, A. (2007) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-006-0173-3 · Cited by 405
Franklin, M. E., Abramowitz, J. S., Kozak, M. J., Levitt, J. T., & Foa, E. B. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.4.594 · Cited by 387 · cited 1×
“In some cases, randomized and nonrandomized patients receiving similar treatments appear to do equally well (Franklin, Abramowitz, Kozak, Levitt, & Foa, 2000), whereas in other cases, treatments administered in a research setting yield outcomes superior to the same treatments administered in a…”
BackgroundGreenland, S. (2001) · Biostatistics · doi:10.1093/biostatistics/2.4.463 · Cited by 359 · cited 1×
“…the quality of reporting with the quality of trial conduct, to assign weights to different items in ways that are dif-ficult to justify, and to yield inconsistent and unpredictable associations with intervention effect estimates (Greenland & O’Rourke, 2001; Juni, Witschi, Bloch, & Egger, 1999).”
BackgroundThe Behavioral Management of Anxiety, Depression and Pain
WEISS, T. (1977) · American Journal of Psychiatry · doi:10.1176/ajp.134.8.945-a · Cited by 346
Cowen, E. L. (1991) · American Psychologist · doi:10.1037//0003-066x.46.4.404 · Cited by 337
A randomized controlled clinical trial of psychoanalytic psychotherapy for panic disorder
Milrod, B., Leon, A. C., Busch, F., Rudden, M., Schwalberg, M., Clarkin, J., Aronson, A., Singer, M., Turchin, W., Klass, E. T., et al. (2007) · American Journal of Psychiatry · doi:10.1176/ajp.2007.164.2.265 · Cited by 323
Perlis, R. H., Perlis, C. S., Wu, Y., Hwang, C., Joseph, M., & Nierenberg, A. A. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.10.1957 · Cited by 318 · cited 2×
“Financial conflict of interest, a demonstrated source of publication bias in pharmaceutical studies (Friedman & Richter, 2004; Lexchin et al., 2003; Perlis et al., 2005), may also be considered in rating risk of bias (Bero, 2013; Roseman et al.”
BackgroundRosen, G. M., & Davison, G. C. (2003) · Behavior Modification · doi:10.1177/0145445503027003003 · Cited by 305
Finding What Works in Health Care: Standards for Systematic Reviews
Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Board on Health Care Services, & Institute of Medicine (2011) · doi:10.17226/13059 · Cited by 289
Acceptance and Commitment Therapy
Hayes, S. C., & Sackett, C. (2014) · Encyclopedia of Behavior Modification and Cognitive Behavior Therapy: Volume 1: Adult Clinical Applications Volume 2: Child Clinical Applications Volume 3: Educational Applications · doi:10.4135/9781412950534.n1 · Cited by 279
Toward a more clinically valid approach to therapy research.
Goldfried, M. R., & Wolfe, B. E. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.66.1.143 · Cited by 278
Psychotherapy practice and research: Repairing a strained alliance.
Goldfried, M. R., & Wolfe, B. E. (1996) · American Psychologist · doi:10.1037/0003-066x.51.10.1007 · Cited by 271
BERNARDY, K., FÜBER, N., KÖLLNER, V., & HÄUSER, W. (2010) · The Journal of Rheumatology · doi:10.3899/jrheum.100104 · Cited by 266
Relationship between conflicts of interest and research results
Friedman, L. S., & Richter, E. D. (2004) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2004.30617.x · Cited by 258
Response versus remission in obsessive-compulsive disorder
Simpson, H. B., Huppert, J. D., Petkova, E., Foa, E. B., & Liebowitz, M. R. (2006) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v67n0214 · Cited by 241 · cited 1×
“…SCIENCE AND PRACTICE V22 N4, DECEMBER 2015 response and remission (Andreasen et al., 2005; Doyle & Pollack, 2003; Frank et al., 1991; McIntyre, Fallu, & Konarski, 2006; Simpson, Huppert, Petkova, Foa, & Liebowitz, 2006), usually operationalized as a cutoff score on a standardized measure.”
BackgroundThe Range of Impaired Functioning Tool (LIFE–RIFT): a brief measure of functional impairment
LEON, A. C., SOLOMON, D. A., MUELLER, T. I., TURVEY, C. L., ENDICOTT, J., & KELLER, M. B. (1999) · Psychological Medicine · doi:10.1017/s0033291799008570 · Cited by 231
Frances, R. J. (2007) · Yearbook of Psychiatry and Applied Mental Health · doi:10.1016/s0084-3970(08)70411-0 · Cited by 220
Psychotherapy practice and research: Repairing a strained relationship.
Goldfried, M. R., & Wolfe, B. E. (1996) · American Psychologist · doi:10.1037//0003-066x.51.10.1007 · Cited by 214
What is the scientific meaning of empirically supported therapy?
Borkovec, T. D., & Costonguay, L. G. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.1.136 · Cited by 211
Empirical evaluation of the assumptions in identifying evidence based treatments in mental health
Blatt, S. J., & Zuroff, D. C. (2005) · Clinical Psychology Review · doi:10.1016/j.cpr.2005.03.001 · Cited by 199 · cited 1×
“Indeed, many have suggested that the therapeutic relationship accounts for greater variance in clinical outcomes than do those aspects of the therapy that are described as “techniques” (Blatt & Zuroff, 2005; Henry, 1998; Lambert & Barley, 2001; Norcross, 1999).”
BackgroundScience as an ally of practice.
Seligman, M. E. P. (1996) · American Psychologist · doi:10.1037//0003-066x.51.10.1072 · Cited by 195
Researcher allegiance and meta-analysis: The case of cognitive therapy for depression.
Gaffan, E. A., Tsaousis, I., & Kemp-Wheeler, S. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.966 · Cited by 194
Perri, M. G., Sears, S. F., & Clark, J. E. (1993) · Diabetes Care · doi:10.2337/diacare.16.1.200 · Cited by 193
Practice Guideline for the Treatment of Patients With Panic Disorder
(2007) · APA Practice Guidelines for the Treatment of Psychiatric Disorders: Comprehensive Guidelines and Guideline Watches · doi:10.1176/appi.books.9780890423363.51396 · Cited by 191
Allegiance in psychotherapy outcome research: Separating association from bias.
Leykin, Y., & DeRubeis, R. J. (2009) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2009.01143.x · Cited by 170 · cited 1×
“The presence of researcher allegiance does not necessarily imply bias (Hollon, 1999; Leykin & DeRubeis, 2009); however, it is a risk factor that has been shown empirically to be associated with some probability of bias.”
BackgroundGuidelines and Principles for Accreditation of Programs in Professional Psychology: (415022008-001)
(1995) · PsycEXTRA Dataset · doi:10.1037/e415022008-001 · Cited by 162
Feeny, N. C., Zoellner, L. A., & Foa, E. B. (2002) · Journal of Personality Disorders · doi:10.1521/pedi.16.1.30.22555 · Cited by 159
Thrive: The Power of Evidence-Based Psychological Therapies
Layard, R. (2014) · Cited by 159
Identifying empirically supported treatments: What if we didn't?
Beutler, L. E. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.66.1.113 · Cited by 150
Krarup, A. L., Ny, L., Gunnarsson, J., Hvid‐Jensen, F., Zetterstrand, S., Simrén, M., Funch-Jensen, P. M., Hansen, M. B., & Drewes, A. M. (2013) · Scandinavian Journal of Gastroenterology · doi:10.3109/00365521.2012.758769 · Cited by 149
Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments
Roseman, M. (2011) · JAMA · doi:10.1001/jama.2011.257 · Cited by 145 · cited 1×
“…bias in pharmaceutical studies (Friedman & Richter, 2004; Lexchin et al., 2003; Perlis et al., 2005), may also be considered in rating risk of bias (Bero, 2013; Roseman et al., 2011, 2012), although conflict of interest may be harder to identify and quantify in studies of psychological treatments.”
BackgroundSelf-citationToward a standard definition of clinically significant change
Jacobson, N., Follette, W., & Revenstorf, D. (1986) · Behavior Therapy · doi:10.1016/s0005-7894(86)80061-2 · Cited by 145
Henry, W. P. (1998) · Psychotherapy Research · doi:10.1093/ptr/8.2.126 · Cited by 125
Susskind, O., Ruzek, J., & Friedman, M. (2012) · Journal of rehabilitation research and development · doi:10.1682/jrrd.2011.12.0241 · Cited by 119
Thrive: the power of evidence-based psychological therapies
Rustin, M. (2015) · European Journal of Psychotherapy & Counselling · doi:10.1080/13642537.2014.1001163 · Cited by 118
Efficacy of eye movement desensitization and reprocessing: Implications for behavior therapy
Lohr, J. M., Tolin, D. F., & Lilienfeld, S. O. (1998) · Behavior Therapy · doi:10.1016/s0005-7894(98)80035-x · Cited by 118
Self-citationBeyond “ESTs”: Problematic assumptions in the pursuit of evidence-based practice.
Wachtel, P. L. (2010) · Psychoanalytic Psychology · doi:10.1037/a0020532 · Cited by 116
Thoma, N. C., McKay, D., Gerber, A. J., Milrod, B. L., Edwards, A. R., & Kocsis, J. H. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11030433 · Cited by 105
Self-citationO’Farrell, T. J., Cutter, H. S. G., Choquette, K. A., Floyd, F. J., & Bayog, R. D. (1992) · Behavior Therapy · doi:10.1016/s0005-7894(05)80220-5 · Cited by 103
Transference-focused psychotherapy.
Yeomans, F. E., Levy, K. N., & Caligor, E. (2013) · Psychotherapy · doi:10.1037/a0033417 · Cited by 100
Can the randomized controlled trial literature generalize to nonrandomized patients?
Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., & Rothman, A. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.1.127 · Cited by 98
Development of Clinical Practice Guidelines
Hollon, S. D., Areán, P. A., Craske, M. G., Crawford, K. A., Kivlahan, D. R., Magnavita, J. J., Ollendick, T. H., Sexton, T. L., Spring, B., Bufka, L. F., et al. (2014) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185529 · Cited by 97
A new scale for assessing the quality of randomized clinical trials of psychotherapy
Kocsis, J. H., Gerber, A. J., Milrod, B. L., Roose, S. P., Barber, J. P., Thase, M. E., Perkins, P., & Leon, A. C. (2009) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2009.07.001 · Cited by 97
Why the Cochrane Risk of Bias Tool Should Include Funding Source as a Standard Item
Bero, L. A. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.ed000075 · Cited by 96
Statistical Analyses of Single-Case Designs: The Shape of Things to Come
Shadish, W. R. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414524773 · Cited by 94
A Randomized Controlled Clinical Trial of Psychoanalytic Psychotherapy for Panic Disorder
Milrod, B. L. (2007) · American Journal of Psychiatry · doi:10.1176/appi.ajp.164.2.265 · Cited by 94
The Science and Practice of Empirically Supported Treatments
Herbert, J. D. (2003) · Behavior Modification · doi:10.1177/0145445503027003008 · Cited by 88
Criteria for evaluating treatment guidelines.
(2002) · American Psychologist · doi:10.1037//0003-066x.57.12.1052 · Cited by 86
The Empirically Validated Treatments Movement: A Practitioner/Educator Perspective
Levant, R. (2006) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.bph075 · Cited by 85
van Loon, A., van Schaik, A., Dekker, J., & Beekman, A. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.12.014 · Cited by 73
Establishment of remission criteria for anxiety disorders.
Doyle, A., & Pollack, M. H. (2003) · PubMed · Cited by 72
Milette, K., Roseman, M., & Thombs, B. D. (2010) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2010.09.015 · Cited by 71
Self-citationPsychotherapy research is not psychotherapy practice.
Fensterheim, H., & Raw, S. D. (1996) · Clinical Psychology Science and Practice · doi:10.1111/j.1468-2850.1996.tb00067.x · Cited by 67
Psychological Mechanisms of Effective Cognitive–Behavioral Treatments for PTSD
Zalta, A. K. (2015) · Current Psychiatry Reports · doi:10.1007/s11920-015-0560-6 · Cited by 66 · cited 1×
“As just one example, a recent review concluded that the mechanisms of prolonged exposure (PE) for posttraumatic stress disorder (PTSD), which is a well-studied and fairly straightforward treatment, remain unclear (Zalta, 2015).”
BackgroundMeasuring quality of life in psychiatry
Hunt, S. M., & McKenna, S. P. (1993) · Springer eBooks · doi:10.1007/978-94-011-2988-6_20 · Cited by 63
Measurable outcomes in psychiatric disorders: Remission as a marker of wellness
McIntyre, R. S., Fallu, A., & Konarski, J. Z. (2006) · Clinical Therapeutics · doi:10.1016/j.clinthera.2006.11.007 · Cited by 59
Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study
Roseman, M., Turner, E. H., Lexchin, J., Coyne, J. C., Bero, L. A., & Thombs, B. D. (2012) · BMJ · doi:10.1136/bmj.e5155 · Cited by 55
Self-citationThe Tangled and Thorny Path of Science to Practice: Tensions in Interpreting and Applying "Evidence"
Gonzales, J. J., Ringeisen, H. L., & Chambers, D. (2002) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.9.2.204 · Cited by 55
Evidence-Based Practice Is Not Synonymous With Delivery of Uniform Health Care
Djulbegović, B., & Guyatt, G. H. (2014) · JAMA · doi:10.1001/jama.2014.10713 · Cited by 51
Interpersonal psychotherapy for depression? The need to define its ecological niche
Parker, G. B., Parker, I., Brotchie, H., & Stuart, S. P. (2006) · Journal of Affective Disorders · doi:10.1016/j.jad.2006.03.019 · Cited by 48
Implementation of Evidence-Based Psychological Treatments in the Veterans Health Administration
Ruzek, J. I., Karlin, B. E., & Zeiss, A. M. (2015) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195389050.003.0005 · Cited by 48
Goldfried, M. R., & Eubanks, C. F. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.4.669 · Cited by 47
Dobson, K. S., & Beshai, S. (2013) · Behavior Therapy · doi:10.1016/j.beth.2013.03.002 · Cited by 47
Keltner, J. R., Vaida, F., Ellis, R. J., Moeller‐Bertram, T., Sanders, C., Duarte, N. A., Robinson‐Papp, J., Dworkin, R. H., Clifford, D. B., McArthur, J. C., et al. (2012) · Psychosomatics · doi:10.1016/j.psym.2012.05.002 · Cited by 45
Empirical evidence for using subjective quality of life as an outcome variable in clinical studies
Vatne, S., & Bjørkly, S. K. (2008) · Clinical Psychology Review · doi:10.1016/j.cpr.2008.01.001 · Cited by 44
Allegiance effects in treatment research: A commentary.
Hollon, S. D. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.1.107 · Cited by 44
Establishment of Remission Criteria for Anxiety Disorders
Doyle, A., & Pollack, M. H. (2003) · The Primary Care Companion to The Journal of Clinical Psychiatry · Cited by 35
High Quality Psychotherapy Research: From Conception to Piloting to National Trials
Areán, P. A., & KRAEMER, H. C. (2013) · Oxford University Press eBooks · doi:10.1093/med:psych/9780199782468.001.0001 · Cited by 34
Collegially validated limitations of empirically validated treatments.
Norcross, J. C. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.4.472 · Cited by 29
Fonagy, P. (1999) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.6.4.442 · Cited by 29
Riehm, K. E., Azar, M., & Thombs, B. D. (2015) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2015.04.010 · Cited by 27
Self-citation2022 VA/DoD clinical practice guideline: The management of major depressive disorder
Sall, J. (2023) · PsycEXTRA Dataset · doi:10.1037/e506062025-001 · Cited by 26
Kushner, S. C., Quilty, L. C., McBride, C., & Bagby, R. M. (2009) · Depression and Anxiety · doi:10.1002/da.20566 · Cited by 24
Measures of clinically significant change.
Lambert, M. J., & Bailey, R. J. (2012) · American Psychological Association eBooks · doi:10.1037/13621-007 · Cited by 17
Gibbons, C. R., Wiltsey Stirman, S., DeRubeis, R. J., Newman, C. F., & Beck, A. T. (2013) · Cognitive Therapy and Research · doi:10.1007/s10608-012-9499-7 · Cited by 15
Baird, J. (2010) · Assessment in Education Principles Policy and Practice · doi:10.1080/09695941003732357 · Cited by 12
Bernard, L., Brandwein, R. A., Braverman, L., Chernesky, R. H., Dinerman, M., Gottlieb, N., Gross, E. R., Meyer, C. H., Queiro‐Tajalli, I., Sancier, B., et al. (1990) · Affilia · doi:10.1177/088610999000500108 · Cited by 8
Trial Registration in Rheumatology: The Next Step
Thombs, B. D., Kwakkenbos, L., & Coronado‐Montoya, S. (2014) · Arthritis Care & Research · doi:10.1002/acr.22335 · Cited by 4
Self-citationThe drive towards evidence-based health care
Barkham, M., & Mellor‐Clark, J. (2013) · Routledge eBooks · doi:10.4324/9780203131602-10 · Cited by 3
Research works printed but not linked: 8
- American Psychological Association, 1995, Template for developing guidelines: Interventions for mental disorders and psychosocial aspects of physical disorders
- Baker, 2000, Evidence-based counseling and psychological therapies: Research and applications, 13
- Chambless, 1998, Update on empirically validated therapies. II, The Clinical Psychologist, 51, 3
- Lambert, 1996, Administration and scoring manual for the Outcome Questionnaire (OQ-45.2)
- National Institute for Clinical Excellence, 2011, Generalised anxiety disorder and panic disorder (with or without agoraphobia) in adults: Management in primary, secondary and community care (National Clinical Practice Guideline number CG113)
- Ruzek, 2012, Dissemination of evidence-based psychological treatments, 78
- Task Force on Promotion and Dissemination of Psychological Procedures, 1993, Training in and dissemination of empirically-validated psychological treatments: Report and recommendation, The Clinical Psychologist, 48, 3
- Tolin, 2014, Evidence-based practice: Three-legged stool or filter system?, The Clinical Psychologist, 67, 1
Cited by
The Future of Intervention Science: Process-Based Therapy
Hofmann, S. G., & Hayes, S. C. (2019) · Clinical Psychological Science · doi:10.1177/2167702618772296 · Cited by 888
Kazdin, A. E. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.06.004 · Cited by 554
Stade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00056-z · Cited by 391
Kazantzis, N., Whittington, C., Zelencich, L., Kyrios, M., Norton, P. J., & Hofmann, S. G. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.05.002 · Cited by 280
Tackett, J. L., Lilienfeld, S. O., Patrick, C. J., Johnson, S. L., Krueger, R. F., Miller, J. D., Oltmanns, T. F., & Shrout, P. E. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617690042 · Cited by 251
Metadata
- Authors
- Tolin, D. F., McKay, D., Forman, E. M., Klonsky, E. D., & Thombs, B. D.
- Affiliations
- Yale University (US)
- Fordham University (US)
- Drexel University (US)
- University of British Columbia (CA)
- Jewish General Hospital (CA)
- Corresponding author
- Tolin, David F.
- Venue
- Clinical Psychology: Science and Practice
- Published
- 2015-11-09
- Volume
- 22
- Issue
- 4
- Pages
- 317-338
- ISSN
- 0969-5893 (ISSN-L)
- OpenAlex
- W2103164132
- Cited by
- 219
- References
- 135
- Citations by year
- 2026: 13 · 2025: 35 · 2024: 22 · 2023: 21 · 2022: 21 · 2021: 14 · 2020: 35 · 2019: 17 · 2018: 16 · 2017: 11 · 2016: 11 · 2015: 3
- FWCI
- 12.47
- Type
- journal article
- Language
- en
- Open access
- Licence
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Locations
- Clinical Psychology Science and Practice (publishedVersion)
- eScholarship@McGill (McGill) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- empirically supported treatments, treatment criteria, treatment guidelines, systematic reviews, methodological quality, treatment outcomes, treatment setting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.