Evidence that therapy works in clincally representative conditions.
The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.
Abstract
This article reports a secondary analysis of past therapy outcome meta-analysis. Fifteen meta-analysis provided effect sizes from 56 studies in previous reviews that met 1 of 3 increasingly stringent levels of criteria for clinical representativeness. The effect sizes were synthesized and compared with results from the original meta-analyses. Effect sizes from more clinically representative studies are the same size at all 3 criteria levels as in past meta-analyses. Almost no studies exist that meet the most stringent level of criteria. Results are interpreted cautiously because of controversy about what criteria best capture the notion of clinical representativeness, because so few experiments have tested therapy in clinical conditions, and because other models for exploring the generalizability of therapy outcome research to clinical conditions might yield different results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Second, clinical representativeness of studies is often rated based on a multitude of criteria, besides recruitment type, such as setting of treatment delivery, experience of therapists, and flexibility in treatment manuals [9,109].”
From Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis (2019) · cites this paper as [109]
Citations
Cites
The Handbook of Social Psychology
Gravatt, A. E., Lindzey, G., & Aronson, F. (1969) · The Family Coordinator · doi:10.2307/582242 · Cited by 12,517
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Handbook of Psychotherapy and Behavior Change
Garfield, S. L., Bergin, A. E., & Dryden, W. (1987) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.1.4.264a · Cited by 3,997
Smith, M. L., Glass, G. V., & Miller, T. I. (1980) · Medical Entomology and Zoology · Cited by 2,123
Self-citationLipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
The effectiveness of psychotherapy: the Consumer Reports study
Seligman, M. E. P. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.965 · Cited by 1,608
Update on empirically validated therapies, II: (619622010-001)
Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun, K. S., Crits-Christoph, P., Daiuto, A., DeRubeis, R., Detweiler, J., Haaga, D. A. F., et al. (1997) · PsycEXTRA Dataset · doi:10.1037/e619622010-001 · Cited by 1,488
Self-citationEffectiveness of psychotherapy
(2011) · Revision MCQs and EMIs for the MRCPsych · doi:10.1201/b13482-76 · Cited by 1,058
Robinson, L. A., Berman, J. S., & Neimeyer, R. A. (1990) · Psychological Bulletin · doi:10.1037//0033-2909.108.1.30 · Cited by 671
Self-citation
Cited by
Morley, S., Eccleston, C., & Williams, A. (1999) · Pain · doi:10.1016/s0304-3959(98)00255-3 · Cited by 1,597
“5, is concordant with those from larger meta-analyses for psychological treatments for a variety of disorders (Shadish et al., 1997), and our conclusion is similar to that reached by Compas et al.”
ResultInfluentialThe psychotherapy dose-response effect and its implications for treatment delivery services.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.3.329 · Cited by 783
“…& Yap, 1997; Gould, Otto, Pollack, & Liang, 1997; Gould, Otto, & Pollack, 1995; Lambert & Bergin, 1994; Nietzal et al., 1987; Roth & Fonagy, 1996; Shadish et al., 1997, 2000; Stevens, Hynan, & Allen, 2000;Taylor, 1996;van Balkom et al., 1997;Wampold et al., 1997), and studies reviewed and…”
BackgroundAcceptance and Values-Based Action in Chronic Pain: A Study of Treatment Effectiveness and Process
Vowles, K. E., & McCracken, L. M. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.76.3.397 · Cited by 500
“These results mirror the degree of success CBT has achieved within medical care settings in general (Compas, Haage, Keefe, Leitenberg, & Williams, 1999; Shadish et al., 1997).”
ResultThe Dodo bird verdict is alive and well-mostly.
Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T. P., Berman, J. S., Levitt, J. T., Seligman, D. A., & Krause, E. D. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.1.2 · Cited by 581
“We will not deal here with the level of efficacy of these treatments, for there is much evidence already for their mostly good level of efficacy (e.g., Lambert & Bergin, 1994; Lipsey & Wilson, 1993; Shadish et al., 1997).”
BackgroundResultPsychotherapy for Children and Adolescents
Kazdin, A. E. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145105 · Cited by 462
Wise, E. A. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8201_10 · Cited by 399
“If, on the other hand, therapy is at all effective (e.g., Nietzel et al., 1987; Robinson et al., 1990; Shadish et al., 1997; Smith, Glass, & Miller, 1980; Wampold, 2001), then the change would not necessarily be symmetrical and could result in more people classified as Improved rather than…”
BackgroundThe effects of psychological therapies under clinically representative conditions: a meta-analysis
Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.4.512 · Cited by 381
Self-citationInternet interventions: Past, present and future
Andersson, G. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.03.008 · Cited by 315
Family interventions in the treatment of alcohol and drug problems
COPELLO, A. G., VELLEMAN, R. D. B., & TEMPLETON, L. J. (2005) · Drug and Alcohol Review · doi:10.1080/09595230500302356 · Cited by 296
“sentativeness [164] of the studies reviewed.”
Barlow, D. H. (2004) · American Psychologist · doi:10.1037/0003-066x.59.9.869 · Cited by 271
“Thus, it is encouraging that recent efforts to implement these specific psychological treatments comprehensively in front-line clinics, where some patients would not meet the inclusion criteria of many research studies, have produced results every bit as robust as those from research clinics when the treatments are ad…”
BackgroundResearch on psychotherapy efficacy and effectiveness: Between Scylla and Charybdis?
Nathan, P. E., Stuart, S. P., & Dolan, S. L. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.6.964 · Cited by 258
Gunter, R. W., & Whittal, M. L. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.11.001 · Cited by 224
“More broadly, Shadish et al. (1997) meta-analyzed 56 treatment studies, categorizing their samples on a continuum from least to most clinically representative.”
BackgroundEffects of Behavioral Marital Therapy: A Meta-Analysis of Randomized Controlled Trials.
Shadish, W. R., & Baldwin, S. A. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.1.6 · Cited by 215
Self-citationWestbrook, D., & Kirk, J. (2005) · Behaviour Research and Therapy · doi:10.1016/j.brat.2004.09.006 · Cited by 183
A vision of the next generation of behavioral therapies research in the addictions*
Carroll, K. M., & Rounsaville, B. J. (2007) · Addiction · doi:10.1111/j.1360-0443.2007.01798.x · Cited by 177
Guthrie, E. (1999) · Archives of General Psychiatry · doi:10.1001/archpsyc.56.6.519 · Cited by 180
“2) [0-42] .”
InfluentialEffectiveness of Guided Internet-Based Cognitive Behavior Therapy in Regular Clinical Settings
Andersson, G., & Hedman, E. (2013) · Verhaltenstherapie · doi:10.1159/000354779 · Cited by 165
“In practice, this means that research participants tend to be highly selected, homogenous in clinical characteristics, self-referred, and therapists are well trained and monitored for adherence to treatment manuals [Shadish et al., 1997].”
Predictors of Engagement in Adolescent Drug Abuse Treatment
DAKOF, G. A., TEJEDA, M., & LIDDLE, H. A. (2001) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200103000-00006 · Cited by 161
“Even though the necessity for and efficacy of behavioral interventions for adolescent problems are well recognized (Kazdin and Weisz, 1998; Shadish et al., 1997), many youths fail to obtain treatment (U.”
BackgroundFoundations of idiographic methods in psychology and applications for psychotherapy
Piccirillo, M. L., & Rodebaugh, T. L. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2019.01.002 · Cited by 160
Nordgren, L. B., Hedman, E., Etienne, J., Bodin, J., Kadowaki, Å., Eriksson, S., Lindkvist, E., Andersson, G., & Carlbring, P. (2014) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.05.007 · Cited by 145
Psychotherapy With Persons Who Have Mental Retardation: An Evaluation of Effectiveness
Prout, H. T., & Nowak-Drabik, K. M. (2003) · American Journal on Mental Retardation · doi:10.1352/0895-8017(2003)108<0082:pwpwhm>2.0.co;2 · Cited by 140
Self-citationKraus, D. R., Seligman, D. A., & Jordan, J. R. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20084 · Cited by 132
Outcomes of patients completing and not completing cognitive therapy for depression
Cahill, J., Barkham, M., Hardy, G., Rees, A., Shapiro, D. A., Stiles, W. B., & Macaskill, N. (2003) · British Journal of Clinical Psychology · doi:10.1348/014466503321903553 · Cited by 129
“…of specific brands of therapy under varying degrees of rigour is essential for assessing the generalizability of efficacy findings to practice (Shadish et al., 1997; Shadish, Navarro, Matt, & Phillips, 2000) and making policy recommendations regarding clients who unilaterally terminate…”
BackgroundPsychological testing and psychological assessment: A review of evidence and issues.
Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2001) · American Psychologist · doi:10.1037//0003-066x.56.2.128 · Cited by 1,469
Minami, T., Wampold, B. E., Serlin, R. C., Hamilton, E. G., Brown, G. S. (., & Kircher, J. C. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.76.1.116 · Cited by 120
(1997) · Psychiatric Services · doi:10.1176/ps.48.12.1543 · Cited by 117
Practice makes perfect and other myths about mental health services.
Bickman, L. (1999) · American Psychologist · doi:10.1037/h0088206 · Cited by 123
The effectiveness of computerized cognitive behavioural therapy in routine care
Cavanagh, K., Shapiro, D. A., Van den Berg, S., Swain, S., Barkham, M., & Proudfoot, J. (2006) · British Journal of Clinical Psychology · doi:10.1348/014466505x84782 · Cited by 113
Benchmarks for psychotherapy efficacy in adult major depression.
Minami, T., Wampold, B. E., Serlin, R. C., Kircher, J. C., & Brown, G. S. (. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.2.232 · Cited by 113
“Despite some evidence from clinical trials suggesting that efficacy corresponds to effectiveness (e.g., Shadish, Matt, Navarro, & Phillips, 2000; Shadish et al., 1997), there are few outcome data from clinical settings to suggest that treatments in these settings (i.e., treatment as usual; TAU)…”
BackgroundSexton, T. L., Gilman, L., & Johnson—Erickson, C. (2007) · Springer eBooks · doi:10.1007/0-387-23846-8_6 · Cited by 113
STARTUP, M., JACKSON, M. C., & BENDIX, S. (2004) · Psychological Medicine · doi:10.1017/s0033291703001211 · Cited by 111
“Indeed, it met criteria for ‘Stage 1’ representative conditions, in the terminology of Shadish et al. (1997), in the following respects : consecutive admissions to hospitals meeting lenient criteria were invited to participate (rather than patients specially solicited or selected for the research);…”
BackgroundRuiz, M. A., Pincus, A. L., Borkovec, T. D., Echemendia, R. J., Castonguay, L. G., & Ragusea, S. A. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8303_05 · Cited by 111
“However, one speculation is that the inconsistencies between these and previously reported findings are due to the constellation of variables that differentiates the PRN from previous studies (Shadish et al., 1997).”
ResultAre waiting-list control groups needed in future marital therapy outcome research?
Baucom, D. H., Hahlweg, K., & Kuschel, A. (2003) · Behavior Therapy · doi:10.1016/s0005-7894(03)80012-6 · Cited by 128
Effectiveness of an empirically supported treatment for social phobia in the field
Lincoln, T. M., Rief, W., Hahlweg, K., Frank, M., von Witzleben, I., Schroeder, B., & Fiegenbaum, W. (2003) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(03)00038-x · Cited by 121
Behavioral self-control training for problem drinkers: A meta-analysis of randomized control studies
Walters, G. D. (2000) · Behavior Therapy · doi:10.1016/s0005-7894(00)80008-8 · Cited by 116
Systematic review of practice‐based research on psychological therapies in routine clinic settings
Cahill, J., Barkham, M., & Stiles, W. B. (2010) · British Journal of Clinical Psychology · doi:10.1348/014466509x470789 · Cited by 92
“A strategic review of psychotherapy services in the UK (Department of Health, 1996) as well as critical scholarly reviews (e.g., Roth & Parry, 1997; Shadish et al., 1997; Shadish, Navarro, Matt, & Phillips, 2000) have emphasized the need for evidence”
BackgroundProgress review of the psychosocial treatment of child conduct problems.
Nock, M. K. (2003) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.10.1.1 · Cited by 88
“fererent settings are the result of methodological artifacts (Shadish, Matt, Navarro, & Phillips, 2000; Shadish et al., 1997).”
BackgroundLegenbauer, T., Petrak, F., de Zwaan, M., & Herpertz, S. (2010) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2010.06.012 · Cited by 95
“7) [18-65] 38.”
ResultShort- and long-term effectiveness of an empirically supported treatment for agoraphobia.
Hahlweg, K., Fiegenbaum, W., Frank, M., Schroeder, B., & von Witzleben, I. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.69.3.375 · Cited by 83
Ree, M. J., & Craigie, M. A. (2007) · Behaviour Change · doi:10.1375/bech.24.2.70 · Cited by 82
Bridging the Enormous Gaps of Theory With Therapy Research and Practice
Kazdin, A. E. (2001) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3001_7 · Cited by 81
“When studies are reviewed that focus on the clients, conditions, and treatments used in clinical work, the number of studies is quite small, and reviewers do not agree whether treatment works in this context (Shadish et al., 1997; Weisz, Donenberg, Han, & Weiss, 1995).”
BackgroundEffectiveness of cognitive behavioral group therapy for depression in routine practice
Thimm, J. C., & Antonsen, L. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0292-x · Cited by 78
“To define the clinical representativeness of a study, the authors suggested the following criteria based on the work of Shadish and colleagues [28,29]: (a) non-university setting; (b) referred patients; (c) professional therapists with regular caseloads; (d) flexible structure; (e) no monitoring of treatment implement…”
MethodA Meta-Analysis of School-Based Studies of Counseling and Psychotherapy
Prout, S. M., & Prout, H. T. (1998) · Journal of School Psychology · doi:10.1016/s0022-4405(98)00007-7 · Cited by 76
Self-citationKopta, S. M., Lueger, R. J., Saunders, S. M., & Howard, K. I. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.441 · Cited by 72
Quarmby, L. M., Rimes, K. A., Deale, A., Wessely, S. C., & Chalder, T. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2006.08.019 · Cited by 73
“Furthermore, the therapists taking part in a trial may receive specific training and supervision and may have resources available to them that it is unrealistic to expect of everyday clinicians (Shadish et al., 1997; Wilson, 1998).”
BackgroundMcGinn, L. K., & Sanderson, W. C. (2001) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.8.1.23 · Cited by 70
Effects of psychological therapies in randomized trials and practice‐based studies
Barkham, M., Stiles, W. B., Connell, J., Twigg, E., Leach, C., Lucock, M., Mellor‐Clark, J., Bower, P., King, M., Shapiro, D. A., et al. (2008) · British Journal of Clinical Psychology · doi:10.1348/014466508x311713 · Cited by 68
Efficacy, Effectiveness and Efficiency in Mental Health Service Delivery
Andrews, G. (1999) · Australian & New Zealand Journal of Psychiatry · doi:10.1046/j.1440-1614.1999.00581.x · Cited by 68
Identifying highly effective psychotherapists in a managed care environment.
Brown, G. S. J., Lambert, M. J., Jones, E. R., & Minami, T. (2005) · PubMed · Cited by 82
Building a Rigorous and Relevant Knowledge Base for the Psychological Therapies
Barkham, M., Stiles, W. B., Lambert, M. J., & Mellor‐Clark, J. (2010) · Wiley eBooks · doi:10.1002/9780470687994.ch2 · Cited by 74
de C Williams, A. C., Nicholas, M. K., Richardson, P. H., Pither, C. E., & Fernandes, J. (1999) · Pain · doi:10.1016/s0304-3959(99)00074-3 · Cited by 62
Guthrie, E. (2000) · British Journal of Psychiatry · doi:10.1192/bjp.177.2.131 · Cited by 61
Romijn, G., Batelaan, N., Kok, R., Koning, J., van Balkom, A., Titov, N., & Riper, H. (2019) · Journal of Medical Internet Research · doi:10.2196/11706 · Cited by 52
Cahill, J., Paley, G., & Hardy, G. E. (2012) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12015 · Cited by 82
So, C., Leung, P. W. L., & Hung, S. (2008) · Behaviour Research and Therapy · doi:10.1016/j.brat.2008.06.007 · Cited by 61
Carson, C. C., Giuliano, F., Goldstein, I., Hatzichristou, D. G., Hellstrom, W. J. G., Lue, T. F., Montorsi, F., Munarriz, R. M., Nehra, A. X., Porst, H., et al. (2004) · International Journal of Impotence Research · doi:10.1038/sj.ijir.3901227 · Cited by 54
The Anatomy of Evidence‐based Practice: Principles and Methods
Reynolds, S. A. (2000) · Wiley eBooks · doi:10.1002/9780470699003.ch2 · Cited by 71
Group cognitive analytic therapy for female survivors of childhood sexual abuse
Calvert, R., Kellett, S., & Hagan, T. (2015) · British Journal of Clinical Psychology · doi:10.1111/bjc.12085 · Cited by 50
Kaldo, V., Haak, T., Buhrman, M., Alfonsson, S., Larsen, H.-C., & Andersson, G. (2013) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2013.769622 · Cited by 48
“The study can be described as an effectiveness trial (Shadish et al., 1997), as regular therapists provided the treatment in a regular clinical setting and no advertisements were used to recruit patients.”
BackgroundThe clinical effectiveness of cognitive therapy for depression in an outpatient clinic
Gibbons, C. J., Fournier, J. C., Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., & Beck, A. T. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.12.030 · Cited by 46
“One area of concern with such evaluations, however, is the extent to which treatment in the “realworld” has been assessed under clinically representative conditions (Shadish et al., 1997; Shadish et al., 2000).”
BackgroundSelf-citationLutz, W., Schiefele, A.-K., Wucherpfennig, F., Rubel, J. A., & Stulz, N. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.08.072 · Cited by 56
Mathiasen, K., Riper, H., Andersen, T. E., & Roessler, K. K. (2018) · Journal of Medical Internet Research · doi:10.2196/10927 · Cited by 45
Gaskell, C., Simmonds‐Buckley, M., Kellett, S., Stockton, C., Somerville, E., Rogerson, E., & Delgadillo, J. (2022) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-022-01225-y · Cited by 70
Dwyer, L. A., Hornsey, M. J., Smith, L. G. E., Oei, T. P. S., & Dingle, G. A. (2011) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2011.30.1.24 · Cited by 40
Durham, R. C., Fisher, P. L., Dow, M. G. T., Sharp, D., Power, K. G., Swan, J. S., & Morton, R. V. (2004) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.405 · Cited by 40
The principle of parallelism in the design of studies to estimate treatment effects.
Reichardt, C. S. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.1.1 · Cited by 42
“But in any given research instance, random selection differences are likely to be nonzero, in which case their effects are confounded with the treatment effect.”
BackgroundHiller, W., Schindler, A., Andor, T., & Rist, F. (2011) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000063 · Cited by 36
Effects of the Transcendental Meditation Program on Substance Use among University Students
Haaga, D. A. F., Grosswald, S. J., Gaylord‐King, C., Rainforth, M. V., Tanner, M., Travis, F., Nidich, S. I., & Schneider, R. H. (2011) · Cardiology Research and Practice · doi:10.4061/2011/537101 · Cited by 33
“Studies of “solicited volunteers” and studies of patients referred through routine clinical channels appear to obtain similar results [24].”
ResultNorquist, G., & Hyman, S. E. (1999) · Health Affairs · doi:10.1377/hlthaff.18.5.32 · Cited by 33
Askey-Jones, S., David, A. S., Silber, E., Shaw, P., & Chalder, T. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.04.001 · Cited by 32
“Given the potential for discrepant results, criteria were developed by Shadish et al. (1997) and Shadish, Matt, Navarro, and Philips (2000) to assess whether studies carried out in routine clinical practice were clinically representative.”
BackgroundBias in Meta-Analytic Estimates of the Absolute Efficacy of Psychotherapy
Staines, G. L., & Cleland, C. M. (2007) · Review of General Psychology · doi:10.1037/1089-2680.11.4.329 · Cited by 35
“To illustrate the magnitude of the bias, when Shadish et al. (1997) reanalyzed Smith et al.’s (1980) raw data by weighting for sample size, the overall ES estimate was reduced from 0.85 to 0.60.”
BackgroundMethodInfluentialFrom the lab to the clinic: effectiveness of cognitive-behavioral treatments for anxiety disorders
van Ingen, D. J., Freiheit, S. R., & Vye, C. S. (2009) · Professional Psychology Research and Practice · doi:10.1037/a0013318 · Cited by 30
Peeters, F. P. M. L., Huibers, M. J. H., Roelofs, J., van Breukelen, G., Hollon, S. D., Markowitz, J. C., van Os, J., & Arntz, A. R. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.08.022 · Cited by 40
“Third, treatments within the context of a RCT are highly controlled in content, duration, number of sessions, therapist qualifications, and patient choice, complicating translation to real-world clinical practice (Cuijpers et al., 2010b; Shadish et al., 2000; Shadish et al., 1997).”
BackgroundWatzke, B., Rüddel, H., Koch, U., Rudolph, M., & Schulz, H. (2008) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.595 · Cited by 36
Evaluating evaluation: Where do we go from here?
Bickman, L. (1997) · Evaluation Practice · doi:10.1016/s0886-1633(97)90003-9 · Cited by 34
Evidence based practice and psychotherapy research
Reynolds, S. A. (2000) · Journal of Mental Health · doi:10.1080/713680248 · Cited by 34
Valdés, N., Krause, M., Vilches, O., Dagnino, P., Echávarri, O., Ben-Dov, P., Arístegui, R., & de la Parra, G. (2005) · Psykhe (Santiago) · doi:10.4067/s0718-22282005000200001 · Cited by 26
Barkham, M., Gilbert, N. L., Richards, A., & Cameron, I. (2005) · Mental Health in Family Medicine · Cited by 26
Henton, I. (2012) · Counselling Psychology Review · doi:10.53841/bpscpr.2012.27.3.11 · Cited by 54
Feaster, D. J., Newman, F. L., & Rice, C. (2003) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.382 · Cited by 25
“…Miller, & Brown, 2001; Miller & Wilbourne, 2002; Moyer, Finney, Swearingen, & Vergun 2002; Prendergast, Podus, & Chang, 2000; Seligman, 1995; Shadish et al., 1993, 1997; Stanton & Shadish, 1997; Wilson, 2000) that inform policy makers on whether psychotherapy of a particular dose is…”
Dissemination of Youth ESTs: Ready for Prime Time?
Shirk, S. R. (2004) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.bph086 · Cited by 25
Falkenström, F. (2009) · Psychotherapy Research · doi:10.1080/10503300903170954 · Cited by 23
“…1050-3307 print/ISSN 1468-4381 online # 2010 Society for Psychotherapy Research DOI: 10.1080/10503300903170954 meta-analyses by Shadish et al. (1997) and Shadish, Navarro, Matt, and Phillips (2000) showed effect sizes for clinic therapy comparable to those yielded by…”
ResultInfluentialOutcome study of brief relational-cultural therapy in a women's mental health center
Oakley, M. A., Addison, S. C., Piran, N., Johnston, G. J., Damianakis, M., Curry, J., Dunbar, C., & Weigeldt, A. (2013) · Psychotherapy Research · doi:10.1080/10503307.2012.745956 · Cited by 27
“Additionally, naturalistic psychotherapy studies are conducted with heterogeneous samples where co-morbidity is present (Shadish et al., 1997; Shadish, Matt, Navarro, & Phillips, 2000; Weersing & Weisz, 2002), increasing their generalizability to real-world clinical populations (Stirman, DeRubeis,…”
BackgroundLambert, M. J. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.69.6.910 · Cited by 23
Historical Paths in Psychotherapy Research
Braakmann, D. (2014) · Springer eBooks · doi:10.1007/978-3-7091-1382-0_3 · Cited by 32
Practice‐informed research and research‐informed psychotherapy
Lueger, R. J. (2002) · Journal of Clinical Psychology · doi:10.1002/jclp.10110 · Cited by 27
ROSENBERG, N. K., & HOUGAARD, E. (2005) · Nordic Journal of Psychiatry · doi:10.1080/08039480510027670 · Cited by 21
“(3) suggest the following criteria for clinically representative studies: 1) treatment is conducted in a non-university setting, 2) it involves patients referred through usual clinical routes rather than solicited by the experimenter, 3) it uses patients heterogeneous in personal characteristics, 4) therapists do not…”
BackgroundCastro, M., Cockerton, T., & Birke, S. (2002) · The British Journal of Forensic Practice · doi:10.1108/14636646200200019 · Cited by 21
The Current View of Evidence and Evidence‐Based Practice
Bower, P., & Gilbody, S. M. (2010) · Wiley eBooks · doi:10.1002/9780470687994.ch1 · Cited by 26
Disorders of Childhood and Adolescence
Mash, E. J., & Wolfe, D. A. (2003) · Wiley eBooks · doi:10.1002/0471264385.wei0802 · Cited by 22
Lucock, M., Kirby, R., & Wainwright, N. (2011) · British Journal of Clinical Psychology · doi:10.1348/014466510x520231 · Cited by 20
“The importance of good quality effectiveness research in routine services to complement efficacy research has been widely advocated (e.g., Roth & Fonagy, 2005; Shadish et al., 1997; Shadish, Navarro, Matt, & Phillips, 2000).”
BackgroundPaley, G., Cahill, J., Barkham, M., Shapiro, D. A., Jones, J., Patrick, S., & REID, E. (2008) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1348/147608307x270889 · Cited by 28
Margraf, J. (2009) · Springer eBooks · doi:10.1007/978-3-540-79541-4_1 · Cited by 20
An Introduction to the Best Practices Section in the Journal of Counseling & Development
Marotta, S. A., & Watts, R. E. (2007) · Journal of Counseling & Development · doi:10.1002/j.1556-6678.2007.tb00617.x · Cited by 19
Antidepressants, Psychotherapy or their Combination: Weighing Options for Depression Treatments
Greenberg, R. P., & Goldman, E. D. (2008) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-008-9092-2 · Cited by 23
“Today, psychotherapy is recognized as an effective treatment option (Shadish et al. 1997; Shadish et al. 2000) with numerous meta-analyses consistently providing evidence of treatment benefits (Lambert and Ogles 2004; Lipsey and Wilson 1993; Wampold 2001).”
BackgroundWise, E. A. (2003) · Psychotherapy · doi:10.1037/0033-3204.40.3.203 · Cited by 23
“Shadish et al. (1997) found that in clinically representative studies utilizing control groups, the average effect size d equaled .59.”
How much do sample characteristics affect the effect size?
Lincoln, T. M., & Rief, W. (2003) · Journal of Anxiety Disorders · doi:10.1016/s0887-6185(03)00040-9 · Cited by 22
Effectiveness of Open-Ended Psychotherapy Under Clinically Representative Conditions
Nordmo, M., Sønderland, N. M., Havik, O. E., Eilertsen, D., Monsen, J. T., & Solbakken, O. A. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00384 · Cited by 20
“(14) categorized studies with a criterion-based approach.”
ResultEmpirically certified treatments or therapists: The issue of separability.
Krause, M. S., Lutz, W., & Saunders, S. M. (2007) · Psychotherapy · doi:10.1037/0033-3204.44.3.347 · Cited by 18
“…(UPR), credible problem-relevant information, fruitful insight-evocative interpretations of problematic actions or experiences, and more (see, e.g., Howard, Krause, Caburnay, Noel, & Saunders, 2001; Lambert & Archer, 2006; Lambert & Ogles, 2004; Lipsey & Wilson, 1993; Shadish et al., 1997, 2000).”
Smakowski, A., Adamson, J., Turner, T., & Chalder, T. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1949049 · Cited by 17
Metadata
- Authors
- Shadish, W. R., Matt, G. E., Navarro, A. M., Siegle, G., Crits-Christoph, P., Hazelrigg, M. D., Jorm, A. F., Lyons, L. C., Nietzel, M. T., Robinson, L., Prout, H. T., Smith, M. L., Svartberg, M., & Weiss, B.
- Affiliations
- University of Memphis (US)
- ORCID
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 1997
- Volume
- 65
- Issue
- 3
- Pages
- 355-365
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2049024183
- Cited by
- 186
- References
- 94
- Citations by year
- 2025: 2 · 2024: 3 · 2023: 3 · 2022: 3 · 2021: 3 · 2020: 6 · 2019: 3 · 2018: 6 · 2017: 1 · 2016: 1 · 2015: 5 · 2014: 7 · 2013: 6 · 2012: 9
- FWCI
- 14.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- generalizability, effect size, clinical conditions
- MeSH terms
- Humans, Psychotherapy, Meta-Analysis as Topic, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.