Departing from the essential features of a high quality systematic review of psychotherapy: A response to Öst (2014) and recommendations for improvement
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Öst's (2014) systematic review and meta-analysis of Acceptance and Commitment Therapy (ACT) has received wide attention. On the basis of his review, Öst argued that ACT research was not increasing in its quality and that, in contradiction to the views of Division 12 of the American Psychological Association (APA), ACT is "not yet well-established for any disorder" (2014, p. 105). We conducted a careful examination of the methods, approach, and data used in the meta-analysis. Based in part on examinations by the authors of the studies involved, which were then independently checked, 91 factual or interpretive errors were documented, touching upon 80% of the studies reviewed. Comparisons of Öst's quality ratings with independent teams rating the same studies with the same scale suggest that Ost's ratings were unreliable. In all of these areas (factual errors; interpretive errors; quality ratings) mistakes and differences were not random: Ost's data were dominantly more negative toward ACT. The seriousness, range, and distribution of errors, and a wider pattern of misinterpreting the purpose of studies and ignoring positive results, suggest that Öst's review should be set aside in future considerations of the evidence base for ACT. We argue that future published reviews and meta-analyses should rely upon diverse groups of scholars rather than a single individual; that resulting raw data should be made available for inspection and independent analysis; that well-crafted committees rather than individuals should design, apply and interpret quality criteria; that the intent of transdiagnostic studies need to be more seriously considered as the field shifts away from a purely syndromal approach; and that data that demonstrate theoretically consistent mediating processes should be given greater weight in evaluating specific interventions. Finally, in order to examine substantive progress since Öst's review, recent outcome and process evidence was briefly examined.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Psychotherapy for depression in children and adolescents
Watanabe, N., Churchill, R., Hunot, V., & Furukawa, T. A. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005334 · Cited by 24
RetractedAcceptance and commitment therapy: model, processes and outcomes
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2005.06.006 · Cited by 6,587
Self-citationPain: a review of three commonly used pain rating scales
Williamson, A., & Hoggart, B. (2005) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2005.01121.x · Cited by 3,083
Defining empirically supported therapies.
Chambless, D. L., & Hollon, S. D. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.1.7 · Cited by 3,067
Mediators and Mechanisms of Change in Psychotherapy Research
Kazdin, A. E. (2007) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091432 · Cited by 2,504
Validity of four pain intensity rating scales
Ferreira-Valente, M. A., Pais-Ribeiro, J. L., & Jensen, M. P. (2011) · Pain · doi:10.1016/j.pain.2011.07.005 · Cited by 1,872
Kazdin, A. E. (2008) · American Psychologist · doi:10.1037/0003-066x.63.3.146 · Cited by 1,112 · cited 1×
“advancing evidence-based therapy (Borkovec & Sibrava, 2005; David & Montgomery, 2011; Herbert & Gaudiano, 2005; Kazdin, 2008; Lohr, 2011) and have pointed with concern to weaknesses in these areas in mainstream CBT (Gaudiano, 2005; Kazdin, 2007; Longmore &”
BackgroundVeehof, M. M., Trompetter, H. R., Bohlmeijer, E. T., & Schreurs, K. M. G. (2016) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2015.1098724 · Cited by 737
Efficacy of the third wave of behavioral therapies: A systematic review and meta- analysis
Öst, L.-G. (2008) · Behaviour Research and Therapy · doi:10.1016/j.brat.2007.12.005 · Cited by 727
Acceptance and commitment therapy: A meta-analytic review
Powers, M. B., Zum Vörde Sive Vörding, M. B., & Emmelkamp, P. M. G. (2009) · Psychotherapy and Psychosomatics · doi:10.1159/000190790 · Cited by 619 · cited 1×
“Ruiz (2012), in a meta-analysis focused specifically on comparing ACT to CBT, found that ACT outperformed CBT overall, for depression and for quality of life in the studies analyzed.”
Background
Research works printed but not linked: 22
- A-Tjak, 2015, A meta-analysis of the efficacy of acceptance and commitment therapy for clinically relevant mental and physical health problems, Psychotherapy and Psychosomatics, 84, 30, 10.1159/000365764 · DOI · Paper page
- Aebi, 2012, The use of the development and well-being assessment (DAWBA) in clinical practice: A randomized trial, European Child & Adolescent Psychiatry, 21, 559, 10.1007/s00787-012-0293-6 · DOI · Paper page
- Arch, 2012, Randomized clinical trial of cognitive behavioral therapy (CBT) versus acceptance and commitment therapy (ACT) for mixed anxiety disorders, Journal of Consulting and Clinical Psychology, 80, 750, 10.1037/a0028310 · DOI · Paper page
- Benowitz, 2003, Nicotine metabolite ratio as a predictor of cigarette consumption, Nicotine & Tobacco Research, 5, 621, 10.1080/1462220031000158717 · DOI · Paper page
- Bethay, 2013, A controlled pilot evaluation of acceptance and commitment training for intellectual disability staff, Mindfulness, 4, 113, 10.1007/s12671-012-0103-8 · DOI
- David, 2011, The scientific status of psychotherapies: A new evaluative framework for evidence-based psychosocial interventions, Clinical Psychology: Science and Practice, 18, 89
- Forman, 2013, Comparison of acceptance-based and standard cognitive-based coping strategies for craving sweets in overweight and obese women, Eating Behaviors, 14, 64, 10.1016/j.eatbeh.2012.10.016 · DOI
- Goodman, 2000, The development and well-being Assessment: Description and initial validation of an integrated assessment of child and adolescent psychopathology, Journal of Child Psychology and Psychiatry, 41, 645, 10.1111/j.1469-7610.2000.tb02345.x · DOI · Paper page
- Han, 2015, Relationship between methamphetamine use history and segmental hair analysis findings of MA users, Forensic Science International, 254, 59, 10.1016/j.forsciint.2015.06.029 · DOI
- Hayes, 2008, Climbing our hills: A beginning conversation on the comparison of acceptance and commitment therapy and traditional cognitive behavioral therapy, Clinical Psychology: Science and Practice, 15, 286
- Insel, 2010, Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders, American Journal of Psychiatry, 167, 748, 10.1176/appi.ajp.2010.09091379 · DOI · Paper page
- Landis, 1977, The measurement of observer agreement for categorical data, Biometrics, 33, 159, 10.2307/2529310 · DOI · Paper page
- Lillis, 2009, Teaching acceptance and mindfulness to improve the lives of the obese: A preliminary test of a theoretical model, Annals of Behavioral Medicine, 37, 58, 10.1007/s12160-009-9083-x · DOI · Paper page
- Lohr, 2011, What is (and what is not) the meaning of evidence-based psychosocial intervention?, Clinical Psychology: Science and Practice, 18, 100
- Overall, 1962, The brief psychiatric rating scale, Psychological Reports, 10, 799, 10.2466/pr0.1962.10.3.799 · DOI · Paper page
- Patrick, 1994, The validity of self-reported smoking: A review and meta-analysis, American Journal of Public Health, 84, 1086, 10.2105/AJPH.84.7.1086 · DOI · Paper page
- Ruiz, 2012, Acceptance and commitment therapy versus traditional cognitive behavioral therapy: A systematic review and meta-analysis of current empirical evidence, International Journal of Psychology and Psychological Therapy, 12, 333
- Silverman, 2008, The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update, Journal of Clinical Child & Adolescent Psychology, 37, 1, 10.1080/15374410701817725 · DOI · Paper page
- Smout, 2010, Psychosocial treatment for methamphetamine use disorders: A preliminary randomized controlled trial of cognitive behavior therapy and acceptance and commitment therapy, Substance Abuse, 31, 98, 10.1080/08897071003641578 · DOI · Paper page
- Stotts, 2012, A stage I pilot study of acceptance and commitment therapy for methadone detoxification, Drug Alcohol Depend, 125, 215, 10.1016/j.drugalcdep.2012.02.015 · DOI · Paper page
- Turk, 2003, Core outcome domains for chronic pain clinical trials: IMMPACT recommendations, PAIN, 106, 337, 10.1016/j.pain.2003.08.001 · DOI · Paper page
- Wolfe, 1990, The American College of Rheumatology 1990 criteria for the classification of fibromyalgia, Arthritis & Rheumatology, 33, 160, 10.1002/art.1780330203 · DOI · Paper page
Cited by
Hayes, S. C., Ciarrochi, J., Hofmann, S. G., Chin, F., & Sahdra, B. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104155 · Cited by 234
Self-citationCurrent status of acceptance and commitment therapy for chronic pain: A narrative review
Feliu Soler, A., Montesinos, F., Gutiérrez-Martínez, O., Scott, W., McCracken, L., & Luciano, J. (2018) · Journal of Pain Research · doi:10.2147/jpr.s144631 · Cited by 198
O'Neill, L., Latchford, G., McCracken, L. M., & Graham, C. D. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.08.008 · Cited by 87
Prudenzi, A., Graham, C. D., Clancy, F., Hill, D., O'Driscoll, R., Day, F., & O'Connor, D. B. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.07.084 · Cited by 64
Dochat, C., Wooldridge, J. S., Herbert, M. S., Lee, M. W., & Afari, N. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.03.003 · Cited by 60
Metadata
- Authors
- Atkins, P. W. B., Ciarrochi, J., Gaudiano, B. A., Bricker, J. B., Donald, J., Rovner, G., Smout, M., Livheim, F., Lundgren, T., & Hayes, S. C.
- Venue
- Behaviour Research and Therapy
- Publisher
- Elsevier BV
- Published
- 2017-10
- Volume
- 97
- Pages
- 259-272
- ISSN
- 0005-7967 (ISSN-L)
- OpenAlex
- W2617641004
- Cited by
- 41
- References
- 71
- FWCI
- 1.58
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Departing from the essential features of a high quality systematic review of psychotherapy: A response to Öst (2014) and recommendations for improvement. https://app.cochat.ai/papers/10.1016/j.brat.2017.05.016
- The paper
- Atkins, P. W. B., Ciarrochi, J., Gaudiano, B. A., Bricker, J. B., Donald, J., Rovner, G., Smout, M., Livheim, F., Lundgren, T., & Hayes, S. C. (2017). Departing from the essential features of a high quality systematic review of psychotherapy: A response to Öst (2014) and recommendations for improvement. Behaviour Research and Therapy. https://doi.org/10.1016/j.brat.2017.05.016
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.