CoChat

Departing from the essential features of a high quality systematic review of psychotherapy: A response to Öst (2014) and recommendations for improvement

Open access
Atkins, Paul W.B., Ciarrochi, Joseph, Gaudiano, Brandon A., Bricker, Jonathan B., Donald, James, Rovner, Graciela, Smout, Matthew, Livheim, Fredrik, Lundgren, Tobias, Hayes, Steven C. · 2017 · Behaviour Research and Therapy
Cited by 41

The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.

Abstract

Öst's (2014) systematic review and meta-analysis of Acceptance and Commitment Therapy (ACT) has received wide attention. On the basis of his review, Öst argued that ACT research was not increasing in its quality and that, in contradiction to the views of Division 12 of the American Psychological Association (APA), ACT is "not yet well-established for any disorder" (2014, p. 105). We conducted a careful examination of the methods, approach, and data used in the meta-analysis. Based in part on examinations by the authors of the studies involved, which were then independently checked, 91 factual or interpretive errors were documented, touching upon 80% of the studies reviewed. Comparisons of Öst's quality ratings with independent teams rating the same studies with the same scale suggest that Ost's ratings were unreliable. In all of these areas (factual errors; interpretive errors; quality ratings) mistakes and differences were not random: Ost's data were dominantly more negative toward ACT. The seriousness, range, and distribution of errors, and a wider pattern of misinterpreting the purpose of studies and ignoring positive results, suggest that Öst's review should be set aside in future considerations of the evidence base for ACT. We argue that future published reviews and meta-analyses should rely upon diverse groups of scholars rather than a single individual; that resulting raw data should be made available for inspection and independent analysis; that well-crafted committees rather than individuals should design, apply and interpret quality criteria; that the intent of transdiagnostic studies need to be more seriously considered as the field shifts away from a purely syndromal approach; and that data that demonstrate theoretically consistent mediating processes should be given greater weight in evaluating specific interventions. Finally, in order to examine substantive progress since Öst's review, recent outcome and process evidence was briefly examined.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.brat.2017.05.016

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

1 retracted

Research works printed but not linked: 19

  1. A-Tjak, 2015, A meta-analysis of the efficacy of acceptance and commitment therapy for clinically relevant mental and physical health problems, Psychotherapy and Psychosomatics, 84, 30, 10.1159/000365764 · DOI · Paper page
  2. Aebi, 2012, The use of the development and well-being assessment (DAWBA) in clinical practice: A randomized trial, European Child & Adolescent Psychiatry, 21, 559, 10.1007/s00787-012-0293-6 · DOI · Paper page
  3. Arch, 2012, Randomized clinical trial of cognitive behavioral therapy (CBT) versus acceptance and commitment therapy (ACT) for mixed anxiety disorders, Journal of Consulting and Clinical Psychology, 80, 750, 10.1037/a0028310 · DOI · Paper page
  4. Benowitz, 2003, Nicotine metabolite ratio as a predictor of cigarette consumption, Nicotine & Tobacco Research, 5, 621, 10.1080/1462220031000158717 · DOI · Paper page
  5. Bethay, 2013, A controlled pilot evaluation of acceptance and commitment training for intellectual disability staff, Mindfulness, 4, 113, 10.1007/s12671-012-0103-8 · DOI
  6. David, 2011, The scientific status of psychotherapies: A new evaluative framework for evidence-based psychosocial interventions, Clinical Psychology: Science and Practice, 18, 89
  7. Forman, 2013, Comparison of acceptance-based and standard cognitive-based coping strategies for craving sweets in overweight and obese women, Eating Behaviors, 14, 64, 10.1016/j.eatbeh.2012.10.016 · DOI
  8. Han, 2015, Relationship between methamphetamine use history and segmental hair analysis findings of MA users, Forensic Science International, 254, 59, 10.1016/j.forsciint.2015.06.029 · DOI
  9. Insel, 2010, Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders, American Journal of Psychiatry, 167, 748, 10.1176/appi.ajp.2010.09091379 · DOI · Paper page
  10. Landis, 1977, The measurement of observer agreement for categorical data, Biometrics, 33, 159, 10.2307/2529310 · DOI · Paper page
  11. Lillis, 2009, Teaching acceptance and mindfulness to improve the lives of the obese: A preliminary test of a theoretical model, Annals of Behavioral Medicine, 37, 58, 10.1007/s12160-009-9083-x · DOI · Paper page
  12. Lohr, 2011, What is (and what is not) the meaning of evidence-based psychosocial intervention?, Clinical Psychology: Science and Practice, 18, 100
  13. Overall, 1962, The brief psychiatric rating scale, Psychological Reports, 10, 799, 10.2466/pr0.1962.10.3.799 · DOI · Paper page
  14. Patrick, 1994, The validity of self-reported smoking: A review and meta-analysis, American Journal of Public Health, 84, 1086, 10.2105/AJPH.84.7.1086 · DOI · Paper page
  15. Ruiz, 2012, Acceptance and commitment therapy versus traditional cognitive behavioral therapy: A systematic review and meta-analysis of current empirical evidence, International Journal of Psychology and Psychological Therapy, 12, 333
  16. Silverman, 2008, The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update, Journal of Clinical Child & Adolescent Psychology, 37, 1, 10.1080/15374410701817725 · DOI · Paper page
  17. Stotts, 2012, A stage I pilot study of acceptance and commitment therapy for methadone detoxification, Drug Alcohol Depend, 125, 215, 10.1016/j.drugalcdep.2012.02.015 · DOI · Paper page
  18. Turk, 2003, Core outcome domains for chronic pain clinical trials: IMMPACT recommendations, PAIN, 106, 337, 10.1016/j.pain.2003.08.001 · DOI · Paper page
  19. Wolfe, 1990, The American College of Rheumatology 1990 criteria for the classification of fibromyalgia, Arthritis & Rheumatology, 33, 160, 10.1002/art.1780330203 · DOI · Paper page

Cited by

See all 41

Metadata

Authors
Atkins, P. W. B., Ciarrochi, J., Gaudiano, B. A., Bricker, J. B., Donald, J., Rovner, G., Smout, M., Livheim, F., Lundgren, T., & Hayes, S. C.
Venue
Behaviour Research and Therapy
Publisher
Elsevier BV
Published
2017-10
Volume
97
Pages
259-272
ISSN
0005-7967 (ISSN-L)
OpenAlex
W2617641004
Cited by
41
References
74
FWCI
1.58
Type
journal article
Language
en
Open access
green
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.