Empirically supported treatments or type I errors? Problems with the analysis of data from group-administered treatments.
Abstract
When treatments are administered in groups, clients interact in ways that lead to violations of a key assumption of most statistical analyses-the assumption of independence of observations. The resulting dependencies, when not properly accounted for, can increase Type I errors dramatically. Of the 33 studies of group-administered treatment on the empirically supported treatments list, none appropriately analyzed their data. The current authors provide corrections that can be applied to improper analyses. After the corrections, only 12.4% to 68.2% of tests that were originally reported as significant remained significant, depending on what assumptions were made about how large the dependencies among observations really are. Of the 33 studies, 6-19 studies no longer had any significant results after correction. The authors end by providing recommendations for researchers planning group-administered treatment research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Statistical tests should account for dependencies of individual participants clustered in groups.
“This assumption is part of most statistical tools and is due to the potential for participants within a group to influence one another beneficially or detrimentally, and thus statistical tests should account for dependencies of individual participants clustered in groups.20”
From Innovations in group-based psilocybin-assisted therapy of major depression in patients with cancer (2024) · cites this paper as [20]
The assumption of independence of observations is part of most statistical tools.
“This assumption is part of most statistical tools and is due to the potential for participants within a group to influence one another beneficially or detrimentally, and thus statistical tests should account for dependencies of individual participants clustered in groups.20”
From Innovations in group-based psilocybin-assisted therapy of major depression in patients with cancer (2024) · cites this paper as [20]
The assumption of independence of observations is due to the potential for participants within a group to influence one another beneficially or detrimentally.
“This assumption is part of most statistical tools and is due to the potential for participants within a group to influence one another beneficially or detrimentally, and thus statistical tests should account for dependencies of individual participants clustered in groups.20”
From Innovations in group-based psilocybin-assisted therapy of major depression in patients with cancer (2024) · cites this paper as [20]
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 63 · Cited by 209
- Authors
- Baldwin, S. A., Murray, D. M., & Shadish, W. R.
- Affiliations
- University of Memphis (US)
- University of California, Merced (US)
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2005
- Volume
- 73
- Issue
- 5
- Pages
- 924-935
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W1969464104
- Cited by
- 209
- References
- 63
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 1 · 2023: 3 · 2022: 8 · 2021: 2 · 2020: 14 · 2019: 11 · 2018: 11 · 2017: 9 · 2016: 12 · 2015: 13 · 2014: 11 · 2013: 3 · 2012: 15
- FWCI
- 2.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (F31MH073203, F31MH73203-01)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- empirically supported treatments, type I error, independence assumption, group therapy
- MeSH terms
- Analysis of Variance, Behavior Therapy, Data Interpretation, Statistical, Humans, Psychotherapy, Group, Research Design, Reproducibility of Results, Bias, Treatment Outcome, Evidence-Based Medicine, Empirical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
