Recommendations for choosing an analysis method that controls Type I error for unbalanced cluster sample designs with Gaussian outcomes
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
We used theoretical and simulation‐based approaches to study Type I error rates for one‐stage and two‐stage analytic methods for cluster‐randomized designs. The one‐stage approach uses the observed data as outcomes and accounts for within‐cluster correlation using a general linear mixed model. The two‐stage model uses the cluster specific means as the outcomes in a general linear univariate model. We demonstrate analytically that both one‐stage and two‐stage models achieve exact Type I error rates when cluster sizes are equal. With unbalanced data, an exact size α test does not exist, and Type I error inflation may occur. Via simulation, we compare the Type I error rates for four one‐stage and six two‐stage hypothesis testing approaches for unbalanced data. With unbalanced data, the two‐stage model, weighted by the inverse of the estimated theoretical variance of the cluster means, and with variance constrained to be positive, provided the best Type I error control for studies having at least six clusters per arm. The one‐stage model with Kenward–Roger degrees of freedom and unconstrained variance performed well for studies having at least 14 clusters per arm. The popular analytic method of using a one‐stage model with denominator degrees of freedom appropriate for balanced data performed poorly for small sample sizes and low intracluster correlation. Because small sample sizes and low intracluster correlation are common features of cluster‐randomized trials, the Kenward–Roger method is the preferred one‐stage approach. Copyright © 2015 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Design and analysis of group-randomized trials: a review of recent methodological developments
Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.423 · Cited by 677 · cited 5×
“[7] and Murray et al.[6] used a two group design, indicating the study design is commonly used in practice.”
MethodInfluentialSelf-citation
Cited by
Cluster randomized trials with a small number of clusters: which analyses should be used?
Leyrat, C., Morgan, K. E., Leurent, B., & Kahan, B. C. (2018) · International Journal of Epidemiology · doi:10.1093/ije/dyx169 · Cited by 244
Review of recent methodological developments in group-randomized trials: part 2—analysis
Turner, E. L., Prague, M., Gallis, J. A., Li, F., & Murray, D. M. (2017) · American Journal of Public Health · doi:10.2105/ajph.2017.303707 · Cited by 195
Self-citationKahan, B. C., Forbes, G., Ali, Y., Jairath, V., Bremner, S., Harhay, M. O., Hooper, R., Wright, N., Eldridge, S. M., & Leyrat, C. (2016) · Trials · doi:10.1186/s13063-016-1571-2 · Cited by 185
“A number of these small-sample corrections have been evaluated in the context of CRTs [13, 16, 17].”
BackgroundMethods for dealing with unequal cluster sizes in cluster randomized trials: a scoping review
Zhan, D., Xu, L., Ouyang, Y., Sawatzky, R., & Wong, H. (2021) · PLOS ONE · doi:10.1371/journal.pone.0255389 · Cited by 18
“[52] investigated Wald tests from two-level linear mixed effect models (i.”
Method
Metadata
- Authors
- Johnson, J. L., Kreidler, S. M., Catellier, D. J., Murray, D. M., Muller, K. E., & Glueck, D. H.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2015-06-18
- Volume
- 34
- Issue
- 27
- Pages
- 3531-3545
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W1843165756
- Cited by
- 29
- References
- 1
- FWCI
- 2.04
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Recommendations for choosing an analysis method that controls Type I error for unbalanced cluster sample designs with Gaussian outcomes. https://app.cochat.ai/papers/10.1002/sim.6565
- The paper
- Johnson, J. L., Kreidler, S. M., Catellier, D. J., Murray, D. M., Muller, K. E., & Glueck, D. H. (2015). Recommendations for choosing an analysis method that controls Type I error for unbalanced cluster sample designs with Gaussian outcomes. Statistics in Medicine. https://doi.org/10.1002/sim.6565
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.