Correcting a Significance Test for Clustering
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
A common mistake in analysis of cluster randomized trials is to ignore the effect of clustering and analyze the data as if each treatment group were a simple random sample. This typically leads to an overstatement of the precision of results and anticonservative conclusions about precision and statistical significance of treatment effects. This article gives a simple correction to the t statistic that would be computed if clustering were (incorrectly) ignored. The correction is a multiplicative factor depending on the total sample size, the cluster size, and the intraclass correlation ρ. The corrected t statistic has Student’s t distribution with reduced degrees of freedom. The corrected statistic reduces to the t statistic computed by ignoring clustering when ρ = 0. It reduces to the t statistic computed using cluster means when ρ = 1. If 0 < ρ < 1, it lies between these two, and the degrees of freedom are in between those corresponding to these two extremes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
An Extension of Box's Results on the Use of the $F$ Distribution in Multivariate Analysis
Geisser, S., & Greenhouse, S. W. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706545 · Cited by 1,098
Design and analysis of group-randomized trials: a review of recent methodological developments
Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.423 · Cited by 677 · cited 3×
“(2000), Klar and Donner (2001), Murray (1998), or Murray, Varnell, & Blitstein (2004) .”
BackgroundMethods to reduce the impact of intraclass correlation in group-randomized trials.
Murray, D. M., & Blitstein, J. L. (2003) · Evaluation Review · doi:10.1177/0193841x02239019 · Cited by 221 · cited 1×
“For example, Murray and Blitstein (2003) reported a summary of intraclass correlations obtained from 17 articles reporting cluster-randomized trials in psychology and public health, and Murray et al. (2004) gave references to 14 very recent studies that provide data on intraclass correlations for…”
Method
Cited by
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787
“This leads to statistical problems: The number of independent data points in each treatment is not equal to the number of students (40).”
BackgroundMethodIntraclass Correlation Values for Planning Group-Randomized Trials in Education
Hedges, L. V., & Hedberg, E. C. (2007) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373707299706 · Cited by 847
Self-citationMol, S. E., Bus, A. G., & de Jong, M. T. (2009) · Review of Educational Research · doi:10.3102/0034654309332561 · Cited by 635
Strain, P. S., & Bovey, E. H. (2011) · Topics in Early Childhood Special Education · doi:10.1177/0271121411408740 · Cited by 292
Moeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W. (2017) · International Journal of Social Research Methodology · doi:10.1080/13645579.2016.1252189 · Cited by 274
Metadata
- Authors
- Hedges, L. V.
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2007-06
- Volume
- 32
- Issue
- 2
- Pages
- 151-179
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2099003806
- Cited by
- 115
- References
- 4
- FWCI
- 6.31
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Correcting a Significance Test for Clustering. https://app.cochat.ai/papers/10.3102/1076998606298040
- The paper
- Hedges, L. V. (2007). Correcting a Significance Test for Clustering. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998606298040
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.