The DOI matches its Crossref record.
Dealing With Dependence (Part I): Understanding the Effects of Clustered Data
Abstract
This article provides a conceptual introduction to the issues surrounding the analysis of clustered (nested) data. We define the intraclass correlation coefficient (ICC) and the design effect, and we explain their effect on the standard error. When the ICC is greater than 0, then the design effect is greater than 1. In such a scenario, the standard error produced under the assumption of independence is underestimated. This increases the Type I error rate. We provide a short illustration of the effect of non-independence on the standard error. We show that after accounting for the design effect, our decision about the statistical significance of the test statistic changes. When we fail to account for the clustered nature of the data, we conclude that the difference between the two groups is statistically significant. However, once we adjust the standard error for the design effect, the difference is no longer statistically significant.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When studies assign classes or schools to condition rather than individual students, the Type I error rate is inflated.
“One such issue occurs when studies assign classes or schools to condition rather than individual students, which inflates the Type I error rate (Hox, 1998;McCoach & Adelson, 2010).”
From Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices. (2023) · cites this paper as McCoach & Adelson, 2010
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 7
- Coyne, M., Journal of Research on Educational Effectiveness
- Glass, G.V., 1996, Statistical Methods in Education and Psychology
- 10.3102/0162373707299706 · DOI · Paper page
- Kish, L., 1965, Survey sampling
- O’Connell, A.A., 2008, Multilevel modeling of educational data
- Raudenbush, S., 2002, Hierarchical linear models
- Snijders, T.A., 1999, Multilevel analysis
Cited by
Sorted by citations
Macnamara, B. N., & Burgoyne, A. P. (2023) · Psychological Bulletin · doi:10.1037/bul0000352 · Cited by 415
Data with Hierarchical Structure: Impact of Intraclass Correlation and Sample Size on Type-I Error
Musca, S. C., Kamiejski, R., Nugier, A., Méot, A., Er-Rafiy, A., & Brauer, M. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00074 · Cited by 259
Using cluster bootstrapping to analyze nested data with a few clusters
Huang, F. L. (2018) · Educational and Psychological Measurement · doi:10.1177/0013164416678980 · Cited by 107
Good-enough level and dose-effect models: variation among outcomes and therapists
Owen, J. J., Adelson, J., Budge, S., Kopta, S. M., & Reese, R. J. (2016) · Psychotherapy Research · doi:10.1080/10503307.2014.966346 · Cited by 58
Self-citationTeachers or psychologists: Who should facilitate depression prevention programs in schools?
Wahl, M., Adelson, J., Patak, M., Pössel, P., & Hautzinger, M. (2014) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph110505294 · Cited by 31
Self-citationLarose, S., Boisclair-Châteauvert, G., De Wit, D. J., DuBois, D., Erdem, G., & Lipman, E. L. (2018) · The Journal of Primary Prevention · doi:10.1007/s10935-018-0509-8 · Cited by 24
Application of the hierarchical bootstrap to multi-level data in neuroscience
Saravanan, V., Berman, G. J., & Sober, S. J. (2020) · arXiv · doi:10.48550/arxiv.2007.07797 · Cited by 26
Metadata
- Authors
- McCoach, D. B., & Adelson, J. L.
- Affiliations
- University of Connecticut (US)
- University of Louisville (US)
- Corresponding author
- McCoach, D. Betsy
- Venue
- Gifted Child Quarterly
- Publisher
- SAGE Publications
- Published
- 2010-03-05
- Volume
- 54
- Issue
- 2
- Pages
- 152-155
- ISSN
- 0016-9862 (ISSN-L)
- 1934-9041
- OpenAlex
- W2115971707
- Cited by
- 166
- References
- 0
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 6 · 2023: 9 · 2022: 11 · 2021: 17 · 2020: 16 · 2019: 16 · 2018: 12 · 2017: 12 · 2016: 9 · 2015: 7 · 2014: 15 · 2013: 6 · 2012: 10
- FWCI
- 6.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Gifted Child Quarterly (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- clustered data, nested data, intraclass correlation coefficient, design effect, type I error, non-independence, statistical significance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
