The DOI matches its Crossref record; 61 of the 65 references it cites were found in a registry or an index.
Using Cluster Bootstrapping to Analyze Nested Data With a Few Clusters
Open accessAbstract
Cluster randomized trials involving participants nested within intact treatment and control groups are commonly performed in various educational, psychological, and biomedical studies. However, recruiting and retaining intact groups present various practical, financial, and logistical challenges to evaluators and often, cluster randomized trials are performed with a low number of clusters (~20 groups). Although multilevel models are often used to analyze nested data, researchers may be concerned of potentially biased results due to having only a few groups under study. Cluster bootstrapping has been suggested as an alternative procedure when analyzing clustered data though it has seen very little use in educational and psychological studies. Using a Monte Carlo simulation that varied the number of clusters, average cluster size, and intraclass correlations, we compared standard errors using cluster bootstrapping with those derived using ordinary least squares regression and multilevel models. Results indicate that cluster bootstrapping, though more computationally demanding, can be used as an alternative procedure for the analysis of clustered data when treatment effects at the group level are of primary interest. Supplementary material showing how to perform cluster bootstrapped regressions using R is also provided.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For small numbers of clusters in particular, Huang [26] showed that a version of a nonparametric bootstrap (cluster bootstrapping) may work well for cRCT data for studies with normally distributed data and designs with 10 or more clusters per treatment group.”
In the citing paper
“We have confirmed Huang et al.’s observation [26] that using bootstrap methods for cRCT should have at least 10 clusters per group to have a valid type I error rate.”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
SPSS and SAS procedures for estimating indirect effects in simple mediation models
Preacher, K. J., & Hayes, A. F. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03206553 · Cited by 17,383
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,820
Bootstrap Methods and their Application
Davison, A. C., & Hinkley, D. V. (1997) · doi:10.1017/cbo9780511802843 · Cited by 7,350
(2010) · Cited by 7,034
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037
Research works printed but not linked: 8
- Bell R., 2002, Survey Methodology, 28, 169
- Kish L., 1965, Survey sampling
- Meijer E., 1998, Assumptions, robustness, and estimation methods in multivariate modeling, 35
- Mundfrom D. J., 2001, Multiple Linear Regression Viewpoints, 27, 3
- Roberts J., 2004, Multiple Linear Regression Viewpoints, 30, 23
- Snijders T., 2012, Multilevel analysis
- Spybrook J., 2011, Optimal design plus empirical evidence: Documentation for the “Optimal Design” software
- StataCorp, 2015, Stata statistical software: Release 14
Cited by
Sorted by citations
McNeish, D. (2017) · Multivariate Behavioral Research · doi:10.1080/00273171.2017.1344538 · Cited by 308
“…2014), a comparison of small sample corrections (McNeish & Stapleton, 2016a), a comparison of multilevel bootstrapping and small sample corrections (Huang, 2017), comparisons of Bayesian and frequentist methods (Hox, van de Schoot, & Matthijsse, 2012), and a review of small sample methods for…”
Application of the Hierarchical Bootstrap to Multi-Level Data in Neuroscience
Saravanan, V., Berman, G. J., & Sober, S. J. (2019) · doi:10.1101/819334 · Cited by 170
“The hierarchical bootstrap [16, 17, 18, 19] is a statistical method that has been applied successfully to awide variety of clustered datasets, including census and polling results, educational and psychological data, and phylogenetic tree information [20, 21, 14].”
BackgroundMethodSteppat, D., Castro, L., & Esser, F. (2021) · European Journal of Communication · doi:10.1177/02673231211012141 · Cited by 104
“To account for the small number of level 2 units (five countries), we follow the approach suggested by Huang (2018) and estimated cluster bootstrapped regressions with 1000 replications (clustered on countries) using the bootcov function in the rms: Regression Modelling Strategies package by…”
MethodRoczniewska, M. A., Smoktunowicz, E., Coo, C., Schwarz, U. V. T., Hasson, H., & Richter, A. (2021) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000311 · Cited by 93
Multilevel Modeling and Ordinary Least Squares Regression: How Comparable Are They?
Huang, F. L. (2018) · The Journal of Experimental Education · doi:10.1080/00220973.2016.1277339 · Cited by 60
“…is often done with FE models, which include dummy-coded variables for dozens or even hundreds of group clusters where coefficients for each and every dummy code are not reported when the group-level variable is merely noise to be accounted for and not of substantive interest (Huang, 2016a).”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Huang, F. L.
- Affiliations
- University of Missouri (US)
- Corresponding author
- Huang, Francis L.
- ORCID
- Huang, Francis L.
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2016-11-24
- Volume
- 78
- Issue
- 2
- Pages
- 297-318
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- PMC
- PMC5965657
- OpenAlex
- W2554437640
- Cited by
- 107
- References
- 65
- Citations by year
- 2026: 22 · 2025: 22 · 2024: 18 · 2023: 12 · 2022: 10 · 2021: 13 · 2020: 3 · 2019: 2 · 2018: 2 · 2017: 3
- FWCI
- 2.36
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Educational and Psychological Measurement (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- small number of clusters, nested data, multilevel models, standard errors, intraclass correlation coefficient, cluster size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
