The DOI matches its Crossref record; 61 of the 65 references it cites were found in a registry or an index.
Using Cluster Bootstrapping to Analyze Nested Data With a Few Clusters
Open accessAbstract
Cluster randomized trials involving participants nested within intact treatment and control groups are commonly performed in various educational, psychological, and biomedical studies. However, recruiting and retaining intact groups present various practical, financial, and logistical challenges to evaluators and often, cluster randomized trials are performed with a low number of clusters (~20 groups). Although multilevel models are often used to analyze nested data, researchers may be concerned of potentially biased results due to having only a few groups under study. Cluster bootstrapping has been suggested as an alternative procedure when analyzing clustered data though it has seen very little use in educational and psychological studies. Using a Monte Carlo simulation that varied the number of clusters, average cluster size, and intraclass correlations, we compared standard errors using cluster bootstrapping with those derived using ordinary least squares regression and multilevel models. Results indicate that cluster bootstrapping, though more computationally demanding, can be used as an alternative procedure for the analysis of clustered data when treatment effects at the group level are of primary interest. Supplementary material showing how to perform cluster bootstrapped regressions using R is also provided.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For small numbers of clusters in particular, Huang [26] showed that a version of a nonparametric bootstrap (cluster bootstrapping) may work well for cRCT data for studies with normally distributed data and designs with 10 or more clusters per treatment group.”
In the citing paper
“We have confirmed Huang et al.’s observation [26] that using bootstrap methods for cRCT should have at least 10 clusters per group to have a valid type I error rate.”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
SPSS and SAS procedures for estimating indirect effects in simple mediation models
Preacher, K. J., & Hayes, A. F. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03206553 · Cited by 17,383
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,820
Bootstrap Methods and their Application
Davison, A. C., & Hinkley, D. V. (1997) · doi:10.1017/cbo9780511802843 · Cited by 7,350
(2010) · Cited by 7,034
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037
Research works printed but not linked: 8
- Bell R., 2002, Survey Methodology, 28, 169
- Kish L., 1965, Survey sampling
- Meijer E., 1998, Assumptions, robustness, and estimation methods in multivariate modeling, 35
- Mundfrom D. J., 2001, Multiple Linear Regression Viewpoints, 27, 3
- Roberts J., 2004, Multiple Linear Regression Viewpoints, 30, 23
- Snijders T., 2012, Multilevel analysis
- Spybrook J., 2011, Optimal design plus empirical evidence: Documentation for the “Optimal Design” software
- StataCorp, 2015, Stata statistical software: Release 14
Cited by
Sorted by citations
McNeish, D. (2017) · Multivariate Behavioral Research · doi:10.1080/00273171.2017.1344538 · Cited by 308
“…2014), a comparison of small sample corrections (McNeish & Stapleton, 2016a), a comparison of multilevel bootstrapping and small sample corrections (Huang, 2017), comparisons of Bayesian and frequentist methods (Hox, van de Schoot, & Matthijsse, 2012), and a review of small sample methods for…”
Application of the Hierarchical Bootstrap to Multi-Level Data in Neuroscience
Saravanan, V., Berman, G. J., & Sober, S. J. (2019) · doi:10.1101/819334 · Cited by 170
“The hierarchical bootstrap [16, 17, 18, 19] is a statistical method that has been applied successfully to awide variety of clustered datasets, including census and polling results, educational and psychological data, and phylogenetic tree information [20, 21, 14].”
BackgroundMethodSteppat, D., Castro, L., & Esser, F. (2021) · European Journal of Communication · doi:10.1177/02673231211012141 · Cited by 104
“To account for the small number of level 2 units (five countries), we follow the approach suggested by Huang (2018) and estimated cluster bootstrapped regressions with 1000 replications (clustered on countries) using the bootcov function in the rms: Regression Modelling Strategies package by…”
MethodRoczniewska, M. A., Smoktunowicz, E., Coo, C., Schwarz, U. V. T., Hasson, H., & Richter, A. (2021) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000311 · Cited by 93
Multilevel Modeling and Ordinary Least Squares Regression: How Comparable Are They?
Huang, F. L. (2018) · The Journal of Experimental Education · doi:10.1080/00220973.2016.1277339 · Cited by 60
“…is often done with FE models, which include dummy-coded variables for dozens or even hundreds of group clusters where coefficients for each and every dummy code are not reported when the group-level variable is merely noise to be accounted for and not of substantive interest (Huang, 2016a).”
BackgroundMethodInfluentialSelf-citationAnalyzing Cross-Sectionally Clustered Data Using Generalized Estimating Equations
Huang, F. L. (2021) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986211017480 · Cited by 67
“Although OLS and MLM coefficients are often similar—they are not always the same and can even be in the opposite direction (Huang, 2018a).”
BackgroundMethodInfluentialSelf-citationUsing cluster-robust standard errors when analyzing group-randomized trials with few clusters
Huang, F. L., & Li, X.-T. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01627-0 · Cited by 67
“We include the use of an RI-MLM as this is a common approach used when investigating treatment effects using a CRT, as coefficients and standard errors are relatively unbiased even with as few as 20 clusters (Huang, 2018a).”
BackgroundMethodSelf-citationLawson, M. B., Partridge, S. C., Hippe, D. S., et al. (2023) · Radiology · doi:10.1148/radiol.230576 · Cited by 61
Steppat, D., Castro, L., & Esser, F. (2021) · Digital Journalism · doi:10.1080/21670811.2021.1939747 · Cited by 55
Racial/Ethnic Parity in Disciplinary Consequences Using Student Threat Assessment
Cornell, D. G., Maeng, J. L., Huang, F. L., Shukla, K. D., & Konold, T. R. (2018) · School Psychology Review · doi:10.17105/spr-2017-0030.v47-2 · Cited by 54
“V47-2 In addition, to avoid making any distributional assumptions of the risk ratios, as the distributions of risk ratios across schools are likely skewed, bootstrapped 95% confidence intervals for the average risk ratios were also computed (see Huang, 2016).”
Self-citationChallenging Conventional Wisdom for Multivariate Statistical Models With Small Samples
McNeish, D. (2017) · Review of Educational Research · doi:10.3102/0034654317727727 · Cited by 54
“Though this is only one example and methodologically advanced readers will be quick to note that the results do not generalize as they would if obtained via a comprehensive simulation, these data demonstrate consistent patterns that have been found previously in such comprehensive simulations (Browne & Draper, 2006; d…”
MethodResultInfluentialAlili, M. E., van Dongen, J. M., Eßer, J., Heymans, M. W., van Tulder, M. W., & Bosmans, J. E. (2022) · Health Economics · doi:10.1002/hec.4603 · Cited by 41
Bootstrap Confidence Intervals for Multilevel Standardized Effect Size
Lai, M. H. C. (2020) · Multivariate Behavioral Research · doi:10.1080/00273171.2020.1746902 · Cited by 41
Liu, C.-H. S., Horng, J., Chou, S.-F., Yu, T., Lee, M.-T., & Lapuz, M. C. B. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100835 · Cited by 39
Cities and regions tackle climate change mitigation but often focus on less effective solutions
Farr, K. B., Song, K., Yeo, Z. Y., Johnson, E., & Hsu, A. (2023) · Communications Earth & Environment · doi:10.1038/s43247-023-01108-6 · Cited by 35
Akaike, T., Thakuria, M., Silk, A. W., Hippe, D. S., Park, S. Y., So, N. A., Maloney, N. J., Gunnell, L., Eschholz, A., Kim, E., et al. (2024) · Journal of Clinical Oncology · doi:10.1200/jco.23.02054 · Cited by 37
Hirshberg, M. J., Colaianne, B. A., Greenberg, M. T., Inkelas, K. K., Davidson, R. J., Germano, D., Dunne, J. D., & Roeser, R. W. (2022) · Mindfulness · doi:10.1007/s12671-022-01952-1 · Cited by 41
Enhancing the Capabilities of Continuous Glucose Monitoring With a Predictive App
Herrero, P., Andorrà, M., Babion, N., Bos, H., Koehler, M., Klopfenstein, Y., Leppäaho, E., Lustenberger, P., Peak, A., Ringemann, C., et al. (2024) · Journal of Diabetes Science and Technology · doi:10.1177/19322968241267818 · Cited by 31
Steppat, D., Castro, L., & Esser, F. (2020) · Media and Communication · doi:10.17645/mac.v8i3.3091 · Cited by 26
Lim, J. T., Mailepessov, D., Chong, C., et al. (2024) · Journal of Travel Medicine · doi:10.1093/jtm/taae103 · Cited by 26
“17 The constrained randomization procedure also helped to prevent chance imbalance in baseline dengue risk due to the small number of intervention ( n =4) locations considered (see Supplementary information).”
BackgroundWu, B., Anderson, R. A., Pei, Y., Xu, H., Nye, K. E., Poole, P., Bunn, M., Downey, C. L., & Plassman, B. L. (2021) · Gerodontology · doi:10.1111/ger.12528 · Cited by 24
Tutorial on Biostatistics: Receiver-Operating Characteristic (ROC) Analysis for Correlated Eye Data
Ying, G., Maguire, M. G., Glynn, R. J., et al. (2021) · Ophthalmic Epidemiology · doi:10.1080/09286586.2021.1921226 · Cited by 22
“Although guidelines have been suggested as to the optimal number of bootstrap replications, 1,000 replications is generally considered acceptable for standard error estimates.(16) When bootstrapping the AUC from a multivariable logistic regression model (i.”
MethodHamori, M. (2023) · Journal of business research · doi:10.1016/j.jbusres.2022.113590 · Cited by 21
Kulkarni, R. U., Wang, C. L., & Bertozzi, C. R. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010061 · Cited by 22
Youn, I., Biswas, D., Hippe, D. S., et al. (2024) · Radiology · doi:10.1148/radiol.232313 · Cited by 19
Argentieri, E. C., Tan, E. T., Whang, J. S., et al. (2021) · Muscle & Nerve · doi:10.1002/mus.27186 · Cited by 18
“resamples to account for the dependency from multiple muscles per subject.(23) Statistical significance of the differences in c indices is implied by bootstrapped 95% CIs that exclude the null value of 0.”
BackgroundRiding smooth: A cost-benefit assessment of surface quality on Copenhagen’s bicycle network
Argyros, D., Jensen, A. F., Rich, J., et al. (2024) · Sustainable Cities and Society · doi:10.1016/j.scs.2024.105473 · Cited by 18
Staffing effectiveness across countries: An institutional perspective
Knappert, L., Peretz, H., Aycan, Z., & Budhwar, P. S. (2021) · Human Resource Management Journal · doi:10.1111/1748-8583.12411 · Cited by 16
Dribin, T. E., Sampson, H. A., Zhang, Y., et al. (2025) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(25)00139-7 · Cited by 12
Barry, S., & Wang, S. Y. (2024) · Translational Vision Science & Technology · doi:10.1167/tvst.13.6.15 · Cited by 12
“Metrics and their confidence intervals were derived using a clustered bootstrapping approach 11,12 to account for the nonindependence within the test set resulting from rare instances where patients underwent multiple glaucoma surgeries.”
Schroeder, K. M., & Remage‐Healey, L. (2020) · Developmental Neurobiology · doi:10.1002/dneu.22802 · Cited by 12
“, Student's t-tests) but retains statistical power with low sample sizes or small effect sizes (Huang, 2018).”
BackgroundMethodBenhamed, A., Batomen, B., Boucher, V., et al. (2023) · BMC Emergency Medicine · doi:10.1186/s12873-023-00863-1 · Cited by 11
“Knot locations were identified heuristically using Harrel’s Regression Modelling Strategies, and 95% CI were obtained using a clustered bootstrap [23].”
MethodGolzarri-Arroyo, L., Dickinson, S. L., Jamshidi-Naeini, Y., Zoh, R. S., Brown, A. W., Owora, A. H., Li, P., Oakes, J. M., & Allison, D. B. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106654 · Cited by 9
Akaike, T., Cahill, K., Akaike, G., Huynh, E. T., Hippe, D. S., Shinohara, M. M., Liao, J. J., Apisarnthanarax, S., Parvathaneni, U., Hall, E. T., et al. (2022) · Cancers · doi:10.3390/cancers14235914 · Cited by 10
“The non-parametric bootstrap was used to calculate 95% confidence intervals (CIs) for the cardiac and non-cardiac mMCC OS curves [34].”
MethodBaliki, G., Weiffen, D., Schreinemachers, P., Shrestha, A., Shrestha, R. M., Schreiner, M., & Brück, T. (2023) · Food and Nutrition Bulletin · doi:10.1177/03795721231194124 · Cited by 9
“47 Again, we control for household-specific characteristics.”
Stallasch, S. E., Lüdtke, O., Artelt, C., Hedges, L. V., & Brunner, M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09898-7 · Cited by 9
“We drew 1000 samples (Huang, 2018, p. 303; Schomaker & Heumann, 2018).”
Tsai, W.-S., Thottarath, S., Gurudas, S., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12051050 · Cited by 9
“Third, the statistical methods employed in this study, namely the generalized estimating equations (GEE) model and bias-corrected bootstrapping strategy, corrected the inter-eye correlation [22,23].”
MethodA comparison of automatic and manual measures of turn-taking in monolingual and bilingual contexts
Ramírez, N. F., Hippe, D. S., Braverman, A., Weiss, Y., & Kuhl, P. K. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02127-z · Cited by 9
“Confidence intervals (CIs) and p -values were calculated for the regression coefficients using the nonparametric bootstrap with resampling done by participant rather than by segment (Huang, 2018).”
Chandra, S., Gurudas, S., Pearce, I., McKibbin, M., Kotagiri, A. K., Menon, G., Burton, B., Talks, J. S., Grabowska, A., Ghanchi, F. D., et al. (2023) · Eye · doi:10.1038/s41433-023-02886-1 · Cited by 8
“…for CST and IRF ± SRF using clustered bootstrap with 1000 replicates to estimate the bias corrected 95% confidence intervals for the area under the curve (AUC), and optimal thresholds for CST were selected based on maximising Youden’s index, defined as sensitivity + specificity – 1 [12, 13].”
MethodTang, Z., Wang, X., Ran, A. R., Yang, D., Ling, A., Yam, J. C. S., Zhang, X. J., Szeto, S., Chan, J., Wong, C. Y., et al. (2024) · British Journal of Ophthalmology · doi:10.1136/bjo-2023-323871 · Cited by 11
Aydın, B., Algina, J. J., & Leite, W. L. (2018) · Behavior Research Methods · doi:10.3758/s13428-018-1080-1 · Cited by 8
“A relatively recent approach, cluster bootstrapping, has been shown to produce results similar to those of multilevel modeling (MLM), but the results are computationally demanding, especially within a Monte Carlo simulation study (Huang, 2016).”
ResultClientelism and ideological competition: the impact on ideological overlapping
Barreda, M., Felipe, P. O., & Rodríguez, L. M. R. (2023) · Journal of Elections Public Opinion and Parties · doi:10.1080/17457289.2023.2190120 · Cited by 8
“13 Bootstrapping provides di ff erent ways of calculating the con fi dence intervals for the parameters obtained in estimated models (Huang 2018).”
MethodHemming, K., Thompson, J. A., Kristunas, C., Forbes, A., Watson, S., Bowden, R. A., McKenzie, J. E., & Kasza, J. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111838 · Cited by 11
Chatterjee, A., Yousuf, A. N., Engelmann, R. M., et al. (2025) · Radiology Imaging Cancer · doi:10.1148/rycan.240156 · Cited by 6
Charnogursky, C. E., Gil, A. I., Ecker, L., Cornejo, R., Rios, S., Ochoa, M., Peña, B., Flores, O., Jones, A., Beeri, K., et al. (2025) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofaf157 · Cited by 7
“The bootstrap method for clustered data was used to construct the 95% CIs based on 2.5% and 97.5% quantiles from 5000 bootstrap samples [26].”
InfluentialWimmesberger, N., Allen, R., Fink, G., Först, G., Kern, W. V., Farin‐Glattacker, E., Rieg, S., the ID ROLL OUT Study group, Solzbach, U., Friedrich, H., et al. (2021) · Infectious Diseases and Therapy · doi:10.1007/s40121-021-00552-1 · Cited by 6
“Analysis of the Primary and Secondary Outcomes Even with a number of just ten clusters, multilevel modeling may result in unbiased estimations for the regression coefficients and standard errors [47].”
BackgroundBates, A., King, C., Dhar, M., Fitzpatrick, C., & Laven, R. A. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-21585 · Cited by 6
A mixed model approach to estimate the survivor average causal effect in cluster‐randomized trials
Wang, W., Tong, G., Hirani, S. P., Newman, S. P., Halpern, S. D., Small, D. S., Li, F., & Harhay, M. O. (2023) · Statistics in Medicine · doi:10.1002/sim.9939 · Cited by 6
“40,41 We randomly sample the treated clusters and the control clusters separately.”
MethodMultisite educational trials: estimating the effect size and its confidence intervals
Singh, A., Uwimpuhwe, G., Li, M., Einbeck, J., Higgins, S., & Kasim, A. (2021) · International Journal of Research & Method in Education · doi:10.1080/1743727x.2021.1882416 · Cited by 6
“However, bootstrapping in education trials requires further research to fully understand the implication of bootstrapping at the level of school or pupils in cluster or multisite randomized trials (Gehlbach et al. 2016; Huang 2018).”
MethodCommon odds ratio test and interval estimation for stratified bilateral and unilateral data
Hua, S., & Ma, C. (2024) · Statistical Methods in Medical Research · doi:10.1177/09622802241267357 · Cited by 5
“For example, cluster bootstrapping, 8,9 though computationally intensive, provides robust standard error and point estimates when dealing with correlated data.”
Multiply robust generalized estimating equations for cluster randomized trials with missing outcomes
Rabideau, D. J., Li, F., & Wang, R. (2024) · Statistics in Medicine · doi:10.1002/sim.10027 · Cited by 7
“…procedure, we resampled entire clusters separately within each randomized group (ie, resampled K ∕ 2 clusters in the treatment arm with replacement and resampled K ∕ 2 clusters in the control group with replacement) to maintain an equal number of clusters per arm across every bootstrap replicate.”
MethodRamírez, N. F., & Hippe, D. S. (2024) · Infant Behavior and Development · doi:10.1016/j.infbeh.2024.101943 · Cited by 6
Sadinski, L. M., Westreich, D., Edmonds, A., Breger, T., Cole, S., Ramirez, C., Brown, T., Ofotokun, I., Konkle-Parker, D., Kassaye, S. G., et al. (2022) · AIDS (London) · doi:10.1097/qad.0000000000003461 · Cited by 3
“We calculated Wald 95% confidence intervals (95% CIs) by employing nonparametric bootstrap methods [43].”
BackgroundMethodWang, C., Kaufman, J. S., Steele, R. J., & Shrier, I. (2024) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2024-002037 · Cited by 6
“39 55 In both a repeated measures RCT and observational study, we would have to account for repeated measures in the analyses (eg, through cluster bootstrapping, 56 mixed models, 57 58 or generalised estimating equations 59 ).”
MethodThe Target Study: A Conceptual Model and Framework for Measuring Disparity
Jackson, J. W., Hsu, Y., Greer, R. C., Boonyasai, R. T., & Howe, C. J. (2025) · Sociological Methods & Research · doi:10.1177/00491241251314037 · Cited by 16
Cluster-robust standard errors with three-level data
Huang, F. L., & Zhang, B. (2025) · Communication in Statistics- Theory and Methods · doi:10.1080/03610926.2025.2461609 · Cited by 6
“However, methodologists (Huang 2016; McNeish, Stapleton, and Silverman 2017) have pointed out that several alternative approaches for the analysis of nested data are possible such as the use of multilevel models (Raudenbush and Bryk 2002), generalized estimating equations (Huang 2022), or cluster…”
Self-citationEvaluating Effects of Multilevel Interventions on Disparity in Health and Healthcare Decisions
Jackson, J. W., Hsu, Y., Zalla, L. C., Carson, K. A., Marsteller, J. A., Cooper, L. A., & the RICH LIFE Project Investigators (2024) · Prevention Science · doi:10.1007/s11121-024-01677-8 · Cited by 5
“For standard errors and confidence intervals that account for the hierarchical structure of the data, we suggest a non-parametric, balanced, stratified cluster bootstrap procedure (Davison & Hinkley, 1997; Field & Welsh, 2007; Gleason, 1988; Huang, 2018; Ren et al., 2010).”
Polyomavirus Antibodies for Merkel Cell Carcinoma Recurrence Detection
Gunnell, L., Hippe, D. S., Park, S. Y., Fu, A., Akaike, T., Lachance, K., Cahill, K., Doolittle‐Amieva, C., & Nghiem, P. (2025) · JAMA Dermatology · doi:10.1001/jamadermatol.2025.3155 · Cited by 4
Political trust and informal traders in African cities
Resnick, D., & Sivasubramanian, B. (2023) · The Journal of Modern African Studies · doi:10.1017/s0022278x23000125 · Cited by 3
Biswas, D., Hippe, D. S., Winter, A. M., et al. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1437506 · Cited by 3
Womack, T. A., Palardy, G. J., & Park, S. (2024) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2024.2342850 · Cited by 3
“The procedure and its rationale are described in Huang (2018).”
Multicenter evaluation of machine and deep learning methods to predict glaucoma surgical outcomes
Barry, S., & Wang, S. Y. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1636410 · Cited by 2
“Confidence intervals were calculated via clustered bootstrapping (Ying et al., 2022; Huang, 2018) to account for within-patient clustering, as some patients underwent multiple surgeries and thus contributed multiple observations to the dataset.”
Liu, F., Schrack, J. A., Walker, K. A., Walston, J. D., Mathias, R. A., Griswold, M. E., Palta, P., Windham, B. G., & Jackson, J. W. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01282-1 · Cited by 2
“We estimated the 95% confidence intervals using 1000 samples from the non-parametric cluster bootstrap [45], where individuals were sampled with replacement and all of their follow-up observations were retained.”
Chen, Y. A., Kazerouni, A. S., Phelps, M. D., et al. (2025) · Journal of Breast Imaging · doi:10.1093/jbi/wbae089 · Cited by 2
Dissanayake, M. V., Jackson, J. W., Martin, C. L., Urrutia, R. P., Funk, M. J., & Wood, M. E. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf072 · Cited by 2
Inference of random effects for linear mixed-effects models with a fixed number of clusters
Chang, C., Huang, H., & Ing, C. (2022) · Annals of the Institute of Statistical Mathematics · doi:10.1007/s10463-022-00825-7 · Cited by 2
“The research of Hsin-Cheng Huang is supported by ROC Ministry of Science and Technology grant MOST 106-2118-M-001-002-MY3.”
BackgroundInfluentialBest Linear Unbiased Prediction of Latent Means in Three-Level Data
Aydın, B., & Algina, J. J. (2021) · The Journal of Experimental Education · doi:10.1080/00220973.2021.1873088 · Cited by 2
“This clustered structure is often accounted by model based or design based methods, with multilevel modeling (also called hierarchical linear modeling and mixed effects models), a model based method, argued to be the predominant choice in the field of social sciences (Gardiner et al., 2009; Hoffman & Rovine, 2007; Hua…”
MethodAderanti, O., Srinivasan, C. S., & Zanello, G. (2025) · Feminist Economics · doi:10.1080/13545701.2025.2548289 · Cited by 1
“Hitomi Komatsu, Hazel J. Malapit, and Sophie Theis (2018) found that women’s agricultural time use is associated with a reduction in the consumption of diverse diets among women in Mozambique.”
Sadinski, L. M., Edwards, J. K., Herce, M. E., et al. (2025) · AIDS · doi:10.1097/qad.0000000000004387 · Cited by 1
Evaluating ESCO skill classifiers
Marconi, G., Baer, I., Silveira, M. D., Gallais, M., & Pruski, C. (2025) · Statistical Journal of the IAOS · doi:10.1177/18747655251395229 · Cited by 5
“35,36 In our case, we need to consider clustering of the standard error at the skill and at the job ad level.”
Hochberg, C. H., Yan, L., Card, M. E., Berry, S. A., Brodie, D., Rotello, L. C., Nader, A., Chatterjee, S., Sahetya, S. K., Iwashyna, T. J., et al. (2026) · Annals of the American Thoracic Society · doi:10.1093/annalsats/aaoaf048 · Cited by 1
Postow, M. A., Zang, P. D., Pal, S. K., Mauguen, A., Farwell, M. D., Gordon, M. S., Hays, D., Wong, J. Y. C., Chen, D. L., Ulaner, G. A., et al. (2026) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2025-012208 · Cited by 2
McCurdy, S., Sheng, E., Jalaleddini, K., et al. (2026) · Multiple Sclerosis Journal · doi:10.1177/13524585261439680 · Cited by 1
“95% Confidence intervals for these estimates were obtained via cluster bootstrapping, 18 using the 2.5 and 97.5 quantiles from 2,000 replicates.”
InfluentialSenevirathne, D. M., Yang, S.-I., Zhao, D., et al. (2026) · Forest Ecology and Management · doi:10.1016/j.foreco.2026.123935 · Cited by 1
Liao, W.-C., Huang, H.-Y., Chen, Y.-H., Huang, H.-B., Liaw, W., & Hsu, Y. (2026) · CardioVascular and Interventional Radiology · doi:10.1007/s00270-026-04533-0 · Cited by 1
“Between-modality differences with 95% CIs were obtained via patient-level cluster bootstrap resampling (B = 1,000) [18], resampling patients and retaining all sites to account for within-patient correlation.”
Park, J. W., Dulin, A., Scarpaci, M. M., Dionne, L. A., Needham, B. L., Sims, M., Kanaya, A. M., Kandula, N. R., Loucks, E. B., Fava, J. L., et al. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad159 · Cited by 1
“To account for within-neighborhood clustering of CVD events in the secondary analyses, we used cluster boot-strapping with 200 repetitions to obtain the 95% confidence intervals (CIs) (40, 43, 44).”
MethodIdentification of Kappaphycus alvarezii seaweed based on phylogenetic and carrageenan conten
Satriani, G. I., Soelistyowati, D. T., Alimuddin, A., et al. (2024) · Jurnal Akuakultur Indonesia · doi:10.19027/jai.23.1.1-11 · Cited by 1
“The method neighbor-joining (NJ) and replication bootstrap 1000 times constructs the phylogenetic tree (Huang, 2018).”
MethodCherkezova, S. (2024) · Environment and Social Psychology · doi:10.54517/esp.v9i5.2313 · Cited by 3
Design of field trials for the evaluation of transmissible vaccines in animal populations
Sheen, J. K., Kennedy‐Shaffer, L., Levy, M. Z., & Metcalf, C. J. E. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012779 · Cited by 1
“Alternatively, investigators may also include all originally sampled animals from each resampled cluster, among other resampling approaches [46].”
Learning Gaussian graphical models from correlated data
Song, Z., Gunn, S., Monti, S., Peloso, G. M., Liu, C., Lunetta, K. L., & Sebastiani, P. (2025) · Frontiers in Systems Biology · doi:10.3389/fsysb.2025.1589079 · Cited by 1
“This is consistent with the well-known limitation that the bootstrap estimate of the standard error is not accurate when the number of clusters is small and the test statistics tends to be too liberal (Huang, 2018).”
Upstrap for Estimating Power and Sample Size in Complex ModelsPreprint
Karas, M., & Crainiceanu, C. M. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.08.21.457220
Tian, R., Stojanovski, E., & Miller, D. (2023) · MODSIM · doi:10.36334/modsim.2023.tian126
Schuetze, B. A., Yan, V. X., & Carvalho, P. F. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/yu8tr
Schuetze, B. A., Yan, V. X., & Carvalho, P. F. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/yu8tr_v2
Schuetze, B. A., Yan, V. X., & Carvalho, P. F. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00508-3 · Cited by 1
“We used cluster bootstrapping (Huang, 2018) with 1000 replicates to calculate mean RMSE, AUC, classification accuracy, and bias to compute appropriate 95% confidence intervals of these statistics.”
Brown, A. R., Gajewski, B. J., Mayo, M. S., Ellerbeck, E. F., & Befort, C. A. (2025) · International Journal of Clinical Biostatistics and Biometrics · doi:10.23937/2469-5831/1510058
Multiparametric breast MRI to problem-solve mammographically detected suspicious calcifications
Park, V. Y., Hippe, D. S., Kazerouni, A. S., et al. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.112467
Nikolla, D. A., Bowers, K. M., Mudrakola, V., et al. (2025) · Women s Health Issues · doi:10.1016/j.whi.2025.10.006
Milella, G., Fiorella, M. L., Velucci, V., Sciancalepore, D., Luisi, F., Fraddosio, A., D’Errico, E., Muroni, A., & Defazio, G. (2025) · Journal of Neural Transmission · doi:10.1007/s00702-025-03069-w
van Lith, L., Ejbye-Ernst, P., Borghuis, E. E., & Hutter, R. I. (2026) · Policing: An International Journal · doi:10.1108/pijpsm-08-2025-0146
Eddy, C. L., Herman, K. C., Huang, F. L., Weigand, T., Venkat, S., & Conway, C. (2026) · Journal of School Psychology · doi:10.1016/j.jsp.2026.101561
Self-citationHsueh, H., Wu, Y., Chuang, T., et al. (2026) · Brain Communications · doi:10.1093/braincomms/fcag113
“33,34 Significance was defined when the confidence interval of the difference did not cross zero, with P values being adjusted for multiple comparisons by using the Holm method.”
Meche, A., Boonyasai, R. T., Hsu, Y., Greer, R. C., Mehta, H., Alexander, G. C., Segal, J. B., Cooper, L. A., & Jackson, J. W. (2026) · Epidemiology · doi:10.1097/ede.0000000000001964
Changsheng, L., Daqing, L., Jiangping, F., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6506531
Volz, S. G., Loewinger, G., Márquez, I., Fevola, S., Kang, M., Reverte, I., Krishnan, A., Gardner, M., Iordanova, M. D., & Esber, G. R. (2026) · eNeuro · doi:10.1523/eneuro.0560-24.2026
“One possible approach would be to construct con fi dence intervals using a cluster-bootstrapping procedure (Huang, 2018, and references therein) that resamples at both the subject and trial levels, re-estimates subject-speci fi c PCs, and fi ts the model to each bootstrap sample.”
ctDNA or Merkel Virus Antibodies for Surveillance of Merkel Cell Carcinoma Recurrence
Thakuria, M., Akaike, T., Silk, A. W., et al. (2026) · JAMA Dermatology · doi:10.1001/jamadermatol.2026.2153
Changsheng, L., Luo, D., Jiangping, F., et al. (2026) · Ecological Indicators · doi:10.1016/j.ecolind.2026.115139
Reimers, P. (2026) · Politische Vierteljahresschrift · doi:10.1007/s11615-026-00672-8
Abdelsalam, Y., Peitek, N., Maurer, A.-M., Toneva, M., & Apel, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773219 · Cited by 3
“The procedure consists of four main steps.”
Metadata
- Authors
- Huang, F. L.
- Affiliations
- University of Missouri (US)
- Corresponding author
- Huang, Francis L.
- ORCID
- Huang, Francis L.
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2016-11-24
- Volume
- 78
- Issue
- 2
- Pages
- 297-318
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- PMC
- PMC5965657
- OpenAlex
- W2554437640
- Cited by
- 107
- References
- 65
- Citations by year
- 2026: 22 · 2025: 22 · 2024: 18 · 2023: 12 · 2022: 10 · 2021: 13 · 2020: 3 · 2019: 2 · 2018: 2 · 2017: 3
- FWCI
- 2.36
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Educational and Psychological Measurement (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- small number of clusters, nested data, multilevel models, standard errors, intraclass correlation coefficient, cluster size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
