Cited by 245Statistical Methods in Clinical TrialsStatistical Methods and Bayesian InferenceAdvanced Causal Inference Techniques

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.

Cluster randomized trials with a small number of clusters: which analyses should be used?

Leyrat, Clémence, Morgan, Katy E, Leurent, Baptiste, Kahan, Brennan C·2018·International Journal of Epidemiology

DOI 10.1093/ije/dyx169

Open this paper in CoChat — free account

Abstract

Background: Cluster randomized trials (CRTs) are increasingly used to assess the effectiveness of health interventions. Three main analysis approaches are: cluster-level analyses, mixed-models and generalized estimating equations (GEEs). Mixed models and GEEs can lead to inflated type I error rates with a small number of clusters, and numerous small-sample corrections have been proposed to circumvent this problem. However, the impact of these methods on power is still unclear. Methods: We performed a simulation study to assess the performance of 12 analysis approaches for CRTs with a continuous outcome and 40 or fewer clusters. These included weighted and unweighted cluster-level analyses, mixed-effects models with different degree-of-freedom corrections, and GEEs with and without a small-sample correction. We assessed these approaches across different values of the intraclass correlation coefficient (ICC), numbers of clusters and variability in cluster sizes. Results: Unweighted and variance-weighted cluster-level analysis, mixed models with degree-of-freedom corrections, and GEE with a small-sample correction all maintained the type I error rate at or below 5% across most scenarios, whereas uncorrected approaches lead to inflated type I error rates. However, these analyses had low power (below 50% in some scenarios) when fewer than 20 clusters were randomized, with none reaching the expected 80% power. Conclusions: Small-sample corrections or variance-weighted cluster-level analyses are recommended for the analysis of continuous outcomes in CRTs with a small number of clusters. The use of these corrections should be incorporated into the sample size calculation to prevent studies from being underpowered.

Integrity signals

Registry

Crossref · 10.1093/ije/dyx169

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

lmerTest Package: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i13 · Cited by 24,346
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552
Models for Longitudinal Data: A Generalized Estimating Equation Approach
Zeger, S. L., Liang, K., & Albert, P. S. (1988) · Biometrics · doi:10.2307/2531734 · Cited by 4,505
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Design and Analysis of Cluster Randomization Trials in Health Research
Davis, W. W. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s433 · Cited by 1,735
lmerTest: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2013) · doi:10.32614/cran.package.lmertest · Cited by 1,327
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review.
Ukoumunne, Gulliford, M. C., Chinn, Sterne, & Burney, P. (1999) · Health Technology Assessment · doi:10.3310/hta3050 · Cited by 1,211
Cluster Randomised Trials
Hayes, R., & Moulton, L. H. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584888178 · Cited by 881
Simple sample size calculation for cluster-randomized trials
Hayes, R., & Bennett, S. (1999) · International Journal of Epidemiology · doi:10.1093/ije/28.2.319 · Cited by 824

Show all 31 references

Research works printed but not linked: 4

Campbell, 2014, How to Design, Analyse and Report Cluster Randomized Trials in Medicine and Health Related Research
PROC MIXED, Mixed Models Theory: SAS/STAT(R) 9.2 User’s Guide
Cook, Small-sample robust variance correction for generalized estimating equations for use in cluster randomized clinical trials
Halekoh, 2016, Parametric Bootstrap and Kenward Roger Based Methods for Mixed Model Comparison

Sorted by citations

A tutorial on sample size calculation for multiple-period cluster randomized parallel, cross-over and stepped-wedge trials using the Shiny CRT Calculator
Hemming, K., Kasza, J., Hooper, R., Forbes, A. B., & Taljaard, M. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyz237 · Cited by 235
“Furthermore, we have relaxed large-sample assumptions by using critical values from the t-distribution, but the corresponding degrees of freedom used have not been verified for designs other than the simple parallel CRT with continuous outcomes.(49,54) For parallel CRTs, these degrees of freedom corrections result in…”
BackgroundMethod
A systematic review and meta-analysis of effects of early life non-cognitive skills on academic, psychosocial, cognitive and health outcomes
Smithers, L. G., Sawyer, A. C. P., Chittleborough, C. R., Davies, N. M., Davey Smith, G., & Lynch, J. W. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0461-x · Cited by 175
Estimands in cluster-randomized trials: choosing analyses that answer the right question
Kahan, B. C., Li, F., Copas, A. J., & Harhay, M. O. (2022) · International Journal of Epidemiology · doi:10.1093/ije/dyac131 · Cited by 127
“Although there are several small-sample adjustment methods that improve standard error estimation in such settings, cluster-level analyses are sometimes preferred as they are less prone to small-sample biases, as shown in simulation studies.(24) However, as described above, these technical decisions can result in the…”
Self-citation
Effect of a Sedation and Ventilator Liberation Protocol vs Usual Care on Duration of Invasive Mechanical Ventilation in Pediatric Intensive Care Units
Blackwood, B., Tume, L. N., Morris, K., Clarke, M. J., McDowell, C., Hemming, K., Peters, M. J., McIlmurray, L., Jordan, J., Agus, A. M., et al. (2021) · JAMA · doi:10.1001/jama.2021.10296 · Cited by 106
Key considerations for designing, conducting and analysing a cluster randomized trial
Hemming, K., & Taljaard, M. (2023) · International Journal of Epidemiology · doi:10.1093/ije/dyad064 · Cited by 100
“92 If an individual-level analysis is preferred, two main approaches to accommodating the non-independence are generalized linear mixed models (GLMM) and generalized estimating equations (GEE).”
Relative Measures of Association for Binary Outcomes: Challenges and Recommendations for the Global Health Researcher
Gallis, J. A., & Turner, E. L. (2019) · Annals of Global Health · doi:10.5334/aogh.2581 · Cited by 86
The effect of online multimedia psychoeducational interventions on the resilience and perceived stress of hospitalized patients with COVID-19: a pilot cluster randomized parallel-controlled trial
Shaygan, M., Yazdani, Z., & Valibeygi, A. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03085-6 · Cited by 83
Male autism spectrum disorder is linked to brain aromatase disruption by prenatal BPA in multimodal investigations and 10HDA ameliorates the related mouse phenotype
Symeonides, C., Vacy, K., Thomson, S., Tanner, S., Chua, H. K., Dixit, S., Mansell, T., O’Hely, M., Novakovic, B., Herbstman, J. B., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-48897-8 · Cited by 66
“Given the small number of clusters (i.e. animals), bootstrapped standard errors were estimated using 200 repeats to maintain a conservative type 1 error rate 104 .”
Effect of an Electronic Medication Reconciliation Intervention on Adverse Drug Events
Tamblyn, R. M., Abrahamowicz, M., Buckeridge, D. L., Bustillo, M., Forster, A. J., Girard, N., Habib, B., Hanley, J. A., Huang, A. R., Kurteva, S., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.10756 · Cited by 71
Effects of a smartphone application for cognitive rehearsal intervention on workplace bullying and turnover intention among nurses
Kang, J., & Jeong, Y. J. (2019) · International Journal of Nursing Practice · doi:10.1111/ijn.12786 · Cited by 58
Multiple imputation by predictive mean matching in cluster-randomized trials
Bailey, B. E., Andridge, R., & Shoben, A. B. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00948-6 · Cited by 52
“not be the most appropriate or most powerful analysis approach [8, 9].”
Background
Comparison of small-sample standard-error corrections for generalised estimating equations in stepped wedge cluster randomised trials with a binary outcome: A simulation study
Thompson, J. A., Hemming, K., Forbes, A. B., Fielding, K., & Hayes, R. (2020) · Statistical Methods in Medical Research · doi:10.1177/0962280220958735 · Cited by 51
Impact of Background Music on Medical Student Anxiety and Performance During Anatomical Dissections: A Cluster Randomized Interventional Trial
Bellier, A., Secheresse, T., Stoeckle, A., Dols, A., & Chaffanjon, P. C. (2019) · Anatomical Sciences Education · doi:10.1002/ase.1918 · Cited by 45
Use of multiple period, cluster randomised, crossover trial designs for comparative effectiveness research
Hemming, K., Taljaard, M., Weijer, C., & Forbes, A. B. (2020) · BMJ · doi:10.1136/bmj.m3800 · Cited by 47
Randomized controlled trials in de-implementation research: a systematic scoping review
Raudasoja, A., Falkenbach, P., Vernooij, R. W. M., Mustonen, J. M. J., Agarwal, A., Aoki, Y., Blanker, M. H., Cartwright, R., García‐Perdomo, H. A., Kilpeläinen, T. P., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01238-z · Cited by 43
Effectiveness of a Household Energy Package in Improving Indoor Air Quality and Reducing Personal Exposures in Rural China
Baumgartner, J. C., Clark, S. N., Carter, E. M., Lai, A. M., Zhang, Y., Shan, M., Schauer, J. J., & Yang, X. (2019) · Environmental Science & Technology · doi:10.1021/acs.est.9b02061 · Cited by 43
A multicomponent family support intervention in intensive care units: study protocol for a multicenter cluster-randomized trial (FICUS Trial)
Naef, R., Filipovic, M., Jeitziner, M., von Felten, S., Safford, J., Riguzzi, M., & Rufer, M. (2022) · Trials · doi:10.1186/s13063-022-06454-y · Cited by 48
Cluster randomised trials with a binary outcome and a small number of clusters: comparison of individual and cluster level analysis method
Thompson, J. A., Leyrat, C., Fielding, K., & Hayes, R. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01699-2 · Cited by 43
“cluster-level analyses and GEEs, and a similar power as inverse variance weighted cluster-level analyses [11].”
BackgroundMethodInfluentialSelf-citation
How the Cluster-randomized Trial “Works”
Hurley, J. C. (2019) · Clinical Infectious Diseases · doi:10.1093/cid/ciz554 · Cited by 36
Education Increases Disposal of Unused Opioids After Total Joint Arthroplasty
Nahhas, C. R., Hannon, C. P., Yang, J., Gerlinger, T. L., Nam, D., & Valle, C. J. D. (2020) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.19.01166 · Cited by 35
Comparing theory and non-theory based implementation approaches to improving referral practices in cancer genetics: a cluster randomised trial protocol
Morrow, A., Hogden, E., Kang, Y., Steinberg, J., Canfell, K., Solomon, M. J., Kench, J. G., Gill, A. J., Shaw, T. J., Pachter, N., et al. (2019) · Trials · doi:10.1186/s13063-019-3457-6 · Cited by 34
A Quality Improvement Intervention to Enhance Access to Kidney Transplantation and Living Kidney Donation (EnAKT LKD) in Patients With Chronic Kidney Disease: Clinical Research Protocol of a Cluster-Randomized Clinical Trial
Yohanna, S., Naylor, K. L., Mucsi, I., McKenzie, S. Q., Belenko, D., Blake, P. G., Coghlan, C., Dixon, S. N., Elliott, L., Getchell, L. E., et al. (2021) · Canadian Journal of Kidney Health and Disease · doi:10.1177/2054358121997266 · Cited by 36
Mobile health technology (WeChat) for the hierarchical management of community hypertension: protocol for a cluster randomized controlled trial
Li, T., Ding, W., Li, X., & Lin, A. (2019) · Patient Preference and Adherence · doi:10.2147/ppa.s215719 · Cited by 32
“01, the Satterthwaite correction is used to obtain an approximation of the degree of freedom to control the type I error rate.(44) When calculating the sample size, the type II error was controlled within 10%, which could appropriately improve the power of the test to some extent.”
Method
Considerations for first field trials of low-threshold gene drive for malaria vector control
Connolly, J. B., Burt, A., Christophides, G. K., Diabaté, A., Habtewold, T., Hancock, P. A., James, A. A., Kayondo, J., Lwetoijera, D. W., Manjurano, A. M., et al. (2024) · Malaria Journal · doi:10.1186/s12936-024-04952-9 · Cited by 33
“As with the design of any cRCT, the amount of variation in entomological measurements between trial locations also will be a factor in determining the number of clusters required in control and intervention arms of the field trials involving low threshold gene drive [15, 50, 158–160] .”
Background
The effectiveness of the Structured Health Intervention For Truckers (SHIFT): a cluster randomised controlled trial (RCT)
Clemes, S. A., Varela-Mato, V., Bodicoat, D. H., Brookes, C. L., Chen, Y., Edwardson, C. L., Gray, L. J., Guest, A. J., Johnson, V., Munir, F., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02372-7 · Cited by 37
“Due to the number of clusters [39], the primary analysis was performed using a mixed effects linear regression model with each participant’s daily average number of steps (measured using the activPAL, across any valid day(s)) at 6 months as the outcome, adjusting for their daily average number of steps at baseline and…”
Method
Revitalizing cultivation and strengthening the seed systems of fonio and Bambara groundnut in Mali through a community biodiversity management approach
Sidibé, A., Meldrum, G., Coulibaly, H., Padulosi, S., Traore, I. S., Diawara, G., Sangaré, A. R., & Mbosso, C. (2020) · Plant Genetic Resources · doi:10.1017/s1479262120000076 · Cited by 30
Demystifying estimands in cluster-randomised trials
Kahan, B. C., Blette, B. S., Harhay, M. O., Halpern, S. D., Jairath, V., Copas, A. J., & Li, F. (2024) · Statistical Methods in Medical Research · doi:10.1177/09622802241254197 · Cited by 37
“Furthermore, cluster-randomised trials often have a small number of clusters, and so further research into these methods to evaluate their performance in small-sample settings is warranted (though we note the cluster-specific, cluster-average estimator has been well studied in this setting [15, 18, 25, 38]).”
InfluentialSelf-citation
Early-Life Nutrition Interventions and Associated Long-Term Cardiometabolic Outcomes: A Systematic Review and Meta-Analysis of Randomized Controlled Trials
He, S., & Stein, A. D. (2020) · Advances in Nutrition · doi:10.1093/advances/nmaa107 · Cited by 29
“, small-sample corrections or varianceweighted cluster-level analyses) (n = 2) (18); or the analysis not based on original assignment, but used difference-indifference modeling strategy (n = 2).”
Background
A Community Health Worker–Led Multicomponent Program to Promote Cervical Cancer Screening in South Asian Women: A Cluster RCT
Wong, C. L., Choi, K. C., Chen, J., Law, B. M. H., Chan, D. N. S., & So, W. K. W. (2021) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2021.01.031 · Cited by 28
Community Health Workers in Early Childhood Well-Child Care for Medicaid-Insured Children
Coker, T. R., Liljenquist, K. S., Lowry, S., Fiscella, K. A., Weaver, M. R., Ortiz, J., LaFontaine, R., Silva, J., Salaguinto, T., Johnson, G., et al. (2023) · JAMA · doi:10.1001/jama.2023.7197 · Cited by 27
Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016–2022
Nevins, P., Ryan, M. M., Davis‐Plourde, K. L., Ouyang, Y., Macedo, J. A. P., Meng, C., Tong, G., Wang, X., Ortiz‐Reyes, L., Caille, A., et al. (2023) · Clinical Trials · doi:10.1177/17407745231208397 · Cited by 30
“9,10 11–13 As in other trial designs, adjusting for prespecified baseline prognostic factors in the analysis can help control for potential confounding, improve power and efficiency, and mitigate potential bias due to attrition, although ability to conduct covariate-adjusted analyses may be limited…”
Background
Reporting of cluster randomised crossover trials: extension of the CONSORT 2010 statement with explanation and elaboration
McKenzie, J. E., Taljaard, M., Hemming, K., Arnup, S. J., Giraudeau, B., Eldridge, S. M., Hooper, R., Kahan, B. C., Li, T., Moher, D., et al. (2025) · BMJ · doi:10.1136/bmj-2024-080472 · Cited by 32
“80 Any adjustments applied should be reported.”
Self-citation
Collaborative community mental health and aged care services with peer support to prevent late-life depression: study protocol for a non-randomised controlled trial
Liu, T., Leung, D. K. Y., Lu, S., Kwok, W.-W., Sze, L., Tse, S. S.-K., Ng, S., Wong, P. W. C., LOU, V. W. Q., Tang, J. Y. ‐., et al. (2022) · Trials · doi:10.1186/s13063-022-06122-1 · Cited by 25
“to maintain the desired type I error rate [52].”
Background
Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial
Kahan, B. C., Li, F., Blette, B. S., Jairath, V., Copas, A. J., & Harhay, M. O. (2023) · Clinical Trials · doi:10.1177/17407745231186094 · Cited by 24
“7,24 To examine the expected variability between the cluster-and participant-average estimators when there was no informative cluster size, we conducted a small simulation study to explore this; details are in the Supplemental Material.”
BackgroundMethodSelf-citation
Impact of Upgraded Lighting on Falls in Care Home Residents
Grant, L. K., Hilaire, M. A. S., Heller, J. P., Heller, R. A., Lockley, S. W., & Rahman, S. A. (2022) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2022.06.013 · Cited by 25
A scoping review described diversity in methods of randomization and reporting of baseline balance in stepped-wedge cluster randomized trials
Nevins, P., Davis‐Plourde, K. L., Macedo, J. A. P., Ouyang, Y., Ryan, M., Tong, G., Wang, X., Meng, C., Ortiz‐Reyes, L., Li, F., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.03.010 · Cited by 23
Design and analysis of three‐arm parallel cluster randomized trials with small numbers of clusters
Watson, S., Girling, A. J., & Hemming, K. (2020) · Statistics in Medicine · doi:10.1002/sim.8828 · Cited by 22
Weather information and agricultural outcomes: Evidence from a pilot field experiment in Benin
Yegbemey, R. N., Bensch, G., & Vance, C. (2023) · World Development · doi:10.1016/j.worlddev.2022.106178 · Cited by 19
Evaluation of a youth mental health literacy and action program: Protocol for a cluster controlled trial
Marinucci, A., Grové, C., Allen, K., & Riebschleger, J. (2021) · Mental Health & Prevention · doi:10.1016/j.mhp.2021.200216 · Cited by 21
Generation Healthy Kids: Protocol for a cluster-randomized controlled trial of a multi-component and multi-setting intervention to promote healthy weight and wellbeing in 6–11-year-old children in Denmark
Thomsen, L. T., Schmidt‐Persson, J., Damsgaard, C. T., Krustrup, P., Grøntved, A., Krølner, R. F., Nielsen, G., Lundbye‐Jensen, J., Skovgaard, T., Mølgaard, C., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0308142 · Cited by 18
Methods for dealing with unequal cluster sizes in cluster randomized trials: a scoping review
Zhan, D., Xu, L., Ouyang, Y., Sawatzky, R., & Wong, H. (2021) · PLOS ONE · doi:10.1371/journal.pone.0255389 · Cited by 18
“[12] extended Johnson et al.’s work by considering also the Satherwaite denominator degrees of freedom in the mixed effects model, the Wilcoxon rank sum test in the cluster-level analysis, and GEEs with or without small-sample corrected robust standard error (SE) estimators.”
BackgroundMethod
Higher survival but smaller size of juvenile Dungeness crab (Metacarcinus magister) in high CO2
McElhany, P., Busch, D. S., Lawrence, A., Maher, M. T., Perez, D., Reinhardt, E. M., Rovinski, K., & Tully, E. M. (2022) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2022.151781 · Cited by 18
The effectiveness of the Positive Mood and Active Life Program on reducing depressive symptoms in long-term care facilities
Luo, H., Lou, V. W. Q., Chen, C., & Chi, I. (2020) · The Gerontologist · doi:10.1093/geront/gny120 · Cited by 18
Type I error control for cluster randomized trials under varying small sample structures
Nugent, J. R., & Kleinman, K. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01236-7 · Cited by 18
“[18] evaluated the power and TIE rates of different degrees of freedom (DF) choices for LMMs with Wald hypothesis tests for CRT designs under various design factors.”
Method
Cluster randomized controlled trial analysis at the cluster level: The clan command
Thompson, J. A., Leurent, B., Nash, S., Moulton, L. H., & Hayes, R. (2023) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x231196294 · Cited by 17
Self-citation
How should a cluster randomized trial be analyzed?
Billot, L., Copas, A. J., Leyrat, C., Forbes, A. B., & Turner, E. L. (2024) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2024.202196 · Cited by 17
“These corrections involve a modification of the number of degrees of freedom used to construct confidence intervals and perform statistical tests but may also involve a correction to the estimates of the standard errors them-selves [17].”
MethodSelf-citation
Impact of baseline covariate imbalance on bias in treatment effect estimation in cluster randomized trials: Race as an example
Yang, S., Starks, M. A., Hernandez, A. F., Turner, E. L., Califf, R. M., O’Connor, C. M., Mentz, R. J., & Choudhury, K. R. (2019) · Contemporary Clinical Trials · doi:10.1016/j.cct.2019.04.016 · Cited by 17
Somatic acupressure for the fatigue-sleep disturbance-depression symptom cluster in breast cancer survivors: A phase II randomized controlled trial
Li, M., Kwok, S. W. H., Tan, J.-Y. B., Bressington, D. T., Liu, X., Wang, T., & Chen, S.-L. (2023) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2023.102380 · Cited by 18
Meta Salud Diabetes for cardiovascular disease prevention in Mexico: a cluster-randomized behavioural clinical trial
Rosales, C. B., Denman, C. A., Bell, M. L., Cornejo, E., Ingram, M., del Carmen Castro Vásquez, M., Gonzalez-Fagoaga, J. E., Aceves, B., Nuño, T., Anderson, E. J., et al. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyab072 · Cited by 15
Clinical and Ecological Impact of an Educational Program to Optimize Antibiotic Treatments in Nursing Homes (PROA-SENIOR): A Cluster, Randomized, Controlled Trial and Interrupted Time-Series Analysis
Peñalva, G., Crespo‐Rivas, J. C., Guisado-Gil, A. B., Rodríguez-Villodres, Á., Pachón-Ibáñez, M. E., Cachero-Alba, B., Rivas-Romero, B., Gil-Moreno, J., Galvá-Borras, M. I., García-Moreno, M., et al. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac834 · Cited by 15
“In order to control this bias, we used random effects, restricted maximum likelihood estimation, and the Satterthwaite degrees-of-freedom correction to minimize the type I error in studies with small samples, allowing improved parameter estimation accuracy [30].”
Method
Effect Partitioning in Cross-Sectionally Clustered Data Without Multilevel Models
McNeish, D. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2019.1602504 · Cited by 15
‘Slash and clear’ vector control for onchocerciasis elimination and epilepsy prevention: a protocol of a cluster randomised trial in Cameroonian villages
Fodjo, J. N. S., Vieri, M. K., Ngarka, L., Njamnshi, W. Y., Nfor, L. N., Mengnjo, M. K., Hendy, A., Enyong, P., Palmer, D., Basáñez, M.-G., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050341 · Cited by 15
“Final anonymised datasets will be deposited in appropriate data repositories and available on reasonable request.”
Background
Effect of desensitizing agents on dentin hypersensitivity after non-surgical periodontal therapy: A systematic review and meta-analysis
de Oliveira, R. P., Alencar, C. D. M., Silva, F. A., Magno, M. B., Maia, L. C., & Silva, C. M. (2020) · Journal of Dentistry · doi:10.1016/j.jdent.2020.103498 · Cited by 14
Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters
Grantham, K. L., Kasza, J., Heritier, S., Carlin, J., & Forbes, A. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01550-8 · Cited by 16
“Our results for REML estimation are consistent with several studies for parallel CRTs where a KR approximation or similar degrees of freedom correction were found to better maintain the type I error rate [9, 11, 12].”
Result
Effectiveness of Cash Transfer Delivered Along With Combination HIV Prevention Interventions in Reducing the Risky Sexual Behavior of Adolescent Girls and Young Women in Tanzania: Cluster Randomized Controlled Trial
Kuringe, E., Christensen, A., Materu, J., Drake, M., Majani, E., Casalini, C., Mjungu, D., Mbita, G., Kalage, E., Komba, A., et al. (2022) · JMIR Public Health and Surveillance · doi:10.2196/30372 · Cited by 14
“Descriptive analysis was done using standard methods for the analysis of a pair-matched cluster randomized trial with a small number of clusters [44].”
Method
Randomization-based confidence intervals for cluster randomized trials
Rabideau, D. J., & Wang, R. (2020) · Biostatistics · doi:10.1093/biostatistics/kxaa007 · Cited by 14
Blurring cluster randomized trials and observational studies: Two-Stage TMLE for subsampling, missingness, and few independent units
Nugent, J. R., Marquez, C., Charlebois, E. D., Abbott, R., & Balzer, L. B. (2023) · Biostatistics · doi:10.1093/biostatistics/kxad015 · Cited by 14
CRTpowerdist: An R package to calculate attained power and construct the power distribution for cross-sectional stepped-wedge and parallel cluster randomized trials
Ouyang, Y., Xu, L., Karim, M. E., Gustafson, P., & Wong, H. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106255 · Cited by 13
Quantitative study of ESBL and carbapenemase producers in wastewater treatment plants in Seville, Spain: a culture-based detection analysis of raw and treated water
Monge-Olivares, L., Peñalva, G., Pulido, M. R., Bárcena, L. G., Doval, M. Á., Ballesta, S., Merchante, N., del Real, P. R., Gómez, L. C., Hortal, G. C., et al. (2025) · Water Research · doi:10.1016/j.watres.2025.123706 · Cited by 13
KEPT-app trial: a pragmatic, single-blind, parallel, cluster-randomised effectiveness study of pelvic floor muscle training among incontinent pregnant women: study protocol
Sidik, S. M., Jaffar, A., Foo, C. N., Muhammad, N. A., Manaf, R. A., Ismail, S. I. F., Alagirisamy, P., Fazlah, A. F. A., Suli, Z. B., & Goodyear‐Smith, F. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-039076 · Cited by 13
A group resilience training program for people with multiple sclerosis: Study protocol of a multi-centre cluster-randomized controlled trial (multi-READY for MS)
Giovannetti, A. M., Pakenham, K. I., Presti, G., Quartuccio, M. E., Confalonieri, P., Bergamaschi, R., Grobberio, M., Di Filippo, M., Micheli, M., Brichetto, G., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0267245 · Cited by 11
“To mitigate the risk of inflating Type I error given the small number of clusters, we will use the KenwardRoger degrees of freedom correction, which does not rely on the assumption of fixed cluster sizes [83].”
Method
Sample size determination for external pilot cluster randomised trials with binary feasibility outcomes: a tutorial
Hemming, K., Taljaard, M., Gkini, E., & Bishop, J. R. B. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01384-1 · Cited by 14
“Analysis of cluster-level proportions, as with sample size justification, should also use t -distributions on k-2 degrees of freedom and might need to consider weighting if cluster sizes vary considerably [15].”
Method
Evaluating tests for cluster‐randomized trials with few clusters under generalized linear mixed models with covariate adjustment: A simulation study
Qiu, H., Cook, A. J., & Bobb, J. F. (2023) · Statistics in Medicine · doi:10.1002/sim.9950 · Cited by 13
“An important area of application where such issues arise is cluster-randomized trials (CRTs).”
Background
A Simulation Study of Statistical Approaches to Data Analysis in the Stepped Wedge Design
Ren, Y., Hughes, J. P., & Heagerty, P. J. (2019) · Statistics in Biosciences · doi:10.1007/s12561-019-09259-x · Cited by 15
Phase I, II and III Trials in Inflammatory Bowel Diseases: A Practical Guide for the Non-specialist
D’Amico, F., Baumann, C., Rousseau, H., Danese, S., & Peyrin‐Biroulet, L. (2019) · Journal of Crohn s and Colitis · doi:10.1093/ecco-jcc/jjz214 · Cited by 11
Association of antenatal vitamin B complex supplementation with neonatal vitamin B12 status: evidence from a cluster randomized controlled trial
Li, S., Mi, B., Qu, P., Liu, D., Lei, F., Wang, D., Zeng, L., Kang, Y., Shen, Y. L., Pei, L., et al. (2020) · European Journal of Nutrition · doi:10.1007/s00394-020-02309-x · Cited by 12
An investigation of the effect of smartphone-based pain management application on pain intensity and the quality-of-life dimensions in adolescents with chronic pain: a cluster randomized parallel-controlled trial
Shaygan, M., Jahandide, Z., & Zarifsanaiey, N. (2021) · Quality of Life Research · doi:10.1007/s11136-021-02891-5 · Cited by 11
“Because of the hierarchical structure of the data, an adjustment for clustering was needed [40].”
Background
Effect of a family-based multimedia intervention on the uptake of faecal immunohistochemical test among South Asian older adults: A cluster-randomised controlled trial
So, W. K. W., Chan, D. N. S., Law, B. M. H., Choi, K. C., Krishnasamy, M., & Chan, C. W. H. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104254 · Cited by 11
Evaluation of the type I error rate when using parametric bootstrap analysis of a cluster randomized controlled trial with binary outcomes and a small number of clusters
Golzarri-Arroyo, L., Dickinson, S. L., Jamshidi-Naeini, Y., Zoh, R. S., Brown, A. W., Owora, A. H., Li, P., Oakes, J. M., & Allison, D. B. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106654 · Cited by 9
Performance of mixed effects models and generalized estimating equations for continuous outcomes in partially clustered trials including both independent and paired data
Lange, K. M., Sullivan, T., Kasza, J., & Yelland, L. N. (2024) · Statistics in Medicine · doi:10.1002/sim.10201 · Cited by 10
“The use of small sample corrections improved the performance of mixed models and GEEs, as has also been seen for cluster randomized trials, 50-54 and should be considered in these settings.”
Short-term, lagged association of airway inflammation, lung function, and asthma symptom score with PM2.5 exposure among schoolchildren within a high air pollution region in South Africa
Buthelezi, M. S., Mentz, G. B., Wright, C. Y., Phaswana, S., Garland, R. M., & Naidoo, R. N. (2024) · Environmental Epidemiology · doi:10.1097/ee9.0000000000000354 · Cited by 12
“42 Thus, the available sample of approximately 700 children clustered into six schools had adequate power to test our hypothesis.”
Background
Effectiveness and scalability of an electronic patient-reported outcome measure and decision support tool for family-centred and participation-focused early intervention: PROSPECT hybrid type 1 trial protocol
Kaelin, V., Villegas, V. C., Chen, Y., Murphy, N. J., Papautsky, E. L., Dooling-Litfin, J. K., Leland, N. E., Maheshwari, V., McManus, B. M., & Khetani, M. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-051582 · Cited by 10
Effectiveness of case-area targeted interventions including vaccination on the control of epidemic cholera: protocol for a prospective observational study
Ratnayake, R., Peyraud, N., Ciglenečki, I., Gignoux, E., Lightowler, M. S., Azman, A. S., Gakima, P., Ouamba, J. P., Sagara, J. A., Ndombe, R., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-061206 · Cited by 10
Ivermectin treatment in humans for reducing malaria transmission
de Souza, D. K., Thomas, R., Bradley, J., Leyrat, C., Boakye, D. A., & Okebe, J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013117.pub2 · Cited by 9
Self-citation
Sample size estimation for stratified individual and cluster randomized trials with binary outcomes
Kennedy‐Shaffer, L., & Hughes, M. D. (2020) · Statistics in Medicine · doi:10.1002/sim.8492 · Cited by 10
“44,45 49 Our determination of the relationship between the within-stratum ICCs and the ICC in the overall unstratified population given by equations (15) and (17), and hence also of the ratio, R C RT , of sample sizes needed with and without stratification, rests on the definition of the ICC used.”
The impact of improved water supply on cholera and diarrhoeal diseases in Uvira, Democratic Republic of the Congo: a protocol for a pragmatic stepped-wedge cluster randomised trial and economic evaluation
Gallandat, K., Jeandron, A., Ross, I., Saidi, J. M., Rumedeka, B. B., Kapepula, L., Cousens, S. N., Allen, E., MacDougall, A. M., & Cumming, O. (2021) · Trials · doi:10.1186/s13063-021-05249-x · Cited by 10
Interim data monitoring in cluster randomised trials: Practical issues and a case study
Hemming, K., Martín, J., Gallos, I. D., Coomarasamy, A., & Middleton, L. (2021) · Clinical Trials · doi:10.1177/17407745211024751 · Cited by 10
“This means not only appropriate consideration of clustering, but this might also require the use of small sample corrections in trials with less than about 40 clusters.(13) In practice, this means there will often be considerable uncertainty around the values of these nuisance parameters at any sample size re-estimati…”
Analysing cluster randomised controlled trials using GLMM, GEE1, GEE2, and QIF: results from four case studies
Offorha, B. C., Walters, S. J., & Jacques, R. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-02107-z · Cited by 9
“When the sample size is small (which is a recipe for imbalance) the robust SE estimator of GEE1 does not provide full protection over incorrect working correlation structure, causing GEE1 to have reduced efficiency in regards to the size of the SE of the estimated intervention effect [23, 40, 42].”
Background
The fixed-effects model for robust analysis of stepped-wedge cluster trials with a small number of clusters and continuous outcomes: a simulation study
Lee, K. M., & Cheung, Y. B. (2024) · Trials · doi:10.1186/s13063-024-08572-1 · Cited by 12
“Alternatively, analyses may also be applied on the cluster period means to target the cluster-level response [6, 27].”
BackgroundMethodResultInfluential
An empirical comparison of methods for analyzing over-dispersed zero-inflated count data from stratified cluster randomized trials
Borhan, S., Kennedy, C., Ioannidis, G., Παπαϊωάννου, Α., Adachi, J. D., & Thabane, L. (2020) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2020.100539 · Cited by 9
Community-based intervention via WeChat official account to improve parental health literacy among primary caregivers of children aged 0 to 3 years: Protocol for a cluster randomized controlled trial
Li, Y., Xiao, Q.-L., Li, M., Zhang, Y., Chen, M., Jiang, C.-H., Kang, S.-R., Zhang, Y., Huang, J. G., & Jiang, H. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1039394 · Cited by 8
“Additionally, given the relatively small number of clusters, the Satterthwaite correction will be used in GLMM to maintain an appropriate Type I error (33).”
Method
Optimal study designs for cluster randomised trials: An overview of methods and results
Watson, S., Girling, A. J., & Hemming, K. (2023) · Statistical Methods in Medical Research · doi:10.1177/09622802231202379 · Cited by 9
“The standard errors for ̂𝛽 are underestimated when the number of clusters is small (see, e.g. Leyrat et al., 46 Kahan et al., 47 and Watson et al. 48 ).”
Background
Effect of a culturally sensitive DEmentia Competence Education for Nursing home Taskforce (DECENT) programme in China: A quasi-experimental study
Zhao, Y., Liu, L., Ding, Y., & Chan, H. Y. L. (2022) · Nurse Education Today · doi:10.1016/j.nedt.2022.105434 · Cited by 9
Optimising engagement in a digital parenting intervention to prevent violence against adolescents in Tanzania: protocol for a cluster randomised factorial trial
Janowski, R., Green, O., Shenderovich, Y., Stern, D., Clements, L., Wamoyi, J., Wambura, M., Lachman, J. M., Meléndez‐Torres, G. J., Gardner, F. E. M., et al. (2023) · BMC Public Health · doi:10.1186/s12889-023-15989-x · Cited by 10
Estimating an adjusted risk difference in a cluster randomized trial with individual-level analyses
Macedo, J. A. P., Giraudeau, B., & collaborators, E. (2024) · Statistical Methods in Medical Research · doi:10.1177/09622802241293783 · Cited by 10
“The parameters were selected from previous simulation studies on CRTs such as the studies of Leyrat et al. 11 and Thompson et al., 17 whose parameter values were based on CRT reviews.”
Influential
Effects of a multidisciplinary team-led school-based human papillomavirus vaccination health-promotion programme on improving vaccine acceptance and uptake among female adolescents
Chau, J. P. C., Lo, S. H. S., Choi, K. C., Lee, V. W. Y., Lui, G. C.-Y., Chan, K. M., & Lau, A. Y. L. (2020) · Medicine · doi:10.1097/md.0000000000022072 · Cited by 8
“The generalized estimating equations (GEE) analysis with small-sample correction for sandwich variance estimate, which can be used to account for the small number of clusters in cluster randomized trial,([37]) will be used for comparing the primary and secondary outcomes between the 2 groups.”
Method
The Effectiveness of Interactive Text Messaging and Structured Psychosocial Support Groups on Developmental Milestones of Children From Adolescent Pregnancies in Kenya: Quasi-Experimental Study
Mwenda, V., Makena, I., Ogweno, V., Obonyo, J., & Were, V. O. (2022) · JMIR Pediatrics and Parenting · doi:10.2196/37359 · Cited by 10
“The GEE model was used with a small-sample correction, due to the small number of clusters [38].”
Method
Permutation‐based multiple testing corrections for P$$ P $$‐values and confidence intervals for cluster randomized trials
Watson, S., Akinyemi, J. O., & Hemming, K. (2023) · Statistics in Medicine · doi:10.1002/sim.9831 · Cited by 8
“uj,q+1 = { ujq − sjα∕q if Hj rejected ujq + sj(1 − α∗)∕q otherwise, (17)”
Leveraging baseline covariates to analyze small cluster‐randomized trials with a rare binary outcome
Zhu, Y., Mitra, N., Hemming, K., Harhay, M. O., & Li, F. (2023) · Biometrical Journal · doi:10.1002/bimj.202200135 · Cited by 8
“There remains considerable interest in improving finite-sample inference of the sandwich variance estimator for GEE analysis of small CRTs (Leyrat et al., 2018).”
Background
Effects of nutritional supplements on the re-infection rate of soil-transmitted helminths in school-age children: A systematic review and meta-analysis
Isah, A. U. J., Ekwunife, O. I., Ejie, I. L., & Mandrik, O. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237112 · Cited by 8
“While cluster RCTs have been reported to have more potential to detect treatment effects in the most affected groups within the clusters [48,49], the risk of bias in such studies should be detailed, exploring the differences in outcomes reported.”
Background
Inclusion of unexposed clusters improves the precision of fixed effects analysis of stepped‐wedge cluster randomized trials
Lee, K. M., Ma, X., Yang, G. M., & Cheung, Y. B. (2022) · Statistics in Medicine · doi:10.1002/sim.9394 · Cited by 9
Mind the gap: covariate constrained randomisation can protect against substantial power loss in parallel cluster randomised trials
Kristunas, C., Grayling, M. J., Gray, L. J., & Hemming, K. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01588-8 · Cited by 7
Evaluation of a multicomponent intervention to shorten thrombolytic door-to-needle time in stroke patients in China (MISSION): A cluster-randomized controlled trial
Zhong, W., Lin, L., Gong, X., Chen, Z., Chen, Y., Yan, S., Zhou, Y., Zhang, X., Hu, H., Tong, L., et al. (2022) · PLoS Medicine · doi:10.1371/journal.pmed.1004034 · Cited by 7
“Generalized estimating equation models with an exchangeable working correlation structure were conducted to account for the correlations of observations within clusters (hospitals) [26].”
Method
Analysis of the RIMDAMAL trial – Authors' reply
Foy, B. D., Rao, S., Parikh, S., Slater, H., & Dabiré, R. K. (2019) · The Lancet · doi:10.1016/s0140-6736(19)32139-7 · Cited by 7
The performance of small sample correction methods for controlling type I error when analyzing parallel cluster randomized trials: a systematic review of simulation studies
Hemming, K., Thompson, J. A., Kristunas, C., Forbes, A., Watson, S., Bowden, R. A., McKenzie, J. E., & Kasza, J. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111838 · Cited by 11
Improving Parental Health Literacy in Primary Caregivers of 0- to 3-Year-Old Children Through a WeChat Official Account: Cluster Randomized Controlled Trial
Li, Y., Xiao, Q., Chen, M., Jiang, C., Kang, S.-R., Zhang, Y., Huang, J., Yang, Y., Li, M., & Jiang, H. (2024) · JMIR Public Health and Surveillance · doi:10.2196/54623 · Cited by 6
“Given the relatively small number of clusters, Satterthwaite correction was used in GLMM to maintain an appropriate type I error [35].”
Method
A Comparison of Variance Estimators for Logistic Regression Models Estimated Using Generalized Estimating Equations ( GEE ) in the Context of Observational Health Services Research
Austin, P. C. (2024) · Statistics in Medicine · doi:10.1002/sim.10260 · Cited by 6
“Second, we assumed that the regression coefficients followed a t -distribution with N clusters —2 degrees of freedom (2 was sub-tracted from N clusters because there were two cluster-level covariates) [10, 22].”
How to design and analyse cluster randomized trials with a small number of clusters? Comment on Leyrat et al.
van Breukelen, G., & Candel, M. J. J. M. (2018) · International Journal of Epidemiology · doi:10.1093/ije/dyy061 · Cited by 6
Analysis of a multifaceted interactive pedagogy program in an upper limb anatomy course: A time series study
Bellier, A., Kaladzé, N., Rabattu, P., Chaffanjon, P. C., & Cavalié, G. (2022) · Anatomical Sciences Education · doi:10.1002/ase.2170 · Cited by 6
Restricted mean survival time to estimate an intervention effect in a cluster randomized trial
Vilain-Abraham, F. L., Tavernier, E., Dantan, É., Desmée, S., & Caille, A. (2023) · Statistical Methods in Medical Research · doi:10.1177/09622802231192960 · Cited by 5
“39 Several methodological reviews indicated that CRTs often randomized fewer than 40 clusters, 40–42 so a permutation test could be relevant to control the type I error rate when estimating the Δ RMST ( t ∗ ) with pseudo-values regression.”
Background

Authors: Leyrat, C., Morgan, K. E., Leurent, B., & Kahan, B. C.
Affiliations: Inserm (FR)
London School of Hygiene & Tropical Medicine (GB)
Queen Mary University of London (GB)
Corresponding author: Leyrat, Clémence
ORCID: Leyrat, Clémence
Morgan, Katy E
Leurent, Baptiste
Kahan, Brennan C
Venue: International Journal of Epidemiology
Publisher: Oxford University Press (OUP)
Published: 2017-08-23
Volume: 47
Issue: 1
Pages: 321-331
ISSN: 0300-5771 (ISSN-L)
1464-3685
DOI: 10.1093/ije/dyx169
OpenAlex: W2746185216
Cited by: 244
References: 31
Citations by year: 2026: 34 · 2025: 33 · 2024: 34 · 2023: 32 · 2022: 39 · 2021: 30 · 2020: 21 · 2019: 17 · 2018: 4
FWCI: 8.02
Type: journal article
Language: en
Open access: closed
Locations: International Journal of Epidemiology (publishedVersion)
PubMed (publishedVersion)
LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: small number of clusters, type I error rate, statistical power, small sample correction, mixed models, generalized estimating equations, intraclass correlation coefficient, sample size determination, continuous outcomes
MeSH terms: Analysis of Variance, Computer Simulation, Data Interpretation, Statistical, Humans, Models, Statistical, Cluster Analysis, Randomized Controlled Trials as Topic, Sample Size

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Cluster randomized trials with a small number of clusters: which analyses should be used?

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks