Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis, and simulation study
Open accessAbstract
BACKGROUND: Cluster randomised trials (CRTs) are commonly analysed using mixed-effects models or generalised estimating equations (GEEs). However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error (finding a statistically significant treatment effect when it does not exist) if appropriate corrections are not used. METHODS: We conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction. RESULTS: Our simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values (P ≤ 0.01) than appropriate methods (P = 0.04-0.15). In our review, of the 99 trials that reported the number of clusters, 64 (65 %) were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction. CONCLUSIONS: CRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550 · see citing works
Multilevel and Longitudinal Modeling Using Stata
Rabe-Hesketh, S., & Skrondal, A. (2005) · Biometrics · doi:10.1111/j.1541-0420.2006.00588_16.x · Cited by 4,626 · see citing works
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · see citing works
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510 · see citing works
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236 · see citing works
Hayes, R., & Moulton, L. H. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584888178 · Cited by 881 · see citing works
Design and analysis of group-randomized trials: a review of recent methodological developments
Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.423 · Cited by 677 · see citing works
A Note on the Efficiency of Sandwich Covariance Matrix Estimation
Kauermann, G., & Carroll, R. J. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382309 · Cited by 630 · see citing works
Design and Analysis of Cluster Randomization Trials in Health Research.
Flynn, T. N. (2001) · International Journal of Epidemiology · doi:10.1093/ije/30.2.407-a · Cited by 482 · see citing works
A Practical Guide to Cluster Randomised Trials in Health Services Research
Eldridge, S. M., & Kerry, S. (2012) · Wiley eBooks · doi:10.1002/9781119966241 · Cited by 365 · see citing works
Self-citationSmall‐Sample Adjustments for Wald‐Type Tests Using Sandwich Estimators
Fay, M. P., & Graubard, B. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.01198.x · Cited by 289 · see citing works
Eldridge, S. M., Ashby, D., Feder, G. S., Rudnicka, A. R., & Ukoumunne, O. C. (2004) · Clinical Trials · doi:10.1191/1740774504cn006rr · Cited by 288 · see citing works
Self-citationJairath, V., Kahan, B. C., Gray, A. J., Doré, C. J., Mora, A., James, M. W., Stanley, A. J., Everett, S. M., Bailey, A. A., Dallal, H. J., et al. (2015) · The Lancet · doi:10.1016/s0140-6736(14)61999-1 · Cited by 257 · see citing works
Self-citationIvers, N. M., Taljaard, M., Dixon, S., Bennett, C., McRae, A., Taleban, J., Skea, Z., Brehaut, J. C., Boruch, R. F., Eccles, M. P., et al. (2011) · BMJ · doi:10.1136/bmj.d5886 · Cited by 193 · see citing works
Li, P., & Redden, D. T. (2014) · Statistics in Medicine · doi:10.1002/sim.6344 · Cited by 189 · see citing works
Cluster randomised trials in the medical literature: two bibliometric surveys
Bland, M. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-21 · Cited by 187 · see citing works
How to Design, Analyse and Report Cluster Randomised Trials in Medicine and Health Related Research
Campbell, M. J., & Walters, S. J. (2014) · Wiley eBooks · doi:10.1002/9781118763452 · Cited by 178 · see citing works
Pan, W., & Wall, M. M. (2002) · Statistics in Medicine · doi:10.1002/sim.1142 · Cited by 176 · see citing works
Li, P., & Redden, D. T. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0026-x · Cited by 142 · see citing works
Design and Analysis of Group-Randomized Trials: A Review of Recent Practices
Varnell, S. P., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.393 · Cited by 140 · see citing works
A comparison of two bias-corrected covariance estimators for generalized estimating equations
Lu, B., Preisser, J. S., Qaqish, B. F., Suchindran, C., Bangdiwala, S. I., & Wolfson, M. (2007) · Biometrics · doi:10.1111/j.1541-0420.2007.00764.x · Cited by 118 · see citing works
Substantial risks associated with few clusters in cluster randomized and stepped wedge designs
Taljaard, M., Teerenstra, S., Ivers, N. M., & Fergusson, D. A. (2016) · Clinical Trials · doi:10.1177/1740774516634316 · Cited by 96 · see citing works
Huang, S., Fiero, M. H., & Bell, M. L. (2016) · Clinical Trials · doi:10.1177/1740774516643498 · Cited by 75 · see citing works
Are missing data adequately handled in cluster randomised trials? A systematic review and guidelines
Diaz‐Ordaz, K., Kenward, M. G., Cohen, A., Coleman, C. L., & Eldridge, S. M. (2014) · Clinical Trials · doi:10.1177/1740774514537136 · Cited by 61 · see citing works
Self-citationDiaz-Ordaz, K., Froud, R., Sheehan, B., & Eldridge, S. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-127 · Cited by 47 · see citing works
Self-citationJohnson, J. L., Kreidler, S. M., Catellier, D., Murray, D. M., Muller, K. E., & Glueck, D. H. (2015) · Statistics in Medicine · doi:10.1002/sim.6565 · Cited by 29 · see citing works
Jairath, V., Kahan, B., Gray, A., Doré, C., Mora, A., Dyer, C., Stokes, E., Llewelyn, C., Bailey, A., Dallal, H., et al. (2013) · Transfusion Medicine Reviews · doi:10.1016/j.tmrv.2013.04.001 · Cited by 24 · see citing works
Self-citationKahan, B. C., Jairath, V., Murphy, M. F., & Doré, C. J. (2013) · Trials · doi:10.1186/1745-6215-14-206 · Cited by 18 · see citing works
Self-citationWhy does clustering matter in orthodontic trials?
Harrison, J. E., & Burnside, G. (2012) · European Journal of Orthodontics · doi:10.1093/ejo/cjs026 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [4]Murray DM Design and analysis of group-randomised trials 1998 London Oxford University Press
Cited by
Sorted by citations
Consensus in the delphi method: What makes a decision change?
Barrios, M., Guilera, G., Nuño, L., & Gómez‐Benito, J. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120484 · Cited by 404 · see citing works
Cluster randomized trials with a small number of clusters: which analyses should be used?
Leyrat, C., Morgan, K. E., Leurent, B., & Kahan, B. C. (2018) · International Journal of Epidemiology · doi:10.1093/ije/dyx169 · Cited by 244 · see citing works
Self-citationde Silva, V. A., Suraweera, C., Ratnatunga, S. S., Dayabandara, M., Wanniarachchi, N., & Hanwella, R. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-1049-5 · Cited by 227 · see citing works
Family-led rehabilitation after stroke in India (ATTEND): a randomised controlled trial
Lindley, R. I., Anderson, C. S., Billot, L., Förster, A., Hackett, M. L., Harvey, L. A., Jan, S., Li, Q., Huiqin, L. I. U., Langhorne, P., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31447-2 · Cited by 146 · see citing works
Frampton, G. K., Shepherd, J., Pickett, K., Griffiths, G., & Wyatt, J. C. (2020) · Trials · doi:10.1186/s13063-020-04358-3 · Cited by 119 · see citing works
Ashton, L. M., Morgan, P. J., Hutchesson, M. J., Rollo, M. E., & Collins, C. E. (2017) · Nutrition Journal · doi:10.1186/s12937-017-0227-8 · Cited by 111 · see citing works
Estimands in cluster-randomized trials: choosing analyses that answer the right question
Kahan, B. C., Li, F., Copas, A. J., & Harhay, M. O. (2022) · International Journal of Epidemiology · doi:10.1093/ije/dyac131 · Cited by 108 · see citing works
Self-citationClyne, B., Cooper, J. A., Boland, F., Hughes, C. M., Fahey, T. P., & Smith, S. M. (2017) · British Journal of General Practice · doi:10.3399/bjgp17x691073 · Cited by 95 · see citing works
Key considerations for designing, conducting and analysing a cluster randomized trial
Hemming, K., & Taljaard, M. (2023) · International Journal of Epidemiology · doi:10.1093/ije/dyad064 · Cited by 90 · see citing works
Romero, P., Günther, P. S., Kowalewski, K., Friedrich, M., Schmidt, M. W., Trent, S., de la Garza, J. R., Müller-Stich, B. P., & Nickel, F. (2017) · Journal of surgical education · doi:10.1016/j.jsurg.2017.07.025 · Cited by 85 · see citing works
Digital Inhalers for Asthma or Chronic Obstructive Pulmonary Disease: A Scientific Perspective
Chan, A. H. Y., Pleasants, R. A., Dhand, R., Tilley, S. L., Schworer, S. A., Costello, R. W., & Merchant, R. K. (2021) · Pulmonary Therapy · doi:10.1007/s41030-021-00167-4 · Cited by 84 · see citing works
Bell, M. L., & Rabe, B. A. (2020) · Trials · doi:10.1186/s13063-020-4114-9 · Cited by 81 · see citing works
Scherer, E. M., Hagaman, A. K., Chung, E. O., Rahman, A., O’Donnell, K. J., & Maselko, J. (2019) · BMC Public Health · doi:10.1186/s12889-019-6571-1 · Cited by 79 · see citing works
Advances in neuroprosthetic management of foot drop: a review
Gil-Castillo, J., Alnajjar, F., Koutsou, A., Torricelli, D., & Moreno, J. C. (2020) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-020-00668-4 · Cited by 65 · see citing works
Analysis of cluster randomised trials with an assessment of outcome at baseline
Hooper, R., Forbes, A. B., Hemming, K., Takeda, A., & Beresford, L. (2018) · BMJ · doi:10.1136/bmj.k1121 · Cited by 79 · see citing works
Self-citationCandlish, J., Teare, M. D., Dimairo, M., Flight, L., Mandefield, L., & Walters, S. J. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0559-x · Cited by 72 · see citing works
Food cue-induced craving in individuals with bulimia nervosa and binge-eating disorder
Meule, A., Küppers, C., Harms, L., Friederich, H.-C., Schmidt, U., Blechert, J., & Brockmeyer, T. (2018) · PLOS ONE · doi:10.1371/journal.pone.0204151 · Cited by 71 · see citing works
What is the role of enhanced recovery after surgery in children? A scoping review
Pearson, K., & Hall, N. (2016) · Pediatric surgery international (Print) · doi:10.1007/s00383-016-3986-y · Cited by 68 · see citing works
Involvement of people with schizophrenia in decision-making in rural Ethiopia: a qualitative study
Souraya, S., Hanlon, C., & Asher, L. (2018) · Globalization and Health · doi:10.1186/s12992-018-0403-4 · Cited by 68 · see citing works
Flight, L., Allison, A., Dimairo, M., Lee, E. C., Mandefield, L., & Walters, S. J. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0249-5 · Cited by 64 · see citing works
Effect of an Electronic Medication Reconciliation Intervention on Adverse Drug Events
Tamblyn, R. M., Abrahamowicz, M., Buckeridge, D. L., Bustillo, M., Forster, A. J., Girard, N., Habib, B., Hanley, J. A., Huang, A. R., Kurteva, S., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.10756 · Cited by 61 · see citing works
Campbell, C. I., Saxon, A. J., Boudreau, D. M., Wartko, P. D., Bobb, J. F., Lee, A. K., Matthews, A. G., McCormack, J., Liu, D. S., Addis, M., et al. (2021) · Addiction Science & Clinical Practice · doi:10.1186/s13722-021-00218-w · Cited by 58 · see citing works
Physical Exercise as Personalized Medicine for Dementia Prevention?
Müllers, P., Taubert, M., & Müller, N. G. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.00672 · Cited by 52 · see citing works
What Affects Attendance and Engagement in a Parenting Program in South Africa?
Shenderovich, Y., Eisner, M., Cluver, L. D., Doubt, J., Berezin, M. N., Majokweni, S., & Murray, A. L. (2018) · Prevention Science · doi:10.1007/s11121-018-0941-2 · Cited by 53 · see citing works
Adverse childhood experiences and depression among women in rural Pakistan
LeMasters, K. H., Bates, L. M., Chung, E. O., Gallis, J. A., Hagaman, A. K., Scherer, E. M., Sikander, S., Staley, B. S., Zalla, L. C., Zivich, P. N., et al. (2021) · BMC Public Health · doi:10.1186/s12889-021-10409-4 · Cited by 53 · see citing works
Doody, C. B., Robertson, L., Cox, K. M., Bogue, J., Egan, J., & Sarma, K. M. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013242.pub2 · Cited by 49 · see citing works
Validation of Online Versions of Tinnitus Questionnaires Translated into Swedish
Müller, K., Edvall, N. K., Idrizbegovic, E., Huhn, R., Cima, R. F. F., Persson, V., Leineweber, C., Westerlund, H., Langguth, B., Schlee, W., et al. (2016) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2016.00272 · Cited by 45 · see citing works
Randomized controlled trials in de-implementation research: a systematic scoping review
Raudasoja, A., Falkenbach, P., Vernooij, R. W. M., Mustonen, J. M. J., Agarwal, A., Aoki, Y., Blanker, M. H., Cartwright, R., García‐Perdomo, H. A., Kilpeläinen, T. P., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01238-z · Cited by 43 · see citing works
Kamal, A. K., Khalid, W., Muqeet, A., Jamil, A., Farhat, K., Gillani, S. R., Zulfiqar, M., Saif, M., Muhammad, A. A., Zaidi, F., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0197671 · Cited by 43 · see citing works
Suhail, M., Saeed, H., Saleem, Z., Younas, S., Hashmi, F. K., Rasool, M. F., Islam, M. F., & Imran, I. (2021) · Health and Quality of Life Outcomes · doi:10.1186/s12955-021-01761-5 · Cited by 42 · see citing works
Two-Stage TMLE to reduce bias and improve efficiency in cluster randomized trials
Balzer, L. B., van der Laan, M. J., Ayieko, J., Kamya, M. R., Chamie, G., Schwab, J., Havlir, D. V., & Petersen, M. L. (2021) · Biostatistics · doi:10.1093/biostatistics/kxab043 · Cited by 42 · see citing works
Hanrahan, V., Gillies, K., & Biesty, L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234783 · Cited by 39 · see citing works
Day, C. B., Bierhals, C. C. B. K., Mocellin, D., Predebon, M. L., dos Santos, N. O., Pizzol, F. L. F. D., Fuhrmann, A. C., Aires, M., & Paskulin, L. M. G. (2020) · Health & Social Care in the Community · doi:10.1111/hsc.13068 · Cited by 41 · see citing works
Day, C. B., Bierhals, C. C. B. K., dos Santos, N. O., Mocellin, D., Predebon, M. L., Pizzol, F. L. F. D., & Paskulin, L. M. G. (2018) · Trials · doi:10.1186/s13063-018-2454-5 · Cited by 41 · see citing works
Assessment and management of peripheral arterial disease: what every cardiologist should know
Tran, B. G. (2021) · Heart · doi:10.1136/heartjnl-2019-316164 · Cited by 41 · see citing works
Khoe, H. C. H., Low, J. W., Wijerathne, S. I., Ann, L. S., Salgaonkar, H. P., Lomanto, D., Choi, J., Baek, J., Tam, W., Pei, H., et al. (2020) · Surgical Endoscopy · doi:10.1007/s00464-019-07331-7 · Cited by 40 · see citing works
Carona, A., Bicker, J., Silva, R., Fonseca, C., Falcão, A. C., & Fortuna, A. C. (2021) · Life Sciences · doi:10.1016/j.lfs.2021.119342 · Cited by 39 · see citing works
Price, J., Hall, E., West, C., & Thomson, D. J. (2019) · Clinical Oncology · doi:10.1016/j.clon.2019.09.052 · Cited by 38 · see citing works
Pannicke, B., Kaiser, T., Reichenberger, J., & Jens Blechert, A. (2021) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-01066-8 · Cited by 38 · see citing works
Thompson, J. A., Leyrat, C., Fielding, K., & Hayes, R. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01699-2 · Cited by 37 · see citing works
Self-citationHow the Cluster-randomized Trial “Works”
Hurley, J. C. (2019) · Clinical Infectious Diseases · doi:10.1093/cid/ciz554 · Cited by 36 · see citing works
Boecking, B., Brüeggemann, P., Kleinjung, T., & Mazurek, B. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.596037 · Cited by 36 · see citing works
From Diagnoses to Ongoing Journey: Parent Experiences Following Congenital Heart Disease Diagnoses
Williams, T. S., McDonald, K. P., Roberts, S. D., Chau, V., Seed, M., Miller, S. P., & Sananes, R. (2019) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsz055 · Cited by 35 · see citing works
Podina, I. R., Fodor, L. A., Cosmoiu, A., & Boian, R. (2017) · Trials · doi:10.1186/s13063-017-2340-6 · Cited by 34 · see citing works
Machine learning techniques in eating behavior e-coaching
Spanakis, G., Weiß, G., Boh, B., Lemmens, L. H. J. M., & Roefs, A. (2017) · Personal and Ubiquitous Computing · doi:10.1007/s00779-017-1022-4 · Cited by 34 · see citing works
Gardner, K., Sibthorpe, B. M., Chan, M., Sargent, G. M., Dowden, M., & McAullay, D. R. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3308-2 · Cited by 33 · see citing works
Bell, M. L., Horton, N. J., Dhillon, H. M., Bray, V., & Vardy, J. L. (2018) · Psycho-Oncology · doi:10.1002/pon.4777 · Cited by 33 · see citing works
Considerations for first field trials of low-threshold gene drive for malaria vector control
Connolly, J. B., Burt, A., Christophides, G. K., Diabaté, A., Habtewold, T., Hancock, P. A., James, A. A., Kayondo, J., Lwetoijera, D. W., Manjurano, A. M., et al. (2024) · Malaria Journal · doi:10.1186/s12936-024-04952-9 · Cited by 33 · see citing works
Contamination: How much can an individually randomized trial tolerate?
Hemming, K., Taljaard, M., Moerbeek, M., & Forbes, A. (2021) · Statistics in Medicine · doi:10.1002/sim.8958 · Cited by 31 · see citing works
Bobb, J. F., Qiu, H., Matthews, A. G., McCormack, J., & Bradley, K. A. (2020) · Trials · doi:10.1186/s13063-020-4148-z · Cited by 30 · see citing works
Demystifying estimands in cluster-randomised trials
Kahan, B. C., Blette, B. S., Harhay, M. O., Halpern, S. D., Jairath, V., Copas, A. J., & Li, F. (2024) · Statistical Methods in Medical Research · doi:10.1177/09622802241254197 · Cited by 29 · see citing works
Self-citationDesign and analysis of group-randomized trials in cancer: A review of current practices
Murray, D. M., Pals, S. L., George, S. M., Kuzmichev, A., Lai, G. Y., Lee, J. A., Myles, R. L., & Nelson, S. M. (2018) · Preventive Medicine · doi:10.1016/j.ypmed.2018.03.010 · Cited by 30 · see citing works
Garmendia, M. L., Corvalán, C., Araya, M., Casanello, P. C., Kusanovic, J. P., & Uauy, R. D. (2020) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqaa185 · Cited by 27 · see citing works
Chasing the personalized medicine dream through biomarker validation in colorectal cancer
Patil, H., Saxena, S. G., Barrow, C. J., Kanwar, J. R., Kapat, A., & Kanwar, R. K. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.09.022 · Cited by 25 · see citing works
Solomon, K. T., Bandara, S. N., Reynolds, I. S., Krawczyk, N., Saloner, B. K., Stuart, E. A., & Connolly, E. (2021) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2021.108424 · Cited by 25 · see citing works
Hooper, R., Kasza, J., & Forbes, A. B. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01155-z · Cited by 24 · see citing works
Self-citationPuertas, E. B., Velandia-González, M., Vulanovic, L., Bayley, L., Broome, K., Ortiz, C., Rise, N., Antelo, M. V., & Rhoda, D. A. (2022) · The Lancet Regional Health - Americas · doi:10.1016/j.lana.2022.100193 · Cited by 24 · see citing works
Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial
Kahan, B. C., Li, F., Blette, B. S., Jairath, V., Copas, A. J., & Harhay, M. O. (2023) · Clinical Trials · doi:10.1177/17407745231186094 · Cited by 24 · see citing works
Self-citationStuart, B. L., Becque, T., Moore, M. V., & Little, P. S. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00971-7 · Cited by 23 · see citing works
Design and analysis of three‐arm parallel cluster randomized trials with small numbers of clusters
Watson, S., Girling, A. J., & Hemming, K. (2020) · Statistics in Medicine · doi:10.1002/sim.8828 · Cited by 22 · see citing works
Hemming, K., Hughes, J. P., McKenzie, J. E., & Forbes, A. B. (2020) · Statistical Methods in Medical Research · doi:10.1177/0962280220948550 · Cited by 21 · see citing works
Universal EHRs Clinical Decision Support for Thromboprophylaxis in Medical Inpatients
Spyropoulos, A. C., Goldin, M., Koulas, I., Solomon, J., Qiu, M., Ngu, S., Smith, K., Leung, T., Ochani, K., Malik, F., et al. (2023) · JACC Advances · doi:10.1016/j.jacadv.2023.100597 · Cited by 19 · see citing works
Gass, J. D., Misra, A., Yadav, M. N. S., Sana, F., Singh, C., Mankar, A., Neal, B. J., Fisher-Bowman, J., Maisonneuve, J. J., Delaney, M. M., et al. (2017) · Trials · doi:10.1186/s13063-017-2159-1 · Cited by 19 · see citing works
Methods for dealing with unequal cluster sizes in cluster randomized trials: a scoping review
Zhan, D., Xu, L., Ouyang, Y., Sawatzky, R., & Wong, H. (2021) · PLOS ONE · doi:10.1371/journal.pone.0255389 · Cited by 18 · see citing works
Core outcome set for stable angina pectoris in traditional Chinese medicine (COS-SAP-TCM)
Zhang, M., Zhang, J., Chua, H. Z., Feng, R., Lu, M., & Tian, Y. (2021) · Acupuncture and Herbal Medicine · doi:10.1097/hm9.0000000000000007 · Cited by 18 · see citing works
Wippert, P., Niederer, D., Drießlein, D., Beck, H., Banzer, W. E., Schneider, C., Schiltenwolf, M., & Mayer, F. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.629474 · Cited by 17 · see citing works
Type I error control for cluster randomized trials under varying small sample structures
Nugent, J., & Kleinman, K. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01236-7 · Cited by 12 · see citing works
Key concepts in clinical epidemiology: Stepped wedge trials
Hooper, R. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.04.003 · Cited by 17 · see citing works
Self-citationCluster randomized controlled trial analysis at the cluster level: The clan command
Thompson, J. A., Leurent, B., Nash, S., Moulton, L. H., & Hayes, R. (2023) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x231196294 · Cited by 17 · see citing works
How should a cluster randomized trial be analyzed?
Billot, L., Copas, A. J., Leyrat, C., Forbes, A. B., & Turner, E. L. (2024) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2024.202196 · Cited by 17 · see citing works
Self-citationCarragher, M., Ryan, B., Worrall, L. E., Thomas, S., Rose, M. L., Simmons‐Mackie, N. N., Khan, A., Hoffmann, T., Power, E., Togher, L., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-023560 · Cited by 17 · see citing works
Organic electrolytic photocapacitors for stimulation of the mouse somatosensory cortex
Missey, F., Botzanowski, B., Migliaccio, L., Acerbo, E., Głowacki, E. D., & Williamson, A. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac37a6 · Cited by 17 · see citing works
What Tinnitus Therapy Outcome Measures Are Important for Patients?– A Discrete Choice Experiment
Rademaker, M. M., Essers, B. A. B., Stokroos, R. J., Smit, A. L., & Stegeman, I. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.668880 · Cited by 16 · see citing works
Copas, A. J., & Hooper, R. (2019) · Clinical Trials · doi:10.1177/1740774519873888 · Cited by 16 · see citing works
Self-citationXu, M., Hu, T., Li, D., Ma, L., Zhang, H., Fan, J., Fan, X., Zeng, X., Qiu, S.-Q., Liu, Z., et al. (2020) · BMC Complementary Medicine and Therapies · doi:10.1186/s12906-020-03077-1 · Cited by 16 · see citing works
Caille, A., Tavernier, E., Taljaard, M., & Desmée, S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.02.004 · Cited by 16 · see citing works
Potter, S., Davies, C. F., Holcombe, C., Weiler-Mithoff, E. M., Skillman, J. M., Vidya, R., Masannat, Y., Weber, W. P., Heil, J., Wilson, S. M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035505 · Cited by 15 · see citing works
Wang, Y., Wang, X., Wang, J., Li, B., Yu, R., Hu, Y., Li, X., Peng, G.-C., Zhang, M., & Zhu, M. (2020) · Trials · doi:10.1186/s13063-020-4208-4 · Cited by 15 · see citing works
A virtual reality study on postretrieval extinction of smoking memory reconsolidation in smokers
Zandonai, T., Benvegnù, G., Tommasi, F., Ferrandi, E., Libener, E., Ferraro, S., Maris, B., & Chiamulera, C. (2021) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2021.108317 · Cited by 14 · see citing works
Grantham, K. L., Kasza, J., Heritier, S., Carlin, J., & Forbes, A. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01550-8 · Cited by 16 · see citing works
A cluster randomized trial of xylitol chewing gum for prevention of preterm birth: The PPaX trial
Valentine, G. C., Antony, K. M., Sangi‐Haghpeykar, H., Wood, A. C., Chirwa, R., Petro, S., Dumba, M., Nanthuru, D., Shope, C. D., Mlotha-Namarika, J., et al. (2024) · Med · doi:10.1016/j.medj.2024.10.016 · Cited by 14 · see citing works
O'Neil, A. E., Cocker, F., Rarau, P., Baptista, S. N., Cassimatis, M., Taylor, C. B., Lau, A. Y. S., Kanuri, N., & Oldenburg, B. F. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw166 · Cited by 15 · see citing works
Nugent, J. R., Marquez, C., Charlebois, E. D., Abbott, R., & Balzer, L. B. (2023) · Biostatistics · doi:10.1093/biostatistics/kxad015 · Cited by 14 · see citing works
Khanal, S., Choi, L., Innes‐Hughes, C., & Rissel, C. E. (2019) · BMC Public Health · doi:10.1186/s12889-019-7094-5 · Cited by 14 · see citing works
Başkaya, E., & Demi̇r, S. (2021) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2021.101504 · Cited by 14 · see citing works
Krieger, Y., Pachevsky, M., Shoham, Y., Biederko, R., Novack, L., & Sarid, O. (2021) · Burns · doi:10.1016/j.burns.2021.06.008 · Cited by 13 · see citing works
Intracorporal knot tying techniques – which is the right one?
Romero, P., Nickel, F., Mantel, M., Frongia, G., Roßler, A., Kowalewski, K., Müller-Stich, B. P., & Günther, P. S. (2016) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2016.11.049 · Cited by 13 · see citing works
Zéqui, S. D. C., Mourão, T. C., de Oliveira, M. M., Curado, M. P., Gueglio, G., da Costa, W. H., Zúñiga, Á., Bengió, R. H., Scorticati, C. H., Rodríguez, F., et al. (2019) · Kidney Cancer · doi:10.3233/kca-190068 · Cited by 13 · see citing works
Becker, J., Zwerenz, R., Johansson, R., Frederick, R. J., Andersson, G., & Beutel, M. E. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.07.003 · Cited by 11 · see citing works
Qiu, H., Cook, A. J., & Bobb, J. F. (2023) · Statistics in Medicine · doi:10.1002/sim.9950 · Cited by 12 · see citing works
Hernon, M., Hall, A. M., O’Mahony, J. F., Normand, C. E. M., & Hurley, D. A. (2017) · Physical Therapy · doi:10.1093/ptj/pzx073 · Cited by 13 · see citing works
Clinical stroke research in resource limited settings: Tips and hints
Pandian, J. D., Liu, H., Gandhi, D. B. C., & Lindley, R. I. (2017) · International Journal of Stroke · doi:10.1177/1747493017743798 · Cited by 11 · see citing works
Stress-Related Tinnitus Treatment Protocols
Cima, R. F. F. (2017) · Springer eBooks · doi:10.1007/978-3-319-58397-6_8 · Cited by 11 · see citing works
Prihartadi, A. S., Licastro, G. I., Pearson, M., Johnson, M. J., Luckett, T., & Swan, F. (2021) · BMJ Supportive & Palliative Care · doi:10.1136/bmjspcare-2021-002962 · Cited by 11 · see citing works
So, W. K. W., Chan, D. N. S., Law, B. M. H., Choi, K. C., Krishnasamy, M., & Chan, C. W. H. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104254 · Cited by 11 · see citing works
Golzarri-Arroyo, L., Dickinson, S. L., Jamshidi-Naeini, Y., Zoh, R. S., Brown, A. W., Owora, A. H., Li, P., Oakes, J. M., & Allison, D. B. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106654 · Cited by 9 · see citing works
Lange, K. M., Sullivan, T., Kasza, J., & Yelland, L. N. (2024) · Statistics in Medicine · doi:10.1002/sim.10201 · Cited by 10 · see citing works
Ivermectin treatment in humans for reducing malaria transmission
de Souza, D. K., Thomas, R., Bradley, J., Leyrat, C., Boakye, D. A., & Okebe, J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013117.pub2 · Cited by 9 · see citing works
Self-citationPowell, J. R., Murray, L. J., Burnet, N. G., Fernandez, S., Lingard, Z., McParland, L., O’Hara, D. J., Whitfield, G. A., & Short, S. C. (2019) · Clinical Oncology · doi:10.1016/j.clon.2019.09.049 · Cited by 10 · see citing works
Parker, K., Nunns, M., Xiao, Z., Ford, T. J., & Ukoumunne, O. C. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044143 · Cited by 11 · see citing works
- Authors
- Kahan, B. C., Forbes, G., Ali, Y., Jairath, V., Bremner, S., Harhay, M. O., Hooper, R., Wright, N., Eldridge, S. M., & Leyrat, C.
- Affiliations
- Queen Mary University of London (GB)
- Western University (CA)
- Brighton and Sussex Medical School (GB)
- University of Pennsylvania (US)
- Inserm (FR)
- Corresponding author
- Kahan, Brennan C.
- ORCID
- Venue
- Trials
- Published
- 2016-09-06
- Volume
- 17
- Issue
- 1
- Pages
- 438-438
- ISSN
- 1745-6215 (ISSN-L)
- PMC
- PMC5013635
- OpenAlex
- W2515120484
- Cited by
- 185
- References
- 29
- Citations by year
- 2026: 10 · 2025: 11 · 2024: 22 · 2023: 12 · 2022: 16 · 2021: 35 · 2020: 26 · 2019: 16 · 2018: 13 · 2017: 15 · 2016: 9
- FWCI
- 17.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Trials (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- Research Portal (King's College London) (submittedVersion, other-oa)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion, cc-by)
- Funders
- National Institute for Health and Care Research (NIHR-CTF-2014-03-05)
- National Institutes of Health
- National Heart, Lung, and Blood Institute (F31HL127947)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- type I error, small sample correction, mixed models, generalized estimating equations, small number of clusters, clustering, statistical power
- MeSH terms
- Computer Simulation, Data Interpretation, Statistical, Humans, Risk, Cluster Analysis, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review