The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research
Abstract
The objective of this research was to identify determinants of the magnitude of intracluster correlation coefficients (ICCs) in cluster randomized trials from the field of implementation research. A survey of experts was conducted to generate a priori hypotheses of factors that might affect ICC size. Hypotheses were tested on empirical estimates of ICCs calculated from 21 implementation research datasets, mainly from the UK. Effects of setting (primary or secondary care), type of variable (process or outcome), type of measurement (objective or subjective), prevalence of outcome and size of cluster were tested. In total, 220 ICCs were available (range 0 to 0.415). Significant differences in ICC magnitude were found. The ICCs were significantly higher for process than for outcome variables, and for secondary care outcomes compared with primary care outcomes. The effects of prevalence and size were less clear cut. There was no evidence to suggest that type of measurement affected ICC size. In conclusion, accurate estimates of ICCs are essential for sample size calculations for cluster randomized trials of professional behaviour change interventions. This study demonstrates that ICCs are sensitive to a number of trial factors, particularly setting and outcome type. These factors must be considered when planning such cluster randomized trials.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Campbell et al extracted 220 ICCs from 21 data sets and reported the results separately for both primary and secondary care settings.”
From Tailored interventions to address determinants of practice (2015) · cites this paper as Campbell 2005
In the citing paper
“In the meta‐regression analyses, we carried out sensitivity analyses assuming a larger clustering effect than had been accounted for in the standard analyses, by using higher ICC estimates (i.e. the reported upper quartile range values) taken from Campbell (Campbell 2005).”
From Tailored interventions to address determinants of practice (2015) · cites this paper as Campbell 2005
Citations
Cites
Ouellette, J. A., & Wood, W. (1998) · Psychological Bulletin · doi:10.1037/0033-2909.124.1.54 · Cited by 3,480
Introducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Regression standard errors in clustered samples
Rogers, W. H. (1993) · Stata technical bulletin · Cited by 2,694
A Note on Robust Variance Estimation for Cluster‐Correlated Data
Williams, R. L. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00645.x · Cited by 2,501
Design and Analysis of Cluster Randomization Trials in Health Research
Davis, W. W. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s433 · Cited by 1,735
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
Ukoumunne, Gulliford, M. C., Chinn, Sterne, & Burney, P. (1999) · Health Technology Assessment · doi:10.3310/hta3050 · Cited by 1,211
Design and Analysis of Group-Randomized Trials
Sonnad, S. S. (1999) · Medicine & Science in Sports & Exercise · doi:10.1097/00005768-199902000-00025 · Cited by 1,130
Design and Analysis of Group-Randomized Trials
Murray, D. M. (2007) · doi:10.1093/oso/9780195120363.001.0001 · Cited by 1,117
Gulliford, M. C., Ukoumunne, O. C., & Chinn, S. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009904 · Cited by 853
Research works printed but not linked: 8
- Donner A, 1998, Appl Stat, 47, 95
- Campbell MK, 2000, JHSRP, 5, 12
- Mason J, 1999, JHSRP, 4, 106
- Rogers WH, 1993, Stata Technical Bulletin, 3, 88
- NHS Cervical Screening Programme, 2001, Cervical Screening: a pocket guide
- Adams G, 2003, JHSRP, 8, 87
- Eccles M, 1999, BJGP, 49, 35
- Turner RM, 2000, Proceedings of the International Society of Clinical Biostatistics, 49
Cited by
Sorted by citations
Items 201–275 of 275
Addressing the Practice Context in Evidence-Based Practice Implementation: Leadership and Climate
Shuman, C. J. (2017) · Deep Blue (University of Michigan) · Cited by 3
Miller, J., Donnelly, C. A., McClintock, C., Varette, K., Camargo, Y., Marsh, J. D., Taljaard, M., Mamun, M. S. A., Bacchus, G., Barber, D., et al. (2026) · JMIR Research Protocols · doi:10.2196/89004 · Cited by 2
Oommen, A. M., Mini, G., & George, K. (2019) · Public Health · doi:10.1016/j.puhe.2018.12.019 · Cited by 2
Design and analysis of cluster randomized trials
Harris, K. E. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-817663-4.00028-3 · Cited by 3
Dibao-Dina, C., Léger, J., Ettori-Ajasse, I., Boivin, E., Chambe, J., Abou-Mrad-Fricquegnon, K., Sun, S., Jego-Sablier, M., Motte, B., Chiron, B., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059464 · Cited by 2
Alidu, L., Al-Khudairy, L., Bharatan, I., Bird, P., Campbell, N., Currie, G., Hemming, K., Jolly, K. B., Kudrna, L., Lilford, R. J., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282848 · Cited by 3
A practical look at cluster-randomized trials
Moulton, L. H. (2005) · Clinical Trials · doi:10.1191/1740774505cn80ed · Cited by 2
Effecten van de NHG-Praktijkaccreditering
Nouwens, E., van Lieshout, J., Bouma, M., Braspenning, J. C. C., & Wensing, M. (2016) · Huisarts en Wetenschap · doi:10.1007/s12445-016-0236-4 · Cited by 2
Farhat, I., Moore, L., Porgo, T. V., Patton, M.-P., Tardif, P., Truchon, C., Berthelot, S., Stelfox, H. T., Gabbe, B. J., Lauzier, F., et al. (2019) · Age and Ageing · doi:10.1093/ageing/afz097 · Cited by 2
Evaluating Effectiveness of Complex System Interventions
Chen, J. (2010) · Springer eBooks · doi:10.1007/978-0-387-92853-1_33 · Cited by 2
Wilchesky, M., Ballard, S. A., Voyer, P., McCusker, J., Lungu, O., Champoux, N., Vu, T. T. M., Cole, M. G., Monette, J., Ciampi, A., et al. (2021) · BMC Geriatrics · doi:10.1186/s12877-021-02558-3 · Cited by 2
Clemes, S. A., Varela-Mato, V., Bodicoat, D. H., Brookes, C. L., Chen, Y., Cox, E., Edwardson, C. L., Gray, L. J., Guest, A. J., Johnson, V., et al. (2022) · Public Health Research · doi:10.3310/pnoy9785 · Cited by 4
Intraclass Correlation Coefficient with Emphasis on Clinical Trials
Campbell, M., Grimshaw, J., & Piaggio, G. G. (2014) · Wiley eBooks · doi:10.1002/9781118445112.stat06890 · Cited by 2
Self-citationTherapist variation within meta-analyses of psychotherapy trials
Walwyn, R. E. A. (2011) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 2
Farhat, I., Moore, L., Porgo, T. V., Assy, C., Belcaïd, A., Berthelot, S., Stelfox, H. T., Gabbe, B. J., Lauzier, F., Clément, J., et al. (2022) · PubMed · doi:10.1097/sla.0000000000003922 · Cited by 1
Crocker, A. M., Natkin, L. W., CALLAS, P. W., Bonnell, L. N., Clifton, J., Hitt, J. R., & Littenberg, B. (2022) · Cureus · doi:10.7759/cureus.30970 · Cited by 1
Giusti, M. F., Linhares, M. M., Riera, R., Kleinubing, D. R., Fonseca, A. V., Takahashi, A. A. R., & Waisberg, J. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014552 · Cited by 1
Improving colonoscopy quality in the national VA healthcare system
Gawron, A. J., Bailey, T. S., Codden, R. R., Dominitz, J. A., Gupta, S., Helfrich, C. D., Kahi, C. J., Krop, L., Malvar, C., McKee, G., et al. (2024) · Contemporary Clinical Trials · doi:10.1016/j.cct.2024.107784 · Cited by 1
Sato, K., Niimi, Y., Ihara, R., Iwata, A., & Iwatsubo, T. (2025) · Journal of Aging Research and Lifestyle · doi:10.1016/j.jarlif.2025.100020 · Cited by 1
Ranjan, S., Fälth, S., Kharat, P., Bassi, A., Bakhshi, G. D., Basak, D., Berg, J., Chatterjee, S., Felländer‐Tsai, L., Hemming, K., et al. (2026) · Trials · doi:10.1186/s13063-026-09491-z · Cited by 1
Wu, M.-J., Tsou, Y.-C., Gautama, M. S. N., & Huang, T. (2026) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2026.103166 · Cited by 1
Miller, J., Donnelly, C. A., McClintock, C., Varette, K., Camargo, Y., Marsh, J. D., Taljaard, M., Mamun, M. S. A., Bacchus, G., Barber, D., et al. (2026) · JMIR Research Protocols · doi:10.2196/89006 · Cited by 1
Current issues in clinical trials: standing on the shoulders of Jerome Cornfield
Ellenberg, S. S. (2012) · Statistics in Medicine · doi:10.1002/sim.5602 · Cited by 1
Klement, S. U. A. (2014) · Journal of Clinical Trials · doi:10.4172/2167-0870.1000192 · Cited by 1
Elbourne, D. R. (2012) · Cited by 1
Quinn, L., Martín, J., Asamane, E. A., Manaseki‐Holland, S., Lilford, R. J., Traoré, L., Thompson, J., Watson, S., & Hemming, K. (2024) · Trials · doi:10.1186/s13063-024-08328-x · Cited by 1
Weighting methods for truncation by death in cluster-randomized trials
Isenberg, D., Harhay, M. O., Mitra, N., & Li, F. (2025) · Statistical Methods in Medical Research · doi:10.1177/09622802241309348 · Cited by 1
Caille, A., Leyrat, C., & Giraudeau, B. (2013) · Annales de Dermatologie et de Vénéréologie · doi:10.1016/j.annder.2013.07.024 · Cited by 1
Impact of implementation strategies: controlled studies
van der Weijden, T., Eccles, M., Grimshaw, J., Campbell, M., Ramsay, C. R., & Grol, R. P. T. M. (2013) · doi:10.1002/9781118525975.ch20 · Cited by 1
Self-citationGoud, R. (2009) · UvA-DARE (University of Amsterdam) · Cited by 1
Towards integrated nurse-led self-management support in routine diabetes care
Vries, A. V. D. (2015) · Research Publications (Maastricht University) · doi:10.26481/dis.20151216ad · Cited by 1
Intraclass Correlation Coefficient in Clinical Trials
Ng, E. S. W., & Zou, L. (2019) · Wiley eBooks · doi:10.1002/9781118445112.stat06890.pub2 · Cited by 1
Watson, S., Asamane, E. A., Lilford, R. J., Hemming, K., Sidibe, C., Rego, R. T. T., Bensassi, S., Diarra, A., Diarra, Y., Diop, S. I., et al. (2021) · medRxiv · doi:10.1101/2021.12.15.21267512 · Cited by 1
Heterogeneity issues in the meta-analysis of cluster randomization trials.
Chen, S. F. (2012) · Scholarship@Western (Western University)
Guzmán, J. C. (2011) · MacSphere (McMaster University) · doi:10.71548/9237
Ensayos clínicos con asignación aleatoria de grupos
Argimón, J. M. (2010) · Elsevier eBooks · doi:10.1016/b978-84-8086-461-9.50015-1
7 Wetenschappelijk onderzoek naar kwaliteitsverbetering in de patiëntenzorg
Wollersheim, H. C. H., & van der Weijden, T. (2011) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-313-8236-1_7
20 Effecten van implementatie: gecontroleerde evaluaties
van der Weijden, T., Eccles, M., Grimshaw, J., & Grol, R. P. T. M. (2015) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-368-1077-7_20
Self-citationMcEachan, R., Santorelli, G., Bryant, M., Sahota, P., Farrar, D., Small, N., Akhtar, S., Sargent, J., Barber, S., Taylor, N. J., et al. (2016)
Evaluatief onderzoek naar kwaliteitsverbetering in de patiëntenzorg
Wollersheim, H. C. H., & van der Weijden, T. (2020) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-368-2360-9_7
Choosing the Target Difference (“effect size”) for a Randomised Controlled Trial - DELTA2 GuidancePreprint
Cook, J. A., Julious, S. A., Sones, W. R., Hampson, L. V., Hewitt, C. E., Berlin, J. A., Ashby, D., Emsley, R., Fergusson, D. A., Walters, S. J., et al. (2018) · Preprints.org · doi:10.20944/preprints201808.0521.v1
Idriss-Hassan, A., Berube, M., Belcaïd, A., Clément, J., Bourgeois, G., Rizzo, C., Neveu, X., Soltana, K., Thakore, J., & Moore, L. (2021) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-021-01681-5
van der Wouden, H., & Dees, M. (2016) · Huisarts en Wetenschap · doi:10.1007/s12445-016-0179-9
Machin, D., & Fayers, P. M. (2010) · doi:10.1002/9780470686232.refs
Self-citationMachin, D., Fayers, P. M., & Tai, B. C. (2021) · Wiley eBooks · doi:10.1002/9781119524687.refs
Self-citationEffecten van implementatie: gecontroleerde evaluaties
van der Weijden, T., Eccles, M., Grimshaw, J., & Grol, R. P. T. M. (2017) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-368-1732-5_20
Self-citationEnhanced recovery after gynecologic surgery
Anna, D. G. J. J. (2020) · doi:10.26481/dis.20200408jg
Evaluating Effectiveness of Complex System Interventions
Chen, J. (2017) · Springer eBooks · doi:10.1007/978-3-319-39391-9_33
Wendt, B., Oostra, D. L., Teerenstra, S., Nieuwboer, M. S., Vermeulen, H., van Dulmen, S. A., & Waal, G. H. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105159
Abrar, S., Hafeez, A., Rahim, S., Doi, S. A. R., & Khan, M. N. (2025) · BMC Public Health · doi:10.1186/s12889-025-21771-y
McCann, C., & Liu, X. (2025) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-025-00352-x · Cited by 1
Sequentially Constrained Randomization in Preclinical Animal Studies
Rigdon, J., Walkup, M. P., Amar, D., Wheeler, M. T., Goodyear, L. J., Bodine, S. C., Esser, K. A., Esserman, D. A., & Miller, M. E. (2025) · Function · doi:10.1093/function/zqaf046
Shakor, A. S. A., Mohamad, M., Ibrahim, K., & Sayed, I. M. (2025) · Cureus · doi:10.7759/cureus.99216
Clemes, S., Varela-Mato, V., Bodicoat, D. H., Brookes, C. L., Chen, Y.-L., Edwardson, C. L., Gray, L. J., Guest, A. J., Johnson, V., Munir, F., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19844692
Turner, S. L., Karahalios, A., Forbes, A. B., Taljaard, M., Grimshaw, J., & McKenzie, J. E. (2021) · Figshare · doi:10.6084/m9.figshare.14853378
Self-citationHemming, K., Taljaard, M., Gkini, E., & Bishop, J. (2024) · Figshare · doi:10.6084/m9.figshare.26618314
Tong, J., Li, F., Harhay, M. O., & Tong, G. (2023) · Figshare · doi:10.6084/m9.figshare.22621735
Amini, M., den Hartog, S. J., van Leeuwen, N., Eijkenaar, F., Kuhrij, L. S., Stolze, L. J., Nederkoorn, P. J., Lingsma, H. F., van Es, A. C. G. M., van den Wijngaard, I. R., et al. (2021) · Figshare · doi:10.6084/m9.figshare.17126164
Asamane, E. A., Quinn, L., Watson, S. I., Lilford, R. J., Hemming, K., Sidibe, C., Rego, R. T. T., Bensassi, S., Diarra, Y., Diop, S. I., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22602755
Asamane, E. A., Quinn, L., Watson, S. I., Lilford, R. J., Hemming, K., Sidibe, C., Rego, R. T. T., Bensassi, S., Diarra, Y., Diop, S. I., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22602752
Turner, S., Karahalios, A., Forbes, A. B., Taljaard, M., Grimshaw, J. M., & McKenzie, J. E. (2021) · Figshare · doi:10.6084/m9.figshare.14853381
Self-citationTurner, S. L., Karahalios, A., Forbes, A. B., Taljaard, M., Grimshaw, J. M., & McKenzie, J. E. (2021) · Figshare · doi:10.6084/m9.figshare.14853384
Self-citationAdditional file 1 of Required sample size to detect mediation in 3-level implementation studies
Williams, N. J., Preacher, K. J., Allison, P. D., Mandell, D. S., & Marcus, S. C. (2022) · Figshare · doi:10.6084/m9.figshare.21258791
Hemming, K., Taljaard, M., Gkini, E., & Bishop, J. (2024) · Figshare · doi:10.6084/m9.figshare.26618311
Asamane, E. A., Quinn, L., Watson, S. I., Lilford, R. J., Hemming, K., Sidibe, C., Rego, R. T. T., Bensassi, S., Diarra, Y., Diop, S. I., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22602758
Clemes, S., Varela-Mato, V., Bodicoat, D. H., Brookes, C. L., Chen, Y.-L., Edwardson, C. L., Gray, L. J., Guest, A. J., Johnson, V., Munir, F., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19844695
Diener, J., Krafft, J., Rayling, S., Krell‐Roesch, J., Wäsche, H., Flagmeier, A. L., Wöll, A., & Wunsch, K. (2026) · Sports · doi:10.3390/sports14010041
Senoo-Dogbey, V. E., Dzansi, G., Akpagloh, M. C., Wuaku, D. A., Darkwa, I. O., Boadu, A. A., Michael, D. A., Dzantor, E. K., & Anatsui, P. J. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-114815
Moore, M. B., Okamuro, K., Bresee, C. J., Eskander, R., Gin, G. T., Grisales, T., Galvez, A., Gregory, K. D., Koola, J., Lukacz, E. S., et al. (2026) · Trials · doi:10.1186/s13063-026-09427-7
Cluster effect expression for binary data
Yepnang, A. M. M. (2020) · doi:10.70675/5e3ed03fzabaaz473bzb89bzdecb583d6809
Scharf, T., Janggen, M.-A., Martin, Y., Jakob, J., Tal, K., Biller‐Andorno, N., Bulliard, J., Selby, K. J., Auer, R., & Rohrbasser, A. (2026) · International Journal of Public Health · doi:10.3389/ijph.2026.1608462
Tong, J., Li, F., & Tong, G. (2026) · Biometrical Journal · doi:10.1002/bimj.70146
Liu, E. M., 甘礼园, Wl, M., Chen, M., Zhang, C., Huang, X., & Deng, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1883769
Statistical Analysis of Correlated Ordinal Data: Application to Cluster Randomization Trials
Gao, R. (2013) · Scholarship@Western (Western University)
Metadata
- Authors
- Campbell, M. K., Fayers, P. M., & Grimshaw, J. M.
- Affiliations
- University of Aberdeen (GB)
- Ottawa Hospital Research Institute (CA)
- Corresponding author
- Campbell, Marion K
- Venue
- Clinical Trials
- Publisher
- SAGE Publications
- Published
- 2005-04
- Volume
- 2
- Issue
- 2
- Pages
- 99-107
- ISSN
- 1740-7745 (ISSN-L)
- 1740-7753
- OpenAlex
- W2164947695
- Cited by
- 275
- References
- 30
- Citations by year
- 2026: 11 · 2025: 9 · 2024: 13 · 2023: 20 · 2022: 18 · 2021: 32 · 2020: 14 · 2019: 15 · 2018: 10 · 2017: 15 · 2016: 14 · 2015: 15 · 2014: 16 · 2013: 8 · 2012: 17
- FWCI
- 9.80
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Trials (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Canada Research Chairs
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- intra-cluster correlation, implementation research, primary care, secondary care, sample size determination, cluster size, measurement types
- MeSH terms
- Data Interpretation, Statistical, Health Services Research, Humans, Surveys and Questionnaires, Research Design, Models, Statistical, Cluster Analysis, Randomized Controlled Trials as Topic, Sample Size, Evidence-Based Medicine, Biomedical Research, Empirical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review