Individually Randomized Group Treatment Trials: A Critical Appraisal of Frequently Used Design and Analytic Approaches
Open accessAbstract
Objectives. We reviewed published individually randomized group treatment (IRGT) trials to assess researchers’ awareness of within-group correlation and determine whether appropriate design and analytic methods were used to test for treatment effectiveness. Methods. We assessed sample size and analytic methods in IRGT trials published in 6 public health and behavioral health journals between 2002 and 2006. Results. Our review included 34 articles; in 32 (94.1%) of these articles, inappropriate analytic methods were used. In only 1 article did the researchers claim that expected intraclass correlations (ICCs) were taken into account in sample size estimation; in most articles, sample size was not mentioned or ICCs were ignored in the reported calculations. Conclusions. Trials in which individuals are randomly assigned to study conditions and treatments administered in groups may induce within-group correlation, violating the assumption of independence underlying commonly used statistical methods. Methods that take expected ICCs into account should be used in reexamining past studies and planning future studies to ensure that interventions are not judged effective solely on the basis of statistical artifacts. We strongly encourage investigators to report ICCs from IRGT trials and describe study characteristics clearly to aid these efforts.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Additional elements for trial designs other than parallel group superiority trials are often not described in final trial reports and are infrequently specified in the protocol.
“Such elements are often not described in final trial reports,299 300 301 302 and they are infrequently specified in the protocol.303”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [299]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550 · see citing works
Bootstrap Methods and Their Application
Loughin, T. M., Davison, A. C., & HINKLEY, D. V. (1999) · Journal of the American Statistical Association · doi:10.2307/2669707 · Cited by 5,478 · see citing works
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · see citing works
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557 · see citing works
Moher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426 · see citing works
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236 · see citing works
Donner, A. P., Birkett, N., & Buck, C. W. (1981) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a113261 · Cited by 698 · see citing works
Design and analysis of group-randomized trials: a review of recent methodological developments
Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.423 · Cited by 677 · see citing works
Self-citationConsequences of violating the independence assumption in analysis of variance.
Kenny, D. A., & Judd, C. M. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.3.422 · Cited by 616 · see citing works
Wiegand, H. (1968) · Biometrische Zeitschrift · doi:10.1002/bimj.19680100122 · Cited by 510 · see citing works
The statistical analysis of data from small groups.
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.83.1.126 · Cited by 470 · see citing works
Crits-Christoph, P. F., & Mintz, J. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.20 · Cited by 352 · see citing works
Design and analysis of clinical trials with clustering effects due to treatment
Roberts, C., & Roberts, S. A. (2005) · Clinical Trials · doi:10.1191/1740774505cn076oa · Cited by 299 · see citing works
Effect of internet support on the long-term maintenance of weight loss
Harvey‐Berino, J., Pintauro, S., Buzzell, P., & Gold, E. C. (2004) · Obesity Research · doi:10.1038/oby.2004.40 · Cited by 283 · see citing works
Lepore, S. J., Helgeson, V. S., Eton, D. T., & Schulz, R. (2003) · Health Psychology · doi:10.1037/0278-6133.22.5.443 · Cited by 274 · see citing works
van den Bosch, L. V. D., Verheul, R., Schippers, G., & van den Brink, W. J. (2002) · Addictive Behaviours · doi:10.1016/s0306-4603(02)00293-9 · Cited by 256 · see citing works
Miller, G. D., Nicklas, B. J., Davis, C., Loeser, R. F., Lenchik, L., & Messier, S. P. (2006) · Obesity · doi:10.1038/oby.2006.139 · Cited by 247 · see citing works
Cornfield, J. (1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112592 · Cited by 235 · see citing works
Methods to reduce the impact of intraclass correlation in group-randomized trials.
Murray, D. M., & Blitstein, J. L. (2003) · Evaluation Review · doi:10.1177/0193841x02239019 · Cited by 221 · see citing works
Self-citationPerformance of Generalized Estimating Equations in Practical Situations
Lipsitz, S. R., Fitzmaurice, G. M., Orav, E. J., & Laird, N. M. (1994) · Biometrics · doi:10.2307/2533218 · Cited by 220 · see citing works
Baldwin, S. A., Murray, D. M., & Shadish, W. R. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.924 · Cited by 209 · see citing works
Self-citationMayer-Davis, E. J., D'Antonio, A. M., Smith, S. M., Kirkner, G., Levin Martin, S., Parra-Medina, D., & Schultz, R. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.10.1736 · Cited by 202 · see citing works
The consequence of ignoring a nested factor on measures of effect size in analysis of variance.
Wampold, B. E., & Serlin, R. C. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.4.425 · Cited by 186 · see citing works
Epstein, L. H., Paluch, R. A., KILANOWSKI, C. K., & Raynor, H. A. (2004) · Health Psychology · doi:10.1037/0278-6133.23.4.371 · Cited by 185 · see citing works
Margolin, A., Avants, S. K., Warburton, L. A., Hawkins, K. A., & Shi, J. M. (2003) · Health Psychology · doi:10.1037/0278-6133.22.2.223 · Cited by 173 · see citing works
Intraclass correlation coefficients for cluster randomized trials in primary care
Smeeth, L. L., & Ng, E. S.-W. (2002) · Controlled Clinical Trials · doi:10.1016/s0197-2456(02)00208-8 · Cited by 168 · see citing works
Wyatt, G. E., Longshore, D. L., Chin, D., Carmona, J. V., Loeb, T. B., Myers, H. F., Warda, U. S., Liu, H., & Rivkin, I. D. (2004) · AIDS and Behavior · doi:10.1007/s10461-004-7329-y · Cited by 159 · see citing works
Design and Analysis of Group-Randomized Trials: A Review of Recent Practices
Varnell, S. P., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.393 · Cited by 140 · see citing works
Self-citationJackson, K. M., & Aiken, L. S. (2006) · Health Psychology · doi:10.1037/0278-6133.25.1.34 · Cited by 124 · see citing works
Parker, D. R., Εvangelou, Ε., & Eaton, C. B. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.01.002 · Cited by 115 · see citing works
Stice, E. M., Presnell, K. E., Groesz, L. M., & Shaw, H. E. (2005) · Health Psychology · doi:10.1037/0278-6133.24.4.402 · Cited by 111 · see citing works
Taylor, K. L., Lamdan, R. M., Siegel, J. E., Shelby, R. A., Moran‐Klimi, K., & Hrywna, M. (2003) · Health Psychology · doi:10.1037/0278-6133.22.3.316 · Cited by 106 · see citing works
An Analysis of Variance Pitfall: The Fixed Effects Analysis in a Nested Design
Zucker, D. M. (1990) · Educational and Psychological Measurement · doi:10.1177/0013164490504002 · Cited by 103 · see citing works
The Impact of Calcium and Dairy Product Consumption on Weight Loss
Harvey‐Berino, J., Gold, B. C., Lauber, R., & Starinski, A. (2005) · Obesity Research · doi:10.1038/oby.2005.210 · Cited by 103 · see citing works
Auricular acupuncture, education, and smoking cessation: a randomized, sham-controlled trial.
Bier, I. D., Wilson, J., Studt, P., & Shakleton, M. (2002) · American Journal of Public Health · doi:10.2105/ajph.92.10.1642 · Cited by 103 · see citing works
Kukkonen-Harjula, K. T., Borg, P. T., Nenonen, A., & Fogelholm, M. (2005) · Preventive Medicine · doi:10.1016/j.ypmed.2005.07.008 · Cited by 93 · see citing works
Sikkema, K. J., Hansen, N. B., Ghebremichael, M. S., Kochman, A., Tarakeshwar, N., Meade, C. S., & Heping Zhang, D. (2006) · Health Psychology · doi:10.1037/0278-6133.25.5.563 · Cited by 92 · see citing works
School-Level Intraclass Correlation for Physical Activity in Adolescent Girls
Murray, D. M., Catellier, D., Hannan, P. J., Treuth, M. S., Stevens, J., Schmitz, K. H., Rice, J. C., & Conway, T. L. (2004) · Medicine & Science in Sports & Exercise · doi:10.1249/01.mss.0000126806.72453.1c · Cited by 87 · see citing works
Self-citationThe use of random effects models to allow for clustering in individually randomized trials
Lee, K. J., & Thompson, S. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn082oa · Cited by 86 · see citing works
Design and Analysis of Cluster Randomization Trials
Zucker, D. M. (2003) · doi:10.1201/9780203912881-5 · Cited by 86 · see citing works
Herman, S., Blumenthal, J. A., Babyak, M. A., Khatri, P., Craighead, W. E., Krishnan, K. R., & Doraiswamy, P. M. (2002) · Health Psychology · doi:10.1037/0278-6133.21.6.553 · Cited by 85 · see citing works
A Monte Carlo Study of Alternative Responses To Intraclass Correlation in Community Trials
Murray, D. M., Hannan, P. J., & Baker, W. L. (1996) · Evaluation Review · doi:10.1177/0193841x9602000305 · Cited by 85 · see citing works
Self-citationCreamer, M. C., Morris, P., Biddle, D., & Elliott, P. (1999) · Journal of Traumatic Stress · doi:10.1023/a:1024702931164 · Cited by 85 · see citing works
Wiers, R. W. H. J., & Kummeling, R. H. C. (2003) · Addictive Behaviors · doi:10.1016/s0306-4603(03)00081-9 · Cited by 83 · see citing works
Efficacy of an HIV Prevention Program Among Female Adolescents Experiencing Gender-Based Violence
Wingood, G. M., DiClemente, R. J., Harrington, K., Lang, D. L., Davies, S. L., Hook, E. W., Oh, M. K., & Hardin, J. W. (2006) · American Journal of Public Health · doi:10.2105/ajph.2004.053595 · Cited by 79 · see citing works
The therapist-as-fixed-effect fallacy in psychotherapy research.
Martindale, C. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.6.1526 · Cited by 73 · see citing works
Copeland, A. L., Martin, P. D., Geiselman, P. J., Rash, C. J., & Kendzor, D. E. (2005) · Addictive Behaviors · doi:10.1016/j.addbeh.2005.04.020 · Cited by 73 · see citing works
Schroeder, J., Epstein, D., Umbricht, A., & Preston, K. (2006) · Addictive Behaviours · doi:10.1016/j.addbeh.2005.07.009 · Cited by 63 · see citing works
Effects of exercise on emerging and traditional cardiovascular risk factors
Duncan, G. E. (2004) · Preventive Medicine · doi:10.1016/j.ypmed.2004.03.012 · Cited by 59 · see citing works
Effect of an STD/HIV Behavioral Intervention on Women's Use of the Female Condom
Devanter, N. V., Gonzáles, V., Merzel, C. R., et al. (2002) · American Journal of Public Health · doi:10.2105/ajph.92.1.109 · Cited by 59 · see citing works
A combined breast health/weight loss intervention for Black women
Fitzgibbon, M., Stolley, M. R., Schiffer, L. A., Sánchez-Johnsen, L., Wells, A. M., & Dyer, A. R. (2004) · Preventive Medicine · doi:10.1016/j.ypmed.2004.06.018 · Cited by 50 · see citing works
Comparing the Benefits of Diet and Exercise in the Treatment of Dyslipidemia
Lalonde, L., Gray‐Donald, K., Lowensteyn, I., Marchand, S., Dorais, M., Michaels, G., Llewellyn‐Thomas, H. A., O'Connor, A. M. C., & Grover, S. A. (2002) · Preventive Medicine · doi:10.1006/pmed.2002.1052 · Cited by 49 · see citing works
Analyzing data from individuals in groups: The past, the present, and the future.
Hoyle, R. H., Georgesen, J. C., & Webster, J. M. (2001) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.5.1.41 · Cited by 48 · see citing works
Aerobic Exercise and Snoring in Overweight Children: A Randomized Controlled Trial
Davis, C. L., Tkacz, J. P., Gregoski, M. J., Boyle, C. A., & Lovrekovic, G. (2006) · Obesity · doi:10.1038/oby.2006.232 · Cited by 48 · see citing works
Murray, D. M., Phillips, G. A., Birnbaum, A. S., & Lytle, L. A. (2001) · Health Education & Behavior · doi:10.1177/109019810102800602 · Cited by 43 · see citing works
Self-citationKorhonen, M. J., Kastarinen, M. J., Uusitupa, M. I. J., Puska, P., & Nissinen, A. M. (2002) · Preventive Medicine · doi:10.1006/pmed.2002.1120 · Cited by 42 · see citing works
An Evaluation of Analysis Options for the One-Group-Per-Condition Design
Varnell, S. P., Murray, D. M., & Baker, W. L. (2001) · Evaluation Review · doi:10.1177/0193841x0102500402 · Cited by 42 · see citing works
Self-citationPrescribed “Breaks” as a Means to Disrupt Weight Control Efforts
Wing, R. R., & Jeffery, R. W. (2003) · Obesity Research · doi:10.1038/oby.2003.43 · Cited by 39 · see citing works
Gurka, M. J., Wolf, A., Conaway, M. R., Crowther, J. Q., Nadler, J. L., & Bovbjerg, V. E. (2006) · Obesity · doi:10.1038/oby.2006.124 · Cited by 36 · see citing works
Clinical trials of behavioural interventions with heterogeneous teaching subgroup effects
Hoover, D. R. (2002) · Statistics in Medicine · doi:10.1002/sim.1139 · Cited by 34 · see citing works
Body image treatment for weight concerned smokers: A pilot study
Clark, M. M., Hays, J. T., Vickers, K. S., Patten, C. A., Croghan, I. T., Berg, E., Wadewitz, S., Schwartz, S. A., Decker, P. A., Offord, K. P., et al. (2004) · Addictive Behaviors · doi:10.1016/j.addbeh.2004.10.009 · Cited by 28 · see citing works
Intraclass correlation among measures related to cigarette use by adolescents
Murray, D. M., Alfano, C. M., Zbikowski, S. M., Padgett, L. S., Robinson, L. A., & Klesges, R. C. (2002) · Addictive Behaviors · doi:10.1016/s0306-4603(01)00189-7 · Cited by 25 · see citing works
Self-citationHerzog, T. A., Lazev, A. B., Irvin, J. E., Juliano, L. M., Greenbaum, P. E., & Brandon, T. H. (2002) · Behavior Therapy · doi:10.1016/s0005-7894(02)80004-1 · Cited by 21 · see citing works
Scheier, L. M., Griffin, K. W., Doyle, M. M., et al. (2002) · Health Education & Behavior · doi:10.1177/109019810202900109 · Cited by 21 · see citing works
A Satisfaction Enhancement Intervention for Long‐Term Weight Loss
Jeffery, R. W., Linde, J. A., Finch, E. A., Rothman, A. J., & King, C. M. (2006) · Obesity · doi:10.1038/oby.2006.100 · Cited by 21 · see citing works
A mother-based intervention trial for osteoporosis prevention in children
Winzenberg, T. M., Oldenburg, B. F., Frendin, S., Wit, L. D., & Jones, G. (2005) · Preventive Medicine · doi:10.1016/j.ypmed.2005.11.006 · Cited by 21 · see citing works
Effect of cognitive behavior therapy on smokers' compensatory coping skills
Thorndike, F. P., Friedman‐Wheeler, D. G., & Haaga, D. A. F. (2006) · Addictive Behaviors · doi:10.1016/j.addbeh.2005.12.005 · Cited by 18 · see citing works
Day treatment for cocaine dependence
Marlowe, D. B., Kirby, K. C., Festinger, D. S., Merikle, E. P., Tran, G. Q., & Platt, J. J. (2003) · Addictive Behaviors · doi:10.1016/s0306-4603(01)00248-9 · Cited by 16 · see citing works
Hoff, C. C., Pals, S. L., Purcell, D. W., et al. (2006) · AIDS and Behavior · doi:10.1007/s10461-006-9077-7 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- [1]Cornfield J. Randomization by group: a formal analysis. Am J Epidemiol. 1978;108:100–102. [DOI] [PubMed] [Google Scholar]
- [3]Murray DM. Design and Analysis of Group-Randomized Trials. New York, NY: Oxford University Press Inc; 1998.
- [8]Kish L. Survey Sampling. New York, NY: John Wiley & Sons Inc; 1965.
Cited by
Sorted by citations
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · see citing works
Review of recent methodological developments in group-randomized trials: part 2—analysis
Turner, E. L., Prague, M., Gallis, J. A., Li, F., & Murray, D. M. (2017) · American Journal of Public Health · doi:10.2105/ajph.2017.303707 · Cited by 195 · see citing works
Self-citationSPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Taylor, S. J., Carnes, D., Homer, K., Pincus, T., Kahan, B. C., Hounsome, N., Eldridge, S., Spencer, A., Diaz-Ordaz, K., Rahman, A., et al. (2016) · Programme Grants for Applied Research · doi:10.3310/pgfar04140 · Cited by 44 · see citing works
Kubik, M. Y., Fulkerson, J. A., Sirard, J. R., Garwick, A., Temple, J., Gurvich, O., Lee, J., & Dudovitz, B. (2018) · Contemporary Clinical Trials · doi:10.1016/j.cct.2018.10.011 · Cited by 26 · see citing works
Guerrini Usubini, A., Cattivelli, R., Giusti, E. M., Riboni, F. V., Varallo, G., Pietrabissa, G., Manzoni, G. M., Consoli, S., Bastoni, I., Granese, V., et al. (2021) · Trials · doi:10.1186/s13063-021-05191-y · Cited by 14 · see citing works
Musanje, K., Kasujja, R., Camlin, C. S., Hooper, N., Hope-Bell, J., Sinclair, D. L., Kibanja, G. M., Mpirirwe, R., Kalyango, J. N., & Kamya, M. R. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301988 · Cited by 10 · see citing works
Cattivelli, R., Guerrini Usubini, A., Manzoni, G. M., Vailati Riboni, F., Pietrabissa, G., Musetti, A., Franceschini, C., Varallo, G., Spatola, C. A. M., Giusti, E., et al. (2021) · doi:10.20944/preprints202107.0156.v1 · Cited by 20 · see citing works
- Authors
- Pals, S. L., Murray, D. M., Alfano, C. M., Shadish, W. R., Hannan, P. J., & Baker, W. L.
- Affiliations
- Centers for Disease Control and Prevention (US)
- Venue
- American Journal of Public Health
- Publisher
- American Public Health Association
- Published
- 2008-08
- Volume
- 98
- Issue
- 8
- Pages
- 1418-1424
- ISSN
- 0090-0036 (ISSN-L)
- 1541-0048
- PMC
- PMC2446464
- OpenAlex
- W2064994313
- Cited by
- 104
- References
- 70
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 9 · 2023: 7 · 2022: 7 · 2021: 11 · 2020: 5 · 2019: 4 · 2018: 4 · 2017: 7 · 2016: 7 · 2015: 3 · 2014: 5 · 2013: 7 · 2012: 4
- FWCI
- 3.78
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- American Journal of Public Health (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- within-group correlation, intraclass correlation coefficient, sample size determination, statistical analysis methods, cluster randomization, treatment effectiveness, public health interventions, behavioral health interventions, reporting quality
- MeSH terms
- Biometry, Data Interpretation, Statistical, Humans, Research Design, Public Health Practice, Randomized Controlled Trials as Topic, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review