Design of Cluster-Randomized Trials of Quality Improvement Interventions Aimed at Medical Care Providers
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Randomized trials aimed at improving the quality of medical care often randomize the provider. Such trials are frequently embedded in health care systems with available automated records, which can be used to enhance the design of the trial. METHODS: We consider how available information from automated records can address each of the following concerns in the design of a trial: whether to randomize individual providers or practices; clustering of outcomes among patients in the same practice and its impact on study size; expected heterogeneity in adherence and the response to the intervention; eligibility criteria and the trade-offs between generalizability and internal validity; and blocking or matching to alleviate covariate imbalance across practices. RESULTS: Investigators can use available information from an automated database to estimate the amount of clustering of patients within providers and practices, and these estimates can inform the decision on whether to randomize at the level of the patient, the provider, or the practice. We illustrate calculation of the anticipated design effect for a proposed cluster-randomized trial and its implications for sample size. With available claims data, investigators can apply focused eligibility criteria to exclude subjects and providers with expected low compliance or lower likelihood of benefit, although possibly at some loss of generalizability. Chance imbalances in covariates are more likely when randomization occurs at the level of the practice than at the level of the patient, so we propose a matching score to limit such imbalances by design. CONCLUSIONS: Challenges to compliance, expected small effects, and covariate imbalances are particularly likely in cluster-randomized trials of quality improvement interventions. When such trials are embedded in medical systems with available automated records, use of these data can enhance the design of the trial.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There are opportunities to embed research within ongoing organizational initiatives at low cost in settings where routine data are available and where mechanisms for audit are in place, and this should be considered in the planning and conduct of A&F studies [41].”
From No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention (2014) · cites this paper as [41]
In the citing paper
“Thus, settings where valid data for audit and mechanisms for feedback are already established are potential implementation research laboratories, enabling relatively rapid testing of multiple intervention components.”
From No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention (2014) · cites this paper as [41]
Citations
Cites
The Design and Analysis of Clinical Experiments
Fleiss, J. L. (1999) · Wiley eBooks · doi:10.1002/9781118032923 · Cited by 4,819
Fundamentals of Biostatistics.
(1969) · Annals of Internal Medicine · doi:10.7326/0003-4819-70-4-873 · Cited by 4,575
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
Pocock, S. J., & Simon, R. (1975) · Biometrics · doi:10.2307/2529712 · Cited by 2,350
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
A review of uses of health care utilization databases for epidemiologic research on therapeutics
Schneeweiss, S., & Avorn, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.012 · Cited by 1,277
Self-citationPrinciples of Educational Outreach ('Academic Detailing') to Improve Clinical Decision Making
Soumerai, S. B. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440040088034 · Cited by 1,084
Avorn, J., & Soumerai, S. B. (1983) · New England Journal of Medicine · doi:10.1056/nejm198306163082406 · Cited by 854
Self-citationPatterns of intra-cluster correlation from primary care research to inform study design and analysis
Adams, G., Gulliford, M. C., Ukoumunne, O. C., Eldridge, S., Chinn, S., & Campbell, M. J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.013 · Cited by 602
Indications for propensity scores and review of their use in pharmacoepidemiology
Glynn, R. J., Schneeweiss, S., & Stürmer, T. (2006) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/j.1742-7843.2006.pto_293.x · Cited by 541 · cited 1×
“Use of such a score is akin to balancing on a propensity score or a multivariate risk score.(26) To illustrate the approach, consider the possible use of a balancing score in the context of the trial of interventions to improve care for osteoporosis.”
MethodSelf-citationThe Exclusion of the Elderly and Women From Clinical Trials in Acute Myocardial Infarction
Gurwitz, J. H. (1992) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1992.03490110055029 · Cited by 532 · cited 1×
“For example, older people and women were frequently excluded from past clinical trials of treatments for myocardial infarction, leaving providers to wonder about the value of treatments in the age group with the highest event rates.(18) Wide eligibility criteria are particularly relevant for evaluation of quality impr…”
BackgroundValidation of diagnostic codes within medical services claims
Wilchesky, M., Tamblyn, R. M., & Huang, A. (2004) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00246-4 · Cited by 439
Puffer, S., Torgerson, D., & Watson, J. (2003) · BMJ · doi:10.1136/bmj.327.7418.785 · Cited by 351
Treasure, T., & MacRae, K. D. (1998) · BMJ · doi:10.1136/bmj.317.7155.362 · Cited by 344
The Design and Analysis of Clinical Experiments.
Armitage, P., & Fleiss, J. L. (1987) · Biometrics · doi:10.2307/2531561 · Cited by 310
Academic Detailing to Improve Use of Broad-Spectrum Antibiotics at an Academic Medical Center
Solomon, D. H., Van Houten, L., Glynn, R. J., Baden, L., Curtis, K., Schrager, H., & Avorn, J. (2001) · Archives of Internal Medicine · doi:10.1001/archinte.161.15.1897 · Cited by 270
Self-citationSteptoe, A., Doherty, S., Rink, E., Kerry, S., Kendrick, T., Hilton, S., & Day, S. (1999) · BMJ · doi:10.1136/bmj.319.7215.943 · Cited by 242
Balance in cluster randomized trials
RAAB, G. M., & Butcher, I. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3<351::aid-sim797>3.0.co;2-c · Cited by 194
Accounting for cluster randomization: a review of primary prevention trials, 1990 through 1993.
Simpson, J. M., Klar, N., & Donnor, A. (1995) · American Journal of Public Health · doi:10.2105/ajph.85.10.1378 · Cited by 177
Bayesian methods of analysis for cluster randomized trials with binary outcome data
Turner, R. M., Omar, R. Z., & Thompson, S. G. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3<453::aid-sim803>3.0.co;2-l · Cited by 126
Yusuf, S., Held, P., Teo, K. K., & Toretsky, E. R. (1990) · Statistics in Medicine · doi:10.1002/sim.4780090114 · Cited by 121
Henderson, M., Wight, D., Raab, G. M., Abraham, C., Parkes, A., Scott, S., & Hart, G. (2007) · BMJ · doi:10.1136/bmj.39014.503692.55 · Cited by 119
Are distinctive ethical principles required for cluster randomized controlled trials?
Hutton, J. L. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3<473::aid-sim805>3.0.co;2-d · Cited by 118
Adherence to Aspirin in the Prevention of Myocardial Infarction: The Physicians' Health Study
Glynn, R. J. (1994) · Archives of Internal Medicine · doi:10.1001/archinte.1994.00420230032005 · Cited by 82
Self-citationImproving Care of Patients At-Risk for Osteoporosis: A Randomized Controlled Trial
Solomon, D. H., Polinski, J. M., Stedman, M., Truppo, C., Breiner, L., Egan, C., Jan, S., Patel, M., Weiss, T. W., Chen, Y.-T., et al. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-006-0099-7 · Cited by 80
Self-citationBrookhart, M. A., Solomon, D. H., Wang, P., Glynn, R. J., Avorn, J., & Schneeweiß, S. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.07.005 · Cited by 38
Self-citationDesign and Analysis of Cluster Randomized Trials
Campbell, M., DONNER, A. P., & Elbourne, D. R. (2001) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · Cited by 21
Osteoporosis action: design of the healthy bones project trial
Solomon, D. H., Brookhart, M. A., Polinski, J., Katz, J. N., Cabral, D., Licari, A., & Avorn, J. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2004.11.012 · Cited by 16 · cited 1×
“Healthy Bones Project provides an example of such a trial.(10) This trial randomized 826 primary care physicians in Pennsylvania either to receive a one-on-one academic detailing encounter covering fall and fracture prevention and osteoporosis diagnosis and treatment, or to receive no intervention.”
BackgroundSelf-citationConcerns About Run-in Periods in Randomized Trials
Glynn, R. J., Buring, J. E., & Hennekens, C. H. (1998) · JAMA · doi:10.1001/jama.279.19.1526 · Cited by 14
Self-citationConcerns about run-in periods in randomized trials.
Glynn, R. J., Buring, J. E., & Hennekens, C. H. (1998) · PubMed · Cited by 13
Self-citationCompliance, bias, and power in clinical trials.
SEIGEL, D. G., & Podgor, M. J. (1991) · PubMed · Cited by 8
Cited by
Ivers, N. M., Sales, A., Colquhoun, H., Michie, S., Foy, R., Francis, J. J., & Grimshaw, J. M. (2014) · Implementation Science · doi:10.1186/1748-5908-9-14 · Cited by 400
“There are opportunities to embed research within ongoing organizational initiatives at low cost in settings where routine data are available and where mechanisms for audit are in place, and this should be considered in the planning and conduct of A&F studies [41].”
BackgroundDempsey, A. F., Pyrznawoski, J., Lockhart, S., Barnard, J., Campagna, E. J., Garrett, K., Fisher, A., Dickinson, L. M., & O’Leary, S. T. (2018) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2018.0016 · Cited by 288
Chaffin, M., Hecht, D., Bard, D., Silovsky, J. F., & Beasley, W. H. (2012) · Pediatrics · doi:10.1542/peds.2011-1840 · Cited by 272
Allocation techniques for balance at baseline in cluster randomized trials: a methodological review
Ivers, N. M., Halperin, I. J., Barnsley, J., Grimshaw, J. M., Shah, B. R., Tu, K., Upshur, R., & Zwarenstein, M. (2012) · Trials · doi:10.1186/1745-6215-13-120 · Cited by 251
Ivers, N. M., Taljaard, M., Dixon, S., Bennett, C., McRae, A., Taleban, J., Skea, Z., Brehaut, J. C., Boruch, R. F., Eccles, M. P., et al. (2011) · BMJ · doi:10.1136/bmj.d5886 · Cited by 193
Metadata
- Authors
- Glynn, R. J., Brookhart, M. A., Stedman, M., Avorn, J., & Solomon, D. H.
- Affiliations
- Brigham and Women's Hospital (US)
- Venue
- Medical Care
- Published
- 2007-10
- Volume
- 45
- Issue
- 10
- Pages
- S38-S43
- ISSN
- 0025-7079 (ISSN-L)
- OpenAlex
- W2039469899
- Cited by
- 72
- References
- 32
- Citations by year
- 2026: 1 · 2024: 2 · 2022: 5 · 2021: 4 · 2020: 4 · 2019: 7 · 2018: 5 · 2017: 2 · 2016: 4 · 2015: 6 · 2014: 8 · 2013: 5 · 2012: 11
- FWCI
- 1.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Care (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute on Aging (AG-18833)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- quality improvement intervention, healthcare workers, design effect, sample size, claims data, eligibility criteria, generalizability, internal validity, covariate imbalance, matching, blocking, intervention adherence
- MeSH terms
- Adult, Aged, Data Collection, Female, Health Services Research, Humans, Male, Management Information Systems, Middle Aged, Osteoporosis, Patient Compliance, Quality Assurance, Health Care, Research Design, United States, Multicenter Studies as Topic, Epidemiologic Factors, Cluster Analysis, Randomized Controlled Trials as Topic, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.