CoChat
Sign Up
Cited by 19Statistical Methods in Clinical TrialsEthics in Clinical ResearchMeta-analysis and systematic reviews

Selection bias and subject refusal in a cluster-randomized controlled trial

Open access
Yang, Rochelle, Carter, Barry L., Gums, Tyler H., Gryzlak, Brian M., Xu, Yinghui, Levy, Barcey T.·2017·BMC Medical Research Methodology
DOI 10.1186/s12874-017-0368-7PMC PMC5504663
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Selection bias and non-participation bias are major methodological concerns which impact external validity. Cluster-randomized controlled trials are especially prone to selection bias as it is impractical to blind clusters to their allocation into intervention or control. This study assessed the impact of selection bias in a large cluster-randomized controlled trial. METHODS: The Improved Cardiovascular Risk Reduction to Enhance Rural Primary Care (ICARE) study examined the impact of a remote pharmacist-led intervention in twelve medical offices. To assess eligibility, a standardized form containing patient demographics and medical information was completed for each screened patient. Eligible patients were approached by the study coordinator for recruitment. Both the study coordinator and the patient were aware of the site's allocation prior to consent. Patients who consented or declined to participate were compared across control and intervention arms for differing characteristics. Statistical significance was determined using a two-tailed, equal variance t-test and a chi-square test with adjusted Bonferroni p-values. Results were adjusted for random cluster variation. RESULTS: There were 2749 completed screening forms returned to research staff with 461 subjects who had either consented or declined participation. Patients with poorly controlled diabetes were found to be significantly more likely to decline participation in intervention sites compared to those in control sites. A higher mean diastolic blood pressure was seen in patients with uncontrolled hypertension who declined in the control sites compared to those who declined in the intervention sites. However, these findings were no longer significant after adjustment for random variation among the sites. After this adjustment, females were now found to be significantly more likely to consent than males (odds ratio = 1.41; 95% confidence interval = 1.03, 1.92). CONCLUSIONS: Though there appeared to be a higher consent rate for females than for males, the overall impact of potential selection bias and refusal to participate was minimal. Without rigorous methodology, selection bias may be a threat to external validity in cluster-randomized trials. TRIAL REGISTRATION: NCT01983813 . Date of registration: Oct. 28, 2013.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12874-017-0368-7

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Knowledge of treatment condition at the time of patient recruitment is known to be a risk factor for differential identification and recruitment of participants across treatment arms, unless recruitment and identification are conducted by someone blind to the treatment allocation or the inclusion criteria are broad.

    “Knowledge of treatment condition at the time of patient recruitment is known to be a risk factor for differential identification and recruitment of participants across treatment arms [1,14,16,37], unless recruitment and identification are conducted by someone blind to the treatment allocation or the inclusion criteria are broad [2,11,14].”

    From Cluster randomized trials of individual-level interventions were at high risk of bias (2021) · cites this paper as [37]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 35 · Cited by 19

Research works printed but not linked: 29

  1. [2]Hill AM McPhail SM Waldron N Etherton-Beer C Ingram K Flicker L Fall rates in hospital rehabilitation units after individualized patient and staff education programs: a pragmatic, stepped-wedge, cluster-randomized controlled trial Lancet 2015 385 9987 2592 2599 10.1016/S0140-6736(14)61945-0 25865864 · DOI
  2. [3]LaBresh KA Lazorick S Ariza AJ Furberg RD Whetstone L Hobbs C Implementation of the NHLBI integrated guidelines for cardiovascular health and risk reduction in children and adolescents: rationale and study design for young hearts, strong starts, a cluster-randomized trial targeting body mass index, blood pressure, and tobacco Contemp Clin Trials 2014 37 1 98 105 10.1016/j.cct.2013.11.011 24295879 PMC3923469 · DOI
  3. [5]Torgerson D Contamination in trials: is cluster randomization the answer? BMJ 2001 322 7282 355 357 10.1136/bmj.322.7282.355 11159665 PMC1119583 · DOI · Paper page
  4. [6]Gums T Carter B Foster E Cluster randomized trials for pharmacy practice research Int J Clin Pharm 2016 38 3 607 614 26715549
  5. [7]Eldridge S Kerry S Torgerson D Bias in identifying and recruiting participants in cluster randomized trials: what can be done? BMJ 2009 339 b4006 10.1136/bmj.b4006 19819928 · DOI · Paper page
  6. [8]Brierley G Brabyn S Torgerson D Watson J Bias in recruitment to cluster randomized trials: a review of recent publications J Eval Clin Pract 2012 18 4 878 886 10.1111/j.1365-2753.2011.01700.x 21689213 · DOI · Paper page
  7. [10]Campbell MJ Challenges of cluster randomized trials J Comp Eff Res 2014 3 3 271 281 10.2217/cer.14.21 24969154 · DOI
  8. [11]Puffer S Torgerson D Watson J Evidence for risk of bias in cluster randomized trials: review of recent trials published in three general medical journals BMJ 2003 327 7418 785 789 10.1136/bmj.327.7418.785 14525877 PMC214092 · DOI · Paper page
  9. [12]Tol WA Komproe IH Susanty D Jordans MJ Macy RD De Jong JT School-based mental health intervention for children affected by political violence in Indonesia: a cluster randomized trial JAMA 2008 300 6 655 662 10.1001/jama.300.6.655 18698064 · DOI · Paper page
  10. [13]Campbell R Starkey F Holliday J Audrey S Bloor M Parry-Langdon N An informal school-based peer-led intervention for smoking prevention in adolescence (ASSIST): a cluster randomized trial Lancet 2008 371 9624 1595 1602 10.1016/S0140-6736(08)60692-3 18468543 PMC2387195 · DOI · Paper page
  11. [14]Jewkes R Nduna M Levin J Jama N Dunkle K Puren A Impact of Stepping Stones on incidence of HIV and HSV-2 and sexual behavior in rural South Africa: cluster randomized controlled trial BMJ 2008 337 a506 10.1136/bmj.a506 18687720 PMC2505093 · DOI
  12. [15]Davies MJ Heller S Skinner TC Campbell MJ Carey ME Cradock S Effectiveness of the diabetes education and self management for ongoing and newly diagnosed (DESMOND) program for people with newly diagnosed type 2 diabetes: cluster randomized controlled trial BMJ 2008 336 7649 491 495 10.1136/bmj.39474.922025.BE 18276664 PMC2258400 · DOI
  13. [16]Carter BL Levy BT Gryzlak B Chrischilles EA Vander Weg MW Christensen AJ A centralized cardiovascular risk service to improve guideline adherence in private primary care offices Contemp Clin Trials 2015 43 25 32 10.1016/j.cct.2015.04.014 25952471 PMC4522340 · DOI
  14. [17]Chobonian AV Bakris GL Black HR Cushman WC Green LA Izzo JL The seventh report of the Joint National Committee on prevention, detection, evaluation, and treatment of high blood pressure: the JNC 7 report JAMA 2003 289 19 2560 2572 10.1001/jama.289.19.2560 12748199 · DOI · Paper page
  15. [18]Grundy SM Becker D Clark LT Cooper RS Denke MA Howard WJ Third report of the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report Circulation 2002 106 25 3144 3421 12485966
  16. [19]American Diabetes Association Standards of medical care in diabetes care—2012 Diabetes Care 2012 35 Suppl 1 S11 S63 22187469 PMC3632172
  17. [20]Klosky JL Tyc VL Lawford J Ashford J Lensing S Buscemi J Predictors of non-participation in a randomized intervention trial to reduce environmental tobacco smoke (ETS) exposure in pediatric cancer patients Pediatr Blood Cancer 2009 52 5 644 649 10.1002/pbc.21946 19156856 PMC2733242 · DOI
  18. [21]Chinn DJ White M Howel D Harland JO Drinkwater CK Factors associated with non-participation in a physical activity promotion trial Public Health 2006 120 4 309 319 10.1016/j.puhe.2005.11.003 16473376 · DOI
  19. [22]Karbalaeifar R Kazempour-Ardebili S Amiri P Ghannadi S Tahmasebinejad Z Amouzegar A Evaluating the effect of knowledge, attitude and practice on self-management in patients with type 2 diabetes Acta Diabetol 2016 53 6 1015 1023 10.1007/s00592-016-0905-6 27624577 · DOI
  20. [23]Weijman I Ros WJ Rutten GE Schaufeli WB Schabracg MJ Winnubst JA Frequency and perceived burden of diabetes self-management activities in employees with insulin-treated diabetes: relationships with health outcomes Diabetes Res Clinl Pract 2005 68 1 56 64 10.1016/j.diabres.2004.08.004 15811566 · DOI
  21. [24]Martinez L Consoli SM Monnier L Simon D Wong O Yomtov B Studying the hurdles of insulin prescription (SHIP): development, scoring and initial validation of a new self-administered questionnaire Health Qual Life Outcomes 2007 5 53 10.1186/1477-7525-5-53 17727695 PMC2042975 · DOI
  22. [25]Groeneveld IR Proper KI Van der Beek AJ Hildebrandt VH van Mechelen W Factors associated with non-participation and drop-out in a lifestyle intervention for workers with an elevated risk of cardiovascular disease Int J Behav Nutr Phys Act 2009 6 80 10.1186/1479-5868-6-80 19951417 PMC3224927 · DOI
  23. [26]Artama M Heiavaara S Sarkeala T Prattala R Pukkala E Malila N Determinants of non-participation in a mass screening program for colorectal cancer in Finaland Acta Oncol 2016 55 7 870 874 10.1080/0284186X.2016.1175658 27152755 · DOI
  24. [28]Baquet CR Commiskey P Daniel Mullins C Mishra SI Recruitment and participation in clinical trials: socio-demographic, rural/urban, and health care access predictors Cancer Detect Prev 2006 30 1 24 33 10.1016/j.cdp.2005.12.001 16495020 PMC3276312 · DOI
  25. [30]Arfken CL Balon R Declining participation in research studies Psychother Psychosom 2011 80 325 328 10.1159/000324795 21829043 · DOI
  26. [31]O’Toole BI Battistutta D Long A Crouch K A comparison of costs and data quality of three health survey methods: mail, telephone, and personal home interview Am J Epidemiol 1986 124 2 317 328 10.1093/oxfordjournals.aje.a114390 3728446 · DOI
  27. [32]Siemiatychi J A comparison of mail, telephone, and home interview strategies for household health surveys Am J Public Health 1979 69 3 238 245 10.2105/AJPH.69.3.238 420369 PMC1619081 · DOI
  28. [33]Barnes M Wiles N Morrison J Kessler D Williams C Kuyken W Exploring patients’ reasons for declining contact in a cognitive behavioral therapy randomized controlled trial in primary care Br J Gen Pract 2012 62 598 e371 e377 10.3399/bjgp12X641492 22546597 PMC3338059 · DOI
  29. [34]Sharp L, Cotton SC, Alexander L, Williams E, Gray NM, Reid JM; TOMBOLA Group. Reasons for participation and non-participation in a randomized controlled trial: postal questionnaire surveys of women eligible for TOMBOLA (Trial of Management of Borderline Other Low-Grade Abnormal smears). Clin Trials 2006;3(5):431-42. 17060217

Authors
Yang, R., Carter, B. L., Gums, T. H., Gryzlak, B. M., Xu, Y., & Levy, B. T.
Affiliations
  • University of Iowa (US)
  • The University of Texas at Austin (US)
Corresponding author
Carter, Barry L.
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2017-07-10
Volume
17
Issue
1
Pages
94-94
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/s12874-017-0368-7
PMC
PMC5504663
OpenAlex
W2734469331
Cited by
19
References
35
Citations by year
2025: 1 · 2023: 3 · 2022: 2 · 2021: 8 · 2019: 1 · 2018: 4
FWCI
0.95
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Iowa Research Online (The University of Iowa) (submittedVersion)
  • Iowa Research Online (University of Iowa) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Heart, Lung, and Blood Institute (R01HL116311)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
selection bias, generalizability, pharmacist intervention, rural primary care, diabetes mellitus, hypertension, ICARE study
MeSH terms
Aged, Cardiovascular Diseases, Diabetes Mellitus, Female, Humans, Hypertension, Male, Middle Aged, Prospective Studies, Rural Population, Selection Bias, Cluster Analysis, Patient Selection, Refusal to Participate

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review