The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
A qualitative study exploring the acceptability of the McNulty-Zelen design for randomised controlled trials evaluating educational interventions
Open accessAbstract
BACKGROUND: Traditional randomised controlled trials evaluating the effect of educational interventions in general practice may produce biased results as participants know they are being evaluated. We aimed to explore the acceptability of a McNulty-Zelen Cluster Randomised Control Trial (CRT) design which conceals from educational participants that they are in a RCT. Consent is obtained from a trusted third party considered appropriate to give consent on participants' behalf, intervention practice staff then choose whether to attend the offered education as would occur with normal continuing professional development. METHODS: We undertook semi structured telephone interviews in England with 16 general practice (GP) staff involved in a RCT evaluating an educational intervention aimed at increasing chlamydia screening tests in general practice using the McNulty-Zelen design, 4 Primary Care (PC) Research Network officers, 5 Primary Care Trust leads in Public or sexual health, and one Research Ethics committee Chair. Interviews were undertaken by members of the original intervention evaluation McNulty-Zelen design RCT study team. These experienced qualitative interviewers used an agreed semi-structured interview schedule and were careful not to lead the participants. To further mitigate against bias, the data analysis was undertaken by a researcher (CR) not involved in the original RCT. RESULTS: We reached data saturation and found five main themes; Support for the design: All found the McNulty-Zelen design acceptable because they considered that it generated more reliable evidence of the value of new educational interventions in real life GP settings. Lack of familiarity with study design: The design was novel to all. GP staff likened the evaluation using the McNulty-Zelen design to audit of their activities with feedback, which were to them a daily experience and therefore acceptable. Ethical considerations: Research stakeholders considered the consent procedure should be very clear and that these trial designs should go through at least a proportionate ethical review. GP staff were happy for the PCT leads to give consent on their behalf. GP research capacity and trial participation: GP staff considered the design increased generalisability, as staff who would not normally volunteer to participate in research due to perceived time constraints and paperwork might do so. Design 'worth it': All interviewees agreed that the advantages of the "more accurate" or "truer" results and information gained about uptake of workshops within Primary Care Trusts (PCTs) outweighed any disadvantages of the consent procedure. DISCUSSION: Our RCT was evaluating the effect of an educational intervention to increase chlamydia screening tests in general practices where there was routine monitoring of testing rates; our participants may have been less enthusiastic about the design if it had been evaluating a more controversial educational area, or if data monitoring was not routine. IMPLICATIONS: The McNulty-Zelen design should be considered for the evaluation of educational interventions, but these designs should have clear consent protocols and proportionate ethical review. TRIAL REGISTRATION: The trial was registered on the UK Clinical Research Network Study Portfolio database. UKCRN9722 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies that adopt the criterion of saturation quite frequently propose at the same time a prior sample size.
“Quite frequently, studies that adopt the criterion of saturation propose at the same time a prior sample size (e.g. McNulty et al. 2015; Long-Sutehall et al. 2011).”
From Saturation in qualitative research: exploring its conceptualization and operationalization (2018) · cites this paper as McNulty et al. 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 3×
“A six-stage Thematic Analysis using an inductive approach was used by the three researchers coding data (CR, CMCN, ER) [12] Additional file 4.”
BackgroundMethodInfluentialMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · cited 1×
“The method used to select participants within a Randomised Controlled Trial (RCT) is a crucial aspect of trial design [1].”
MethodMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345
Ethical issues in the design and conduct of cluster randomised controlled trials
Edwards, S., Braunholtz, D., Lilford, R. J., & Stevens, A. J. (1999) · BMJ · doi:10.1136/bmj.318.7195.1407 · Cited by 256 · cited 3×
“We suggest that the McNulty-Zelen design and consent process is similar to a cluster randomised trial, if the PCT were considered the cluster and the GP practices the participants [3].”
BackgroundMethodGjelstad, S., Hoye, S., Straand, J., Brekke, M., Dalen, I., & Lindbæk, M. (2013) · BMJ · doi:10.1136/bmj.f4403 · Cited by 153 · cited 1×
“The consent procedure can lead to loss of eligible clusters, because of low recruitment [2, 3].”
BackgroundAbraham, N. S., Young, J. M., & Solomon, M. J. (2006) · Surgery · doi:10.1016/j.surg.2005.08.014 · Cited by 150
Review of randomised trials using the post-randomised consent (Zelen's) design
Adamson, J., Cockayne, S., Puffer, S., & Torgerson, D. J. (2006) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.11.003 · Cited by 146 · cited 1×
“Our interviewees’ views agreed with a review of 44 studies using the Zelen design [17], in which half stated that they used the design to reduce bias, and is highlighted by the less than 50 % recruitment in several other GP educational cluster randomised trials [5, 18].”
Understanding controlled trials: What is Zelen's design?
Torgerson, D. J., & Roland, M. O. (1998) · BMJ · doi:10.1136/bmj.316.7131.606 · Cited by 141
When is informed consent required in cluster randomized trials in health research?
McRae, A. D., Weijer, C., Binik, A., Grimshaw, J., Boruch, R. F., Brehaut, J., Donner, A. P., Eccles, M., Saginur, R., White, A., et al. (2011) · Trials · doi:10.1186/1745-6215-12-202 · Cited by 109 · cited 1×
“also discussed the ethics of lack of consent in cluster randomised trials [20].”
BackgroundKontopantelis, E., Springate, D. A., Reeves, D., Ashcroft, D. M., Valderas, J. M., & Doran, T. (2014) · BMJ · doi:10.1136/bmj.g330 · Cited by 91 · cited 1×
“GP staff likened the study design to audit and feedback which were common, familiar and acceptable processes [14].”
BackgroundTactics of tobacco industry in Belgium (Rapid response letter)
Nemery, B., Joossens, L., Bärtsch, P., Vermeire, P. A., & Ninane, V. (2005) · Cited by 90
Snowdon, C., Elbourne, D., & Garcia, J. (1999) · Controlled Clinical Trials · doi:10.1016/s0197-2456(98)00049-x · Cited by 74
Rognstad, S., Brekke, M., Fetveit, A., Dalen, I., & Straand, J. (2013) · British Journal of General Practice · doi:10.3399/bjgp13x670688 · Cited by 53 · cited 1×
“Our interviewees’ views agreed with a review of 44 studies using the Zelen design [17], in which half stated that they used the design to reduce bias, and is highlighted by the less than 50 % recruitment in several other GP educational cluster randomised trials [5, 18].”
McNulty, C. A. M., Hogan, A. H., Ricketts, E. J., WALLACE, L. M., Oliver, I., Campbell, R. M., Kalwij, S., O'Connell, E. E., & Charlett, A. (2013) · Sexually Transmitted Infections · doi:10.1136/sextrans-2013-051029 · Cited by 44 · cited 7×
“tion” (practices were given information after the study end) [6].”
BackgroundMethodInfluentialSelf-citationCampbell, R. M., Peters, T., Grant, C., Quilty, B., & Dieppe, P. A. (2005) · Journal of Health Services Research & Policy · doi:10.1258/135581905774414150 · Cited by 38 · cited 1×
“In the single consent Zelen design all participants are randomised to intervention or control, but consent is only sought from the intervention group [8]; the control group receive standard care, and do not know about the intervention or the trial [7].”
BackgroundSelf-citationChang, R. W., Falconer, J., Stulberg, S. D., Arnold, W. J., & Dyer, A. R. (1990) · Journal of Bone and Joint Surgery · doi:10.2106/00004623-199072100-00004 · Cited by 34 · cited 1×
“The use of the Zelen design has been used to increase recruitment resulting in a 6-fold increase in recruitment in an arthroscopy study [16].”
MethodInteractive workshops increase chlamydia testing in primary care--a controlled study
McNulty, C. A. M., Thomas, M. D., Bowen, J., Buckley, C., Charlett, A., Gelb, D., Foy, C. J. W., Sloss, J. H., & Smellie, S. (2008) · Family Practice · doi:10.1093/fampra/cmn032 · Cited by 20 · cited 1×
“As far as we know this McNulty-Zelen Design has only been used by our group [6, 9], and a recent editorial critiquing the study questioned “was that deception necessary?” [10] Neither its acceptability, nor that of the Zelen Design, has been evaluated with participants involved in such a study.”
MethodSelf-citationCommunity-based intervention trials
Kirkwood, B. R., & Morrow, R. H. (1989) · Journal of Biosocial Science · doi:10.1017/s0021932000025281 · Cited by 17 · cited 1×
“We suggest a similar procedure for McNulty-Zelen designs, as this may go some way to allay the fears of stakeholders about the possible risk to the GPs relationship with a service commissioner [19].”
MethodRelative or absolute? A significant intervention for chlamydia screening with small absolute benefit
Miller, W. C., & Nguyen, N. (2014) · Sexually Transmitted Infections · doi:10.1136/sextrans-2013-051426 · Cited by 8 · cited 1×
“As far as we know this McNulty-Zelen Design has only been used by our group [6, 9], and a recent editorial critiquing the study questioned “was that deception necessary?” [10] Neither its acceptability, nor that of the Zelen Design, has been evaluated with participants involved in such a study.”
Method
Flagged first, then by citations
Research works printed but not linked: 2
- Hutchison D, Styles B. A Guide to Running Randomised Controlled Trials for Educational Researchers. Slough: National Foundation for Educational Research; 2010.
- DH Research and Development Directorate (England); National Institute for Social Care and Health Research (Wales); Chief Scientist Office (Scotland); R&D Division, Public Health Agency (Northern Ireland). Governance arrangements for research ethics committees: a harmonised edition. May 2011
Cited by
Sorted by citations
Saturation in qualitative research: exploring its conceptualization and operationalization
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0574-8 · Cited by 12,033
“Quite frequently, studies that adopt the criterion of saturation propose at the same time a prior sample size (e.g. McNulty et al. 2015; Long-Sutehall et al. 2011).”
BackgroundJones, L. F., Meyrick, J., Bath, J., Dunham, O., & McNulty, C. A. M. (2018) · Journal of Hospital Infection · doi:10.1016/j.jhin.2018.10.013 · Cited by 29
Self-citationAshiru-Oredope, D. A. I., Doble, A., Thornley, T., Saei, A., Gold, N., Sallis, A., McNulty, C. A. M., Lecky, D. M., Umoh, E., & Klinger, C. (2020) · Pharmacy · doi:10.3390/pharmacy8010044 · Cited by 24
Self-citationEley, C. V., Sharma, A. K., Lee, H., Charlett, A., Owens, R., & McNulty, C. A. M. (2020) · Eurosurveillance · doi:10.2807/1560-7917.es.2020.25.44.1900408 · Cited by 20
“We performed a service evaluation using the McNulty– Zelen clustered randomised controlled trial (RCT) design [19] in practices within a high-prescribing CCG in Northern England.”
Self-citationTheoretical Saturation in Grounded Theory Studies: An Evaluative Tool
Ünlü, Z., & Qureshi, H. A. (2023) · Anadolu Üniversitesi Sosyal Bilimler Dergisi · doi:10.18037/ausbd.1272631 · Cited by 15
Wilson, M. G., Asselbergs, F. W., Miguel, R., Brealey, D. A., & Harris, S. K. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059995 · Cited by 4
Wen, Y.-Q., Liu, K., Zhang, Y., Zhang, X.-H., Liu, T.-X., Kim, T., Shi, Z.-Y., Song, Y.-K., & Liu, Z.-L. (2026) · Evidence-Based Chinese Medicine and Technology Assessment · doi:10.26599/ecmta.2026.9570037
Metadata
- Authors
- on behalf of the CIRT study group, McNulty, C., Ricketts, E. J., Rugman, C., Hogan, A., Charlett, A., & Campbell, R.
- Affiliations
- Public Health England (GB)
- University of Bristol (GB)
- Corresponding author
- on behalf of the CIRT study group
- Venue
- BMC Family Practice
- Published
- 2015-11-17
- Volume
- 16
- Issue
- 1
- Pages
- 169-169
- ISSN
- 1471-2296 (ISSN-L)
- PMC
- PMC4647292
- OpenAlex
- W2204364253
- Cited by
- 7
- References
- 19
- Citations by year
- 2026: 1 · 2023: 1 · 2022: 1 · 2020: 2 · 2018: 1 · 2017: 1
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Family Practice (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Explore Bristol Research (publishedVersion, other-oa)
- Open Research Online (The Open University) (acceptedVersion)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- Funders
- National Institute for Health and Care Research (NF-SI-0514-10114)
- Public Health England
- Medical Research Council (MR/K025643/1, MR/K023233/1)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Keywords
- educational intervention, general practice, research acceptability, informed consent, proxy consent, research ethics, chlamydia screening, continuing professional development, stakeholder perspectives, audit and feedback, research generalisability, primary care trusts
- MeSH terms
- Chlamydia Infections, Humans, Interviews as Topic, Research Design, Randomized Controlled Trials as Topic, Qualitative Research, General Practitioners, General Practice, Reproductive Health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
