Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Objectives: Designing studies with an internal pilot phase may optimise the use of pilot work to inform more efficient randomised controlled trials (RCTs). Careful selection of preagreed decision or ‘progression’ criteria at the juncture between the internal pilot and main trial phases provides a valuable opportunity to evaluate the likely success of the main trial and optimise its design or, if necessary, to make the decision not to proceed with the main trial. Guidance on the appropriate selection and application of progression criteria is, however, lacking. This paper outlines the key issues to consider in the optimal development and review of operational progression criteria for RCTs with an internal pilot phase. Design: A structured literature review and exploration of stakeholders' opinions at a Medical Research Council (MRC) Hubs for Trials Methodology Research workshop. Key stakeholders included triallists, methodologists, statisticians and funders. Results: There is considerable variation in the use of progression criteria for RCTs with an internal pilot phase, although 3 common issues predominate: trial recruitment, protocol adherence and outcome data. Detailed and systematic reporting around the decision-making process for stopping, amending or proceeding to a main trial is uncommon, which may hamper understanding in the research community about the appropriate and optimal use of RCTs with an internal pilot phase. 10 top tips for the development, use and reporting of progression criteria for internal pilot studies are presented. Conclusions: Systematic and transparent reporting of the design, results and evaluation of internal pilot trials in the literature should be encouraged in order to facilitate understanding in the research community and to inform future trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In effectiveness trials, power calculations and sample size decisions are primarily based on the detection of a clinically meaningful difference in measures of the effects of the intervention on the patient or public health outcomes such as behaviour, disease, symptomatology or functional outcomes [24].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Feasibility and acceptability were operationalized a priori according to the following progression criteria thresholds: stop (benchmark: < 70%), adapt (> 70%- < 90%), and go (> 90%).”
From Yoga-Based Group Intervention for Inpatients with Schizophrenia Spectrum Disorders—Feasibility, Acceptability, and Preliminary Outcomes of a Rater-Blinded Randomized Controlled Trial. (2026) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These results give solid ground for the likely success of a following fully powered RCT investigating the intervention’s efficacy32 and align with previous findings on MBIs for SSD.22”
From Yoga-Based Group Intervention for Inpatients with Schizophrenia Spectrum Disorders—Feasibility, Acceptability, and Preliminary Outcomes of a Rater-Blinded Randomized Controlled Trial. (2026) · cites this paper as [32]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Thus, it aligns with the main aims and objectives of pilot and feasibility studies and with the progression stop-amend-go recommendations of Eldridge et al. [2] and Avery et al. [18].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Avery et al. [18] published recommendations for internal pilots emphasising a traffic light (stop-amend-go/red-amber-green) approach to progression with focus on process assessment (recruitment, protocol adherence, follow-up) and transparent reporting around the choice of trial design and the decision-making processes for stopping, amending or proceeding to a main trial.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [18]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
CONSORT 2010 statement: extension to randomised pilot and feasibility trials.
Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · BMJ · doi:10.1136/bmj.i5239 · Cited by 3,404
A tutorial on pilot studies: the what, why and how
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-1 · Cited by 3,384
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. J. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,930
CONSORT 2010 statement: extension to randomised pilot and feasibility trials
on behalf of the PAFS consensus group, Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0105-8 · Cited by 2,843
Design and analysis of pilot studies: recommendations for good practice
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j..2002.384.doc.x · Cited by 2,572 · cited 1×
“In addition, the assumption that pilot studies are less rigorous than main studies and are not designed to test the main trial hypothesis means that their findings are underdisseminated, thereby preventing triallists from accessing information that may be valuable to the success of future main trials.(3) 10 The system…”
BackgroundSelf-citationFundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
No surgical innovation without evaluation: the IDEAL recommendations
McCulloch, P., Altman, D. G., Campbell, W. B., Flum, D. R., Glasziou, P., Marshall, J. C., & Nicholl, J. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61116-8 · Cited by 1,820
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2013) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.09.010 · Cited by 1,658
Eldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L., & Bond, C. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0150205 · Cited by 1,581
Cited by
Global Vascular Guidelines on the management of chronic limb-threatening ischemia.
Conte, M. S., Bradbury, A. W., Kolh, P., White, J. V., Dick, F., Fitridge, R., Mills, J. L., Ricco, J.-B., Suresh, K. R., Murad, M. H., et al. (2019) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2019.02.016 · Cited by 2,015
Global vascular guidelines on the management of chronic limb-threatening ischemia
Conte, M. S., Bradbury, A. W., Kolh, P., White, J. V., Dick, F., Fitridge, R., Mills, J. L., Ricco, J.-B., Suresh, K. R., Murad, M. H., et al. (2019) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2019.05.006 · Cited by 1,582
Guidance for conducting feasibility and pilot studies for implementation trials
Pearson, N., Naylor, P.-J., Ashe, M. C., Fernandez, M., Yoong, S. L., & Wolfenden, L. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00634-w · Cited by 475
Moore, G. F., Evans, R. E., Hawkins, J., Littlecott, H., Melendez-Torres, G. J., Bonell, C., & Murphy, S. (2019) · Evaluation · doi:10.1177/1356389018803219 · Cited by 471
“Movement toward pre-specifying objective progression criteria (Avery et al., 2017) perhaps reinforces this.”
BackgroundHaywood, K., Whitehead, L., Nadkarni, V. M., Achana, F., Beesems, S., Böttiger, B. W., Brooks, A., Castrén, M., Ong, M. E., Hazinski, M. F., et al. (2018) · Circulation · doi:10.1161/cir.0000000000000562 · Cited by 362
Metadata
- Authors
- Avery, K. N. L., Williamson, P. R., Gamble, C., O'Connell Francischetto, E., Metcalfe, C., Davidson, P., Williams, H., & Blazeby, J. M.
- Affiliations
- University of Bristol (GB)
- University of Liverpool (GB)
- Medical Research Council ConDuCT-II Hub for Trials Methodology Research, School of Social and Community Medicine, University of Bristol, Bristol, UK
- National Institute for Health and Care Research (GB)
- Nottingham University Hospitals NHS Trust (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Corresponding author
- Avery, Kerry N L
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2017-02
- Volume
- 7
- Issue
- 2
- Pages
- e013537
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC5318608
- OpenAlex
- W2588182016
- Semantic Scholar
- c03117980919b954181d0ea10371c6c6561bde41
- Cited by
- 575
- References
- 48
- Citations by year
- 2026: 83 · 2025: 100 · 2024: 90 · 2023: 83 · 2022: 69 · 2021: 67 · 2020: 49 · 2019: 20 · 2018: 13 · 2017: 1
- FWCI
- 15.85
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMJ Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Portal (Queen's University Belfast) (publishedVersion, cc-by)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- ResearchOnline (Glasgow Caledonian University) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Stirling Online Research Repository (University of Stirling) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Ulster University Research Portal (Ulster University) (submittedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion, other-oa)
- Explore Bristol Research (submittedVersion, cc-by)
- Funders
- National Institute for Health and Care Research (DRF-2011-04-016, NF-SI-0611-10020, DRF-2012-05-182)
- Medical Research Council (MR/K025635/1, MR/L004933/1, MR/K025643/1, MR/L004933/2)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- internal pilot study, progression criteria, randomized controlled trials, trial design, trial recruitment, protocol adherence, outcome data, trial reporting, stakeholder perspectives, trial efficiency
- MeSH terms
- Attitude, Clinical Protocols, Decision Making, Humans, Pilot Projects, Research Design, Research Personnel, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Patient Selection, Guideline Adherence, Clinical Trials Data Monitoring Committees, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies. https://app.cochat.ai/papers/10.1136/bmjopen-2016-013537
- The paper
- Avery, K. N. L., Williamson, P. R., Gamble, C., O'Connell Francischetto, E., Metcalfe, C., Davidson, P., Williams, H., & Blazeby, J. M. (2017). Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies. BMJ Open. https://doi.org/10.1136/bmjopen-2016-013537
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.