The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 10 of the 14 sources its claims cite were checked for support.
Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting
Open accessAbstract
Background: External randomised pilot trials aim to assess whether a future definitive Randomised Controlled Trial (RCT) is feasible. Prespecified progression criteria help guide the interpretation of pilot trial findings to decide whether, and how, a definitive RCT should be conducted. This commentary presents a set of proposed recommendations for progression criteria to guide researchers when (i) designing, (ii) conducting, (iii) analysing and (iv) reporting external randomised pilot trials. Methods: Recommendations were developed following a mixed methods approach. This involved (i) a methodological review of pilot trial publications, (ii) a cross-sectional study of pilot trial research funding applications, (iii) qualitative interviews with pilot trial researchers and (iv) a survey of corresponding authors of identified pilot trial publications. Initial recommendations were refined following two consultation stakeholder workshops held in July 2022. Recommendations for progression criteria for external randomised pilot trials: i. Design : consider progression criteria from the earliest opportunity; map progression criteria to feasibility objectives; consider quantitative and qualitative interpretations of feasibility; provide justification; develop guidelines rather than rules; seek input from relevant stakeholders. ii. Conduct : regularly monitor pilot trial data against progression criteria. iii. Analysis : avoid considering each progression criterion in isolation; engage in discussion with relevant stakeholders; consider context and other factors external to the pilot trial; consider feasibility (can we?) and progression (will we?). iv. Reporting : we propose a reporting checklist in relation to progression criteria and recommend reporting in a table format for clarity. Conclusion: These recommendations provide a helpful resource for researchers to consider progression criteria at different stages of external randomised pilot trials. We have produced a simple infographic tool to summarise these recommendations for researchers to refer to. Further research is needed to evaluate whether these proposed recommendations should inform future development, or update, of established guidelines for the design, conduct, analysis and reporting of external randomised pilot trials.
Overview
CoChat Summary
TL;DR
This commentary proposes recommendations for progression criteria across the design, conduct, analysis, and reporting of external randomised pilot trials, developed from four mixed-methods studies and stakeholder consultation. It offers practical guidance and a reporting checklist/table format, while calling for further evaluation before updating established guidelines.
Key findings
- Recommendations span four stages: design (consider early, map to feasibility objectives, use quantitative and qualitative interpretations, justify criteria, treat as guidelines rather than rules, seek stakeholder input); conduct (regularly monitor data against criteria); analysis (avoid considering criteria in isolation, engage stakeholders, consider context, distinguish feasibility from progression); and reporting (use a proposed checklist and table format).
- Development drew on four studies over three years (2019–2022): a methodological review of pilot trial publications, a cross-sectional study of NIHR Research for Patient Benefit funding applications, qualitative interviews, and a survey of corresponding authors.
- Initial recommendations were refined after two external stakeholder workshops in July 2022; stakeholders included three trial investigators, four trial statisticians, six trial methodologists/health services researchers, one research funder manager, and one professor of primary care pharmacy.
- The paper proposes a reporting checklist (Table 1) and recommends reporting progression criteria in a table format for clarity; an infographic tool was also produced.
Methods
- Mixed methods approach with four distinct studies: (i) methodological review of external randomised pilot trial protocol and result publications (January 2018–December 2019); (ii) cross-sectional study of progression criteria in NIHR RfPB funding applications (funding decision July 2017–July 2019); (iii) qualitative interviews with pilot trial researchers/stakeholders (December 2020–July 2021); and (iv) survey of corresponding authors from study one (January–February 2022).
- Findings were iteratively triangulated; the lead author drafted initial recommendations, which were refined by the wider study team and following feedback from two external stakeholder workshops in July 2022.
- Stakeholder consultation included three trial investigators, four trial statisticians, six trial methodologists/health services researchers, one research funder manager, and one professor of primary care pharmacy.
Limitations
- Stated: Further research is needed to evaluate whether the proposed recommendations should inform future development or update of established guidelines for the design, conduct, analysis, and reporting of external randomised pilot trials.
- Stated: The recommendations were developed with external randomised pilot trials in mind, though they might also apply to non-randomised or single-arm pilot or feasibility studies.
- Evident: The recommendations are proposed from triangulation of four studies and stakeholder workshops rather than a formal consensus or empirical evaluation of their implementation; no data are reported on whether adopting them improves feasibility assessment or reduces research waste.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
7 Valid · 0 Not valid · 3 Not checked · 5 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“If progression criteria are not set a priori, there is a risk that pilot trials may be optimistic in reporting that a future RCT is feasible [11] and progress to a definitive RCT without modification or acknowledgement of potential limitations [12].”
§ Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting > Background
What the sources say
- Supportedcites [12] Mbuagbaw L, Kosa SD, Lawson DO, et al. The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study. Pilot feasibility Stud. 2019;5:120. https://doi.org/10.1186/s40814-019-0500-z. source
The Discussion explicitly warns that, without a priori progression criteria, studies that did not do well in the pilot stage may move to a larger trial without modification or due acknowledgement of potential limitations. This supports the portion of the citing sentence attributed to this source [12]. (LLM, conf=0.95)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“If progression criteria are not set a priori, there is a risk that pilot trials may be optimistic in reporting that a future RCT is feasible [11] and progress to a definitive RCT without modification or acknowledgement of potential limitations [12].”
§ Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting > Background
What the sources say
- Supportedcites [11] Morgan B, Hejdenberg J, Kuleszewicz K, et al. Are some feasibility studies more feasible than others? A review of the outcomes of feasibility studies on the ISRCTN registry. Pilot Feasibility Stud. 2021;71:1–8. https://doi.org/10.1186/S40814-021-00931-Y. source
The Discussion (page not provided) says many reviewed studies lacked progression criteria or used unclear criteria and that some may therefore have been optimistic in reporting a future RCT feasible. This supports the claim’s qualified description of a risk, rather than asserting that optimism occurred in every such trial. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Note that unlike an internal pilot trial, any outcome data collected during an external pilot trial does not contribute to the definitive trial analysis [8].”
§ Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting > Recommendations for progression criteria > Recommendations for progression criteria during pilot trial conduct
What the sources say
- Partly supportedcites [8] Eldridge SM, Lancaster GA, Campbell MJ, et al. Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework. PLoS One. 2016;11:e0150205. https://doi.org/10.1371/journal.pone.0150205. sourceExperimental — accuracy not yet measured
The Introduction distinguishes stand-alone pilot studies completed before a definitive RCT from internal pilots designed as its early stage. It does not state whether outcome data from either type contribute to the definitive trial analysis, so the claim about data inclusion is not established by this paper. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
In the paper
“Investigators should be aware that small pilot trial sample sizes might mean that any estimates of rates are subject to considerable uncertainty [25].”
§ Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting > Recommendations for progression criteria > Recommendations for progression criteria during pilot trial design
What the sources say
- Not checked yetcites [25] Netz Y, Lidor R, Ziv G. Small samples and increased variability-discussing the need for restricted types of randomization in exercise interventions in old age. Eur Rev Aging Phys Act. 2019;16:1–8. https://doi.org/10.1186/S11556-019-0224-3/TABLES/4. doi:10.1186/s11556-019-0224-3/tables/4
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
1 reference still to check
A free account verifies the references that are still open.
- Probably not a substantial claim. 11
- The quote only partly states this claim. 2
- It may state two separate claims as one. 1
Citations
Cites
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“This format of evidence synthesis and reporting strikes a balance between simplicity of information presentation and completeness (or transparency) [31], and some pilot trial publications have effectively reported pilot trial findings in a similar format [32–35].”
ResultNorderyd, O. M., Koch, G., Papias, A., Köhler, A. A., Helkimo, A. N., Brahm, C., Lindmark, U., Lindfors, N., Mattsson, A., Rolander, B., et al. (2015) · PubMed · Cited by 3,584
CONSORT 2010 statement: extension to randomised pilot and feasibility trials.
Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · BMJ · doi:10.1136/bmj.i5239 · Cited by 3,404
Self-citationCONSORT 2010 statement: extension to randomised pilot and feasibility trials
on behalf of the PAFS consensus group, Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0105-8 · Cited by 2,843 · cited 5×
“ADePT A process for Decision-making after Pilot and feasibility Trials CI Chief Investigator CONSORT Consolidated Standards of Reporting Trials CReDECI Criteria for Reporting the Development and Evaluation of Complex Interventions in healthcare CTU Clinical Trials Unit PAFS Pilot and Feasibility Studies PPI Patient an…”
BackgroundInfluentialSelf-citationEldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L., & Bond, C. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0150205 · Cited by 1,581 · cited 4×
“Note that unlike an internal pilot trial, any outcome data collected during an external pilot trial does not contribute to the definitive trial analysis [8].”
BackgroundResultInfluentialSelf-citationPublication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818
Avery, K. N. L., Williamson, P. R., Gamble, C., O'Connell Francischetto, E., Metcalfe, C., Davidson, P., Williams, H., & Blazeby, J. M. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013537 · Cited by 575 · cited 3×
“a ‘rescue plan’ that outlines any problems encountered and how they were addressed [26].”
BackgroundMethodInfluentialO’Cathain, A., Hoddinott, P., Lewin, S., Thomas, K. J., Young, B., Adamson, J., Jansen, Y. J., Mills, N., Moore, G., & Donovan, J. L. (2015) · Pilot and Feasibility Studies · doi:10.1186/s40814-015-0026-y · Cited by 458
Understanding randomised controlled trials
Akobeng, A. K. (2005) · Archives of Disease in Childhood · doi:10.1136/adc.2004.058222 · Cited by 458 · cited 1×
“Background RCTs are integral to the practice of evidence-based medicine [1], however they are often expensive, complicated, and take a long time to set up, conduct, analyse and disseminate findings [2–5].”
Background
Cited by
Consolidated guidance for behavioral intervention pilot and feasibility studies
Pfledderer, C. D., von Klinggraeff, L., Burkart, S., Bandeira, A. D. S., Lubans, D., Jago, R., Okely, A., van Sluijs, E., Ioannidis, J. P., Thrasher, J., et al. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01485-5 · Cited by 56
“We also provide a crosswalk of 161 guidelines, checklists, frameworks, and recommendations, mapped on to the subthemes identified and an additional publication that was published after the Delphi process, but was relevant to include in the list [15].”
BackgroundMethodDetermining sample size for pilot trials: a tutorial
Ying, X.-J., Freedland, K., Powell, L. H., Stuart, E. A., Ehrhardt, S., & Mayo-Wilson, E. (2025) · British medical journal · doi:10.1136/bmj-2024-083405 · Cited by 62
Thurston, C., Humphries, S., Bezuidenhout, L., Johansson, S., Holmlund, L., von Koch, L., English, C., & Conradsson, D. M. (2025) · International Journal of Stroke · doi:10.1177/17474930251315628 · Cited by 16
“These were developed based on recommendations and similar studies 16–18 and modified following feedback from the wider research group.”
Suglo, J. N., Winkley, K., & Sturt, J. A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0302385 · Cited by 21
Schootemeijer, S., de Vries, N. M., Darweesh, S. K. L., Ascherio, A., Schwarzschild, M. A., Macklin, E. A., & Bloem, B. R. (2025) · Neurology Report · doi:10.1097/npt.0000000000000507 · Cited by 14
Metadata
- Authors
- Mellor, K., Albury, C., Dutton, S. J., Eldridge, S., & Hopewell, S.
- Affiliations
- University of Oxford (GB)
- University of London (GB)
- Corresponding author
- Mellor, Katie
- Venue
- Pilot and Feasibility Studies
- Published
- 2023-04-15
- Volume
- 9
- Issue
- 1
- Pages
- 59-59
- ISSN
- 2055-5784 (ISSN-L)
- PMC
- PMC10105402
- OpenAlex
- W4365814045
- Cited by
- 145
- References
- 37
- Citations by year
- 2026: 70 · 2025: 54 · 2024: 16 · 2023: 5
- FWCI
- 29.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Pilot and Feasibility Studies (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Medical Research Foundation (MR/N013468/1)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- progression criteria, feasibility assessment, trial design, trial conduct, trial analysis, trial reporting, reporting checklist, stakeholder engagement, grant applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting. https://app.cochat.ai/papers/10.1186/s40814-023-01291-5
- The paper
- Mellor, K., Albury, C., Dutton, S. J., Eldridge, S., & Hopewell, S. (2023). Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting. Pilot and Feasibility Studies. https://doi.org/10.1186/s40814-023-01291-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.