Progression criteria in trials with an internal pilot: an audit of publicly funded randomised controlled trials
Open accessAbstract
BACKGROUND: With millions of pounds spent annually on medical research in the UK, it is important that studies are spending funds wisely. Internal pilots offer the chance to stop a trial early if it becomes apparent that the study will not be able to recruit enough patients to show whether an intervention is clinically effective. This study aims to assess the use of internal pilots in individually randomised controlled trials funded by the Health Technology Assessment (HTA) programme and to summarise the progression criteria chosen in these trials. METHODS: Studies were identified from reports of the HTA committees' funding decisions from 2012 to 2016. In total, 242 trials were identified of which 134 were eligible to be included in the audit. Protocols for the eligible studies were located on the NIHR Journals website, and if protocols were not available online then study managers were contacted to provide information. RESULTS: Over two-thirds (72.4%) of studies said in their protocol that they would include an internal pilot phase for their study and 37.8% of studies without an internal pilot had done an external pilot study to assess the feasibility of the full study. A typical study with an internal pilot has a target sample size of 510 over 24 months and aims to recruit one-fifth of their total target sample size within the first one-third of their recruitment time. There has been an increase in studies adopting a three-tiered structure for their progression rules in recent years, with 61.5% (16/26) of studies using the system in 2016 compared to just 11.8% (2/17) in 2015. There was also a rise in the number of studies giving a target recruitment rate in their progression criteria: 42.3% (11/26) in 2016 compared to 35.3% (6/17) in 2015. CONCLUSIONS: Progression criteria for an internal pilot are usually well specified but targets vary widely. For the actual criteria, red/amber/green systems have increased in popularity in recent years. Trials should justify the targets they have set, especially where targets are low.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The review of Herbert et al. reported that the use of progression criteria (including recruitment rate) and traffic light stop-amend-go as opposed to simple stop-go is increasing for internal pilot studies.
“The review of Herbert et al. [17] reported that the use of progression criteria (including recruitment rate) and traffic light stop-amend-go as opposed to simple stop-go is increasing for internal pilot studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [17]
The Results report increased use of red/amber/green progression rules and recruitment-rate targets from 2015 to 2016; the Discussion describes the traffic-light approach as increasingly popular. The cited passages do not establish that use of progression criteria overall increased, as distinct from changes in the types of criteria used. Page numbers are not available in our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measuredHerbert et al. reported that targets within internal pilots are often slightly larger and different from external pilots, being based on percentages of total sample size and timeline rather than fixed sample requirement.
“Herbert et al. [17] reported that targets within internal as opposed to external pilots are often slightly larger and somewhat different, being based on percentages of the total sample size and timeline rather than any fixed sample requirement.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [17]
The Results report internal-pilot recruitment targets as proportions of the full sample size and show that those proportions vary with the pilot’s share of the recruitment timeline (Fig. 7; page not available). They do not establish that internal-pilot targets are larger than external-pilot targets. The Results also say a fixed number of patients was the most common recruitment-target format, contrary to the claim’s ‘rather than any fixed sample requirement’ wording. (LLM, conf=0.88)
Experimental — accuracy not yet measuredCurrent recommendations for sample sizes of pilot/feasibility studies vary, have a single rather than a multi-criterion basis, and do not necessarily link directly to formal progression criteria.
“Current recommendations for sample sizes of pilot/feasibility studies vary, have a single rather than a multi-criterion basis, and do not necessarily link directly to formal progression criteria.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [17]
The Conclusions report that progression targets in audited internal-pilot trials vary widely, but do not assess whether published sample-size recommendations vary. The Methods > Analyses and Discussion passages address how recruitment targets can inform progression decisions; they do not establish that sample-size recommendations have a single-criterion basis or lack a direct link to formal progression criteria. Searched for sample-size recommendations and their relationship to progression criteria. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 10 · Cited by 81
- Authors
- Herbert, E., Julious, S. A., & Goodacre, S.
- Affiliations
- University of Sheffield (GB)
- Corresponding author
- Herbert, Esther
- Venue
- Trials
- Published
- 2019-08-09
- Volume
- 20
- Issue
- 1
- Pages
- 493-493
- ISSN
- 1745-6215 (ISSN-L)
- PMC
- PMC6688224
- OpenAlex
- W2967253043
- Cited by
- 81
- References
- 10
- Citations by year
- 2026: 14 · 2025: 8 · 2024: 21 · 2023: 13 · 2022: 6 · 2021: 15 · 2020: 3 · 2017: 1
- FWCI
- 4.68
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Trials (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute for Health Research (NIHR–RMFI–2016–07–25)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- early stopping, sample size, feasibility assessment, health technology assessment, research funding efficiency
- MeSH terms
- Humans, Medical Audit, Pilot Projects, Technology Assessment, Biomedical, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
