CoChat
Sign Up
Cited by 426Statistical Methods in Clinical TrialsStatistical Methods and Bayesian InferenceStatistical Methods in Epidemiology

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

The statistical interpretation of pilot trials: should significance thresholds be reconsidered?

Open access
Lee, Ellen C, Whitehead, Amy L, Jacques, Richard M, Julious, Steven A·2014·BMC Medical Research Methodology
DOI 10.1186/1471-2288-14-41PMC PMC3994566
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: In an evaluation of a new health technology, a pilot trial may be undertaken prior to a trial that makes a definitive assessment of benefit. The objective of pilot studies is to provide sufficient evidence that a larger definitive trial can be undertaken and, at times, to provide a preliminary assessment of benefit. METHODS: We describe significance thresholds, confidence intervals and surrogate markers in the context of pilot studies and how Bayesian methods can be used in pilot trials. We use a worked example to illustrate the issues raised. RESULTS: We show how significance levels other than the traditional 5% should be considered to provide preliminary evidence for efficacy and how estimation and confidence intervals should be the focus to provide an estimated range of possible treatment effects. We also illustrate how Bayesian methods could also assist in the early assessment of a health technology. CONCLUSIONS: We recommend that in pilot trials the focus should be on descriptive statistics and estimation, using confidence intervals, rather than formal hypothesis testing and that confidence intervals other than 95% confidence intervals, such as 85% or 75%, be used for the estimation. The confidence interval should then be interpreted with regards to the minimum clinically important difference. We also recommend that Bayesian methods be used to assist in the interpretation of pilot trials. Surrogate endpoints can also be used in pilot trials but they must reliably predict the overall effect on the clinical outcome.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1471-2288-14-41

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Mean difference in SF-36 GH dimension scores between treatment and control with confidence intervals (based on n = 31 patients).
Figure 1 Mean difference in SF-36 GH dimension scores between treatment and control with confidence intervals (based on n = 31 patients).CC BY · PubMed Central
Figure 2 Prior, observed and posterior distributions for non-informative, pessimistic and optimistic priors.
Figure 2 Prior, observed and posterior distributions for non-informative, pessimistic and optimistic priors.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 1

  1. International Conference on Harmonisation: ICH E9 statistical principals for clinical trials. 1998, http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf,

Cited by

Sorted by citations

See all 426

Related

See all

Metadata

Authors
Lee, E. C., Whitehead, A. L., Jacques, R. M., & Julious, S. A.
Affiliations
University of Sheffield (GB)
Corresponding author
Julious, Steven A
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2014-03-20
Volume
14
Issue
1
Pages
41-41
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/1471-2288-14-41
PMC
PMC3994566
OpenAlex
W2108052506
Cited by
423
References
31
Citations by year
2026: 26 · 2025: 60 · 2024: 50 · 2023: 59 · 2022: 52 · 2021: 40 · 2020: 34 · 2019: 30 · 2018: 31 · 2017: 16 · 2016: 14 · 2015: 9 · 2014: 2
FWCI
14.45
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
  • ePrints Soton (University of Southampton) (acceptedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
pilot trial, significance threshold, confidence intervals, Bayesian analysis, surrogate endpoints, minimal clinically important difference, health technology assessment, treatment effect estimation, hypothesis testing
MeSH terms
Bayes Theorem, Data Interpretation, Statistical, Humans, Leg Ulcer, Pilot Projects, Research Design, Biomarkers, Confidence Intervals, Treatment Outcome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review