Pilot trials in physical activity journals: a review of reporting and editorial policy
Open accessAbstract
BACKGROUND: Since the early 2000s, a number of publications in the medical literature have highlighted inadequacies in the design, conduct and reporting of pilot trials. This work led to two notable publications in 2016: a conceptual framework for defining feasibility studies and an extension to the CONSORT 2010 statement to include pilot trials. It was hoped that these publications would educate researchers, leading to better use of pilot trials and thus more rigorously planned and informed randomised controlled trials. The aim of the present work is to evaluate the impact of these publications in the field of physical activity by reviewing the literature pre- and post-2016. This first article presents the pre-2016 review of the reporting and the current editorial policy applied to pilot trials published in physical activity journals. METHODS: Fourteen physical activity journals were screened for pilot and feasibility studies published between 2012 and 2015. The CONSORT 2010 extension to pilot and feasibility studies was used as a framework to assess the reporting quality of the studies. Editors of the eligible physical activity journals were canvassed regarding their editorial policy for pilot and feasibility studies. RESULTS: Thirty-one articles across five journals met the eligibility criteria. These articles fell into three distinct categories: trials that were carried out in preparation for a future definitive trial (23%), trials that evaluated the feasibility of a novel intervention but did not explicitly address a future definitive trial (23%) and trials that did not have any clear objectives to address feasibility (55%). Editors from all five journals stated that they generally do not accept pilot trials, and none gave reference to the CONSORT 2010 extension as a guideline for submissions. CONCLUSION: The result that over half of the studies did not have feasibility objectives is in line with previous research findings, demonstrating that these findings are not being disseminated effectively to researchers in the field of physical activity. The low standard of reporting across most reviewed articles and the neglect of the extended CONSORT 2010 statement by the journal editors highlight the need to actively disseminate these guidelines to ensure their impact.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Horne et al. reviewed 31 pilot trials published in physical therapy journals between 2012 and 2015 and found that only 4/31 (13%) carried out a valid sample size calculation on effectiveness/efficacy outcomes but 26/31 (84%) used hypothesis testing.
“Horne et al. [21] reviewed 31 pilot trials published in physical therapy journals between 2012 and 2015 and found that only 4/31 (13%) carried out a valid sample size calculation on effectiveness/efficacy outcomes but 26/31 (84%) used hypothesis testing.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [21]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredA common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [21]
Hypothesis testing in pilot studies, particularly for effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations.
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329 · see citing works
CONSORT 2010 statement: extension to randomised pilot and feasibility trials.
Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · BMJ · doi:10.1136/bmj.i5239 · Cited by 3,404 · see citing works
Self-citationA tutorial on pilot studies: the what, why and how
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-1 · Cited by 3,384 · see citing works
CONSORT 2010 statement: extension to randomised pilot and feasibility trials
on behalf of the PAFS consensus group, Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0105-8 · Cited by 2,843 · see citing works
Self-citationDesign and analysis of pilot studies: recommendations for good practice
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j..2002.384.doc.x · Cited by 2,572 · see citing works
Self-citationMeta-analyses involving cross-over trials: methodological issues
Elbourne, D. R., Altman, D. G., Higgins, J. P., Curtin, F., Worthington, H. V., & Vail, A. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.140 · Cited by 2,305 · see citing works
Eldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L., & Bond, C. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0150205 · Cited by 1,581 · see citing works
Self-citationWhat is a pilot or feasibility study? A review of current practice and editorial policy
Arain, M., Campbell, M. J., Cooper, C. L., & Lancaster, G. A. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-67 · Cited by 1,483 · see citing works
Self-citationKyu, H. H., Bachman, V. F., Alexander, L. T., Mumford, J. E., Afshin, A., Estep, K., Veerman, J. L., Delwiche, K., Iannarone, M. L., Moyer, M. L., et al. (2016) · BMJ · doi:10.1136/bmj.i3857 · Cited by 1,272 · see citing works
Wahid, A., Manek, N., Nichols, M., Kelly, P., Foster, C., Webster, P., Kaur, A., Friedemann Smith, C., Wilkins, E., Rayner, M., et al. (2016) · Journal of the American Heart Association · doi:10.1161/jaha.115.002495 · Cited by 715 · see citing works
The Continuing Unethical Conduct of Underpowered Clinical Trials
Halpern, S. D. (2002) · JAMA · doi:10.1001/jama.288.3.358 · Cited by 548 · see citing works
Greaves, C. J., Gillison, F., Stathi, A., Bennett, P., Reddy, P., Dunbar, J., Perry, R. A., Messom, D., Chandler, R., Francis, M., et al. (2015) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0159-z · Cited by 299 · see citing works
Pilot and feasibility studies come of age!
Lancaster, G. A. (2015) · Pilot and Feasibility Studies · doi:10.1186/2055-5784-1-1 · Cited by 260 · see citing works
Self-citationQuestions asked and answered in pilot and feasibility randomized controlled trials
Shanyinde, M., Pickering, R. M., & Weatherall, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-117 · Cited by 229 · see citing works
Effects of Resistance Training on Adiposity and Metabolism after Spinal Cord Injury
Gorgey, A. S., Mather, K. J., Cupp, H., & Gater, D. R. (2011) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e31822672aa · Cited by 164 · see citing works
Egger, M., Ebrahim, S., & Smith, G. D. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.1 · Cited by 160 · see citing works
Chau, J. Y., Daley, M., Dunn, S., Srinivasan, A., Do, A., Bauman, A. E., & van der Ploeg, H. P. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0127-7 · Cited by 157 · see citing works
Riley, N., Lubans, D. R., Morgan, P. J., & Young, M. D. (2014) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2014.09.005 · Cited by 91 · see citing works
Biobehavioral Factors Mediate Exercise Effects on Fatigue in Breast Cancer Survivors
Rogers, L. Q., Vicari, S. K., Trammell, R. A., Hopkins-Price, P. L., Fogleman, A. J., Spenner, A., Rao, K., Courneya, K. S., Hoelzer, K. L., ROBBS, R. S., et al. (2013) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000000210 · Cited by 87 · see citing works
Yardley, L., Ware, L. J., Smith, E. R., Williams, S., Bradbury, K. J., Arden-Close, E. J., Mullee, M. A., Moore, M. V., Peacock, J. L., Lean, M. E., et al. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-11-67 · Cited by 84 · see citing works
Gray, C. M., Hunt, K., Mutrie, N., Anderson, A. S., Treweek, S. P., & Wyke, S. (2013) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-10-121 · Cited by 84 · see citing works
Currie, K. D., Bailey, K. J., Jung, M. E., McKelvie, R. S., & MacDonald, M. J. (2014) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2014.09.013 · Cited by 82 · see citing works
A Pilot Walking Program Promotes Moderate-Intensity Physical Activity during Pregnancy
Kong, K. L., Campbell, C. G., Foster, R. C., Peterson, A., & Lanningham‐Foster, L. (2013) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000000141 · Cited by 78 · see citing works
Exercise Training Improves HR Responses and V˙O2peak in Predialysis Kidney Patients
HEADLEY, S., GERMAIN, M., MILCH, C., PESCATELLO, L., COUGHLIN, M. A., NINDL, B. C., CORNELIUS, A., SULLIVAN, S., GREGORY, S., & WOOD, R. (2012) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e318268c70c · Cited by 73 · see citing works
Baruth, M., Wilcox, S., Ananian, C. D., & Heiney, S. P. (2015) · Journal of Physical Activity and Health · doi:10.1123/jpah.2012-0339 · Cited by 57 · see citing works
Shuttle-run sprint training in hypoxia for youth elite soccer players: a pilot study.
Gatterer, H., Philippe, M., Menz, V., et al. (2014) · PubMed · Cited by 56 · see citing works
Bristol Girls Dance Project Feasibility Trial: outcome and process evaluation results
Jago, R., Sebire, S. J., Cooper, A. R., Haase, A. M., Powell, J. E., Davis, L., McNeill, J., & Montgomery, A. A. (2012) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-9-83 · Cited by 54 · see citing works
Self-citationJago, R., Sebire, S. J., Turner, K. M., Bentley, G. F., Goodred, J. K., Fox, K. R., Stewart-Brown, S. L., & Lucas, P. J. (2013) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-10-31 · Cited by 46 · see citing works
Anderson, A. S., Macleod, M., Mutrie, N., Sugden, J., Dobson, H., Treweek, S. P., O’Carroll, R. E., Thompson, A. M., Kirk, A. F., Brennan, G., et al. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0156-2 · Cited by 45 · see citing works
Jago, R., Sebire, S. J., Davies, B. R., Wood, L., Edwards, M. J., Banfield, K., Fox, K. R., Thompson, J. L., Powell, J. E., & Montgomery, A. A. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0114-z · Cited by 44 · see citing works
Suh, Y., Motl, R. W., Olsen, C. D., & Joshi, I. (2014) · Journal of Physical Activity and Health · doi:10.1123/jpah.2014-0018 · Cited by 42 · see citing works
Yates, T. E., Griffin, S. J., Bodicoat, D. H., Brierly, G., Dallosso, H. M., Davies, M. J., Eborall, H., Edwardson, C. L., Gillett, M., Gray, L. J., et al. (2015) · Trials · doi:10.1186/s13063-015-0813-z · Cited by 40 · see citing works
Sedgwick, P. M. (2014) · BMJ · doi:10.1136/bmj.g3191 · Cited by 39 · see citing works
Steeves, J. A., Bassett, D. R., Fitzhugh, E. C., Raynor, H. A., & Thompson, D. L. (2012) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-9-95 · Cited by 38 · see citing works
Feasibility and Preliminary Efficacy of the MADE4Life Program: A Pilot Randomized Controlled Trial
Barnes, A. T., Plotnikoff, R. C., Collins, C. E., et al. (2015) · Journal of Physical Activity and Health · doi:10.1123/jpah.2014-0331 · Cited by 31 · see citing works
Martens, M. P., Buscemi, J., Smith, A. E., & Murphy, J. G. (2012) · Journal of Physical Activity and Health · doi:10.1123/jpah.9.4.525 · Cited by 30 · see citing works
Mark, R. S., & Rhodes, R. E. (2013) · Journal of Physical Activity and Health · doi:10.1123/jpah.10.2.211 · Cited by 25 · see citing works
Hutchison, L., Scharfbillig, R., Uden, H., et al. (2014) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2014.11.007 · Cited by 24 · see citing works
Martin, E., Battaglini, C. L., Groff, D., & Naumann, F. (2012) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2012.08.012 · Cited by 23 · see citing works
Kass, L. S., Skinner, P., & Poeira, F. (2013) · Europe PMC (PubMed Central) · Cited by 22 · see citing works
Thøgersen‐Ntoumani, C., Loughren, E. A., Duda, J. L., et al. (2014) · Journal of Physical Activity and Health · doi:10.1123/jpah.2012-0243 · Cited by 19 · see citing works
Voglar, M., & Sarabon, N. (2014) · PubMed · Cited by 19 · see citing works
Gennuso, K. P., Zalewski, K. R., Cashin, S. E., & Strath, S. J. (2013) · Journal of Physical Activity and Health · doi:10.1123/jpah.10.6.769 · Cited by 16 · see citing works
Prescribing and Regulating Exercise with RPE after Heart Transplant
Ciolac, E. G., Castro, R. E., Greve, J. M. D., et al. (2014) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000000553 · Cited by 13 · see citing works
The effects of vibration during maximal graded cycling exercise: a pilot study.
Filingeri, D., Jemni, M., Bianco, A., et al. (2012) · PubMed · Cited by 13 · see citing works
Volaklis, K. A., Smilios, I., Spassis, A. T., Zois, C. E., Douda, H. T., Halle, M., & Tokmakidis, S. P. (2015) · PubMed Central · Cited by 11 · see citing works
Reproducibility of lactate markers during 4 and 8min stage incremental running: A pilot study
Gavin, J. P., Willems, M. E. T., & Myers, S. (2013) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2013.08.006 · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [10]Guy P Joosen AMCP Arnold KD Goris AHC Westerterp KR Measuring free-living energy expenditure and physical activity with triaxial accelerometry Obes Res 2012 13 8 16129718
- [31]Pascale M Murray N Bachmann M Barton G Clark A Howe A Study Protocol: the Norfolk Diabetes Prevention Study [NDPS]: a 46 month multi-centre, randomised, controlled parallel group trial of a lifestyle intervention [with or without additional support from lay lifestyle mentors with type 2 diabetes] to preve BMC Public Health 2017 17 1 10.1186/s12889-016-3929-5 28056894 PMC5217324 · DOI
- [35]Thabane L Hopewell S Lancaster GA Bond CM Coleman CL Campbell MJ Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials Pilot Feasibility Stud 2016 2 1 10.1186/s40814-015-0043-x 27965844 PMC5153862 · DOI
Cited by
Sorted by citations
Lewis, M., Bromley, K., Sutton, C. J., McCray, G., Myers, H. L., & Lancaster, G. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00770-x · Cited by 330 · see citing works
Self-citation
- Authors
- Horne, E., Lancaster, G. A., Matson, R., Cooper, A., Ness, A., & Leary, S.
- Affiliations
- University Hospitals Bristol NHS Foundation Trust (GB)
- Keele University (GB)
- Corresponding author
- Horne, Elsie
- Venue
- Pilot and Feasibility Studies
- Published
- 2018-07-17
- Volume
- 4
- Issue
- 1
- Pages
- 125-125
- ISSN
- 2055-5784 (ISSN-L)
- PMC
- PMC6048748
- OpenAlex
- W2884170664
- Cited by
- 23
- References
- 47
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 4 · 2022: 3 · 2021: 3 · 2020: 5 · 2019: 4
- FWCI
- 0.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Pilot and Feasibility Studies (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Explore Bristol Research (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute for Health Research (RMFI-2015-06-03)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- pilot trial, feasibility study, physical activity research, reporting quality, editorial policy, trial planning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
