Should treatment effects be estimated in pilot and feasibility studies?
Open accessAbstract
BACKGROUND: Feasibility studies and external pilot studies are used increasingly to inform planning decisions related to a definitive randomized controlled trial. These studies can provide information on process measures, such as consent rates, treatment fidelity and compliance, and methods of outcome measurement. Additionally, they can provide initial parameter estimates for a sample size calculation, such as a standard deviation or the 'success' rate for a binary outcome in the control group. However, the issue of estimating treatment effects in pilot or feasibility studies is controversial. METHODOLOGICAL DISCUSSION: Between-group estimates of treatment effect from pilot studies are sometimes used to calculate the sample size for a main trial, alongside estimated standard deviations. However, whilst estimating a standard deviation is an empirical matter, a targeted treatment effect should be established in terms of clinical judgement, as a minimum important difference (MID), not through analysis of pilot data. Secondly, between-group effects measured in pilot studies are sometimes used to indicate the magnitude of an effect that might be obtained in a main trial, and a decision on progression made with reference to the associated confidence interval. Such estimates will be imprecise in typically small pilot studies and therefore do not allow a robust decision on a main trial; both a decision to proceed and a decision not to proceed may be made too readily. Thirdly, a within-group change might be estimated from a pilot or a feasibility study in a desire to assess the potential efficacy of a novel intervention prior to testing it in a main trial, but again such estimates are liable to be imprecise and do not allow sound causal inferences. CONCLUSION: Treatment effects calculated from pilot or feasibility studies should not be the basis of a sample size calculation for a main trial, as the MID to be detected should be based primarily on clinical judgement rather than statistics. Deciding on progression to a main trial based on these treatment effects is also misguided, as they will normally be imprecise, and may be biased if the pilot or feasibility study is unrepresentative of the main trial.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hypothesis testing in pilot studies, particularly for effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations.
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A tutorial on pilot studies: the what, why and how
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-1 · Cited by 3,384 · see citing works
CONSORT 2010 statement: extension to randomised pilot and feasibility trials
on behalf of the PAFS consensus group, Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0105-8 · Cited by 2,843 · see citing works
Design and analysis of pilot studies: recommendations for good practice
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j..2002.384.doc.x · Cited by 2,572 · see citing works
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237 · see citing works
Whitehead, A. L., Julious, S. A., Cooper, C. L., & Campbell, M. J. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280215588241 · Cited by 1,936 · cited 1× · see citing works
“They may also function to estimate parameters required for a sample size calculation, such as the standard deviation of a continuous outcome measure [5, 6], or possibly the control group proportion for a binary”
BackgroundThe role and interpretation of pilot studies in clinical research
Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2010.10.008 · Cited by 1,917 · see citing works
Eldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L., & Bond, C. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0150205 · Cited by 1,581 · see citing works
What is a pilot or feasibility study? A review of current practice and editorial policy
Arain, M., Campbell, M. J., Cooper, C. L., & Lancaster, G. A. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-67 · Cited by 1,483 · see citing works
Sim, J., & Lewis, M. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.07.011 · Cited by 1,036 · see citing works
Self-citation. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · see citing works
The statistical interpretation of pilot trials: should significance thresholds be reconsidered?
Lee, E. C., Whitehead, A. L., Jacques, R. M., & Julious, S. A. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-41 · Cited by 423 · see citing works
Understanding controlled trials: Baseline imbalance in randomised controlled trials
Roberts, C., & Torgerson, D. J. (1999) · BMJ · doi:10.1136/bmj.319.7203.185 · Cited by 287 · see citing works
Questions asked and answered in pilot and feasibility randomized controlled trials
Shanyinde, M., Pickering, R. M., & Weatherall, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-117 · Cited by 229 · see citing works
The design and interpretation of pilot trials in clinical research in critical care
Arnold, D. M., Burns, K. E. A., Adhikari, N. K. J., Kho, M. E., Meade, M. O., & Cook, D. J. (2008) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181920e33 · Cited by 214 · see citing works
Abbott, J. H. (2014) · Journal of Orthopaedic & Sports Physical Therapy · doi:10.2519/jospt.2014.0110 · Cited by 204 · see citing works
Cook, J. A., Julious, S. A., Sones, W., Hampson, L. V., Hewitt, C., Berlin, J. A., Ashby, D., Emsley, R., Fergusson, D. A., Walters, S. J., et al. (2018) · Trials · doi:10.1186/s13063-018-2884-0 · Cited by 197 · see citing works
How big should the pilot study for my cluster randomised trial be?
Eldridge, S. M., Costelloe, C. E., Kahan, B. C., Lancaster, G. A., & Kerry, S. M. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280215588242 · Cited by 135 · see citing works
Sample size re‐estimation in clinical trials
Proschan, M. A. (2009) · Biometrical Journal · doi:10.1002/bimj.200800266 · Cited by 132 · see citing works
Abdulatif, M., Mukhtar, A. M., & Obayah, G. M. (2015) · British Journal of Anaesthesia · doi:10.1093/bja/aev166 · Cited by 84 · see citing works
Chester, R., Costa, M. L., Shepstone, L., Cooper, A., & Donell, S. T. (2007) · Manual Therapy · doi:10.1016/j.math.2007.05.014 · Cited by 73 · see citing works
Kemp, J., Coburn, S. L., Jones, D. M., & Crossley, K. (2018) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2018.7941 · Cited by 69 · see citing works
Galantino, M. L., Callens, M. L., Cardena, G. J., Piela, N. L., & Mao, J. J. J. (2013) · PubMed · Cited by 56 · see citing works
Kaur, N., Figueiredo, S. M., Bouchard, V., Moriello, C., & Mayo, N. E. (2017) · Clinical Rehabilitation · doi:10.1177/0269215517692129 · Cited by 51 · see citing works
Julious, S. A., & Swank, D. J. (2005) · Pharmaceutical Statistics · doi:10.1002/pst.149 · Cited by 40 · see citing works
The Nonuse, Misuse, and Proper Use of Pilot Studies in Experimental Evaluation Research
Westlund, E., & Stuart, E. A. (2016) · American Journal of Evaluation · doi:10.1177/1098214016651489 · Cited by 39 · see citing works
Acupuncture for Treatment of Uncontrolled Pain in Cancer Patients
Garcia, M. K., Driver, L., Haddad, R., Lee, R. T., Palmer, J. L., Wei, Q., Frenkel, M. A., & Cohen, L. G. (2013) · Integrative Cancer Therapies · doi:10.1177/1534735413510558 · Cited by 37 · see citing works
Hislop, J., Adewuyi, T. E., Vale, L. D., Harrild, K., Fraser, C. M., Gurung, T., Altman, D. G., Briggs, A. H., Fayers, P. M., Ramsay, C. R., et al. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001645 · Cited by 36 · see citing works
Sheehan, J., Winzeler‐Mercay, U., & Mudie, M. H. (2006) · Clinical Rehabilitation · doi:10.1177/0269215506071267 · Cited by 28 · see citing works
Uszynski, M., Purtill, H., Donnelly, A., & Coote, S. B. (2015) · Clinical Rehabilitation · doi:10.1177/0269215515595522 · Cited by 17 · see citing works
Mollart, L. J., Skinner, V., & Foureur, M. J. (2016) · Midwifery · doi:10.1016/j.midw.2016.02.020 · Cited by 10 · see citing works
Ruzicka, M., McCormick, B. B., Magner, P. O. C., Ramsay, T., Edwards, C., Bugeja, A. L., & Hiremath, S. S. (2018) · Pilot and Feasibility Studies · doi:10.1186/s40814-018-0263-y · Cited by 4 · see citing works
Lancaster (2004) · doi:10.1111/j.2002.384.doc.x
Flagged first, then by citations
Cited by
Sorted by citations
Lewis, M., Bromley, K., Sutton, C. J., McCray, G., Myers, H. L., & Lancaster, G. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00770-x · Cited by 330 · see citing works
Totton, N., Lin, J., Julious, S. A., Chowdhury, M., & Brand, A. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01416-w · Cited by 160 · see citing works
Incorporating feasibility protocols in intervention research
Gadke, D. L., Kratochwill, T. R., & Gettinger, M. (2020) · Journal of School Psychology · doi:10.1016/j.jsp.2020.11.004 · Cited by 140 · see citing works
Consolidated guidance for behavioral intervention pilot and feasibility studies
Pfledderer, C. D., von Klinggraeff, L., Burkart, S. A., Bandeira, A. D. S., Lubans, D. R., Jago, R., Okely, A. D., van Sluijs, E. M. F., Ioannidis, J. P. A., Thrasher, J. F., et al. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01485-5 · Cited by 65 · see citing works
Determining sample size for pilot trials: a tutorial
Ying, X., Freedland, K. E., Powell, L. H., Stuart, E. A., Ehrhardt, S., & Mayo‐Wilson, E. (2025) · BMJ · doi:10.1136/bmj-2024-083405 · Cited by 57 · see citing works
Geirhos, A., Domhardt, M., Lunkenheimer, F., Temming, S., Holl, R. W., Minden, K., Warschburger, P., Meissner, T., Mueller-Stierlin, A. S., & Baumeister, H. (2022) · BMC Pediatrics · doi:10.1186/s12887-022-03134-3 · Cited by 37 · see citing works
Tönnies, J., Hartmann, M., Wensing, M., Szecsenyi, J., Peters-Klimm, F., Brinster, R., Weber, D., Vomhof, M., Icks, A., Friederich, H.-C., et al. (2021) · JMIR Mental Health · doi:10.2196/22569 · Cited by 40 · see citing works
Gumley, A. I., Bradstreet, S., Ainsworth, J. D., Allan, S., Álvarez‐Jiménez, M., Birchwood, M. J., Briggs, A. H., Bucci, S., Cotton, S., Engel, L., et al. (2022) · Health Technology Assessment · doi:10.3310/hlze0479 · Cited by 37 · see citing works
O'Hayer, C. V., O'Loughlin, C. M., Nurse, C. N., Smith, P. J., & Stephen, M. J. (2021) · Journal of Cystic Fibrosis · doi:10.1016/j.jcf.2020.11.013 · Cited by 34 · see citing works
Hubbard, G., Beeken, R. J., Taylor, C., Watson, A. J. M., Munro, J., & Goodman, W. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0461-2 · Cited by 26 · see citing works
Howell, D., Pond, G. R., Bryant‐Lukosius, D., Powis, M., McGowan, P., Makuwaza, T., Kukreti, V., Rask, S., Hack, S., & Krzyzanowska, M. K. (2023) · Journal of the National Comprehensive Cancer Network · doi:10.6004/jnccn.2022.7095 · Cited by 24 · see citing works
Integrating the Digital and the Traditional to Deliver Therapy for Depression
Stawarz, K., Preist, C., Tallon, D., Thomas, L., Turner, K. M., Wiles, N., Kessler, D. S., Shafran, R., & Coyle, D. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376510 · Cited by 24 · see citing works
A Brief Online Implicit Bias Intervention for School Mental Health Clinicians
Liu, F. F., Coifman, J., McRee, E., Stone, J., Law, A., Gaias, L. M., Reyes, R., Lai, C., Blair, I., Yu, C.-L., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19020679 · Cited by 16 · see citing works
Pilot Trial Characteristics, Postpilot Design Modifications, and Feasibility of Full-Scale Trials
Ying, X., & Ehrhardt, S. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.33642 · Cited by 20 · see citing works
Hubbard, G., Taylor, C., Watson, A. J. M., Munro, J., Goodman, W., & Beeken, R. J. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-0560-0 · Cited by 23 · see citing works
Precision Implementation: An Approach to Mechanism Testing in Implementation Research
Frank, H. E., Kemp, J. J., Benito, K., & Freeman, J. B. (2022) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-022-01218-x · Cited by 20 · see citing works
Kinnear, F. J., Lithander, F. E., Searle, A., Bayly, G. R., Wei, C. H., Stensel, D. J., Thackray, A. E., Hunt, L. P., & Hamilton-Shield, J. P. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-044200 · Cited by 17 · see citing works
Hagström, J., Woodford, J., von Essen, A., Lähteenmäki, P. M., & von Essén, L. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056758 · Cited by 17 · see citing works
Mauritz, M. W., van Gaal, B. G. I., Goossens, P. J. J., Jongedijk, R. A., & Vermeulen, H. (2020) · BJPsych Open · doi:10.1192/bjo.2020.124 · Cited by 16 · see citing works
von Klinggraeff, L., Dugger, R., Okely, A. D., Lubans, D. R., Jago, R., Burkart, S. A., Weaver, R. G., Armstrong, B., Pfledderer, C. D., & Beets, M. W. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-00991-8 · Cited by 17 · see citing works
Machado, P., Pimenta, S., Garcia, A. L., Nogueira, T., Silva, S., Oliveiros, B., Martins, R. A., & Cruz, J. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12082971 · Cited by 18 · see citing works
Eccles, F. J. R., Craufurd, D., Smith, A. M. S., Davies, R., Glenny, K., Homberger, M., Peeren, S., Rogers, D., Rose, L., Skitt, Z., et al. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00631-z · Cited by 16 · see citing works
Lapierre, A., Lavoie, P., Castonguay, V., Lonergan, A.-M., & Arbour, C. (2023) · International Emergency Nursing · doi:10.1016/j.ienj.2022.101261 · Cited by 16 · see citing works
Hackett, S., Zubala, A., Doorn, K. A., Chadwick, T. J., Harrison, T. L., Bourne, J., Freeston, M. H., Jahoda, A. J., Taylor, J. L., Ariti, C. A., et al. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00703-0 · Cited by 15 · see citing works
O’Sullivan, I. C., Crossley, K. M., Kamper, S. J., van Middelkoop, M., Vicenzino, B., Smith, M. M. F., Menz, H. B., Smith, A. J., Tucker, K. J., O'Leary, K. T., et al. (2021) · Journal of Foot and Ankle Research · doi:10.1186/s13047-021-00498-0 · Cited by 14 · see citing works
Kusec, A., Murphy, F. C., Peers, P. V., Lawrence, C., Cameron, E., Morton, C., Bateman, A., Watson, P., & Manly, T. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00660-8 · Cited by 13 · see citing works
Signore, A., Jung, M. E., Semenchuk, B. N., Kullman, S. M., Tefft, O., Webber, S. C., Ferguson, L. J., Kowalski, K. C., Fortier, M., McGavock, J., et al. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01072-6 · Cited by 13 · see citing works
Kaur, H., Schneider, J. M., & Orrell, M. W. (2020) · JMIR Research Protocols · doi:10.2196/24628 · Cited by 13 · see citing works
A systematic review and meta-analysis of sample size methodology for traumatic hemorrhage trials
Ghossein, J., Fernando, S., Rochwerg, B., Inaba, K., Lampron, J., & Tran, A. (2023) · Journal of Trauma and Acute Care Surgery · doi:10.1097/ta.0000000000003944 · Cited by 11 · see citing works
Adaptation and Feasibility of the Mandarin Version of PEERS® for Autistic Adolescents
Lao, U., Li, Y., Bai, W., Wang, Y., Li, Y., Xie, Y., Huang, X., Zhu, H., & Zou, X. (2023) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-023-06056-w · Cited by 11 · see citing works
Sensory Stimulation of the Foot and Ankle Early Post-stroke: A Pilot and Feasibility Study
Aries, A., Pomeroy, V. M., Sim, J., Read, S. M., & Hunter, S. M. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.675106 · Cited by 11 · see citing works
Self-citationJansen, S., Niyonsenga, J., Ingabire, M. C., Jansen, A., Nzabonimpa, E., Ingabire, N., Kangabe, J., Sarabwe, E., Richters, A., Rutayisire, T., et al. (2022) · Trials · doi:10.1186/s13063-022-06994-3 · Cited by 10 · see citing works
Ng, L. C., Miller, A. N., Bowers, G., Cheng, Y., Brigham, R., Tai, M. H., Smith, A. M., Mueser, K. T., Fortuna, L. R., & Coles, M. S. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104310 · Cited by 10 · see citing works
McClure, J. B., Heffner, J. L., Krakauer, C. A., Mun, S., Klasnja, P., & Catz, S. L. (2023) · JMIR mhealth and uhealth · doi:10.2196/46155 · Cited by 9 · see citing works
Wolf, F., Nielsen, J., Saliger, J., Hennecken, E., Kröber, P., Eschweiler, M., Folkerts, A., Karbe, H., & Zimmer, P. (2023) · BMC Neurology · doi:10.1186/s12883-023-03436-8 · Cited by 10 · see citing works
Mosher, C. E., Seçinti, E., Kroenke, K., Helft, P. R., Turk, A. A., Loehrer, P. J., Sehdev, A., Al-Hader, A. A., Champion, V. L., & Johns, S. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00837-9 · Cited by 10 · see citing works
Principles of Experimental Design
Kaltenbach, H. (2021) · Statistics for biology and health · doi:10.1007/978-3-030-69641-2_1 · Cited by 10 · see citing works
von Klinggraeff, L., Ramey, K. T., Pfledderer, C. D., Burkart, S. A., Armstrong, B., Weaver, R. G., & Beets, M. W. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01345-8 · Cited by 9 · see citing works
Ying, X., Robinson, K. A., & Ehrhardt, S. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112358 · Cited by 9 · see citing works
Sahely, A., Sintler, C., Soundy, A., & Rosewilliam, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0286611 · Cited by 10 · see citing works
Sample size planning for pilot studiesPreprint
Tseng, C., & Sim, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05483 · Cited by 21 · see citing works
Schuster‐Amft, C., Kool, J., Möller, J. C., Schweinfurther, R., Ernst, M. J., Reicherzer, L., Ziller, C., Schwab, M. E., Wieser, S., Wirz, M., et al. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01086-0 · Cited by 8 · see citing works
Video-guided exercise after stroke: a feasibility randomised controlled trial
Kenny, M., Gilmartin, J., & Thompson, C. A. (2020) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2020.1790072 · Cited by 8 · see citing works
Rogerson, D., Mora, F. A. A., Young, J. S., & Klonizakis, M. (2022) · European Journal of Clinical Nutrition · doi:10.1038/s41430-022-01115-4 · Cited by 8 · see citing works
Distinctive aspects of consent in pilot and feasibility studies
Sim, J. (2021) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13556 · Cited by 7 · see citing works
Self-citationWu, C., Chan, Y., Yen, L.-Y., Chen, Y.-H., Lo, C., Tseng, L., & Wang, Y. (2022) · Healthcare · doi:10.3390/healthcare10040688 · Cited by 7 · see citing works
Essers, B., Veerbeek, J. M., Luft, A. R., & Verheyden, G. (2024) · Disability and Rehabilitation · doi:10.1080/09638288.2024.2313121 · Cited by 7 · see citing works
Machleid, F., Michnevich, T., Huang, L., Schröder-Frerkes, L., Wiegmann, C., Muffel, T., & Kaminski, J. A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0297929 · Cited by 7 · see citing works
Danesh, V. C., Zuniga, J. A., Timmerman, G. M., Radhakrishnan, K., Cuevas, H. E., Young, C. C., Henneghan, A. M., Morrison, J., & Kim, M. T. (2020) · Applied Nursing Research · doi:10.1016/j.apnr.2020.151345 · Cited by 7 · see citing works
Feasibility of a blended therapy approach in the treatment of patients with inflammatory myopathies
Pfister, P. B., Knols, R. H., de Bie, R. A., & de Bruin, E. D. (2021) · Archives of Physiotherapy · doi:10.1186/s40945-021-00108-z · Cited by 7 · see citing works
Lapid, M. I., Pagali, S. R., Basso, M. R., Croarkin, P. E., Geske, J. R., Huston, J., Islam, K., Joseph, B., Kennebeck, W. W., Kang, D., et al. (2025) · Pilot and Feasibility Studies · doi:10.1186/s40814-025-01625-5 · Cited by 7 · see citing works
Koskan, A. M., Larkey, L., Todd, M. W., & Kim, S. W. (2024) · PLoS ONE · doi:10.1371/journal.pone.0299787 · Cited by 6 · see citing works
McIntyre, L., Fergusson, D. A., McArdle, T., Fox‐Robichaud, A., English, S. W., Martin, C. M., Marshall, J. C., Cook, D. J., Graham, I. D., Hawken, S., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-067142 · Cited by 6 · see citing works
A website for pilot and feasibility studies: giving your research the best chance of success
Chan, C. L. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0522-6 · Cited by 6 · see citing works
Volunteering via Smart-Phone for People With Psychosis—Protocol of a Feasibility Trial
da Costa, M. P., & Groups, P. P. A. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.742202 · Cited by 5 · see citing works
Swales, B., Ryde, G. C., & Phillips, A. C. (2024) · Journal of Aging and Physical Activity · doi:10.1123/japa.2022-0170 · Cited by 5 · see citing works
Treatment Effect Estimates From Pilot Trials Are Unreliable
Troy, J. D., Neely, M. L., Pomann, G., Grambow, S. C., & Samsa, G. P. (2023) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2023.08.020 · Cited by 4 · see citing works
A wellbeing program to promote mental health in paediatric burn patients: Study protocol
Wickens, N., McGivern, L., Belinelo, P. D. G., Milroy, H., Martin, L. J., Wood, F. M., Bullman, I., van Rensburg, E. J., & Woolard, A. J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0294237 · Cited by 4 · see citing works
Eisman, A. B., Palinkas, L. A., Koffkey, C., Herrenkohl, T. I., Abbasi, U., Fridline, J., Lundahl, L. H., & Kilbourne, A. M. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01145-6 · Cited by 4 · see citing works
Rush, K. L., Burton, L., Seaton, C. L., Loewen, P. S., O’Connor, B. P., Corman, K., Phillips, R., Moroz, L., & Andrade, J. G. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20186792 · Cited by 4 · see citing works
Moradian, S., Krzyzanowska, M. K., Maguire, R., Kukreti, V., Amir, E., Morita, P. P., Liu, G., & Howell, D. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035648 · Cited by 4 · see citing works
Evaluation of Interventions: Feasibility, Efficacy and Effectiveness Trials
Kwakkenbos, L. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00219-3 · Cited by 5 · see citing works
Zemp, D. D., Pfister, P. B., Knols, R. H., Quadri, P., Bianchi, G., Giunzioni, D., Lavorato, S., Giannini, O., & de Bruin, E. D. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.1054932 · Cited by 4 · see citing works
Ainsworth, H. R., Marshman, Z., Whiteside, K., Sykes, D., Fairhurst, C. M., Turner, E., Chestnutt, I. G., Day, P. F., Dey, D., Elliott, L., et al. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01243-z · Cited by 3 · see citing works
Small numbers, big decisions: guidance for sample size in pilot trials
Lewis, M., Bromley, K., & Joseph, R. (2025) · BMJ · doi:10.1136/bmj.r1885 · Cited by 3 · see citing works
Ahmed, L., Biddle, K., Blundell, A., Koushesh, S., Kiely, P. D. W., Mein, G., Sedgwick, P. M., & Sofat, N. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01505-4 · Cited by 4 · see citing works
Li, G., Wu, D., Chen, X., Zeng, J., Li, Z., & Thabane, L. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00602-4 · Cited by 3 · see citing works
Frost, G., Strodl, E. S., Swannell, S., Macdonald, K., & Madson, K. (2023) · Behaviour Change · doi:10.1017/bec.2023.1 · Cited by 3 · see citing works
Geerts, J. A., Siemerink, E., Sniehotta, F. F., Loman, L. J. M., Bode, C., & Pieterse, M. E. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01571-8 · Cited by 3 · see citing works
A scoping review of insomnia treatments for people living with HIV
Miranda, J., Sidani, S., Côté, J., & Fredericks, S. (2021) · International Health Trends and Perspectives · doi:10.32920/ihtp.v1i1.1422 · Cited by 2 · see citing works
Bialostocki, L. S., Adhia, D. B., Mudiyanselage, D. R., Smith, M. L., Çakmak, Y. Ö., Ridder, D. D., Mani, R., & Mathew, J. (2025) · JMIR Research Protocols · doi:10.2196/78806 · Cited by 2 · see citing works
El Cálculo del Tamaño Muestral en Ciencias de la Salud: Recomendaciones y Guía Práctica
Fernández-Matías, R. (2023) · Journal of MOVE and Therapeutic Science · doi:10.37382/jomts.v5i1.915 · Cited by 4 · see citing works
Wolf, F., Nielsen, J., Saliger, J., Hennecken, E., Eschweiler, M., Folkerts, A., Karbe, H., & Zimmer, P. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062160 · Cited by 2 · see citing works
Wilson, D. T., Hall, A., Brown, J. M., & Walwyn, R. E. A. (2025) · Statistical Methods in Medical Research · doi:10.1177/09622802251322987 · Cited by 1 · see citing works
Huang, J., Yang, S., Xie, Z., Guo, B., Lan, Y., Li, Y., Jiang, Y., Forman, L. S., Lunze, K., Liang, B., et al. (2025) · AIDS and Behavior · doi:10.1007/s10461-025-04823-7 · Cited by 1 · see citing works
Billias, N., Pouliopoulou, D. V., Lawson, A., D’Alessandro, V., Bryant, D., Peters, S., Rushton, A. B., Miller, E., Brunton, L. K., McGuire, S., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-102112 · Cited by 1 · see citing works
Bitonti, T. M., Durr, K. M., Fernando, S. M., Rochwerg, B. N., Carrier, M., Siegal, D. M., & Tran, A. (2025) · Anaesthesia and Intensive Care · doi:10.1177/0310057x251397167 · Cited by 1 · see citing works
Accounting for Pilot Study Uncertainty in Sample Size Determination of Randomized Controlled Trials
Shi, Y., Liu, F., Li, S., & Chen, J. (2020) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2020.1831951 · Cited by 2 · see citing works
Cadilhac, D. A.-M., Ross, A. G., Bagot, K. L., Blennerhassett, J. M., Kilkenny, M. F., Kim, J., Purvis, T. L., Barclay, K. M., Ellery, F., Morrison, J., et al. (2024) · Pilot and Feasibility Studies · doi:10.21203/rs.3.rs-3708649/v1 · Cited by 1 · see citing works
Cairney, M. V., Crampin, L., Campbell, L., & Cleland, J. (2025) · Pilot and Feasibility Studies · doi:10.1186/s40814-025-01640-6 · Cited by 1 · see citing works
Salisbury, T. T., Maguele, M. S. B., Chissale, F., de Melo, M. S. I., Hanselmann, M., Lamahewa, K., Langat, E. C., Mandlate, F., Nyaga, L., Seward, N., et al. (2025) · Pilot and Feasibility Studies · doi:10.1186/s40814-025-01617-5 · Cited by 1 · see citing works
Pilot and feasibility studies in ophthalmology: fundamental keys to success
Nanji, K., Phillips, M., Thabane, L., Tham, Y. C., Wong, T. Y., STEEL, D. H., Munk, M. R., Wykoff, C. C., Chaudhary, V., On behalf of the Retina Evidence and Trials INternational Alliance (R.E.T.I.N.A.) Study Group, et al. (2024) · Eye · doi:10.1038/s41433-024-03413-6 · Cited by 1 · see citing works
Arai, H., Okawa, S., Seki, Y., Shimizu, E., Takahashi, S., Korte, K. J., & Schmidt, N. B. (2020) · Research Square · doi:10.21203/rs.3.rs-107578/v1
Signore, A., & Strachan, S. M. (2021) · Mspace (University of Manitoba)
Nurses' research literacy: A multi-methods exploration
Hines, S. J. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.229970
Arai, H., Seki, Y., Okawa, S., Shimizu, E., Takahashi, S., Ishikawa, S., Korte, K. J., & Schmidt, N. B. (2021) · Japanese Psychological Research · doi:10.1111/jpr.12366
Mardan-Dezfouli, S., Reid, O. G., Fliss, M. D., Stevenson, J., Harris, D., Lyon, M. R., Koehle, M. S., & Mitchell, C. J. (2022) · Aesthetic Surgery Journal · doi:10.1093/asj/sjac220
Hitching, R., Kluge, M. G., Walker, F. R., Gordon, C. J., & Lampe, L. A. (2025) · doi:10.2196/preprints.77094
Arden-Close, E. J., Thomas, S., Bush‐Evans, R., Hamson-Ford, A., Bolat, E., Hodge, S., Phalp, K. T., Wang, R., & McAlaney, J. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-104784 · Cited by 1 · see citing works
Hansen, M. C., Ingstad, K., Uhrenfeldt, L., & Pedersen, P. U. (2025) · BMC Nutrition · doi:10.1186/s40795-025-01203-1
Mosher, C. E., Seçinti, E., Kroenke, K., Helft, P. R., Turk, A. A., Loehrer, P. J., Sehdev, A., Al-Hader, A. A., Champion, V. L., & Johns, S. A. (2021) · Figshare · doi:10.6084/m9.figshare.14457122
Li, G., Wu, D., Chen, X., Zeng, J., Li, Z., & Thabane, L. (2020) · Figshare · doi:10.6084/m9.figshare.12177828
Signore, A., Jung, M. E., Semenchuk, B. N., Kullman, S. M., Tefft, O., Webber, S., Ferguson, L. J., Kowalski, K. C., Fortier, M., McGavock, J., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19914936
Wolf, F., Nielsen, J., Saliger, J., Hennecken, E., Kröber, P., Eschweiler, M., Folkerts, A.-K., Karbe, H., & Zimmer, P. (2023) · Figshare · doi:10.6084/m9.figshare.24454957
Lewis, M., Bromley, K., Sutton, C., McCray, G., Myers, H. L., & Lancaster, G. A. (2021) · Figshare · doi:10.6084/m9.figshare.13710841
von Klinggraeff, L., Dugger, R., Okely, A. D., Lubans, D., Jago, R., Burkart, S., Weaver, R. G., Armstrong, B., Pfledderer, C. D., & Beets, M. W. (2022) · Figshare · doi:10.6084/m9.figshare.19136116
Totton, N., Lin, J., Julious, S., Chowdhury, M., & Brand, A. (2024) · Figshare · doi:10.6084/m9.figshare.26643079
von Klinggraeff, L., Ramey, K. T., Pfledderer, C. D., Burkart, S. A., Armstrong, B., Weaver, R. G., & Beets, M. W. (2023) · Figshare · doi:10.6084/m9.figshare.23647454
Jansen, S., Niyonsenga, J., Ingabire, M. C., Jansen, A., Nzabonimpa, E., Ingabire, N., Kangabe, J., Sarabwe, E., Richters, A., Rutayisire, T., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22599410
von Klinggraeff, L., Dugger, R., Okely, A. D., Lubans, D., Jago, R., Burkart, S., Weaver, R. G., Armstrong, B., Pfledderer, C. D., & Beets, M. W. (2022) · Figshare · doi:10.6084/m9.figshare.19136113
- Authors
- Sim, J.
- Affiliations
- Keele University (GB)
- Corresponding author
- Sim, Julius
- ORCID
- Sim, Julius
- Venue
- Pilot and Feasibility Studies
- Published
- 2019-08-28
- Volume
- 5
- Issue
- 1
- Pages
- 107-107
- ISSN
- 2055-5784 (ISSN-L)
- PMC
- PMC6712606
- OpenAlex
- W2970940014
- Cited by
- 123
- References
- 31
- Citations by year
- 2026: 15 · 2025: 13 · 2024: 13 · 2023: 23 · 2022: 20 · 2021: 18 · 2020: 19 · 2019: 2
- FWCI
- 11.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Pilot and Feasibility Studies (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- pilot study, feasibility study, treatment effect estimation, sample size determination, minimal clinically important difference, clinical judgment, trial planning, confidence intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review