Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Objectives: Designing studies with an internal pilot phase may optimise the use of pilot work to inform more efficient randomised controlled trials (RCTs). Careful selection of preagreed decision or ‘progression’ criteria at the juncture between the internal pilot and main trial phases provides a valuable opportunity to evaluate the likely success of the main trial and optimise its design or, if necessary, to make the decision not to proceed with the main trial. Guidance on the appropriate selection and application of progression criteria is, however, lacking. This paper outlines the key issues to consider in the optimal development and review of operational progression criteria for RCTs with an internal pilot phase. Design: A structured literature review and exploration of stakeholders' opinions at a Medical Research Council (MRC) Hubs for Trials Methodology Research workshop. Key stakeholders included triallists, methodologists, statisticians and funders. Results: There is considerable variation in the use of progression criteria for RCTs with an internal pilot phase, although 3 common issues predominate: trial recruitment, protocol adherence and outcome data. Detailed and systematic reporting around the decision-making process for stopping, amending or proceeding to a main trial is uncommon, which may hamper understanding in the research community about the appropriate and optimal use of RCTs with an internal pilot phase. 10 top tips for the development, use and reporting of progression criteria for internal pilot studies are presented. Conclusions: Systematic and transparent reporting of the design, results and evaluation of internal pilot trials in the literature should be encouraged in order to facilitate understanding in the research community and to inform future trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In effectiveness trials, power calculations and sample size decisions are primarily based on the detection of a clinically meaningful difference in measures of the effects of the intervention on the patient or public health outcomes such as behaviour, disease, symptomatology or functional outcomes [24].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Feasibility and acceptability were operationalized a priori according to the following progression criteria thresholds: stop (benchmark: < 70%), adapt (> 70%- < 90%), and go (> 90%).”
From Yoga-Based Group Intervention for Inpatients with Schizophrenia Spectrum Disorders—Feasibility, Acceptability, and Preliminary Outcomes of a Rater-Blinded Randomized Controlled Trial. (2026) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These results give solid ground for the likely success of a following fully powered RCT investigating the intervention’s efficacy32 and align with previous findings on MBIs for SSD.22”
From Yoga-Based Group Intervention for Inpatients with Schizophrenia Spectrum Disorders—Feasibility, Acceptability, and Preliminary Outcomes of a Rater-Blinded Randomized Controlled Trial. (2026) · cites this paper as [32]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Thus, it aligns with the main aims and objectives of pilot and feasibility studies and with the progression stop-amend-go recommendations of Eldridge et al. [2] and Avery et al. [18].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Avery et al. [18] published recommendations for internal pilots emphasising a traffic light (stop-amend-go/red-amber-green) approach to progression with focus on process assessment (recruitment, protocol adherence, follow-up) and transparent reporting around the choice of trial design and the decision-making processes for stopping, amending or proceeding to a main trial.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [18]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
CONSORT 2010 statement: extension to randomised pilot and feasibility trials.
Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · BMJ · doi:10.1136/bmj.i5239 · Cited by 3,404
A tutorial on pilot studies: the what, why and how
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-1 · Cited by 3,384
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. J. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,930
CONSORT 2010 statement: extension to randomised pilot and feasibility trials
on behalf of the PAFS consensus group, Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0105-8 · Cited by 2,843
Design and analysis of pilot studies: recommendations for good practice
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j..2002.384.doc.x · Cited by 2,572 · cited 1×
“In addition, the assumption that pilot studies are less rigorous than main studies and are not designed to test the main trial hypothesis means that their findings are underdisseminated, thereby preventing triallists from accessing information that may be valuable to the success of future main trials.(3) 10 The system…”
BackgroundSelf-citationFundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
No surgical innovation without evaluation: the IDEAL recommendations
McCulloch, P., Altman, D. G., Campbell, W. B., Flum, D. R., Glasziou, P., Marshall, J. C., & Nicholl, J. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61116-8 · Cited by 1,820
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2013) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.09.010 · Cited by 1,658
Eldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L., & Bond, C. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0150205 · Cited by 1,581
Cited by
Items 101–200 of 577
Rosala‐Hallas, A., Gamble, C. L., Blazeby, J. M., & Williamson, P. R. (2019) · Trials · doi:10.1186/s13063-019-3669-9 · Cited by 11
Self-citationBarzideh, A., Marzolini, S., Danells, C. J., Jagroop, D., Huntley, A. H., Inness, E. L., Mathur, S., Mochizuki, G., OH, P. I., & Mansfield, A. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035740 · Cited by 12
Busse, M. E., Playle, R., Latchem-Hastings, J., Button, K., Lowe, R., Barlow, C., Lloyd, B. J., Andrew, D.-Y., Poile, V., Dawes, H., et al. (2021) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2021.103388 · Cited by 12
“A traffic light system of progression criteria for recruitment and retention 31 (see Table 2) guided our decisions as to future evaluations.”
Craddock, N., Garbett, K. M., Haywood, S., Nasution, K., White, P., Saraswati, L. A., Rizkiah, C., Medise, B. E., & Diedrichs, P. C. (2021) · BMC Public Health · doi:10.1186/s12889-021-11770-0 · Cited by 16
“Criteria will be rated for each parameter with a traffic light system from red (stop, major issue for the progress of the trial) through to amber (amend and proceed with caution) through to green (continue, no issues threatening the success of the trial) [55].”
BackgroundPotter, C., Leggat, F. J., Lowe, R., Pallmann, P., Riaz, M. A., Barlow, C., Edwards, A., Siriwardena, A. N., Sevdalis, N., Sewell, B., et al. (2023) · Trials · doi:10.1186/s13063-023-07090-w · Cited by 13
McGuier, E. A., Feldman, J. A., Bay, M., Ascione, S., Tatum, M., Salas, E., & Kolko, D. J. (2023) · Children and Youth Services Review · doi:10.1016/j.childyouth.2023.107096 · Cited by 13
Mutedzi, B., Langhaug, L. F., Hunt, J., Nkhoma, K. B., & Harding, R. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0450-5 · Cited by 12
“(2) Will we be able to retain at least 75% of both the trial participants and the interventionists in the total duration of the study? (3) Can we deliver the 9-cell bereavement intervention as intended? (4) Will the processes of a randomized cluster trial be possible? (5) Does process data suggest that the anticipated…”
BackgroundDwyer, C. P., Alvarez‐Iglesias, A., Joyce, R. A., Counihan, T. J., Casey, D., & Hynes, S. M. (2020) · Trials · doi:10.1186/s13063-020-4179-5 · Cited by 11
“A traffic light system—green (go), amber (amend) and red (stop)—which allows for modification will be used [66], in consultation with the TSC.”
MethodSchormann, A. L. A., Pillny, M., Haß, K., & Lincoln, T. M. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01284-4 · Cited by 11
Smith, T. O., Khoury, R., Hanson, S., Welsh, A., Grant, K., Clark, A., Ashford, P., Hopewell, S., Pfeiffer, K., Logan, P., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073611 · Cited by 10
Methodological progress note: Pilot randomized controlled trials
Corley, A., Marsh, N. M., & Keogh, S. J. (2024) · Journal of Hospital Medicine · doi:10.1002/jhm.13376 · Cited by 13
Sahely, A., Sintler, C., Soundy, A., & Rosewilliam, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0286611 · Cited by 11
“The feasibility of conducting this study was determined by some pre-defined criteria following the RAG (Red, Amber, Green) rating system described by Avery et al., 2017 [41].”
Bryant, M., Dharni, N., Dickerson, J., Willan, K., McEachan, R., Duffy, J., & Howell, M. (2019) · BMC Public Health · doi:10.1186/s12889-019-7149-7 · Cited by 11
“Recent guidance supports which components are important in the progression of pilot and feasibility studies to definitive trials [11].”
BackgroundMethodInfluentialfor the Canadian Critical Care Trials Group, Newman, A. N. L., Kho, M. E., Harris, J. E., Zamir, N., McDonald, E., Fox‐Robichaud, A., & Solomon, P. E. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00760-5 · Cited by 11
“and employed a traffic light system for specifying our progression criteria [18].”
MethodBolzani, A., Ramsenthaler, C., Hodiamont, F., Burner-Fritsch, I., & Bausewein, C. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-042266 · Cited by 10
Smith, T. O., Clark, L., Khoury, R., Man, M., Hanson, S., Welsh, A., Clark, A., Hopewell, S., Pfeiffer, K., Logan, P., et al. (2021) · Bone & Joint Open · doi:10.1302/2633-1462.211.bjo-2021-0136 · Cited by 10
Griva, K., Chia, J. M. X., Goh, Z. Z. S., Wong, Y. P., Loei, J., Thach, T., Chua, W. B., & Khan, B. A. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053588 · Cited by 13
Brekke, A. F., Overgaard, S., Mussmann, B. R., Poulsen, E., & Holsgaard‐Larsen, A. (2022) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2022.102613 · Cited by 12
Moore, S. A., Flynn, D., Jones, S., Price, C., & Avery, L. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01139-4 · Cited by 11
Tyler, E., Lobban, F. A., Sutton, C., Hadarag, M. B., Johnson, S. L., Depp, C. A., Duncan, D., & Jones, S. H. (2022) · BJPsych Open · doi:10.1192/bjo.2022.582 · Cited by 14
Moffett, B., Pozuelo, J. R., van Heerden, A. C., O’Mahen, H., Craske, M. G., Sodi, T., Lund, C., Orkin, K., Kilford, E. J., Blakemore, S., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-065977 · Cited by 12
Nymand, S. B., Hartmann, J. P., Rasmussen, I. E., Iepsen, U. W., Ried‐Larsen, M., Christensen, R. H., & Berg, R. M. G. (2023) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2022-001486 · Cited by 10
Lewis, N. V., Gregory, A. C., Feder, G. S., Angill-Williams, A., Bates, S., Glynn, J., Halliwell, G., Hawcroft, C., Kessler, D. S., Lawton, M., et al. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01335-w · Cited by 20
Thomson, S., Ainsworth, G., Selvanathan, S. K., Kelly, R., Collier, H., Mujica-Mota, R. E., Talbot, R., Brown, S. T., Croft, J., Rousseau, N. S., et al. (2023) · Health Technology Assessment · doi:10.3310/otoh7720 · Cited by 11
Riordan, F., Racine, E., Smith, S. M., Murphy, A., Browne, J. P., Kearney, P. M., Bradley, C., James, M., Murphy, M. E., & McHugh, S. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00608-y · Cited by 15
Strong, H., Hood, A., Johnson, Y., Hackworth, R., Reed-Shackelford, M., Ramaswamy, R., Varughese, T. E., Quinn, C. T., & Crosby, L. E. (2023) · Clinical Trials · doi:10.1177/17407745231154199 · Cited by 9
Schulte, C. E., Sachser, C., Rosner, R., Ebert, D. D., & Zarski, A. (2024) · European Journal of Psychotraumatology · doi:10.1080/20008066.2024.2364469 · Cited by 9
“Finally, future pilot studies should define formal progression criteria, for instance, akin to a traffic light system (Avery et al., 2017).”
BackgroundPozuelo, J. R., Nabulumba, C., Sikoti, D., Davis, M., Gumikiriza‐Onoria, J. L., Kinyanda, E., Moffett, B., van Heerden, A. C., O’Mahen, H., Craske, M. G., et al. (2024) · JMIR Serious Games · doi:10.2196/59381 · Cited by 11
Hemrage, S., Kalk, N. J., Shah, N. K., Parkin, S., Deluca, P., & Drummond, C. (2025) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.70018 · Cited by 9
“Each of these criteria was assessed according to a “traffic light system” proposed by Avery et al. ( 2017 ).”
Dwyer, C. P., Joyce, R. A., Bane, E., Moses, A., Alvarez‐Iglesias, A., & Hynes, S. M. (2020) · HRB Open Research · doi:10.12688/hrbopenres.12981.2 · Cited by 9
Frawley, E., Cowman, M., Cella, M., Cohen, D. R., Ryan, E., Hallahan, B. P., Bowie, C. R., McDonald, C., Fowler, D. G., Wykes, T., et al. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01064-6 · Cited by 10
“A system of proceed, amend, or stop will be utilised modeled on previously used traffic light systems [35] (see Table 2).”
MethodShaw, L. C., McCue, P., Brown, P. M., Buckley, C., Din, S. D., Francis, R. P., Hunter, H., Lambert, A., Lord, S., Price, C., et al. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01193-y · Cited by 12
“To determine whether to proceed to a definitive RCT, progression criteria based on published recommendations [12] were set for recruitment rate, treatment adherence and data completeness (Table 1).”
BackgroundUrquhart, D. S., Cunningham, S., Taylor, E., Vogiatzis, I., Allen, L. J., Lewis, S. C., Neilson, A. R., Soilemezi, D., Akooji, N., & SAYNOR, Z. L. (2022) · NIHR Open Research · doi:10.3310/nihropenres.13347.1 · Cited by 11
“These pre-specified criteria are set in order to judge the viability of completing the main trial within the planned timetable and budget and usually address the key areas of uncertainty or risk that could influence the success of the trial 45 .”
BackgroundMethodCollings, R., Freeman, J. A., Latour, J. M., Hosking, J., & Paton, J. S. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01252-y · Cited by 14
“The completion and performance rates of the measures were excellent, clearly meeting our pre-de ned progression criteria for a de nitive trial (43).”
McGlashan, J. A., Aaen, M., White, A., & Sadolin, C. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01317-y · Cited by 12
“The aim of this feasibility study is to address the following uncertainties before proceeding with a pilot randomised clinical trial of CVT-VT versus SLT-VT: (1) Can CVT-VT applied by a CVT-P improve the voice and vocal function in patients with MTD? (2) Is it feasible to perform a pilot study using CVT-VT administere…”
BackgroundMethodInfluentialBartoszko, J. M., Miles, S. J., Ansari, S., Grewal, D., Li, M., Callum, J. L., McCluskey, S. A., Lin, Y., & Karkouti, K. (2024) · BJA Open · doi:10.1016/j.bjao.2024.100303 · Cited by 9
“41,42”
BackgroundMbuagbaw, L. C. E., Chen, L. H., Aluko, E., Stevens-Uninsky, M., Onuorah, A. C., Garcia, M. C., Stech, L., Atkin-Jones, T., Rehman, N., & Raifu, A. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01342-x · Cited by 9
“Existing guidance cites the use of prevalence or incidence rates and pre-existing observational data for recruitment rates [8].”
BackgroundFakolade, A. O., Cameron, J., McKenna, O. J., Finlayson, M. L., Freedman, M. S., Latimer‐Cheung, A. E., & Pilutti, L. A. (2021) · JMIR Research Protocols · doi:10.2196/18410 · Cited by 9
Nordlund, J., Henry, R. S., Kwakkenbos, L., Carrier, M., Levis, B., Nielson, W. R., Bartlett, S. J., Dyas, L., Tao, L., Fedoruk, C., et al. (2021) · Trials · doi:10.1186/s13063-021-05827-z · Cited by 9
“To test our revised trial design and program delivery method, we will conduct a new feasibility trial with direct progression to full-scale trial [28, 29] if certain criteria are met.”
MethodRøpke, A., Morville, A.-L., Møller, T. E., Delkus, E. C. G., & Juhl, C. B. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03039-x · Cited by 12
“Research progression criteria were based on a traffic light of green (go), amber (amend) and red (stop) [15].”
BackgroundInfluentialSorsdahl, K. R., van der Westhuizen, C., Hornsby, N., Jacobs, Y., Poole, M., Neuman, M., Weiss, H. A., & MYERS, B. J. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2169083 · Cited by 11
“Based on these seven primary outcomes, a priori criteria for progressing to a full RCT were established (Avery et al., 2017), see Table III.”
BackgroundLightfoot, C. J., Wilkinson, T. J., Vadaszy, N., Graham-Brown, M. P. M., Davies, M. J., Yates, T. E., & Smith, A. C. (2024) · Journal of Renal Care · doi:10.1111/jorc.12488 · Cited by 8
Coffey, T., Williamson, P. R., & Gillies, K. (2024) · Trials · doi:10.1186/s13063-024-07968-3 · Cited by 9
Self-citationZhang, Q., Dimairo, M., Julious, S. A., Lewis, J., & Yu, Z. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02339-7 · Cited by 11
“Considering this, reporting additional information on the sample size and planned trial adaptation could potentially reduce the allocated trial budget, if the trial is very likely to stop early [10, 32–34].”
BackgroundBayesian design and analysis of external pilot trials for complex interventions
Wilson, D. T., Wason, J. M. S., Brown, J. M., Farrin, A. J., & Walwyn, R. E. A. (2021) · Statistics in Medicine · doi:10.1002/sim.8941 · Cited by 10
Kidd, S. A., McKenzie, K. J., Wang, W., Agrawal, S., & Voineskos, A. N. (2021) · JMIR Research Protocols · doi:10.2196/24736 · Cited by 9
Thwaites, S., Thewlis, D., Hall, K. A., & Rickman, M. S. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01057-5 · Cited by 8
“See Table 2 for a description of feasibility outcome metrics and progression criteria based on the traffic light system [73]: Green: continue to full trial without changes Amber: continue to full trial with changes to the protocol as deemed necessary by the Trial Steering Committee relating to enrolment rate, retentio…”
MethodLawson, G. M., Owens, J. S., Mandell, D. S., Tavlin, S., Rufe, S. A., Lyon, A. R., Eiraldi, R. B., & Power, T. J. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01381-4 · Cited by 10
“Consistent with recommendations [2] we will use traffic light criteria to determine whether the research protocol should proceed to a full trial unchanged (“green”), with adaptations (“amber”), or not at all (“red”).”
MethodBowyer‐Crane, C., Nielsen, D., Bryant, M., Dharni, N., Heald, R., Storr, C., & Dickerson, J. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0498-2 · Cited by 8
Lapid, M. I., Pagali, S. R., Basso, M. R., Croarkin, P. E., Geske, J. R., Huston, J., Islam, K., Joseph, B., Kennebeck, W. W., Kang, D., et al. (2025) · Pilot and Feasibility Studies · doi:10.1186/s40814-025-01625-5 · Cited by 7
“While some studies use structured traffic light systems (e.g., Go/Amend/Stop decisions), our approach follows recommendations from Eldridge et al. [43] and Avery et al. [42], which emphasize that feasibility data should guide trial refinement rather than enforce rigid stop/go thresholds.”
InfluentialRapolthy‐Beck, A., Fleming, J., Turpin, M. J., Sosnowski, K., Dullaway, S., & White, H. T. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00795-2 · Cited by 9
Kirwan, C., Hynes, L., Hart, N. D., Mulligan, S., Leathem, C. S., McQuillan, L., Maxwell, M., Carr, E., Roche, K. F., Walkin, S., et al. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01107-y · Cited by 10
“These were assessed using predetermined progression criteria [3, 23].”
BackgroundMethodSevere COVID anxiety among adults in the United Kingdom: cohort study and nested feasibility trial
Crawford, M., King, J. D., McQuaid, A., Bassett, P., Leeson, V. C., Tella, O., Simplicio, M. C. D., Tyrer, P. J., Tyrer, H., Watt, R. G., et al. (2024) · BMC Psychiatry · doi:10.1186/s12888-023-05446-9 · Cited by 8
“The criteria for determining the success of the feasibility study, were based on thresholds used in other feasibility and pilot trials, [38, 39] recruitment of at least 32 participants (80% of the target study sample of 40 participants), uptake of the intervention by at least 60% of participants in…”
MethodLangdon, P. E., Apanasionok, M. M., Scripps, E., Barrowcliff, A. L., Biswas, A. B., Bunning, K., Burbidge, C., Byron‐Daniel, K., Cookson, A., Croom, S., et al. (2024) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.13282 · Cited by 8
Hider, S., Bucknall, M., Cooke, K., Cooke, K., Finney, A. G., Goddin, D., Healey, E. L., Hennings, S., Herron, D., Jinks, C., et al. (2018) · Journal of Comorbidity · doi:10.1177/2235042x18792373 · Cited by 9
“The success of the pilot trial will be assessed using criteria adapted from Avery et al.40: a. Recruitment rates: Uptake of eligible patients (red: <25%, amber: uptake 25–50% and Green: uptake >50% of eligible patients); b. Retention rates: follow-up rates across 3 and 6 months (red: <50%, amber: 50–70% and green: fol…”
MethodAlmulhim, A. N., Goyder, E., & Caton, S. J. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192215089 · Cited by 10
Oldershaw, A., Basra, R. S., Lavender, T., & Startup, H. M. (2023) · European Eating Disorders Review · doi:10.1002/erv.3034 · Cited by 11
Barrett, E., Gillespie, P., Newell, J., & Casey, D. (2018) · Trials · doi:10.1186/s13063-018-2848-4 · Cited by 10
Hider, S., Bucknall, M., Jinks, C., Cooke, K., Cooke, K., Desilva, E. E., Finney, A. G., Healey, E. L., Herron, D., Machin, A. R., et al. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00750-7 · Cited by 7
“[29], following discussion with the TSC.”
Amaefule, C. E., Bolou, A., Drymoussi, Z., Carreras, F. J. G., Pardo, M. D. C., Lanz, D., Dodds, J. P., Sweeney, L., Pizzo, E., D’Amico, M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036198 · Cited by 9
Josefsson, C., Liljeroos, T., Hellgren, M. I., Pöder, U., Hedström, M., & Olsson, E. M. G. (2024) · JMIR Formative Research · doi:10.2196/46222 · Cited by 7
Rus‐Calafell, M., Luker, M., Marzinzik, M., Nguyen, P.-M., Schneider, S., Teismann, T., Ehrbar, N., Tas, E., Zhang, X. C., Edwards, C., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-098004 · Cited by 8
Plourde, V., Corriveau, H., & Ahmadi, S. (2025) · Pilot and Feasibility Studies · doi:10.1186/s40814-025-01696-4 · Cited by 6
O'Neill, B. M., Bradley, J. M., Connolly, B. A., Bruce, J., Underwood, M. R., Lall, R., Ji, C., Costley, J., Clarke, R., Dark, P. M., et al. (2025) · NIHR Open Research · doi:10.3310/nihropenres.13910.2 · Cited by 7
Lodder, A., Mehay, A., Pavlickova, H., Hoare, Z., Box, L., Butt, J., Weaver, T. D., Crawford, M., Clutterbuck, D., Westbrook, N., et al. (2021) · BMC Public Health · doi:10.1186/s12889-021-11912-4 · Cited by 8
“RAG (red/amber/green) progression criteria [47] were used to assess the randomisation, retention and intervention attendance to determine whether the stage 2 main trial could proceed (see Table 1).”
MethodSmith, R., Kenworthy, Y., Astbury, N. M., Birks, J., Bateman, P. A., Dyson, P. A., Hirst, J. E., Jebb, S. A., Michalopoulou, M., Pulsford, R. M., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062525 · Cited by 8
Achten, J., Marques, E. M. R., Pinedo‐Villanueva, R., Whitehouse, M. R., Eardley, W., Costa, M. L., Kearney, R. S., Keene, D. J., & Griffin, X. L. (2024) · Bone & Joint Open · doi:10.1302/2633-1462.53.bjo-2023-0099.r1 · Cited by 6
Smith, R., Gould, R. L., Kenworthy, Y., Astbury, N. M., Smith, I. B. J., Birks, J., Bateman, P. A., Hirst, J. E., Jebb, S. A., Michalopoulou, M., et al. (2024) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-024-06508-w · Cited by 9
“This system has been suggested to be preferable to the stop/go pass/ fail approach [28].”
Castro, Ó., Mair, J. L., Zheng, S., Tan, S. Y. X., Jabir, A. I., Yan, X., Chakraborty, B., Tai, E. S., van Dam, R. M., von Wangenheim, F., et al. (2025) · Contemporary Clinical Trials · doi:10.1016/j.cct.2025.107833 · Cited by 6
Wilson, D. T., Brown, J., Farrin, A. J., & Walwyn, R. E. A. (2021) · Statistics in Medicine · doi:10.1002/sim.9091 · Cited by 6
“A recent workshop identified that recruitment, follow-up, and intervention adherence have emerged as common targets of progression criteria [6].”
BackgroundMethodInfluentialMeek, C., Moghaddam, N. G., Evangelou, N., Oates, L. L., Topcu, G., Allen, C. M., & das Nair, R. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.07.001 · Cited by 11
Mellor, K., Dutton, S., Hopewell, S., & Albury, C. (2022) · Trials · doi:10.1186/s13063-022-06063-9 · Cited by 6
“Internal pilot trials are embedded within an RCT forming the first phase, for example, the first 6 months, where the feasibility of the RCT is being assessed whilst the RCT is being conducted [2].”
BackgroundMethodInfluentialTallon, D., Thomas, L., Brabyn, S., Ching, B. C. F., Hahn, J. S., Jude, B., Logan, M. X., Burrage, A., Fox, F. E., Gilbody, S. M., et al. (2023) · Trials · doi:10.1186/s13063-023-07396-9 · Cited by 9
“[52] and account for the fact that two recruiting sites (London and Hull/York) will have had only a short period to commence recruitment.”
BackgroundVirtual reality-based attention training in patients with neurological damage: A pilot study
Lorentz, L., Müller, K., & Boris Suchan, A. (2023) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2023.2236349 · Cited by 7
“Following the framework put forward by Avery et al. (2017), three main progression criteria should always be considered when moving from a pilot study to the main study phase.”
MethodGibson, I., Jennings, C. S., Neubeck, L., Corcoran, M., Wood, D., Sharif, F., Hynes, L., Murphy, A. W., Byrne, M., & McEvoy, J. W. (2023) · HRB Open Research · doi:10.12688/hrbopenres.13781.1 · Cited by 9
Stopping anticoagulation for isolated or incidental pulmonary embolism: the STOPAPE RCT protocol
Lasserson, D. S., Gaddu, P., Mehta, S. P., Ignatowicz, A., Greenfield, S., Prince, C., Cummins, C., Robinson, G., Rodrigues, J. C. L., Noble, S., et al. (2024) · Health Technology Assessment · doi:10.3310/hrcw7937 · Cited by 6
Shah, A., Marian, I. R., Dutton, S., Barber, V. S., Griffith, D. M., McKechnie, S., Chapman, G., Robbins, P. A., Young, D., Walsh, T. S., et al. (2021) · Journal of the Intensive Care Society · doi:10.1177/17511437211012163 · Cited by 6
Latchem-Hastings, J., Randell, E., Button, K., Jones, F., Lowe, R., Dawes, H., Wood, F., Davies, F., Poile, V., O’Halloran, R., et al. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00852-w · Cited by 6
“[66] will be utilised (see Table 2) to guide our decisions as to future evaluations.”
MethodTechnology clinical trials: Turning innovation into patient benefit
Royle, J., Hughes, A. M., Laura, S., & Landers, D. (2021) · Digital Health · doi:10.1177/20552076211012131 · Cited by 7
“A red/amber/green (stop/amend/continue) trafficlight approach could be applied to the criteria used to determine whether to progress with Part B: BenefitRisk Assessment.34 If progression criteria are not met and cannot be mitigated (usually using Part A: Feasibility feedback as a guide), then the study should not prog…”
MethodCrawford, M., Leeson, V. C., McQuaid, A., Samuel, O., King, J. D., Simplicio, M. C. D., Tyrer, P. J., Tyrer, H., Watt, R. G., & Barnicot, K. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059321 · Cited by 8
Mortensen, S. R., Pedersen, M. E., Skou, S. T., & Ried‐Larsen, M. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20042893 · Cited by 8
“The research progression criteria were based on the traffic light system [22].”
BackgroundMethodInfluentialMcGovern, R., Stamp, E., Javanbakht, M., McColl, E., Hickman, M., & Kaner, E. (2018) · Pilot and Feasibility Studies · doi:10.1186/s40814-018-0305-5 · Cited by 7
“These are common parameters considered when decided upon progression to definitive trial [50].”
BackgroundBrown, M., MURPHY, M. H., McDermott, L., McAneney, H., O’Sullivan, J. M., Jain, S., & Prue, G. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0486-6 · Cited by 7
“Our methods for determining progression to a definitive RCT are based on recent recommendations [30].”
MethodHarris, R., Lowers, V., Best, A., Burnside, G., Clarkson, J., & Hulme, C. (2023) · doi:10.21203/rs.3.rs-2687212/v1 · Cited by 6
Xu, W., Lin, X., Lai, H., Ren, Y., Peng, S., & Lin, T. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-06563-8 · Cited by 5
“The specific progression criteria were [30]: (1) 20% of older adults meeting the eligibility criteria; (2) 80% of intervention participants attending at least six sessions; and (3) 80% of participants completing both baseline and post-intervention data collection.”
Smith, T. O., Khoury, R., Ashford, P., Hanson, S., Welsh, A., Clark, A., Dures, E. K., & Adams, J. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-070865 · Cited by 5
Cunningham, C., O’Doherty, M., Neill, R. D., Adams, M., Kee, F., Cupples, M., Hardeman, W., Simpson, E. E. A., MURPHY, M. H., Callaghan, C., et al. (2024) · Journal of Ageing and Longevity · doi:10.3390/jal4010003 · Cited by 5
“An internal pilot is included with pre-specified stop-go criteria using a ‘traffic light’ system recommended by Avery et al. [15] and assessed by an independent trial steering committee.”
MethodNagappa, M., Subramani, Y., Yang, H., Wood, N. M., Querney, J., Fochesato, L.-A., Nguyen, D., Fatima, N., Martin, J. E., John‐Baptiste, A., et al. (2025) · JMIR Perioperative Medicine · doi:10.2196/58998 · Cited by 5
“We followed the traffic light approach criteria for reporting feasibility outcomes [25-27].”
Jointly assessing multiple endpoints in pilot and feasibility studies
Montgomery, R. N., Bodde, A. E., & Vidoni, E. D. (2025) · Statistical Methods in Medical Research · doi:10.1177/09622802241311219 · Cited by 5
Rice, J. A., Foweather, L., Magill, C., Foulkes, J. D., Meester, A. D., Lenoir, M., Stodden, D. F., & Davies, K. F. (2025) · European Physical Education Review · doi:10.1177/1356336x251388628 · Cited by 7
“Table 1 presents the traf fi c light criteria (Avery et al., 2017), informed by previous work (Fairclough et al., 2024), used to assess feasibility (green = acceptable, proceed; amber = discuss and modify; red = do not proceed, re-design).”
Lucchini‐Raies, C., Doren, F. M., Beca, P., Pérez, J. C., Campos, S., & López‐Dicastillo, O. (2020) · Journal of Advanced Nursing · doi:10.1111/jan.14534 · Cited by 5
Hopewell, S., Kenealy, N., Knight, R., Rangan, A., Dutton, S., Srikesavan, C. S., Feldmann, M., Lamb, S. E., & Nanchahal, J. (2022) · NIHR Open Research · doi:10.3310/nihropenres.13275.1 · Cited by 5
“…a traffic light system whereby ‘Green’ indicates it is feasible to proceed to a definitive trial with the current procedures, ‘Amber’ indicates modification to one or more aspect of the study is required before proceeding to the future definitive trial, and ‘Red’ indicates it is not feasible 41 .”
Fakolade, A. O., Awadia, Z., Cardwell, K. L., McKenna, O. J., Venasse, M., Hume, T. A., Ludgate, J., Freedman, M. S., Finlayson, M. L., Latimer‐Cheung, A. E., et al. (2023) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2023.101222 · Cited by 5
“The following criteria were used to define acceptable feasibility and progression to a definitive trial [47]: 1) a minimum of 50% of the intended 20 dyads, recruited within a 6-month recruitment window; 2) ≥ 80% retention; 3) < 10% of participants report a serious AE; 4) a minimum of 70%…”
BackgroundMethodHarris, R. V., Lowers, V., Best, A., Burnside, G., Clarkson, J., & Hulme, C. T. (2024) · BMC Oral Health · doi:10.1186/s12903-024-03942-2 · Cited by 6
Almulhim, A. N., Alhowaish, A., Madani, A., AlQaddan, M., Altuwalah, A. S., & Goyder, E. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-078631 · Cited by 9
Bartoszko, J. M., Peer, M., Grewal, D., Ansari, S., Callum, J., & Karkouti, K. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01518-z · Cited by 5
Howlett, N., Bottoms, L. M., Chater, A., Clark, A., Clarke, T., David, L., Irvine, K., Jones, A., Jones, J., Mengoni, S. E., et al. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00734-7 · Cited by 6
“A traffic-light system, relating to recruitment, retention, adherence, and completion will inform whether we “stop”, “proceed”, or “proceed but with protocol changes” [64].”
BackgroundEvaluation of Interventions: Feasibility, Efficacy and Effectiveness Trials
Kwakkenbos, L. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00219-3 · Cited by 5
Arden-Close, E. J., Thomas, S., Bush‐Evans, R., Wang, R., Bolat, E., Hodge, S., Phalp, K. T., & McAlaney, J. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-065400 · Cited by 6
Shah, A., Marian, I. R., Dutton, S., Barber, V. S., Griffith, D. M., McKechnie, S., Chapman, G., Robbins, P. A., Young, D., Walsh, T. S., et al. (2019) · Journal of the Intensive Care Society · doi:10.1177/1751143719870080 · Cited by 6
Metadata
- Authors
- Avery, K. N. L., Williamson, P. R., Gamble, C., O'Connell Francischetto, E., Metcalfe, C., Davidson, P., Williams, H., & Blazeby, J. M.
- Affiliations
- University of Bristol (GB)
- University of Liverpool (GB)
- Medical Research Council ConDuCT-II Hub for Trials Methodology Research, School of Social and Community Medicine, University of Bristol, Bristol, UK
- National Institute for Health and Care Research (GB)
- Nottingham University Hospitals NHS Trust (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Corresponding author
- Avery, Kerry N L
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2017-02
- Volume
- 7
- Issue
- 2
- Pages
- e013537
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC5318608
- OpenAlex
- W2588182016
- Semantic Scholar
- c03117980919b954181d0ea10371c6c6561bde41
- Cited by
- 575
- References
- 48
- Citations by year
- 2026: 83 · 2025: 100 · 2024: 90 · 2023: 83 · 2022: 69 · 2021: 67 · 2020: 49 · 2019: 20 · 2018: 13 · 2017: 1
- FWCI
- 15.85
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMJ Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Portal (Queen's University Belfast) (publishedVersion, cc-by)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- ResearchOnline (Glasgow Caledonian University) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Stirling Online Research Repository (University of Stirling) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Ulster University Research Portal (Ulster University) (submittedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion, other-oa)
- Explore Bristol Research (submittedVersion, cc-by)
- Funders
- National Institute for Health and Care Research (DRF-2011-04-016, NF-SI-0611-10020, DRF-2012-05-182)
- Medical Research Council (MR/K025635/1, MR/L004933/1, MR/K025643/1, MR/L004933/2)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- internal pilot study, progression criteria, randomized controlled trials, trial design, trial recruitment, protocol adherence, outcome data, trial reporting, stakeholder perspectives, trial efficiency
- MeSH terms
- Attitude, Clinical Protocols, Decision Making, Humans, Pilot Projects, Research Design, Research Personnel, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Patient Selection, Guideline Adherence, Clinical Trials Data Monitoring Committees, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.