The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The role and interpretation of pilot studies in clinical research
Open accessAbstract
Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 5 No basis given · 2 No full text of the sources · 1 Sources not found
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“When event rates are reported and no adverse event is observed for a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
What the sources say
- Not matchedcites Jovanovic & Levy, 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Jovanovic et al., 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Basing a decision to proceed or terminate evaluation of a particular intervention on pilot data is perilous because there is a very good chance that the decision will be derived from false positive or false negative results.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each aspect of feasibility could be quite different from an uncontrolled study when intervention assignment is randomized and blinded, particularly if placebo is a distinct possibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these issues in mind, a pilot study can better address its goals if a control group is part of the design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A case series estimate tends to be steeped in optimism, particularly if an estimate of such magnitude is seldom, if ever, seen in full scale trials for psychiatric disorders.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(It is the population effect size, not the sample effect size, that an RCT is designed to detect with sufficient power.)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There has been a venerable tradition of using pilot studies to estimate between group effect sizes that, in turn, are used to inform the design of subsequent larger scale hypothesis testing studies. Despite it widespread use, it has been argued that the tradition is ill-founded. (Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot sample size is instead based on the pragmatics of recruitment and the necessities for examining feasibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With no inferential statistical tests, a pilot study will not provide p-values.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot study data generally should not be combined with data from the subsequent larger scale study. This is because it is quite likely that the methods will be modified after the pilot, even if minimally. Such changes in protocol risk adding additional, perhaps unknown, source of variation.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Are Pilot Data Included in the Larger Trial?
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Methods will likely be modified after a pilot study, even if minimally.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 13
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study.”
- Also stated in: § DISCUSSION · plain
“Pilot results inform feasibility, which in turn, is instructive in that it points to modifications needed in the planning and design of a larger efficacy trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The definition of clinically meaningful is not entirely empirically-based, but instead requires input from clinicians who treat the patient population of interest and perhaps from patients with the disorder.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These good clinical practices include the refinement of source documentation, informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring/oversight procedures, especially when multiple sites and investigators are engaged in the study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety, efficacy and effectiveness are not evaluated in a pilot.”
- Also stated in: § DISCUSSION · plain
“a pilot study is not a hypothesis testing study and therefore safety and efficacy are not evaluated.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot studies generate pilot data, their design need not be guided with the support of prior pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible, but highly unlikely, that the between group effect size (d) from a pilot study sample will provide a reasonable estimate of the population effect size (Δ), but that cannot be known based on the pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study can be critical in research staff training and provide experiences that strengthen and confirm competencies and skills required for the investigation to be conducted with accuracy and precision.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An alternative approach is to base sample size estimates for a full scale RCT on what is deemed a clinically meaningful effect.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > An Alternative to Using Pilot Data for Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study provides opportunity to develop consistent practices to enhance data integrity and the protection of human subjects.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We learn little if anything about the efficacy of an intervention with data from a small sample; yet, as discussed earlier, a great deal can be learned from a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This applies to effects sizes for any type of outcome, be it a severity rating, a response status, or survival status.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, with imprecision comes a vast disparity in sample size estimates and, if sample size determination for a subsequent study is based on an imprecise estimate, there is an enormous risk of underpowered or overpowered design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, pilot studies are a necessary first step in exploring novel interventions and novel applications of interventions – whether in a new patient population or with a novel delivery system (e.g., transdermal patch).”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although a pilot study will undoubtedly incorporate relevant outcome measures and can serve a vital role in treatment development, it is not, and should not, be considered a preliminary test of the intervention hypothesis.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The primary role of a pilot study is to examine the feasibility of a research endeavor.”
- Also stated in: § INTRODUCTION · plain
“The fundamental purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to ultimately be used in a larger scale study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This does not preclude the need for a theoretical rationale for the intervention or the methodology being proposed for a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a pilot study effect size is unduly large (i.e., a false positive result), subsequent trials will be designed with an inadequate number of participants to provide the statistical power needed to detect clinically meaningful effects and that would lead to negative trials.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, pilot studies provide an opportunity to implement and examine the feasibility of the adverse event reporting system.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is quite reasonable and expected that a pilot study is proposed with no pilot or other preliminary data supporting the proposal and that its proposed sample size is based on pragmatics such as patient flow and budgetary constraints.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention.”
- Also stated in: § INTRODUCTION · plain
“evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and/or implementation of the novel intervention.”
- Also stated in: § WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY · plain
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, and implementation of the novel intervention and each of these can be quantified”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are two fundamental reasons that hypothesis testing is not used in a pilot study: the limited state of knowledge about the methods or intervention in the patient population to be studied and the smaller proposed sample size.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a consequence, pilot studies can reduce the proportion of failed trials and allow research funds to be spent on projects for which feasibility has been demonstrated and quantified.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Well-conceived and implemented pilot studies will reduce the risk of several problems that are commonly faced in clinical trials. These include the inability to recruit the proposed sample size and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures or interventions, and the need to modify a protocol midway through a trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Estimates become more precise with larger sample sizes.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The true population effect size is unknown when a pilot study is designed.
“An essential challenge of therapeutic development is that the true population effect size is unknown at the time a pilot study is designed.”
The paper relies on this without support.
Methods will likely be modified after a pilot study, even if minimally.
“This is because it is quite likely that the methods will be modified after the pilot, even if minimally.”
The paper relies on this without support.
Pilot results presented to grant review committees tend to be selective, overly optimistic, and at times misrepresentational.
“Furthermore, pilot results that are presented to grant review committees tend to be selective, overly optimistic and, at times, misrepresentational.”
The paper relies on this without support.
Many pilot studies have far fewer than 18 participants per group.
“Many pilot studies have far fewer than 18 participants/group and therefore even greater imprecision.”
The paper relies on this without support.
It is not unusual for research grant applications and pilot study publications to convey optimism based on pilot data.
“Nevertheless, it is not unusual for research grant applications and pilot study publications to convey such optimism, particularly when based on pilot data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The planned sample size was 30 allowing for attrition. This study will inform the sample size for any future RCT (Leon et al., 2011).”
From A mixed methods pilot and feasibility open trial of internet-delivered cognitive behaviour therapy (iCanADAPT Advanced) for people with advanced cancer with depression and/or anxiety (2021) · cites this paper as Leon et al., 2011
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was a pilot study which like other pilot studies of MI did not have a power calculation. 31,32”
From A Pilot Randomized Controlled Trial to Assess the Impact of Motivational Interviewing on Initiating Behavioral Therapy for Migraine (2020) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data analyses in the context of a pilot study are not hypothesis-driven but serve to provide information regarding feasibility and acceptability of the intervention [55].”
From Acceptance and Commitment Therapy Wellness Program for Latine Adults Who Smoke and Have Psychological Distress: Protocol for a Feasibility Study (2023) · cites this paper as [55]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
From PandaMom – Feasibility and acceptability of an internet- and mobile-based intervention to enhance peripartum mental well-being and to prevent postpartum depression (2024) · cites this paper as Leon et al., 2011
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our use of effect sizes was, however, deemed appropriate for this pilot trial (55, 63).”
From EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU (2024) · cites this paper as [63]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work suggests early studies often test feasibility of methods (e.g., recruitment, randomization, retention, and assessment), not hypotheses [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
Citations
Cites
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · doi:10.1007/978-1-4757-2915-3 · Cited by 873
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · cited 1×
“(Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006) Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small samples.”
Self-citationWolfstein (1905) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-190502000-00009 · Cited by 346
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · cited 1×
“(Leon et al., 2006; Leon & Davis, 2009).”
Self-citationSixteen S ‐squared over D ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · cited 1×
“(N/group ≈ 16/d2; Lehr, 1992) The clinical interpretation of HAMD change of 1.2 to 3.0 would drive the choice among possible sample sizes in planning a study.”
BackgroundInfluentialJovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.1080/00031305.1997.10473947 · Cited by 179 · cited 1×
“…a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n. (Jovanovic & Levy, 1997; Jovanovic et al., 1997) For example, consider a study with N=15 receiving medication and no suicidal ideation was reported…”
BackgroundHarvey, I. M. (2009) · Public Health · doi:10.1016/j.puhe.2009.05.009 · Cited by 173
Jovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.2307/2685405 · Cited by 50
Cited by
Sorted by citations
Items 1,001–1,100 of 1,903
Maas, J., Simeunovic-Ostojic, M., & Bodde, N. M. G. (2023) · Journal of Eating Disorders · doi:10.1186/s40337-023-00784-1 · Cited by 4
Park, S.-W., Lim, D.-H., Kim, J.-H., Kim, S., & Han, Y.-O. (2025) · Healthcare · doi:10.3390/healthcare13111327 · Cited by 4
“Larger-scale follow-up studies encompassing diverse regions, age groups, and cultural contexts are needed to validate these results [26,38].”
Conroy, A. A., Butterfield, R. M., Chibi, B., Hahn, J. A., Neilands, T. B., Msimango, L., van Heerden, A. C., Humphries, H., & Starks, T. J. (2025) · AIDS and Behavior · doi:10.1007/s10461-025-04804-w · Cited by 4
Ro, B., Castanhas, L. G., Crenshaw, F., Janczy, A. M., Swift, P., Song, Q., Zhou, X., Parker, J. G., & Roseguini, B. T. (2025) · Journal of Applied Physiology · doi:10.1152/japplphysiol.00285.2025 · Cited by 5
Szeszulski, J., Schaefers, A., de Mello, G. T., Gardner, J., George, A., Uribe, A. L. M., Rethorst, C. D., Seguin‐Fowler, R. A., & Xin, L. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1652485 · Cited by 4
Healing the heart naturally: The role of bioactive phytocompounds in cardiovascular therapeutics
Anand, A. K., Srivastava, S., Sharma, D., Sridhar, S. B., Tariq, M., Rao, G. S. N. K., Mruthunjaya, K., Manjula, S. N., Sahu, R., & Hamida, M. B. B. (2026) · Phytomedicine · doi:10.1016/j.phymed.2026.158084 · Cited by 4
Organisational Capabilities Influencing SMEs’ Export Performance in Tanzania
Okangi, F. P. (2023) · Business Management Review · doi:10.56279/bmrj.v26i2.3 · Cited by 4
A wellbeing program to promote mental health in paediatric burn patients: Study protocol
Wickens, N., McGivern, L., Belinelo, P. D. G., Milroy, H., Martin, L. J., Wood, F. M., Bullman, I., van Rensburg, E. J., & Woolard, A. J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0294237 · Cited by 4
“Due to the nature of this pilot study, a hypothesis is not tested and therefore safety and efficacy are not evaluated [80].”
Shrier, L. A., McCaskill, N. H., Smith, M. C., O’Connell, M., Gluskin, B. S., Parker, S., Everett, V., Burke, P. J., & Harris, S. K. (2024) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2024.209472 · Cited by 4
Möller, S., Hagberg, K., & Ramstrand, N. (2024) · Journal of Rehabilitation Medicine · doi:10.2340/jrm.v56.40111 · Cited by 4
“As a pilot study, results of the present research are limited by a small sample size, which does not allow for testing of hypotheses or generalization of results (40).”
Woods, C., Lukersmith, S., & Salvador‐Carulla, L. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080078 · Cited by 4
Kayrouz, R., Karin, E., Staples, L., Nielssen, O. B., Cross, S. P. M., Dear, B. F., & Titov, N. (2024) · Journal of Cross-Cultural Psychology · doi:10.1177/00220221241269994 · Cited by 5
Bao, L., Kristen, A., Liu, H., Dys, S. P., Gao, Y., Wu, Q., Xiao, B., Xu, J.-J., Zhang, H., Zhang, W., et al. (2025) · Family Process · doi:10.1111/famp.70040 · Cited by 4
“Considering our small sample size, effect sizes may be unstable and should be interpreted with caution (Leon et al. 2011).”
Weisman, A., Yona, T., & Masharawi, Y. (2025) · PeerJ · doi:10.7717/peerj.19448 · Cited by 4
“Previous literature has suggested that extensive instrument validation may be deferred until preliminary hypotheses have been generated and the construct under investigation is better understood ( Leon, Davis & Kraemer, 2011 ).”
Kim, H., Kwon, S., Park, S., Kang, C., & Kim, H. (2025) · BMC Nursing · doi:10.1186/s12912-025-03432-y · Cited by 4
Modrego‐Alarcón, M., del Hoyo, Y. L., Beltrán-Ruiz, M., García‐Campayo, J., & Montero‐Marín, J. (2025) · Virtual Reality · doi:10.1007/s10055-025-01218-w · Cited by 4
“Accordingly, no formal power analysis was performed, in line with recommendations for pilot studies (Julious 2005; Leon et al. 2011; Billingham et al. 2013).”
The Children in Action Pilot Study
Nicklas, T. A., Nguyen, T. T., Butte, N. F., & Liu, Y. (2013) · International Journal of Child Health and Nutrition · doi:10.6000/1929-4247.2013.02.04.3 · Cited by 4
“Thus, it is important that one understands the limitations in the interpretation of pilot studies [63-66].”
BackgroundHawkins, E., Danner, A. N., Malte, C., Painter, J. M., Lott, A. M. K., & Baer, J. (2018) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2018.07.001 · Cited by 1
Ingalls, A., Rebman, P., Martin, L., Kushman, E., Leonard, A., Cisler, A., Gschwind, I., Brayak, A., Amsler, A. M., & Haroz, E. E. (2022) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-022-05057-4 · Cited by 5
“In addition to examining the feasibility of recruitment (number approached and number enrolled per month), randomization (proportion of those approached and eligible who enroll), and assessment procedures (proportion of planned assessments completed) [27], the following primary outcomes were defined: program acceptabi…”
BackgroundResultA Smartphone Physical Activity App for Patients in Alcohol Treatment: Single-Arm Feasibility Trial
Abrantes, A. M., Meshesha, L. Z., Blevins, C. E., Battle, C. L., Lindsay, C., Marsh, E. L., Feltus, S. R., Buman, M. P., Agu, E., & Stein, M. D. (2022) · JMIR Formative Research · doi:10.2196/35926 · Cited by 3
“Therefore, we refrained from conducting significance testing and instead calculated effect sizes [54] in the form of Cohen d [55].”
MethodPérez, J. C., Aldoney, D., García, M. I., Olhaberry, M., Fernández, O., Álamo, N., Franco, P., Pérez, F., Fisher, J., Rowe, H. J., et al. (2022) · Health Informatics Journal · doi:10.1177/14604582221135440 · Cited by 4
“Considering this is a feasibility study, we will not test any hypotheses; therefore, a power calculation is not needed.(50,51) Each of the two health centers involved in the study receives approximately 15 first-time mothers per month.”
BackgroundKavalieratos, D., Lowers, J., Moreines, L. T., Hoydich, Z., Arnold, R. M., Yabes, J. G., Richless, C., Ikejiani, D. Z., Teuteberg, W., & Pilewski, J. M. (2022) · Journal of Palliative Medicine · doi:10.1089/jpm.2022.0349 · Cited by 4
“24,27,28 Second, this was a single site study; results may differ in more heterogeneous treatment settings and populations.”
Sileo, K. M., Muhumuza, C., Sekamatte, S., Lule, H., Wanyenze, R. K., Kershaw, T., & Kiene, S. M. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01226-6 · Cited by 4
“Following stage 1b pilot study guidelines [48, 49], the primary aims of our pilot trial are to assess acceptability and feasibility of the trial procedures and intervention content/structure, with the exploratory aim of assessing the intervention’s potential efficacy, detailed in the specific aims:”
MethodReynaud, D., & Bruneau, L. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-066749 · Cited by 4
Clasby, B., Hughes, N., Clasby, E., et al. (2023) · Neurorehabilitation · doi:10.3233/nre-220218 · Cited by 4
“…to establish the feasibility of an intervention, there was also little to no evaluation of costs and resources required to conduct specific interventions thereby making it impossible to make strong conclusions about feasibility (Leon, Davis, & Kraemer, 2011) and suitability for more widespread use.”
Caldarella, P., Richards, E. J., Williams, L., & Warburton, E. J. (2023) · Education and Treatment of Children · doi:10.1007/s43494-023-00108-7 · Cited by 4
“Pilot studies are an important initial step in exploring novel applications of interventions, with results helping to inform feasibility and potential modifications needed in large scale studies (Leon et al., 2011).”
Fulbrook, P., Miles, S., & Williams, D. M. (2023) · International Journal of Nursing Practice · doi:10.1111/ijn.13229 · Cited by 4
“As this was a pilot study, hypothesis testing was not appropriate (Lancaster et al., 2004; Lee et al., 2014; Leon et al., 2011).”
Choudhary, V., Satapathy, S. S., & Sagar, R. (2024) · Journal of Child Sexual Abuse · doi:10.1080/10538712.2024.2319668 · Cited by 4
“To assess whether REECASST intervention demonstrated statistically significant change, pre and post-intervention scores were compared along with a review of the percentage of change and qualitative feedback from children (Dobkin, 2009; Leon et al., 2011; Orsmond & Cohn, 2015).”
MethodAn Analysis of Students’ Speaking Anxiety in English Foreign Language (EFL)
Sundayanti, H., Surmiyati, S., Bunau, E., et al. (2023) · Journal of English as a Foreign Language Education (JEFLE) · doi:10.26418/jefle.v4i1.68239 · Cited by 4
Design and pilot test of an implicit bias mitigation curriculum for clinicians
Svetkey, L., Bennett, G. G., Reese, B., Corsino, L., Pinheiro, S. O., Fischer, J., Seidenstein, J., Olsen, M., Brown, T. H., Ezem, N., et al. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1316475 · Cited by 4
Travis, A., O’Donnell, A. E., Giraldo‐Santiago, N., Stone, S. M., Torres, D., Adler, S. R., Vranceanu, A., & Ritchie, C. S. (2024) · JMIR Research Protocols · doi:10.2196/58356 · Cited by 4
“Given the exploratory nature of this open pilot, we do not anticipate statistical significance [39].”
Dang, T. H., Wickramasinghe, N. S., Jayaraman, P. P., Burbury, K., Alexander, M., Whitechurch, A., Quinn, S., Rowan, G. D., Brooks, S. L., & Schofield, P. E. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-079122 · Cited by 5
“49 50 Recruitment feasibility will be assessed based on the number of patients recruited divided by the number of patients offered to join the study.”
BackgroundRabasco, A., Wallace, G. T., & Andover, M. S. (2024) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10516-7 · Cited by 4
“The primary purpose of this pilot-RCT was to examine the feasibility and acceptability of Mind-REST, a brief, daily mindfulness intervention aimed at reducing suicidality among individuals with recent suicidal ideation.”
InfluentialRobertson, M. C., Swartz, M. C., Basen‐Engquist, K., Li, Y., Jennings, K., Thompson, D. I., Baranowski, T., Volpi, E., & Lyons, E. (2024) · BMC Public Health · doi:10.1186/s12889-024-19662-9 · Cited by 5
“Because effect size estimates taken directly from pilot studies can produce biased power estimates when used alone [62, 63] and our pilot trials could not provide the range of outcomes needed (e.g., steps at 6, 12, and 18 months), we used several sources to estimate the range of possible…”
BackgroundBest, P., Maddock, A., Ean, N., Montgomery, L., Armour, C., Mulholland, C. C., & Blair, C. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1372062 · Cited by 4
“As a result, the study ’ s uptake rate of 78% exceeds the acceptable threshold of 70% for feasibility studies (36, 62).”
BackgroundBecker, M. E., Levey, N., Yeh, G. Y., Giacino, J. T., Iverson, G. L., Silverberg, N. D., Parker, R. A., McKinnon, E., Siravo, C., Shah, P., et al. (2024) · JMIR Formative Research · doi:10.2196/64540 · Cited by 4
“This approach is designed to provide an initial and tentative indication of potential program effects to be tested in a future RCT, in line with guidelines for feasibility trials [37].”
Shin, L. J., Choi, E., Dawkins-Moultin, L., Wong, C. C. Y., Borjas, M., Fei, F., Xu, Y., Chen, M., Barcenas, C. H., Li, Y., et al. (2025) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2025.102790 · Cited by 4
Nahata, L., Roche, C. I., Griffith, M. M., Karkare, T., Quinn, G. P., O’Brien, S. H., Boone, K. M., Audino, A. N., Yeager, N. D., Whiteside, S. L., et al. (2025) · Pediatric Blood & Cancer · doi:10.1002/pbc.31795 · Cited by 4
“Overall, our notable effect sizes are consistent with the goal of pilot work: to refine research methodology and prepare for larger hypothesis testing and implementation [49].”
Srivastava, S., Wang, P. W., Hill, S. J., Childers, M. E., Keller, K. L., & Ketter, T. A. (2012) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2012.04.004 · Cited by 4
“Despite the limitation of reduced statistical power due to small sample size, pilot studies such as ours can serve the important function of providing initial explorations that may inform the feasibility and design of subsequent clinical trials e indeed, there is increasing attention being paid to the need for such st…”
BackgroundKavale, S., Mugambi, F., & Namusonge, G. (2014) · Global Journal of Commerce & Management Perspective · Cited by 4
White Paper: Games for Health for Children—Current Status and Needed Research
Baranowski, T., Blumberg, F. C., Buday, R., Fiellin, L. E., Green, C. S., Kato, P. M., Lu, A. S., Maloney, E. K., Mellecker, R. R., Morrill, B. A., et al. (2015) · Cited by 4
Safety and pH measurements of sushi rice in Japanese restaurants in Burnaby BC, Canada
Lee, C. C., Sciences, E. H. B. S. O. H., & Heacock, H. J. (2014) · BCIT Environmental Public Health Journal · doi:10.47339/ephj.2014.158 · Cited by 4
“A pilot study was conducted to examine feasibility of an approach and identify modifications needed in the design of the larger hypothesis testing study (Leon et al., 2011).”
MethodDeterminants of SMEs Performance of the Gem and Jewelry Industry in Thailand: A Pilot Study
Ahmad, N. R., Shariff, M. N. M., & Hafeez, M. H. (2017) · Asian Journal of Social Science Studies · doi:10.20849/ajsss.v2i3.173 · Cited by 5
“It serves as an earlier-phase developmental function in improving the probability of success on a larger scale (Leon et al., 2011).”
Background50 Tips for Clinical Trialists
Harvey, L. A., Glinsky, J. V., & Herbert, R. (2017) · Brain Impairment · doi:10.1017/brimp.2017.30 · Cited by 4
Clark, B. J., Sorrell, T. R., Hodapp, R. M., Reed, K., Moss, M., Aagaard, L. M., & Cook, P. F. (2019) · Critical Care Explorations · doi:10.1097/cce.0000000000000051 · Cited by 4
“Consistent with the approach to pilot studies proposed by Leon et al (34), we did not calculate a priori power or use inferential tests.”
MethodStephens, M., & Bartley, C. (2020) · Nursing Older People · doi:10.7748/nop.2020.e1216 · Cited by 4
Practice of Postpartum Warming Among Mothers in Brunei
Wa’ie, I. R. M. W., Koh, D. S. Q., Sharbini, S. H., Noor, S. H. M., Kisut, R., Tuah, N. A., Rahman, H. A., Kifli, N., Hamid, M. R. W. H. A., & Mumin, K. H. A. (2020) · Journal of Transcultural Nursing · doi:10.1177/1043659620914713 · Cited by 5
“Several rounds of discussions by expert panel feedback generated comprehensive and relevant items from the initial broad topics that ensured content validity of the questionnaire (Leon et al., 2011).”
BackgroundEnrique, Á., Bretón-López, J. M., Molinari, G., Llorca, G., & Botella, C. (2020) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-020-00908-5 · Cited by 5
“Regarding the study limitations, it is important to note that, given the small sample size, the results derived from the implementation of the exercise can hardly be generalized to other populations [16], although they can be considered when designing novel interventions focused on improving quality of life and well-b…”
BackgroundColonoscopy Bowel Preparation—Is There an App for That?
Mahmud, N., & Mehta, S. J. (2020) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2020.05.029 · Cited by 4
Carr, R. M., Quested, E., Stenling, A., Thøgersen‐Ntoumani, C., Prestwich, A., Gucciardi, D. F., McVeigh, J. A., & Ntoumanis, N. (2021) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2021.1902815 · Cited by 6
“Yet, a pilot study should not be considered a run-through of whether the hypotheses were supported (Leon et al., 2011).”
BackgroundBoak, J., Blackberry, I., & Rasekaba, T. M. (2021) · Methods and Protocols · doi:10.3390/mps4040070 · Cited by 4
“Frailty is described as a physical wellbeing state, linked to fatigue, resistance and aerobic capability, other illnesses, loss of weight and the client’s awareness of their health [31].”
BackgroundEvaluation of Interventions: Feasibility, Efficacy and Effectiveness Trials
Kwakkenbos, L. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00219-3 · Cited by 5
Snowdon, D. A., Vincent, P., Callisaya, M. L., Collyer, T. A., Wang, Y. T., & Taylor, N. F. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-054298 · Cited by 4
Braus, N., Kewitz, S., & Hunger‐Schoppe, C. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.804567 · Cited by 6
“Limitations While this study is a pilot trial with limited sample size, this publication follows current research ethics that claim the significance of discussing such kind of studies before the realization of a fully powered research project (Arain et al., 2010; Thabane et al., 2010; Leon et al., 2011).”
BackgroundElms, B., & Higgins, A. R. (2022) · Educational Psychology in Practice · doi:10.1080/02667363.2022.2045906 · Cited by 6
“A pilot study was conducted independently to explore the feasibility of the research approach used, following which appropriate amendments were made (Leon et al., 2011).”
MethodKhazanov, G. K., Jager‐Hyman, S., Harrison, J. D., Candon, M. K., Buttenheim, A. M., Pieri, M. F., Oslin, D. W., & Wolk, C. B. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01131-y · Cited by 4
Corpas‐López, V., Díaz‐Sáez, V., Morillas‐Márquez, F., et al. (2022) · Animals · doi:10.3390/ani12192700 · Cited by 4
“This is usually the aim of the pilot, exploratory or proof of concept trials [15,16].”
BackgroundCella, M., Tomlin, P., Robotham, D., Green, P. A., Griffiths, H., Stahl, D. R., & Valmaggia, L. (2023) · Efficacy and Mechanism Evaluation · doi:10.3310/hnvc2041 · Cited by 5
Wong, A., Tomazin, R., Walker, K., Desai, R. H., Hollingsworth, H. H., Newland, P., & Morgan, K. A. (2023) · Disability and health journal · doi:10.1016/j.dhjo.2023.101549 · Cited by 4
Little, M. A., Reid, T. A., MONCRIEF, M. A., Cohn, W. F., Wiseman, K. P., Wood, C. H., You, W., Anderson, R. T., & Krukowski, R. A. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01465-9 · Cited by 4
“Feasibility of randomization was determined by the number of smokers that were approached and screened in order to randomize 24 smokers (e.g., ineligible smokers, smokers that do not consent) [45, 46].”
BackgroundSzota, K., van der Meer, A. S., Bourdeau, T., Chorpita, B. F., Chavanon, M., & Christiansen, H. (2024) · Scientific Reports · doi:10.1038/s41598-024-67407-w · Cited by 4
“These are important observations, given that one goal of our pilot implementation study was to determine which measures might be appropriate for a larger implementation evaluation (see 48 ).”
Choice versus no choice: Practical considerations for increasing choices
Kim, J. Y., Morris, C., Ellsworth, M. E., et al. (2024) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.2920 · Cited by 4
“These experimenters may have used arbitrarily selected numbers or arrangements to assess the feasibility of an assessment procedure or the implementation of novel interventions (Leon et al., 2011).”
The term ‘Pilot Study’ is misused in veterinary medicine: a critical assessment
Rishniw, M., & White, M. E. (2019) · Veterinary Record · doi:10.1136/vr.105377 · Cited by 4
Linking Root Words and Derived Forms for Adult Struggling Readers:A Pilot Study
Gray, S. S. (2019) · ADULT LITERACY EDUCATION THE INTERNATIONAL JOURNAL OF LITERACY LANGUAGE AND NUMERACY · doi:10.35847/sgray.1.1.19 · Cited by 4
Evaluating Customer Satisfaction Using the ACSI Model in a Themed Restaurant in Bangkok
Mao, M., & James, P. T. (2020) · Business Management and Strategy · doi:10.5296/bms.v11i1.16546 · Cited by 4
Zlámal, J., Gjevjon, E. R., Fossum, M., Steindal, S. A., & Nes, A. A. G. (2021) · JMIR Research Protocols · doi:10.2196/31646 · Cited by 4
Pasqualucci, P. L., Seifert, G., Filho, V. O., & dos Santos, A. C. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01176-z · Cited by 3
“A sample of 40 participants recruited is expected to achieve the goal of 30 participants who complete the study, which may be sufficient to provide useful information about the feasibility of the protocol [46, 47].”
BackgroundWard, L., Stebbings, S. M., Athens, J., Cherkin, D. C., & Baxter, D. G. (2014) · Physical Therapy Reviews · doi:10.1179/1743288x14y.0000000139 · Cited by 4
Green, J., Jester, R., McKinley, R. K., & Pooler, A. (2018) · British Journal of Community Nursing · doi:10.12968/bjcn.2018.23.sup6.s20 · Cited by 4
“Application of the Wilcoxon Signed Rank Test converts scores at each time interval to ranks (Z) and compares them, allowing for the calculation of an effect size (Lancaster et al, 2002; Thabane, 2010; Leon et al, 2011).”
MethodDeLacy, J., Saheb, R., Rutherford, E., et al. (2019) · Journal of the Australian and New Zealand Student Services Association · doi:10.30688/janzssa.2019.05 · Cited by 4
Nymberg, P., Calling, S., Stenman, E., Palmér, K., Hansson, E. E., Sundquist, K., Sundquist, J., & Zöller, B. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00810-6 · Cited by 4
Lamprecht, A. (2018) · doi:10.51415/10321/3079 · Cited by 4
Francis, J. D. (2015) · ScholarWorks (Walden University) · Cited by 4
Evaluating the feasibility of a multicenter teleneonatology clinical effectiveness trial
Fang, J. L., Umoren, R. A., Whyte, H. E. A., Limjoco, J. J., Makkar, A., Behl, S., Lo, M. D., White, L., Čuljat, M., Taylor, J. S., et al. (2023) · Pediatric Research · doi:10.1038/s41390-023-02659-2 · Cited by 3
“20 They especially bene fi tted from gaining experience in maintaining a subject identi fi cation linking log, collecting data and entering it into the de-identi fi ed study database, using source documents, and undergoing trial monitoring procedures.”
BackgroundMethodExposure Therapy Consortium: Outcomes of the Proof-of-Principle Study
Smits, J. A. J., Abramowitz, J. S., Anderson, R., Arch, J. J., Badeja, D., Barzilay, S., Belanger, A., Borchert, T., Bryant, E., Burger, A. S., et al. (2024) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2024.102962 · Cited by 3
“Specifically, early small-sample studies often serve primarily to assess feasibility - such as screening, recruitment, treatment fidelity and adherence (Leon et al., 2010).”
Usability of Videoconferencing for Physical Exercise Interventions in Older Adults: Scoping Review
Robin, L., Mandigout, S., Batcho, C. S., Gelineau, A., & Borel, B. (2025) · JMIR Rehabilitation and Assistive Technologies · doi:10.2196/65552 · Cited by 4
Conde, C., Zatzick, D., Wong, D., Russo, J. E., Wang, J., Engstrom, A., & Hernandez, A. H. (2025) · Psychiatry · doi:10.1080/00332747.2025.2503031 · Cited by 3
Zinzow, H. M., Pericot‐Valverde, I., Smalls, L., Brancato, M. G., Chapman, G., Smith, A., Thompson, A., Greco, C., Shank, M., Eichelberger, K. Y., et al. (2025) · JMIR Research Protocols · doi:10.2196/68673 · Cited by 3
“Interviews will be recorded, transcribed, and uploaded to NVivo (Lumivero) [85] for thematic analysis [86].”
Stowe, C. L., Kennedy, K., Emilson, S. S., Neiberg, R. H., Kritchevsky, S. B., Miller, M. E., Houston, D. K., Nicklas, B. J., Fanning, J. T., & Rejeski, W. J. (2025) · Contemporary Clinical Trials · doi:10.1016/j.cct.2025.108049 · Cited by 3
Raley, S. K., Edwards, B., Shogren, K. A., Bumble, J. L., Hicks, T. A., Beuoy, A. M., Ferreira, H. R., Dean, E. E., & Mosconi, M. W. (2025) · Exceptionality · doi:10.1080/09362835.2025.2593667 · Cited by 4
“These results underscore the value of Bayesian inference in small-sample research, where traditional null hypothesis testing is often unfeasible in pilot studies (Leon et al., 2011).”
Major, M., Polyák, M., Würsching, T., et al. (2025) · Oral · doi:10.3390/oral5040098 · Cited by 3
“The sample size (n = 20; 10 per group) was chosen based on feasibility considerations and aligned with accepted principles for pilot study design, which prioritizes procedural refinement and estimation of effect sizes for future confirmatory trials [28].”
Lafortune, D., Lapointe, V. A., Canivet, C., Bonneau, J., Hassan, G., Boislard, M.-A., Calazana, F., Labrie, C., & Dubé, S. (2026) · Archives of Sexual Behavior · doi:10.1007/s10508-025-03356-3 · Cited by 3
Srisawet, S., Mesukko, J., Niyomkar, S., et al. (2026) · Journal of Neonatal Nursing · doi:10.1016/j.jnn.2026.101767 · Cited by 3
Fasting-Based Dietary Interventions in Cancer Patients and Survivors: A Scoping Review
Wen, K., Freedman, J., Tafese, A. M., et al. (2026) · Nutrients · doi:10.3390/nu18071035 · Cited by 3
“Pilot studies are preparatory investigations designed to test feasibility, acceptability and operational readiness rather than providing evidence of effectiveness [42].”
Jake‐Schoffman, D. E., Seoane, M. C., Cooper, K. B., Rajoria, M., & McVay, M. A. (2022) · Psychology and Health · doi:10.1080/08870446.2022.2087073 · Cited by 3
“This sample size is consistent with published guidance for pilot trials (hertzog, 2008; leon et al., 2011; Moore et al., 2011; Thabane et al., 2010).”
ResultCo-creation through non-dyadic service experiences using service design
Gottlieb, U. R., Beatson, A., Chamorro‐Koc, M., & Camilleri, B. A. (2022) · Heliyon · doi:10.1016/j.heliyon.2022.e10318 · Cited by 3
“To maximise objectivity of responses and reduce research bias, any participant who completed the preliminary activities for Stage 1, and therefore was involved in the development of the material used for the cognitive map activity of Stage 2, was excluded from Stage 2 (Leon et al., 2011).”
BackgroundBroome, S., Williams, K. W., & Hendrix, K. H. (2022) · Journal of Food Allergy · doi:10.2500/jfa.2022.4.220035 · Cited by 4
Oakley, B. F. M., Boatman, C. A., Doswell, S., Dittner, A. J., Clarke, A. D., Ozsivadjian, A., Kent, R., Judd, A., Baldoza, S. M., Hearn, A., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286792 · Cited by 3
“As this is a pilot study, we considered approximately 100 participants to be both a pragmatic (in terms of recruitment/ onboarding rate) and sufficient sample size to examine key feasibility targets—such as, establishing the proportion of those eligible who consent to take part, the proportion of participants who cont…”
BackgroundPromising Impact of Telenovela Intervention for Caregivers of Hospice Patients: A Pilot Study
Cruz-Oliver, D. M., Milner, G. E., Mensh, K., Bugayong, M. T., Blinka, M. D., Durkin, N., Saylor, M. A., Budhathoki, C. B., & Oliver, D. P. (2024) · American Journal of Hospice and Palliative Medicine® · doi:10.1177/10499091241228835 · Cited by 3
“6,7 This pilot study aims to explore NOVELA ’ s estimate of intervention effect and feasibility 15,16 factors needed for a randomized control trial (RCT).”
Patterson, M., Campbell–Adams, T., & Johnson, B. (2024) · Radiography · doi:10.1016/j.radi.2024.02.016 · Cited by 3
von Klinggraeff, L., Burkart, S. A., Pfledderer, C. D., McLain, A. C., Armstrong, B., Weaver, R. G., & Beets, M. W. (2024) · Translational Behavioral Medicine · doi:10.1093/tbm/ibae009 · Cited by 3
“Preliminary health behavioral interventions were defined as studies designed to test the feasibility of a behavioral intervention and/or provide evidence of a preliminary effect(s) in the hypothesized direction [4, 11, 12].”
BackgroundEFFECT OF CORE STRENGTHENING FOR KNEE OSTEOARTHRITIS ON BALANCE, PAIN AND FUNCTION: A PILOT STUDY
Lam, J. E., Mokhtar, A. H., & Mohafez, H. (2023) · Journal of Health and Translational Medicine · doi:10.22452/jummec.vol26no1.18 · Cited by 3
Swanson, C. W., Vial, S. E., Manini, T. M., Sibille, K. T., & Clark, D. J. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01516-1 · Cited by 3
“…rate (per month), randomization (screen eligible who enroll), retention rate, and compliance (percent of completed intervention sessions) [38] Secondary outcome measures • Questionnaires ABC scale [39], MSRS [40], PSQI [41], MIQ-RS [42], SF-36 [43], Katz ADL [44], CHAMPS [45],…”
BackgroundMethodParamedic-Assisted Community Evaluation After Discharge: The PACED Intervention
O'Connor, L. C., Sison, S. D. M., Eisenstock, K., et al. (2024) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2024.105165 · Cited by 3
Caregiver Wellness after Traumatic Brain Injury (CG-Well): Protocol for a randomized clinical trial
Kreitzer, N. P., Fink, S., Adeoye, O. M., Kurowski, B. G., Wade, S. L., Sucharew, H. J., & Bakas, T. (2024) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2024.101356 · Cited by 3
“This sample size was also determined by balancing realistic recruitment estimates and minimal sample requirements for planned analyses [53].”
MethodLevey, N., Chen, N. C., Ditre, J. W., Sylvia, L. G., Mudgal, C. S., Bhashyam, A. R., Garg, R., Ring, D., Vranceanu, A., & Bakhshaie, J. (2024) · JMIR Research Protocols · doi:10.2196/64547 · Cited by 3
“We will also execute exploratory correlations to test the processes (eg, within-group changes in risky substance use and pain or disability) [57-59].”
The impact of university housing on student success in higher education: the Ghanaian perspective
Attiogbe, E. J. K., Agyabeng, A. N., Preko, A., et al. (2024) · SN Social Sciences · doi:10.1007/s43545-024-01025-1 · Cited by 3
“This pilot phase not only assessed the feasibility of the research, as suggested by Leon et al. (2011 ), but also allowed for fine-tuning of the items.”
Das, J., Barry, G., Walker, R. W., Vitório, R., Çelik, Y., McDonald, C., Storey, B., Oman, P., Morris, R., & Stuart, S. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000696 · Cited by 4
Chan, S. F., Patten, J., & Lau, N. (2025) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01590-5 · Cited by 3
Furulund, E., Carlsen, S.-E. L., Alpers, S. E., Vold, J. H., Druckrey-Fiskaaen, K. T., Madebo, T., Dyrstad, S. M., Lid, T. G., & Fadnes, L. T. (2025) · Addiction Science & Clinical Practice · doi:10.1186/s13722-025-00583-w · Cited by 3
“Neverthe-less, this study provides important information relating to intervention experiences and feasibility of the intervention [48, 49].”
Metadata
- Authors
- Leon, A. C., Davis, L. L., & Kraemer, H. C.
- Affiliations
- Weill Cornell Medicine (US)
- University of Alabama (US)
- Stanford University (US)
- Corresponding author
- Leon, Andrew C.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2011-05
- Volume
- 45
- Issue
- 5
- Pages
- 626-629
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC3081994
- OpenAlex
- W2168434488
- Cited by
- 1,917
- References
- 17
- Citations by year
- 2026: 203 · 2025: 190 · 2024: 143 · 2023: 169 · 2022: 163 · 2021: 217 · 2020: 169 · 2019: 136 · 2018: 128 · 2017: 120 · 2016: 92 · 2015: 80 · 2014: 52 · 2013: 31 · 2012: 17
- FWCI
- 14.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pilot study, feasibility assessment, clinical trials, research design, sample size determination, effect size estimation, generalizability, recruitment and retention, intervention implementation, research funding review
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Feasibility Studies, Humans, Pilot Projects, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
