The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The role and interpretation of pilot studies in clinical research
Open accessAbstract
Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 5 No basis given · 2 No full text of the sources · 1 Sources not found
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“When event rates are reported and no adverse event is observed for a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
What the sources say
- Not matchedcites Jovanovic & Levy, 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Jovanovic et al., 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Basing a decision to proceed or terminate evaluation of a particular intervention on pilot data is perilous because there is a very good chance that the decision will be derived from false positive or false negative results.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each aspect of feasibility could be quite different from an uncontrolled study when intervention assignment is randomized and blinded, particularly if placebo is a distinct possibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these issues in mind, a pilot study can better address its goals if a control group is part of the design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A case series estimate tends to be steeped in optimism, particularly if an estimate of such magnitude is seldom, if ever, seen in full scale trials for psychiatric disorders.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(It is the population effect size, not the sample effect size, that an RCT is designed to detect with sufficient power.)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There has been a venerable tradition of using pilot studies to estimate between group effect sizes that, in turn, are used to inform the design of subsequent larger scale hypothesis testing studies. Despite it widespread use, it has been argued that the tradition is ill-founded. (Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot sample size is instead based on the pragmatics of recruitment and the necessities for examining feasibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With no inferential statistical tests, a pilot study will not provide p-values.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot study data generally should not be combined with data from the subsequent larger scale study. This is because it is quite likely that the methods will be modified after the pilot, even if minimally. Such changes in protocol risk adding additional, perhaps unknown, source of variation.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Are Pilot Data Included in the Larger Trial?
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Methods will likely be modified after a pilot study, even if minimally.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 13
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study.”
- Also stated in: § DISCUSSION · plain
“Pilot results inform feasibility, which in turn, is instructive in that it points to modifications needed in the planning and design of a larger efficacy trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The definition of clinically meaningful is not entirely empirically-based, but instead requires input from clinicians who treat the patient population of interest and perhaps from patients with the disorder.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These good clinical practices include the refinement of source documentation, informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring/oversight procedures, especially when multiple sites and investigators are engaged in the study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety, efficacy and effectiveness are not evaluated in a pilot.”
- Also stated in: § DISCUSSION · plain
“a pilot study is not a hypothesis testing study and therefore safety and efficacy are not evaluated.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot studies generate pilot data, their design need not be guided with the support of prior pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible, but highly unlikely, that the between group effect size (d) from a pilot study sample will provide a reasonable estimate of the population effect size (Δ), but that cannot be known based on the pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study can be critical in research staff training and provide experiences that strengthen and confirm competencies and skills required for the investigation to be conducted with accuracy and precision.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An alternative approach is to base sample size estimates for a full scale RCT on what is deemed a clinically meaningful effect.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > An Alternative to Using Pilot Data for Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study provides opportunity to develop consistent practices to enhance data integrity and the protection of human subjects.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We learn little if anything about the efficacy of an intervention with data from a small sample; yet, as discussed earlier, a great deal can be learned from a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This applies to effects sizes for any type of outcome, be it a severity rating, a response status, or survival status.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, with imprecision comes a vast disparity in sample size estimates and, if sample size determination for a subsequent study is based on an imprecise estimate, there is an enormous risk of underpowered or overpowered design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, pilot studies are a necessary first step in exploring novel interventions and novel applications of interventions – whether in a new patient population or with a novel delivery system (e.g., transdermal patch).”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although a pilot study will undoubtedly incorporate relevant outcome measures and can serve a vital role in treatment development, it is not, and should not, be considered a preliminary test of the intervention hypothesis.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The primary role of a pilot study is to examine the feasibility of a research endeavor.”
- Also stated in: § INTRODUCTION · plain
“The fundamental purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to ultimately be used in a larger scale study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This does not preclude the need for a theoretical rationale for the intervention or the methodology being proposed for a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a pilot study effect size is unduly large (i.e., a false positive result), subsequent trials will be designed with an inadequate number of participants to provide the statistical power needed to detect clinically meaningful effects and that would lead to negative trials.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, pilot studies provide an opportunity to implement and examine the feasibility of the adverse event reporting system.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is quite reasonable and expected that a pilot study is proposed with no pilot or other preliminary data supporting the proposal and that its proposed sample size is based on pragmatics such as patient flow and budgetary constraints.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention.”
- Also stated in: § INTRODUCTION · plain
“evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and/or implementation of the novel intervention.”
- Also stated in: § WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY · plain
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, and implementation of the novel intervention and each of these can be quantified”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are two fundamental reasons that hypothesis testing is not used in a pilot study: the limited state of knowledge about the methods or intervention in the patient population to be studied and the smaller proposed sample size.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a consequence, pilot studies can reduce the proportion of failed trials and allow research funds to be spent on projects for which feasibility has been demonstrated and quantified.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Well-conceived and implemented pilot studies will reduce the risk of several problems that are commonly faced in clinical trials. These include the inability to recruit the proposed sample size and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures or interventions, and the need to modify a protocol midway through a trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Estimates become more precise with larger sample sizes.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The true population effect size is unknown when a pilot study is designed.
“An essential challenge of therapeutic development is that the true population effect size is unknown at the time a pilot study is designed.”
The paper relies on this without support.
Methods will likely be modified after a pilot study, even if minimally.
“This is because it is quite likely that the methods will be modified after the pilot, even if minimally.”
The paper relies on this without support.
Pilot results presented to grant review committees tend to be selective, overly optimistic, and at times misrepresentational.
“Furthermore, pilot results that are presented to grant review committees tend to be selective, overly optimistic and, at times, misrepresentational.”
The paper relies on this without support.
Many pilot studies have far fewer than 18 participants per group.
“Many pilot studies have far fewer than 18 participants/group and therefore even greater imprecision.”
The paper relies on this without support.
It is not unusual for research grant applications and pilot study publications to convey optimism based on pilot data.
“Nevertheless, it is not unusual for research grant applications and pilot study publications to convey such optimism, particularly when based on pilot data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The planned sample size was 30 allowing for attrition. This study will inform the sample size for any future RCT (Leon et al., 2011).”
From A mixed methods pilot and feasibility open trial of internet-delivered cognitive behaviour therapy (iCanADAPT Advanced) for people with advanced cancer with depression and/or anxiety (2021) · cites this paper as Leon et al., 2011
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was a pilot study which like other pilot studies of MI did not have a power calculation. 31,32”
From A Pilot Randomized Controlled Trial to Assess the Impact of Motivational Interviewing on Initiating Behavioral Therapy for Migraine (2020) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data analyses in the context of a pilot study are not hypothesis-driven but serve to provide information regarding feasibility and acceptability of the intervention [55].”
From Acceptance and Commitment Therapy Wellness Program for Latine Adults Who Smoke and Have Psychological Distress: Protocol for a Feasibility Study (2023) · cites this paper as [55]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
From PandaMom – Feasibility and acceptability of an internet- and mobile-based intervention to enhance peripartum mental well-being and to prevent postpartum depression (2024) · cites this paper as Leon et al., 2011
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our use of effect sizes was, however, deemed appropriate for this pilot trial (55, 63).”
From EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU (2024) · cites this paper as [63]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work suggests early studies often test feasibility of methods (e.g., recruitment, randomization, retention, and assessment), not hypotheses [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
Citations
Cites
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · doi:10.1007/978-1-4757-2915-3 · Cited by 873
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · cited 1×
“(Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006) Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small samples.”
Self-citationWolfstein (1905) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-190502000-00009 · Cited by 346
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · cited 1×
“(Leon et al., 2006; Leon & Davis, 2009).”
Self-citationSixteen S ‐squared over D ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · cited 1×
“(N/group ≈ 16/d2; Lehr, 1992) The clinical interpretation of HAMD change of 1.2 to 3.0 would drive the choice among possible sample sizes in planning a study.”
BackgroundInfluentialJovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.1080/00031305.1997.10473947 · Cited by 179 · cited 1×
“…a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n. (Jovanovic & Levy, 1997; Jovanovic et al., 1997) For example, consider a study with N=15 receiving medication and no suicidal ideation was reported…”
BackgroundHarvey, I. M. (2009) · Public Health · doi:10.1016/j.puhe.2009.05.009 · Cited by 173
Jovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.2307/2685405 · Cited by 50
Cited by
Sorted by citations
Items 801–900 of 1,903
McKeon, G., Palmer, E. E., Macintosh, R., Nevin, S. M., Wheatley, L., & Rosenbaum, S. (2021) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2021.108022 · Cited by 8
Blood-flow restricted exercise following ankle fractures — A feasibility study
Larsen, P. F., Platzer, O. J., Lollesgaard, L. F., Pedersen, S. K., Nielsen, P. K. A., Rathleff, M. S., Bandholm, T., Jensen, S. T., & Elsøe, R. (2021) · Foot and Ankle Surgery · doi:10.1016/j.fas.2021.08.010 · Cited by 11
Pilot Study of the MAP Curriculum for Psychotherapy Competencies in Child and Adolescent Psychiatry
Becker, K. D., Daleiden, E. L., Kataoka, S., Edwards, S. M., Best, K. M., Donohue, A., & Chorpita, B. F. (2021) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20210010 · Cited by 8
Hamadi, L., Hilson, R., Lunn, A., Ralph, E., Rodrigues, E., & Sohail, R. (2023) · Eating Disorders · doi:10.1080/10640266.2022.2099606 · Cited by 8
“However, effect sizes from pilot studies are not considered to be reliable due to the very small sample sizes (Leon et al., 2011).”
BackgroundGriggs, A. C., Lazzara, E. H., Doherty, S. M., et al. (2022) · Simulation & Gaming · doi:10.1177/10468781221123595 · Cited by 7
Battle, C. L., Cardemil, E. V., Rossi, R., O’Hara, M. W., & Miller, I. W. (2022) · Archives of Women s Mental Health · doi:10.1007/s00737-022-01282-0 · Cited by 12
vanDellen, M. R., Wright, J. W. C., Zhao, B., Cullinan, C. A., Beach, S. R. H., Shen, Y., Haskins, L. B., Schiavone, W. M., & MacKillop, J. M. (2023) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntad183 · Cited by 7
Barudin-Carreiro, A. M., Camhi, S. M., Stanish, H. I., & Wright, J. A. (2024) · Children · doi:10.3390/children11030341 · Cited by 8
“Before moving to a randomized, controlled trial, pilot studies are a necessary first step in exploring the feasibility of recruitment, randomization, intervention implementation, assessment procedures, and retention [42].”
MethodMsimango, L., Butterfield, R. M., Starks, T. J., van Heerden, A. C., Neilands, T. B., Hahn, J., Chibi, B., Humphries, H., & Conroy, A. A. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-083390 · Cited by 7
The role of value co-creation on retailer loyalty and omni-channel shopping frequency
Kesenduran, N., Hüseyinoğlu, I. Ö. Y., & Erboz, G. (2024) · International Journal of Retail & Distribution Management · doi:10.1108/ijrdm-10-2023-0622 · Cited by 9
Virtual Reality for Subacute Pain After Orthopedic Traumatic Musculoskeletal Injuries
Mace, R. A., Brewer, J. R., Cohen, J. E., Ly, T. V., Weaver, M. J., & Borsook, D. (2024) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000001231 · Cited by 7
“Consistent with pilot study guidelines, 44,45 we used nonparametric tests when appropriate and refrained from interpreting statistical signi fi cance in the results.”
InfluentialMetal contaminant risk at active floating photovoltaic sites and future research roadmap
Akomea-Ampeh, M., Atekwana, E. A., Steele, E. P., et al. (2025) · Journal of Environmental Management · doi:10.1016/j.jenvman.2025.125216 · Cited by 7
Feasibility of a Mind-Body Program for Chronic Pain
Greenberg, J., Hooker, J. E., McDermott, K. A., Camera, D. E. L., Brewer, J. R., Szapary, C. L., Somers, T. J., Keefe, F. J., Kelleher, S. A., Fisher, H. M., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.15685 · Cited by 7
Sandroff, B. M., Motl, R. W., Wylie, G. R., Wells, G. E., Wender, C. L. A., Picone, M. A., Pandey, K. S., DeLuca, J., & Cutter, G. (2025) · Brain and Cognition · doi:10.1016/j.bandc.2025.106349 · Cited by 8
Fiorentini, G., Sarti, D., Patriti, A., et al. (2020) · World Journal of Clinical Oncology · doi:10.5306/wjco.v11.i6.397 · Cited by 7
Chandler, J., Christon, L., Benfield, K., Pairet, S., Hoffman, M., Treiber, F. A., Mueller, M., & James, W. E. (2023) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2023.101062 · Cited by 7
“Thus, sample size was determined for pragmatic reasons such as assessment of recruitment, drop-out and adherence rates [69].”
BackgroundTuthill, E. L., Maltby, A. E., Odhiambo, B. C., Hoffmann, T. J., Nyaura, M., Shikari, R., Cohen, C. R., & Weiser, S. D. (2023) · International Breastfeeding Journal · doi:10.1186/s13006-023-00600-1 · Cited by 7
“Sample size was determined based on standardized feasibility procedures [20, 21].”
MethodLarkin, L., McKenna, S. G., Pyne, T., Comerford, P., Moses, A., Folan, A., Gallagher, S. M., Glynn, L. G., Fraser, A., Esbensen, B. A., et al. (2024) · Rheumatology International · doi:10.1007/s00296-024-05544-1 · Cited by 7
“This sample size was expected to provide sufficient data to meet the primary outcomes and was in the appropriate range for pilot studies [36].”
Educators’ experiences of teaching learners with hearing loss in inclusive classrooms
Moustache, H. M. T. E., & Makhoba, M. (2024) · South African Journal of Childhood Education · doi:10.4102/sajce.v14i1.1358 · Cited by 18
“A pilot study was conducted with two participants, following the same recruitment and data collection process as the main study (Leon, Davis & Kraemer 2011).”
MethodMildner, S., Hotz, I., Kübler, F., Rausch, L. K., Stampfer‐Kountchev, M., Panzl, J., Brenneis, C., & Seebacher, B. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304788 · Cited by 8
“Hence, we did not conduct hypothesis testing to assess treatment effects within or between groups [44].”
MethodDusek, J. A., Kallenberg, G., Storrow, A. B., Hughes, R. M., Coyne, C. J., Vago, D. R., Nielsen, A., Karasz, A., Kim, R. S., Surdam, J., et al. (2024) · Integrative Medicine Research · doi:10.1016/j.imr.2024.101095 · Cited by 9
Pharmacotherapy of Anorexia Nervosa: More Questions than Answers
Hebebrand, J. (2011) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2011.06.016 · Cited by 7
Mohamed, N., Leung, T. M., Shah, Q. N., Pisipati, S., Berry, D. L., Benn, E. K. T., Lee, C. T., Hall, S. J., Mehrazin, R., & Sfakianos, J. P. (2019) · Journal of Cancer Education · doi:10.1007/s13187-019-01534-1 · Cited by 8
Wilcox, S. L. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22881 · Cited by 7
Danesh, V. C., Zuniga, J. A., Timmerman, G. M., Radhakrishnan, K., Cuevas, H. E., Young, C. C., Henneghan, A. M., Morrison, J., & Kim, M. T. (2020) · Applied Nursing Research · doi:10.1016/j.apnr.2020.151345 · Cited by 8
“The success of pilot and feasibility trials can contribute to refining the subsequent implementation of larger trials to avoid wasting researchers’ and participants’ efforts, to addressing participant burden, and to justifing expanded research endeavors (Leon et al., 2011).”
BackgroundBaig, M. R., Beck, R. D., Wilson, J. L., Lemmer, J. A., Meraj, A., Meyer, E. C., Mintz, J., Peterson, A. L., & Roache, J. D. (2020) · Military Medical Research · doi:10.1186/s40779-020-00278-0 · Cited by 7
de Nooijer, K., Pivodic, L., Noortgate, N. J. V. D., Pype, P. F., & den Block, L. V. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043663 · Cited by 7
Gabriel, I., Creedy, D., & Coyne, E. (2021) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2021.100802 · Cited by 7
Wierenga, K. L., Fresco, D. M., Alder, M. L., Sattar, A., & Moore, S. M. (2021) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0000000000000837 · Cited by 8
“We did not seek to identify efficacy between groups with statistical significance as this study was not powered for efficacy testing but was rather a pilot study.(67) As our sample size was small, we utilized Cohen f effect size(68) to characterize findings along with more traditional assessment of probability values.”
MethodWhiteside, L., Huynh, L., Morse, S. C., Hall, J. P., Meurer, W. J., Banta‐Green, C. J., Scheuer, H., Cunningham, R. J., McGovern, M. P., & Zatzick, D. (2021) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2021.108666 · Cited by 7
Chu, W., Becker, K. D., Boustani, M. M., Park, A. L., & Chorpita, B. F. (2022) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2021.11.003 · Cited by 7
Lester, E. G., Fishbein, N. S., Peterson, A., & Vranceanu, A. (2022) · PEC Innovation · doi:10.1016/j.pecinn.2022.100076 · Cited by 7
“These limitations are similar to other pilot studies [33], including several of our own previous studies in NF [10,19].”
ResultJones, H. J., Kovacic, M. B., Lambert, J., Almallah, W. R., Becker, R. C., Fuentes, L. D. L., & Bakas, T. (2022) · Translational Behavioral Medicine · doi:10.1093/tbm/ibac075 · Cited by 9
Wolff, J. L., Cagle, J. G., Echavarria, D. M., Dy, S. M., Giovannetti, E. R., Boyd, C. M., Hanna, V., Hussain, N., Reiff, J. S., Scerpella, D. L., et al. (2023) · Contemporary Clinical Trials · doi:10.1016/j.cct.2023.107208 · Cited by 9
Bonar, E. E., Goldstick, J. E., Tan, C. Y., Bourque, C. A., Carter, P. M., Duval, E. R., McAfee, J., & Walton, M. A. (2023) · Addictive Behaviors · doi:10.1016/j.addbeh.2023.107829 · Cited by 7
Idrus, S. B., Nizam, N. Z., Liew, Y. S., et al. (2023) · International Journal of Professional Business Review · doi:10.26668/businessreview/2023.v8i7.2070 · Cited by 7
Vance, D. E., Fazeli, P. L., Azuero, A., Frank, J., Wadley, V. G., Raper, J. L., Pope, C. N., Jacob, A., & Ball, K. (2024) · Journal of the Association of Nurses in AIDS Care · doi:10.1097/jnc.0000000000000449 · Cited by 7
Bernardo, D. F. S., Bobadilla-Agouborde, C., Festas, C., Carvalho, C., Abdalla, P. P., Amezcua‐Prieto, C., Naia, Z., Mesquita, C., Mota, J. A. P. S., & Santos, P. C. (2024) · Clinical and Experimental Obstetrics & Gynecology · doi:10.31083/j.ceog5103070 · Cited by 9
“This study was expected to have 24 participants, as the pilot study did not require a large sample size [36].”
ResultNavas, J. M. M., Escandón, C. L., Herrero, J., Bozal, R. G., & la Torre, G. G. D. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01148 · Cited by 7
“Following the suggestions from Leon et al. (2011) about how to proceed in pilot studies, we prefer to address objectives rather than research questions and hypothesis.”
BackgroundMethodSmith, D. C., Ureche, D. J., Davis, J. P., & Walters, S. T. (2014) · Substance Abuse · doi:10.1080/08897077.2014.988838 · Cited by 7
“These are all important indices for determining the feasibility of conducting a larger clinical trial.(50) Additionally, independent-samples t tests and chi-square analyses were used to determine whether the participants differed across conditions on numerous demographic, substance use, and psychosocial variables at b…”
Pacheco-Velázquez, S. C., Gallardo‐Pérez, J. C., Aguilar-Ponce, J. L., Villarreal, P., Ruiz-Godoy, L. M., Pérez-Sánchez, M., Marín‐Hernández, Á., Ruíz‐García, E., Meneses-García, A., Moreno‐Sánchez, R., et al. (2014) · International Journal of Oncology · doi:10.3892/ijo.2014.2676 · Cited by 7
From pilot studies to confirmatory studies.
Duan, N. (2013) · PubMed Central · doi:10.3969/j.issn.1002-0829.2013.05.011 · Cited by 7
“The framing of pilot studies has received extensive discussion in recent mental health statistics literature,([1,2]) leading to major additions to the directives provided in several recent program announcements from the U.”
BackgroundLotfi, A. S., Eftekhari, H., Atreya, A. R., Kashikar, A., Sivalingam, S. K., Giannoni, M., Visintainer, P., & Engelman, D. T. (2016) · World Journal of Cardiology · doi:10.4330/wjc.v8.i10.615 · Cited by 7
Miller, C. K., Pawelczyk, K. M., Cheavens, J. S., Fujita, K., & Moss, S. A. (2017) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2017.1377618 · Cited by 9
Irvine, L. A., Crombie, I. K., Swanson, V., Dimova, E. D., Melson, A. J., Fraser, T. M., Barbour, R. S., Rice, P. M., & Allan, S. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193434 · Cited by 7
“Although a feasibility study should not attempt to estimate the effectiveness of the intervention, even if a control arm is present [65], there are other benefits of including one [67].”
BackgroundMoreira, N. C. F., Dutra, K. L., Feres, M. F. N., et al. (2018) · Journal of Dental Education · doi:10.21815/jde.018.065 · Cited by 7
Izumi, S., Basin, B., Presley, M., et al. (2018) · Journal of Palliative Medicine · doi:10.1089/jpm.2017.0666 · Cited by 7
A review of prospective trial registration in the Journal of Advanced Nursing in 2018
Gray, R. J., Brown, E., & Gray, G. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.14090 · Cited by 7
Jackson, M. A., Brown, A. L., Baker, A., Gould, G. S., & Dunlop, A. J. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032330 · Cited by 7
Higa-McMillan, C. K., Nakamura, B. J., Daleiden, E. L., Edwall, G. E., Nygaard, P., & Chorpita, B. F. (2020) · Journal of Family Social Work · doi:10.1080/10522158.2019.1694341 · Cited by 7
Agile science: what and how in digital diabetes research
Nelson, L. A., Threatt, A. L., Martínez, W., Acuff, S. W., & Mayberry, L. S. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-817485-2.00004-3 · Cited by 8
McMaster, C. M., Wade, T. D., Basten, C., Franklin, J., Ross, J., & HART, S. G. E. (2020) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-020-00955-y · Cited by 7
Hubbard, E. A., Motl, R. W., & Elmer, D. J. (2020) · Trials · doi:10.1186/s13063-020-04887-x · Cited by 9
Tajdini, M., Aminorroaya, A., Tavolinejad, H., Tofighi, S., Jalali, A., Sadeghian, S., Vasheghani‐Farahani, A., Yadangi, S., Shahmansouri, N., Akhondzadeh, S., et al. (2021) · IJC Heart & Vasculature · doi:10.1016/j.ijcha.2021.100789 · Cited by 7
“We reported primary efficacy and safety endpoints in this randomized controlled trial and also presented a subgroup analysis; nonetheless, this is a pilot study which mainly delineates the feasibility of a larger study [29,30].”
MethodMathew, A. S., Rech, M. A., & Lee, H. (2021) · Journal of Behavioral Addictions · doi:10.1556/2006.2021.00031 · Cited by 7
Chow, W. S. E., & Sharma, U. (2022) · International Journal of Inclusive Education · doi:10.1080/13603116.2022.2118379 · Cited by 7
“Before administering the TPSN questionnaire to a large group of participants, a pilot test was conducted to examine the procedures and recruitment (Leon, Davis, and Kraemer 2011).”
Harned, M., Dhami, M., & Reger, G. M. (2023) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-023-00299-1 · Cited by 7
Blanton, S., Clark, P. C., Lyles, R. H., Cotsonis, G., Jones, B. D., Reiss, A., Wolf, S. L., & Dunbar, S. B. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0439-0 · Cited by 7
Schmidt-Hantke, J., Vollert, B., Nacke, B., Hagner, F., Brüderl, H., & Jacobi, C. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100765 · Cited by 7
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
BackgroundSippel, L. M., Khalifian, C. E., Knopp, K. C., Webster, K., Maglione, J., Holcomb, J., Flanagan, J. C., Monson, C. M., Holtzheimer, P. E., & Morland, L. A. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.03.001 · Cited by 7
Cho, Y., Jung, Y. L., Im, A., Hong, S. J., & Kim, K. (2025) · Pain Management Nursing · doi:10.1016/j.pmn.2024.12.010 · Cited by 8
Achuodho, H. O., Berki, T., & Piko, B. F. (2025) · Education Sciences · doi:10.3390/educsci15020135 · Cited by 10
Costela-Ruiz, V. J., González-Vigil, E., Ibáñez, O. E., Alcázar-Caballero, R. M., Melguizo‐Rodríguez, L., Fernández-López, O., & Arias‐Santiago, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0323173 · Cited by 8
“Finally, given the exploratory nature of this study and resource constraints, a smaller sample size is ethical and appropriate at this early stage of research development [82].”
Fanning, J. T., Rejeski, W. J., Nicklas, B. J., Kritchevsky, S. B., Miller, M. E., Houston, D. K., Walkup, M. P., Kennedy, K., Stowe, C. L., Ford, S. A., et al. (2025) · Clinical Interventions in Aging · doi:10.2147/cia.s520273 · Cited by 8
Revising the Concept of a Perceived Brand Globalness: Beyond Broad Market Reach
Jadach, R., & Tłuczak, A. (2025) · Journal of International Consumer Marketing · doi:10.1080/08961530.2025.2507647 · Cited by 7
“Such an approach is valid because the purpose of a pilot study is to evaluate the feasibility of applying the planned method in the target large-scale study (Leon, Davis, and Kraemer 2011).”
Kiyingi, J., Ssewamala, F. M., Neilands, T. B., Tebbetts, S., Mulauzi, N., Mkandawire, J., & Conroy, A. A. (2025) · BMC Public Health · doi:10.1186/s12889-025-23911-w · Cited by 6
Taylor, P. J., Adeyemi, I., Marlow, K., Cottam, S., Airnes, Z., Howells, V., Dunn, B. D., Elliott, R. A., Hann, M., Latham, C., et al. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116859 · Cited by 6
Karnieli‐Miller, O., & Elyoseph, Z. (2026) · Advances in Simulation · doi:10.1186/s41077-026-00429-8 · Cited by 7
“Consistent with established guidance for pilot studies of novel interventions [44] this study employed a prospective, single arm, pre-post design, which incorporates both quantitative and qualitative measures.”
Collins, L. M. (2018) · Statistics for social and behavioral sciences · doi:10.1007/978-3-319-72206-1_2 · Cited by 6
Exercise and PTSD Symptoms in Emergency Service and Frontline Medical Workers: A Systematic Review
McKeon, G., Steel, Z., Wells, R., Fitzpatrick, A., Vancampfort, D., & Rosenbaum, S. (2021) · Translational Journal of the American College of Sports Medicine · doi:10.1249/tjx.0000000000000189 · Cited by 6
“The primary outcome of the pilot study, however, was feasibility, and therefore it was underpowered to detect a significant change in PTSD symptoms (53).”
BackgroundResultThomas, K. S., Dosa, D. M., Fisher, A., Gadbois, E. A., Harrison, J., Hilgeman, M. M., Largent, E. A., Lima, J. C., McAuliff, K., McCreedy, E. M., et al. (2022) · Contemporary Clinical Trials · doi:10.1016/j.cct.2022.106897 · Cited by 6
Newman, S. D., Toatley, S., Rodgers, M. D., Qanungo, S., Mueller, M., Denny, B., & Rodriguez, A. (2022) · JMIR Research Protocols · doi:10.2196/42688 · Cited by 6
“…of feasibility studies, the objective of this study was not to test the efficacy of the PHOENIX intervention but to determine whether the novel telehealth intervention and the study design could be feasibly implemented, and to obtain estimates of the variability of relevant outcome measures [15].”
InfluentialChandler, R. D., Farinu, O., Guillaume, D., Francis, S., Parker, A. G., Shah, K., Hernandez, N., & Science, S. S. I. (2023) · JMIR Research Protocols · doi:10.2196/42712 · Cited by 5
Liu, S., Wegner, L., Haucke, M. N., Gates, J., Adam, M., & Bärnighausen, T. (2023) · JMIR Formative Research · doi:10.2196/43036 · Cited by 5
“Our sample size was at least 50 participants per arm, which would be sufficient to evaluate the study design feasibility [57-60].”
Koskan, A. M., Larkey, L., Todd, M. W., & Kim, S. W. (2024) · PLoS ONE · doi:10.1371/journal.pone.0299787 · Cited by 6
“Formal and informal testing of intervention efficacy testing is inappropriate in feasibility studies as is the derivation of effect size estimates [34–37].”
BackgroundGopalakrishnan, L., Mulauzi, N., Mkandawire, J., Ssewamala, F. M., Tebbetts, S., Neilands, T. B., & Conroy, A. A. (2025) · SSM - Population Health · doi:10.1016/j.ssmph.2025.101768 · Cited by 7
“…designed to test health effects due to the need for design adjustments to optimize recruitment, retention, and participation (Kraemer et al., 2006; Leon et al., 2011; Moore et al., 2011; Pilot Studies, n.d.; Thabane et al., 2010), we conducted an analysis to explore potential effects on attitudes…”
Hilbert, A., Klotz, U., Sadeghi, S., Juarascio, A. S., & Kirsten, T. (2025) · International Journal of Eating Disorders · doi:10.1002/eat.24479 · Cited by 8
“While reliable sample size estimations cannot be derived from this study (Leon et al. 2011), design and methods may inform a future confirmatory RCT.”
Versluis, A., Verkuil, B., Spinhoven, P., & BROSSCHOT, J. F. (2018) · British Journal of Guidance and Counselling · doi:10.1080/03069885.2017.1421310 · Cited by 6
“As small pilot studies are believed to yield biased estimators of effect sizes (Leon et al., 2011), we believe that it is pivotal to examine the effectiveness of the currently developed, theory-based EMI in a RCT using the active control group as well as a waitlist control group.”
BackgroundRecruitment for exercise or physical activity interventions: a protocol for systematic review
Hoover, J. C., Alenazi, A. M., Alothman, S., Alshehri, M. M., RUCKER, J. L., & Kluding, P. M. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019546 · Cited by 6
“These studies often examine whether the research design, intervention and even recruitment methods are reasonable to complete.(24) These studies also require a financial budget, although they often require less time, money and resources.”
BackgroundUnick, J. L., Pellegrini, C. A., DUNSIGER, S. I., Demos, K. E., Thomas, J. G., Bond, D. S., Webster, J., & Wing, R. R. (2020) · Contemporary Clinical Trials · doi:10.1016/j.cct.2020.105953 · Cited by 7
Blumen, H. M., Ayers, E. I., Wang, C., Ambrose, A. F., & Verghese, J. (2020) · Neurodegenerative Disease Management · doi:10.2217/nmt-2020-0002 · Cited by 9
Woodward, E. N., Willging, C. E., Landes, S. J., Hausmann, L. R. M., Drummond, K. L., Ounpraseuth, S., Ball, I. A., & Kirchner, J. E. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-050107 · Cited by 7
Pott, S. L., Kellett, S., Green, S. A., Daughters, S. B., & Delgadillo, J. (2022) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2022.108769 · Cited by 7
Shah, R. P., Herekar, V., Metgud, D., Kim, H., Atkins, M. S., & Dhaded, S. M. (2022) · European Journal of Pediatrics · doi:10.1007/s00431-022-04485-w · Cited by 6
“Feasibility studies are recommended as a critical step prior to more robust effectiveness testing because they collect formative data regarding the potential of a proposed intervention to be relevant and sustainable, such as implementation and acceptability [29, 30].”
BackgroundBaig, M. R., Wilson, J. L., Beck, R. D., Lemmer, J. A., Hernandez, A. L., Meraj, A., Tapia, R. N., Meyer, E. C., Mintz, J., Peterson, A. L., et al. (2022) · Journal of Behavioral and Cognitive Therapy · doi:10.1016/j.jbct.2022.04.001 · Cited by 6
Swindle, T., Rutledge, J. M., Zhang, D., Martin, J., Johnson, S. L., Selig, J. P., Yates, A. M., Gaulden, D., & Curran, G. M. (2022) · Nutrients · doi:10.3390/nu14142861 · Cited by 7
Liu, J., Li, L., Luo, X., Qin, X., Zhao, L., Zhao, J., Zhou, X., Liu, Y., Deng, K., Ma, Y., et al. (2022) · Acupuncture in Medicine · doi:10.1177/09645284221117848 · Cited by 7
Campbell, S. B., Rau, H., Fortney, J. C., Batten, L., & Simpson, T. L. (2022) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2022.08.003 · Cited by 6
Technology-Based Smoking Cessation for Youth Exiting Foster Care: A Pilot Randomized Trial
Braciszewski, J. M., Colby, S. M., Franklin, M. J., Stout, R. L., Vose-O’Neal, A., & Bock, B. C. (2022) · Journal of Public Child Welfare · doi:10.1080/15548732.2022.2155754 · Cited by 7
“Following the seminal guidelines in this area (Leon, Davis, & Kraemer, 2011), feasibility and acceptability were measured through multiple components.”
MethodRethorst, C. D., Trombello, J. M., Chen, P. M., Carmody, T. J., Lazalde, A., & Trivedi, M. H. (2023) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2023.101103 · Cited by 6
“[50] suggests the primary goal of pilot studies is to determine the feasibility for an approach to be used in a subsequent larger clinical trial.”
BackgroundPfledderer, C. D., von Klinggraeff, L., Burkart, S. A., Bandeira, A. D. S., Armstrong, B., Weaver, R. G., Adams, E. L., & Beets, M. W. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01389-w · Cited by 9
Giummarra, M. J., Reeder, S. C., Williams, S., Devlin, A. L., Knol, R., Ponsford, J., Arnold, C. A., Konstantatos, A., Gabbe, B. J., Clarke, H. A., et al. (2023) · Disability and Rehabilitation · doi:10.1080/09638288.2023.2254235 · Cited by 6
Okulicz‐Kozaryn, K., Segura, L., Bruguera, C., et al. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1243904 · Cited by 6
“Although our project is a pilot–focused on feasibility, process, and reaching different groups, rather than focusing purely on the effectiveness of a single prescriptive approach (52, 53), it No change 58 1 Followupriskyalcoholuse < Baselineriskyalcoholuse.”
Nguyen, M. X., Li, C., Muessig, K. E., Gaynes, B. N., & Go, V. F.-L. (2023) · AIDS and Behavior · doi:10.1007/s10461-023-04166-1 · Cited by 6
“Although the pilot phase is a requisite first step to test new interventions, pilot studies have critical limitations in their ability to draw inferences due to small sample sizes and convenient recruitment methods, and their results should be interpreted with caution [82].”
Bu, Z., Wan, S.-R., Steinmann, P., Yin, Z.-T., Tan, J.-P., Li, W., Tang, Z.-Y., Jiang, S., Ye, M.-M., Xu, J.-Y., et al. (2024) · Journal of Cancer · doi:10.7150/jca.91301 · Cited by 8
“Lastly, due to the restricted sample size, certain potential therapeutic impacts or adverse reactions may remain undetected, possibly influencing the refinement and enhancement of treatment strategies [55, 56].”
BackgroundTowards a new protocol of scoliosis assessments and monitoring in clinical practice: A pilot study
Lukovic, T. Z., Ćuković, S., Luković, V., et al. (2015) · Journal of Back and Musculoskeletal Rehabilitation · doi:10.3233/bmr-140574 · Cited by 6
“In this stage of the research, we performed measurements on healthy subjects, in order to examine the feasibility of an approach that is intended to be used in a larger scale study [66].”
Promoting Healthy Nutrition Behavior Using Mobile Devices and Ubiquitous Computing
Besoaín, F., Pérez-Navarro, A., Ojeda, F., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-26410-3_9 · Cited by 6
Hellum, R., Jensen, S., & Nielsen, A. S. (2017) · Nordic Studies on Alcohol and Drugs · doi:10.1177/1455072517696507 · Cited by 6
Larkin, L., Gallagher, S. M., Fraser, A., & Kennedy, N. (2017) · Rheumatology International · doi:10.1007/s00296-017-3850-y · Cited by 6
“The behaviour change wheel (BCW), a theoretical framework which explicitly integrates behavioural theory with the development and description of behavioural change interventions [17], facilitates the identification of factors which should be targeted in the behaviour change intervention.”
Enhancing student competency in risky clinical environments: Evaluating an online education program
Graj, E., Sheen, J. A., Dudley, A. L., Sutherland‐Smith, W., & McGillivray, J. (2018) · Australian Psychologist · doi:10.1111/ap.12364 · Cited by 6
Metadata
- Authors
- Leon, A. C., Davis, L. L., & Kraemer, H. C.
- Affiliations
- Weill Cornell Medicine (US)
- University of Alabama (US)
- Stanford University (US)
- Corresponding author
- Leon, Andrew C.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2011-05
- Volume
- 45
- Issue
- 5
- Pages
- 626-629
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC3081994
- OpenAlex
- W2168434488
- Cited by
- 1,917
- References
- 17
- Citations by year
- 2026: 203 · 2025: 190 · 2024: 143 · 2023: 169 · 2022: 163 · 2021: 217 · 2020: 169 · 2019: 136 · 2018: 128 · 2017: 120 · 2016: 92 · 2015: 80 · 2014: 52 · 2013: 31 · 2012: 17
- FWCI
- 14.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pilot study, feasibility assessment, clinical trials, research design, sample size determination, effect size estimation, generalizability, recruitment and retention, intervention implementation, research funding review
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Feasibility Studies, Humans, Pilot Projects, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
