The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The role and interpretation of pilot studies in clinical research
Open accessAbstract
Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 5 No basis given · 2 No full text of the sources · 1 Sources not found
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“When event rates are reported and no adverse event is observed for a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
What the sources say
- Not matchedcites Jovanovic & Levy, 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Jovanovic et al., 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Basing a decision to proceed or terminate evaluation of a particular intervention on pilot data is perilous because there is a very good chance that the decision will be derived from false positive or false negative results.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each aspect of feasibility could be quite different from an uncontrolled study when intervention assignment is randomized and blinded, particularly if placebo is a distinct possibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these issues in mind, a pilot study can better address its goals if a control group is part of the design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A case series estimate tends to be steeped in optimism, particularly if an estimate of such magnitude is seldom, if ever, seen in full scale trials for psychiatric disorders.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(It is the population effect size, not the sample effect size, that an RCT is designed to detect with sufficient power.)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There has been a venerable tradition of using pilot studies to estimate between group effect sizes that, in turn, are used to inform the design of subsequent larger scale hypothesis testing studies. Despite it widespread use, it has been argued that the tradition is ill-founded. (Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot sample size is instead based on the pragmatics of recruitment and the necessities for examining feasibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With no inferential statistical tests, a pilot study will not provide p-values.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot study data generally should not be combined with data from the subsequent larger scale study. This is because it is quite likely that the methods will be modified after the pilot, even if minimally. Such changes in protocol risk adding additional, perhaps unknown, source of variation.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Are Pilot Data Included in the Larger Trial?
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Methods will likely be modified after a pilot study, even if minimally.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 13
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study.”
- Also stated in: § DISCUSSION · plain
“Pilot results inform feasibility, which in turn, is instructive in that it points to modifications needed in the planning and design of a larger efficacy trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The definition of clinically meaningful is not entirely empirically-based, but instead requires input from clinicians who treat the patient population of interest and perhaps from patients with the disorder.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These good clinical practices include the refinement of source documentation, informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring/oversight procedures, especially when multiple sites and investigators are engaged in the study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety, efficacy and effectiveness are not evaluated in a pilot.”
- Also stated in: § DISCUSSION · plain
“a pilot study is not a hypothesis testing study and therefore safety and efficacy are not evaluated.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot studies generate pilot data, their design need not be guided with the support of prior pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible, but highly unlikely, that the between group effect size (d) from a pilot study sample will provide a reasonable estimate of the population effect size (Δ), but that cannot be known based on the pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study can be critical in research staff training and provide experiences that strengthen and confirm competencies and skills required for the investigation to be conducted with accuracy and precision.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An alternative approach is to base sample size estimates for a full scale RCT on what is deemed a clinically meaningful effect.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > An Alternative to Using Pilot Data for Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study provides opportunity to develop consistent practices to enhance data integrity and the protection of human subjects.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We learn little if anything about the efficacy of an intervention with data from a small sample; yet, as discussed earlier, a great deal can be learned from a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This applies to effects sizes for any type of outcome, be it a severity rating, a response status, or survival status.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, with imprecision comes a vast disparity in sample size estimates and, if sample size determination for a subsequent study is based on an imprecise estimate, there is an enormous risk of underpowered or overpowered design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, pilot studies are a necessary first step in exploring novel interventions and novel applications of interventions – whether in a new patient population or with a novel delivery system (e.g., transdermal patch).”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although a pilot study will undoubtedly incorporate relevant outcome measures and can serve a vital role in treatment development, it is not, and should not, be considered a preliminary test of the intervention hypothesis.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The primary role of a pilot study is to examine the feasibility of a research endeavor.”
- Also stated in: § INTRODUCTION · plain
“The fundamental purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to ultimately be used in a larger scale study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This does not preclude the need for a theoretical rationale for the intervention or the methodology being proposed for a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a pilot study effect size is unduly large (i.e., a false positive result), subsequent trials will be designed with an inadequate number of participants to provide the statistical power needed to detect clinically meaningful effects and that would lead to negative trials.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, pilot studies provide an opportunity to implement and examine the feasibility of the adverse event reporting system.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is quite reasonable and expected that a pilot study is proposed with no pilot or other preliminary data supporting the proposal and that its proposed sample size is based on pragmatics such as patient flow and budgetary constraints.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention.”
- Also stated in: § INTRODUCTION · plain
“evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and/or implementation of the novel intervention.”
- Also stated in: § WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY · plain
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, and implementation of the novel intervention and each of these can be quantified”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are two fundamental reasons that hypothesis testing is not used in a pilot study: the limited state of knowledge about the methods or intervention in the patient population to be studied and the smaller proposed sample size.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a consequence, pilot studies can reduce the proportion of failed trials and allow research funds to be spent on projects for which feasibility has been demonstrated and quantified.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Well-conceived and implemented pilot studies will reduce the risk of several problems that are commonly faced in clinical trials. These include the inability to recruit the proposed sample size and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures or interventions, and the need to modify a protocol midway through a trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Estimates become more precise with larger sample sizes.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The true population effect size is unknown when a pilot study is designed.
“An essential challenge of therapeutic development is that the true population effect size is unknown at the time a pilot study is designed.”
The paper relies on this without support.
Methods will likely be modified after a pilot study, even if minimally.
“This is because it is quite likely that the methods will be modified after the pilot, even if minimally.”
The paper relies on this without support.
Pilot results presented to grant review committees tend to be selective, overly optimistic, and at times misrepresentational.
“Furthermore, pilot results that are presented to grant review committees tend to be selective, overly optimistic and, at times, misrepresentational.”
The paper relies on this without support.
Many pilot studies have far fewer than 18 participants per group.
“Many pilot studies have far fewer than 18 participants/group and therefore even greater imprecision.”
The paper relies on this without support.
It is not unusual for research grant applications and pilot study publications to convey optimism based on pilot data.
“Nevertheless, it is not unusual for research grant applications and pilot study publications to convey such optimism, particularly when based on pilot data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The planned sample size was 30 allowing for attrition. This study will inform the sample size for any future RCT (Leon et al., 2011).”
From A mixed methods pilot and feasibility open trial of internet-delivered cognitive behaviour therapy (iCanADAPT Advanced) for people with advanced cancer with depression and/or anxiety (2021) · cites this paper as Leon et al., 2011
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was a pilot study which like other pilot studies of MI did not have a power calculation. 31,32”
From A Pilot Randomized Controlled Trial to Assess the Impact of Motivational Interviewing on Initiating Behavioral Therapy for Migraine (2020) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data analyses in the context of a pilot study are not hypothesis-driven but serve to provide information regarding feasibility and acceptability of the intervention [55].”
From Acceptance and Commitment Therapy Wellness Program for Latine Adults Who Smoke and Have Psychological Distress: Protocol for a Feasibility Study (2023) · cites this paper as [55]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
From PandaMom – Feasibility and acceptability of an internet- and mobile-based intervention to enhance peripartum mental well-being and to prevent postpartum depression (2024) · cites this paper as Leon et al., 2011
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our use of effect sizes was, however, deemed appropriate for this pilot trial (55, 63).”
From EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU (2024) · cites this paper as [63]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work suggests early studies often test feasibility of methods (e.g., recruitment, randomization, retention, and assessment), not hypotheses [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
Citations
Cites
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · doi:10.1007/978-1-4757-2915-3 · Cited by 873
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · cited 1×
“(Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006) Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small samples.”
Self-citationWolfstein (1905) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-190502000-00009 · Cited by 346
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · cited 1×
“(Leon et al., 2006; Leon & Davis, 2009).”
Self-citationSixteen S ‐squared over D ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · cited 1×
“(N/group ≈ 16/d2; Lehr, 1992) The clinical interpretation of HAMD change of 1.2 to 3.0 would drive the choice among possible sample sizes in planning a study.”
BackgroundInfluentialJovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.1080/00031305.1997.10473947 · Cited by 179 · cited 1×
“…a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n. (Jovanovic & Levy, 1997; Jovanovic et al., 1997) For example, consider a study with N=15 receiving medication and no suicidal ideation was reported…”
BackgroundHarvey, I. M. (2009) · Public Health · doi:10.1016/j.puhe.2009.05.009 · Cited by 173
Jovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.2307/2685405 · Cited by 50
Cited by
Sorted by citations
Items 201–300 of 1,903
Morin, K. H. (2013) · Journal of Nursing Education · doi:10.3928/01484834-20130920-10 · Cited by 66
Beidas, R. S., Becker‐Haimes, E. M., Adams, D. R., Skriner, L. C., Stewart, R. E., Wolk, C. B., Buttenheim, A. M., Williams, N. J., Inacker, P., Richey, E., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0684-7 · Cited by 43
“This is consistent with the primary role of a pilot study which is to examine the feasibility of a research endeavor [9].”
BackgroundAllicock, M. A., Kendzor, D. E., Sedory, A., Gabriel, K. P., Swartz, M. D., Thomas, P., Yudkin, J. S., & Rivers, A. (2020) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-020-00767-x · Cited by 40
“[47], feasibility was assessed 8 weeks post-baseline using the following criteria: (1) Retention: the percentage of participants who completed the study and EMAs and (2) Adherence: the number of days of completed selfmonitoring and number of valid wear days with the accelerometer.”
MethodInfluentialBiello, K. B., Hill-Rorie, J., Valente, P. K., Futterman, D. C., Sullivan, P. S., Hightow‐Weidman, L., Muessig, K. E., Dormitzer, J., Mimiaga, M. J., & Mayer, K. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25107 · Cited by 41
“Sample sizes for technical pilots are often determined based on practical considerations rather than inferential statistical power calculations [65].”
Effect of Morning and Evening Exercise on Energy Balance: A Pilot Study
Creasy, S. A., Wayland, L. T., Panter, S. L., Purcell, S. A., Rosenberg, R. C., Willis, E. A., Shiferaw, B. A., Grau, L., Breit, M. J., Bessesen, D. H., et al. (2022) · Nutrients · doi:10.3390/nu14040816 · Cited by 44
O’Connor, S. S., Comtois, K. A., Wang, J., Russo, J. E., Peterson, R. S., Lapping-Carr, L. R., & Zatzick, D. (2015) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.05.001 · Cited by 45
“The decision to randomize patients is informed by Leon et al [13], wherein we sought to measure randomization as a marker of feasibility for future efficacy trials.”
BackgroundMethodInfluentialPreliminary Study of Behavioral Therapy for Tourette Syndrome Patients in Japan
Nonaka, M., Matsuda, N., Kono, T., Fujio, M., Scahill, L. D., & Kano, Y. (2014) · Children s Health Care · doi:10.1080/02739615.2014.979922 · Cited by 41
“(2013). The awareness training in CBIT apparently did not increase subjective distress for these subjects (Scahill et al.”
BackgroundResultInfluentialExercise as an intervention for sedentary hazardous drinking college students: A pilot study
Weinstock, J., Capizzi, J. A., Weber, S. M., Pescatello, L. S., & Petry, N. M. (2014) · Mental health and physical activity · doi:10.1016/j.mhpa.2014.02.002 · Cited by 40
Dual task training in persons with Multiple Sclerosis: a feasability randomized controlled trial
Sosnoff, J. J., Wajda, D. A., Sandroff, B. M., et al. (2017) · Clinical Rehabilitation · doi:10.1177/0269215517698028 · Cited by 40
Smartphone delivery of a hope intervention: Another way to flourish
Daugherty, D. A., Runyan, J. D., Steenbergh, T. A., Fratzke, B. J., Fry, B. N., & Westra, E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0197930 · Cited by 45
Williams, J., Stubbs, B., Richardson, S., Flower, C., Barr-Hamilton, L., Grey, B., Hubbard, K., Spaducci, G., Gaughran, F., & Craig, T. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2274-5 · Cited by 44
Steinberg, D. M., Kay, M. C., Svetkey, L. P., Askew, S., Christy, J., Burroughs, J., Ahmed, H., & Bennett, G. G. (2020) · JMIR mHealth and uHealth · doi:10.2196/17536 · Cited by 41
“This is consistent with guidelines from Leon et al [21], which state that conducting power analyses is not needed when testing for feasibility.”
MethodResultIto, M., Horikoshi, M., Kato, N., Oe, Y., Fujisato, H., Nakajima, S., Kanie, A., Miyamae, M., Takebayashi, Y., Horita, R., et al. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.02.005 · Cited by 39
Hoey, E., Staines, A. S., Walsh, D., Corby, D., Bowers, K., Belton, S., Meegan, S., McVeigh, T., McKeon, M., Trépel, D., et al. (2016) · Journal of Intellectual Disabilities · doi:10.1177/1744629516657946 · Cited by 40
Verhelst, H., Linden, C. V., Vingerhoets, G., & Caeyenberghs, K. (2016) · Games for Health Journal · doi:10.1089/g4h.2016.0043 · Cited by 40
“42,43 There were nine feasibility measures: four re-lated to the novel training intervention (accessibility, training motivation, technical smoothness, and training compliance) and five concerning study design and procedures (participation willingness, participation rates, loss to follow-up,…”
Famelia, R., Tsuda, E., Bakhtiar, S., & Goodway, J. (2018) · Journal of Motor Learning and Development · doi:10.1123/jmld.2016-0072 · Cited by 33
Rehabilitation robotics after stroke: a bibliometric literature review
Zuccon, G., Lenzo, B., Bottin, M., & Rosati, G. (2022) · Expert Review of Medical Devices · doi:10.1080/17434440.2022.2096438 · Cited by 39
“As the purpose of a pilot study is to increase the likelihood of a successful future randomized controlled trial (RCT) by exploring the efficiency, internal validity and, fundamentally, the delivery of proposed trials [102],”
BackgroundDevelopment of an Intervention for Nonsuicidal Self-Injury in Young Adults: An Open Pilot Trial
Andover, M. S., Schatten, H. T., Morris, B. W., & Miller, I. W. (2014) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2014.05.003 · Cited by 38
“Although this is appropriate for an open pilot trial (Leon et al., 2011; Rounsaville et al., Cogn Behav Pract.”
MethodKim, S. H., Seong, D. H., Yoon, S. M., Choi, Y. D., Choi, E., Song, Y., & Song, H. (2017) · Cancer Nursing · doi:10.1097/ncc.0000000000000530 · Cited by 38
Corti, C., Poggi, G., Romaniello, R., Strazzer, S., Urgesi, C., Borgatti, R., & Bardoni, A. (2018) · PLoS ONE · doi:10.1371/journal.pone.0199001 · Cited by 40
Gallinat, C., Moessner, M., Haenssle, H. A., Winkler, J. K., Backenstrass, M., & Bauer, S. (2019) · Journal of Medical Internet Research · doi:10.2196/15011 · Cited by 38
“The sample size in this pilot trial was determined on the basis of practical considerations and questions of feasibility, as Leon et al [38] recommend for pilot studies.”
Wal, J. S. V., Maraldo, T. M., Vercellone, A. C., & Gagne, D. A. (2015) · Appetite · doi:10.1016/j.appet.2015.01.024 · Cited by 38
Shrestha, R., Altice, F. L., Khati, A., Azwa, I., Gautam, K., Gupta, S., Sullivan, P. S., Ni, Z., Kamarulzaman, A., Phiphatkhunarnon, P., et al. (2023) · JMIR mhealth and uhealth · doi:10.2196/44468 · Cited by 37
“RenderX sample size of 50 was determined based on the pragmatics of recruitment and the need to examine feasibility [32-34].”
MethodFactors associated with home hazards: Findings from the Malaysian Elders Longitudinal Research study
Romli, M. H., Tan, M. P., Mackenzie, L. A., Lovarini, M., Kamaruzzaman, S. B., & Clemson, L. M. (2017) · Geriatrics and gerontology international/Geriatrics & gerontology international · doi:10.1111/ggi.13189 · Cited by 37
McCall, A., Jones, M. D., Gelis, L., Duncan, C. S., Ehrmann, F., Dupont, G., & Duffield, R. (2017) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2017.12.002 · Cited by 37
Hutchesson, M., Taylor, R., Shrewsbury, V. A., Vincze, L. J., Campbell, L., Callister, R., Park, F. J., Schumacher, T. L., & Collins, C. E. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17165779 · Cited by 43
“A powered sample is not required for a pilot study [36], so a maximum target of 90 participants (45 per group) was set a priori, as this was anticipated to be feasible within the funding timeline and budget.”
BackgroundBeukes, E. W., Andersson, G., Fagelson, M., & Manchaiah, V. (2021) · American Journal of Audiology · doi:10.1044/2021_aja-20-00222 · Cited by 34
Cella, M., Tomlin, P., Robotham, D., Green, P. A., Griffiths, H., Stahl, D. R., & Valmaggia, L. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2022.07.013 · Cited by 38
Myers, K., Stoep, A. V., & Lobdell, C. (2013) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2013.0020 · Cited by 36
Westad, K., Tjoestolvsen, F., & Hebron, C. (2018) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2018.12.001 · Cited by 37
Reese, J. B., Lepore, S. J., Daly, M. B., Handorf, E. A., Sorice, K. A., Porter, L. S., Tulsky, J. A., & Beach, M. C. (2019) · Psycho-Oncology · doi:10.1002/pon.5036 · Cited by 36
“As the study was not powered for formal comparisons, we did not conduct statistical tests.(26,27)”
MethodPlumb Vilardaga, J. C., Winger, J. G., Teo, I., Owen, L., Sutton, L. M., Keefe, F. J., & Somers, T. J. (2020) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2019.09.005 · Cited by 36
“However, given that this was a pilot feasibility study, these analyses were exploratory and should be interpreted cautiously.(55)”
BackgroundKharasch, E. D., Brunt, L. M., Blood, J., & Komen, H. (2023) · Anesthesiology · doi:10.1097/aln.0000000000004663 · Cited by 36
Ford, J. D., Grasso, D. J., Greene, C. A., Slivinsky, M. D., & DeViva, J. C. (2018) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2194 · Cited by 37
Mobile App Use for Insomnia Self-Management: Pilot Findings on Sleep Outcomes in Veterans
Reilly, E. D., Robinson, S. A., Petrakis, B. A., Kuhn, E. R., Pigeon, W. R., Wiener, R. S., McInnes, D. K., & Quigley, K. S. (2019) · Interactive Journal of Medical Research · doi:10.2196/12408 · Cited by 35
“…impacted the effect sizes of the subjective and objective sleep findings; small sample sizes can drastically affect the ability to detect statistical power and reduce the likelihood that a statistically significant difference reflects a reproducible difference by overestimating effect size [82].”
BackgroundBombardier, C. H., Dyer, J. R., Burns, P. A., Crane, D. A., Takahashi, M. M., Barber, J. K., & Nash, M. S. (2020) · Spinal Cord · doi:10.1038/s41393-020-0523-6 · Cited by 39
The East Side Table Make-at-Home Meal-Kit Program is feasible and acceptable: A pilot study
Horning, M. L., Hill, T., Martin, C. L., Hassan, A., Petrovskis, A., & Bohen, L. (2020) · Appetite · doi:10.1016/j.appet.2020.105087 · Cited by 37
Vidovich, G. Z. D., Muffatti, R., Monaco, J., Caramia, N., Broglia, D., Caverzasi, E., Barale, F., & D'Angelo, E. U. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00582 · Cited by 38
“This is a pilot study, recently defined (Leon et al., 2011) as a requisite initial step in exploring a novel intervention or an innovative application of an intervention.”
BackgroundTexting to Increase Contraceptive Initiation Among Adolescents in the Emergency Department
Chernick, L. S., Stockwell, M. S., Wu, M., Castaño, P. M., Schnall, R., Westhoff, C. L., Santelli, J. S., & Dayan, P. S. (2017) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2017.07.021 · Cited by 35
“Our pilot sample size was based on recruitment pragmatics, such as our grant time frame and research coordinator availability [14].”
MethodAbbade, L. P. F., Abbade, J. F., & Thabane, L. (2018) · The Journal of venomous animals and toxins including tropical diseases · doi:10.1186/s40409-018-0142-2 · Cited by 36
“On the other hand, a definitive or main trial has a primary objective of determining the efficacy or effectiveness of a new intervention [4, 5].”
BackgroundKayrouz, R., Dear, B. F., Kayrouz, B., Karin, E., Gandy, M., & Titov, N. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1445124 · Cited by 33
Gibson, B. S., Gondoli, D. M., Kronenberger, W. G., Johnson, A. C., Steeger, C. M., & Morrissey, R. A. (2013) · Memory & Cognition · doi:10.3758/s13421-013-0295-8 · Cited by 33
Breastfeeding among Adolescent Mothers
Sipsma, H. L., Jones, K., & Cole-Lewis, H. J. (2014) · Journal of Human Lactation · doi:10.1177/0890334414561264 · Cited by 34
“We excluded pilot studies as they are designed primarily to assess feasibility and not for testing an intervention’s efficacy or effectiveness.(21) We also limited our review to studies conducted in high-income countries, as we believed these studies would be more generalizable to the United States than studies conduc…”
ResultAuslander, W., McGinnis, H., Tlapek, S. M., Smith, P., Foster, A., Edmond, T. E., & Dunn, J. (2016) · American Journal of Orthopsychiatry · doi:10.1037/ort0000233 · Cited by 34
“Because the current study was a pilot implementation study with a small sample size (N=27), statistical significance testing between conditions over time was problematic and likely to result in incorrect conclusions as discussed by Leon, Davis, & Kraemer (2011).”
Ritvo, P., Obadia, M., Mina, D. S., Alibhai, S. M. H., Sabiston, C. M., OH, P. I., Campbell, K. L., McCready, D. R., Auger, L. E., & Jones, J. M. (2017) · JMIR Research Protocols · doi:10.2196/resprot.6615 · Cited by 33
Bártolo, A., Pacheco, E., Rodrigues, F., Pereira, A., Monteiro, S. O. M., & Santos, I. M. (2017) · Disability and Rehabilitation · doi:10.1080/09638288.2017.1411534 · Cited by 34
“Telephonically delivered psycho-education [34] in the survival phase seems to have a medium effect size also in this domain, although this finding is limited by the small sample size in this study [39].”
MethodResultKamody, R. C., Thurston, I. B., Pluhar, E., Han, J. C., & Burton, E. T. (2018) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-018-0580-4 · Cited by 34
One-year pilot study on the effects of nitisinone on melanin in patients with OCA-1B
Adams, D. R., Menezes, S. M., Jauregui, R., Valivullah, Z. M., Power, B., Abraham, M., Jeffrey, B. G., Garced, A., Alur, R. P., Cunningham, D., et al. (2019) · JCI Insight · doi:10.1172/jci.insight.124387 · Cited by 34
“Discussion Pilot clinical trials may serve many purposes, all with the aim of informing the design of a larger, definitive clinical trial (26).”
Karp, J. F., Zhang, J., Wahed, A. S., Anderson, S., Dew, M. A., Fitzgerald, G. K., Weiner, D. K., Albert, S. M., Gildengers, A. G., Butters, M. A., et al. (2019) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2019.03.011 · Cited by 34
“The fundamental purpose of conducting an intervention development study, however, is to examine the feasibility of an approach for use in a confirmatory trial (47), and this goal was achieved based on our ability to: 1) successfully complete this project in both rehabilitation and late-life mental health clinics; 2) i…”
MethodChemical tools for epichaperome-mediated interactome dysfunctions of the central nervous system
Bolaender, A., Zatorska, D., He, H., Joshi, S., Sharma, S., Digwal, C. S., Patel, H. J., Sun, W., Imber, B. S., Ochiana, S. O., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-24821-2 · Cited by 34
McPherson, A. C., McAdam, L. C., Keenan, S., Schwellnus, H., Biddiss, E. A., DeFinney, A., & English, K. (2017) · Developmental Neurorehabilitation · doi:10.1080/17518423.2017.1289271 · Cited by 34
“For example, the study involved a small number of participants, although the sample was sufficient for the aims of feasibility work.(46) We could not test whether the increased goal scores were statistically significant as feasibility studies are not powered to detect such differences.”
BackgroundResultInfluentialUsing the Habit App for Weight Loss Problem Solving: Development and Feasibility Study
Pagoto, S., Tulu, B., Agu, E., Waring, M. E., Oleski, J. L., & Jake‐Schoffman, D. E. (2018) · JMIR mhealth and uhealth · doi:10.2196/mhealth.9801 · Cited by 33
“” As they and others recommend [29,30], we based the sample size on necessities for examining feasibility.”
Lindow, J. C., Hughes, J. L., South, C. C., Gutierrez, L., Bannister, E., Trivedi, M. H., & Byerly, M. (2019) · Archives of Suicide Research · doi:10.1080/13811118.2019.1624667 · Cited by 33
“Feasibility studies aim to measure factors important for intervention implementation (recruitment, data collection) and delivery (fidelity) to inform the design of future, larger-scale RCTs, which then definitively test the efficacy of the intervention (Bowen et al., 2009; Leon et al., 2011).”
O’Connor, T. M., Beltran, A., Musaad, S. M. A., Pérez, O., Flores, A., Galdamez-Calderon, E., Isbell, T., Arredondo, E. M., Parra‐Cardona, J. R., Cabrera, N., et al. (2020) · Childhood Obesity · doi:10.1089/chi.2020.0006 · Cited by 35
Juarascio, A. S., Parker, M. N., Hunt, R., Murray, H. B., Presseller, E. K., & Manasse, S. M. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23512 · Cited by 33
“Effect sizes are often unstable in pilot trials and must be interpreted with caution (Leon et al., 2011).”
BackgroundMethodMeyer, K., Glassner, A. A., Norman, R. S., James, D., Sculley, R., LealVasquez, L., Hepburn, K. W., Liu, J., & White, C. L. (2022) · Geriatric Nursing · doi:10.1016/j.gerinurse.2022.03.013 · Cited by 33
Newman, S. D., Gillenwater, G., Toatley, S., Rodgers, M. D., Todd, N. R., Epperly, D., & Andrews, J. O. (2014) · Disability and health journal · doi:10.1016/j.dhjo.2014.04.006 · Cited by 33
Sirey, J. A., Berman, J., Salamone, A., DePasquale, A., Halkett, A. E., Raeifar, E., Banerjee, S., BRUCE, M. L., & Raue, P. J. (2015) · Journal of Elder Abuse & Neglect · doi:10.1080/08946566.2015.1008086 · Cited by 34
Sylvia, L. G., Gold, A. K., Stange, J. P., Peckham, A., Deckersbach, T., Calabrese, J. R., Weiss, R. D., Perlis, R. H., Nierenberg, A. A., & Ostacher, M. J. (2016) · American Journal on Addictions · doi:10.1111/ajad.12346 · Cited by 33
Morris, N. S., & Rosenbloom, D. A. (2017) · AJN American Journal of Nursing · doi:10.1097/01.naj.0000513261.75366.37 · Cited by 33
“However, good work in these areas can help researchers obtain funding for larger studies, especially when feasibility has been demonstrated and quantified.(5) From subsequent larger randomized studies, researchers may be able to draw conclusions that can be generalized.”
BackgroundGuisado‐Vasco, P., Cardenal, G. D. L., Garrido, I. M., et al. (2016) · Internal and Emergency Medicine · doi:10.1007/s11739-016-1582-8 · Cited by 32
“It also evaluates the feasibility of recruitment, randomization, retention, assessment procedures, and implementation [16].”
BackgroundValue of surgical pilot and feasibility study protocols
Fairhurst, K., Blazeby, J. M., Potter, S., Gamble, C. L., Rowlands, C., & AVERY, K. N. L. (2019) · British journal of surgery · doi:10.1002/bjs.11167 · Cited by 33
Adu, M. D., Malabu, U. H., Malau‐Aduli, A. E. O., Drovandi, A. D., & Malau‐Aduli, B. S. (2020) · JMIR mhealth and uhealth · doi:10.2196/17802 · Cited by 32
Im, A., Yeon, S. H., Lee, J. S., Um, K. A., Ahn, Y. J., & Chae, S. (2016) · BMC Complementary and Alternative Medicine · doi:10.1186/s12906-016-1218-6 · Cited by 33
“A pilot study provides opportunity to enhance data integrity and the protection of human subjects in terms of toxicology [23].”
BackgroundThe conundrum of depression clinical trials: one size does not fit all
Khan, A., Mar, K. F., & Brown, W. A. (2018) · International Clinical Psychopharmacology · doi:10.1097/yic.0000000000000229 · Cited by 30
“As Leon et al. (2011) stated: ‘A pilot study is not a hypothesis testing study…[because] a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples.’”
BackgroundSaberi, P., McCuistian, C., Agnew, E., Wootton, A. R., Legnitto Packard, D. A., Dawson-Rose, C., Johnson, M. O., Gruber, V. A., & Neilands, T. B. (2021) · Telemedicine Reports · doi:10.1089/tmr.2020.0014 · Cited by 30
“with recommendations in the research methods literature and from NIH, inferential analyses were not performed on these pilot study data.(41,42) All analyses were conducted using Stata, version 15 (StataCorp, College Station, TX).”
MethodA social media intervention for cannabis use among emerging adults: randomized controlled trial
Bonar, E. E., Goldstick, J. E., Chapman, L., Bauermeister, J. A., Young, S. D., McAfee, J., & Walton, M. A. (2022) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2022.109345 · Cited by 32
Implementation and adherence issues in a workplace treadmill desk intervention
Tudor-Locke, C. E., Hendrick, C. A., Duet, M., et al. (2014) · Applied Physiology Nutrition and Metabolism · doi:10.1139/apnm-2013-0435 · Cited by 31
Rickard, C. M., Marsh, N. M., Webster, J., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015291 · Cited by 31
Erguera, X. A., Johnson, M. O., Neilands, T. B., Ruel, T., Berrean, B., Thomas, S., & Saberi, P. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030473 · Cited by 30
Barnett, M. L., Miranda, J. M., Kia‐Keating, M., Saldana, L., Landsverk, J. A., & Lau, A. S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-028988 · Cited by 32
“Pilot studies are limited in their ability to test effectiveness given small sample sizes, but they provide a critical phase of research design that can examine the feasibility of the approach to be used in a large-scale study.(64) A focus will be placed on measuring engagement outcomes at a client and agency levels t…”
BackgroundMcDonnell, K. K., Gallerani, D. G., Newsome, B. R., Owens, O. L., Beer, J. M., Myhren-Bennett, A. R., Regan, E. W., Hardin, J. W., & Webb, L. A. (2020) · Integrative Cancer Therapies · doi:10.1177/1534735420969829 · Cited by 32
Pasalich, D. S., Moretti, M. M., Hassall, A., & Curcio, A. L. (2021) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2021.105178 · Cited by 31
McKeon, G., Steel, Z., Wells, R., Newby, J., Hadzi-Pavlovic, D., Vancampfort, D., & Rosenbaum, S. (2021) · JMIR Formative Research · doi:10.2196/23432 · Cited by 29
Bricker, J. B., Sullivan, B. M., Mull, K. E., Santiago‐Torres, M., & Ferres, J. L. (2024) · JMIR mhealth and uhealth · doi:10.2196/57318 · Cited by 42
“Following expert recommendations for pilot RCT design [70,71], the feasibility outcomes were the study’s primary focus to inform the further development of QuitBot and design of a future full-scale trial of QuitBot.”
Smith, J. R., McGrath, J. M., Brotto, M., & Inder, T. E. (2014) · Advances in Neonatal Care · doi:10.1097/anc.0000000000000093 · Cited by 30
Cook, P. F., Schmiege, S. J., Mansberger, S. L., Sheppler, C. R., Kammer, J. A., Fitzgerald, T., & Kahook, M. Y. (2016) · Psychology and Health · doi:10.1080/08870446.2016.1244537 · Cited by 30
Compassion focused therapy for anger: A pilot study of a group intervention for veterans with PTSD
Grodin, J., Clark, J. L., Kolts, R. L., & Lovejoy, T. I. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.06.004 · Cited by 35
Glisenti, K., Strodl, E. S., King, R., & Greenberg, L. S. (2021) · Journal of Eating Disorders · doi:10.1186/s40337-020-00358-5 · Cited by 37
Jaffar, A., Sidik, S. M., Foo, C. N., Muhammad, N. A., Manaf, R. A., & Suhaili, N. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19042332 · Cited by 31
“It is not required to have a powered sample size for the pilot study [32].”
BackgroundDanielson, C. K., McCauley, J. L., Jones, A. M., Borkman, A. L., Miller, S., & Ruggiero, K. J. (2013) · AIDS Education and Prevention · doi:10.1521/aeap.2013.25.5.394 · Cited by 30
“In line with the state purposes of open pilot trials (Leon et al., 2011), participants were reimbursed for their time in completing the site as a component of evaluating the feasibility of methods; however, participant remuneration limits the ability of the current pilot to speak to the potential effectiveness of recr…”
BackgroundMethodInfluentialPromotion of Well-being During Treatment for Childhood Cancer
Derman, Y. E., & Deatrick, J. A. (2015) · Cancer Nursing · doi:10.1097/ncc.0000000000000318 · Cited by 30
“The 2 pilot studies(28,29) illustrate good-quality pilot technique: experiments integrated with operations, using pragmatic recruitment.(35) In the pediatric oncology treatment environment, there is difficulty with accruing, randomization, and retaining participants because of the fallout inherent in the morbidity of…”
BackgroundO'Connor, S. S., Mcclay, M. M., Choudhry, S., Shields, A. D., Carlson, R., Alonso, Y., Lavin, K., Venanzi, L., Comtois, K. A., Wilson, J. E., et al. (2020) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2018.08.001 · Cited by 29
Aimer, P., Treharne, G. J., Stebbings, S. M., Frampton, C. M., Cameron, V. A., Kirby, S., & Stamp, L. K. (2016) · Arthritis Care & Research · doi:10.1002/acr.22960 · Cited by 29
“Despite this, the sample size in this pilot study was adequate based on published guidelines (31,43).”
BackgroundMethodTelephone-assisted, parent-mediated CBT for rural Latino youth with anxiety: A feasibility trial.
Chavira, D. A., Bustos, C. E., Garcia, M., Segovia, F. A. R., Baig, A., Ng, B., & Camacho, A. (2018) · Cultural Diversity & Ethnic Minority Psychology · doi:10.1037/cdp0000186 · Cited by 29
Dorsey, S., Meza, R. D., Martin, P., Gray, C. L., Triplett, N. S., Soi, C., Woodard, G. S., Lucid, L., Amanya, C., Wasonga, A. I., et al. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00860 · Cited by 29
“analytic approach Given the sample size of 60 counselors, we followed best practices for small samples and did not conduct null hypothesis significance testing (43, 44).”
MethodMore pilot trials could plan to use qualitative data: a meta-epidemiological study
Baldeh, T., MacDonald, T. A., Kosa, S. D., Lawson, D. O., Stalteri, R., Olaiya, O. R., Alotaibi, A., Thabane, L., & Mbuagbaw, L. C. E. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00712-z · Cited by 29
“recommendations to use qualitative and quantitative methods concurrently for all pilot studies [5].”
MethodHaven, T. L., Bouter, L. M., Mennen, L. I., & Tijdink, J. K. (2022) · Accountability in Research · doi:10.1080/08989621.2022.2071153 · Cited by 43
“It should be noted that the purpose of our pilot study was to examine the feasibility of this new training method for implementation (Leon, Davis, and Kraemer 2011).”
Beets, M. W., Weaver, R. G., Ioannidis, J. P. A., Pfledderer, C. D., Jones, A., von Klinggraeff, L., & Armstrong, B. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02184-7 · Cited by 33
“For these reasons pilot/feasibility studies are often underpowered and not designed to yield effects expected in the larger trial [12].”
BackgroundMoradian, S., Walshe, C., Shahidsales, S., Nasiri, M. R., Pilling, M. A., & Molassiotis, A. (2014) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2014.10.016 · Cited by 30
“A sample size for a pilot study is usually based on pragmatic issues and the necessity of examining feasibility (Leon et al., 2011).”
BackgroundLong-Term Effects of Randomization to a Weight Loss Intervention in Older Adults: A Pilot Study
Houston, D. K., Miller, M. E., Kitzman, D. W., Rejeski, W. J., Messier, S. P., Lyles, M. F., Kritchevsky, S. B., & Nicklas, B. J. (2019) · Journal of Nutrition in Gerontology and Geriatrics · doi:10.1080/21551197.2019.1572570 · Cited by 29
“Since this was a pilot study, our analyses focused on the feasibility of recall, estimating means (SD), and longitudinal change (95% CI); hypothesis tests were not conducted on the pilot sample (39).”
MethodBecker, K. D., Park, A. L., Boustani, M. M., & Chorpita, B. F. (2019) · Journal of School Psychology · doi:10.1016/j.jsp.2019.07.013 · Cited by 33
Passive social sensing with smartphones: a systematic review
Zhang, H., Ibrahim, A., Bijan Parsia, A., Poliakoff, E., & Harper, S. (2022) · Computing · doi:10.1007/s00607-022-01112-2 · Cited by 32
“Usually, small-sample-size studies provide opportunities to enhance data integrity [94], but only one of the reviewed studies demonstrated the completeness of its data, which was 85.3% [27].”
BackgroundInfluentialMethodologies for optimizing behavioral interventions: introduction to special section
Riley, W. T., & Rivera, D. E. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0281-0 · Cited by 28
“estimate effect sizes for the larger study [9].”
BackgroundHuckels-Baumgart, S., Niederberger, M., Manser, T., Meier, C. R., & Meyer‐Massetti, C. (2017) · Journal of Nursing Management · doi:10.1111/jonm.12491 · Cited by 28
Mailhot, T., Goulet, M., Maheu‐Cadotte, M., Fontaine, G., Lequin, P., & Lavoie, P. (2019) · Journal of research in nursing · doi:10.1177/1744987119883404 · Cited by 30
Pruiksma, K. E., Taylor, D. J., Mintz, J., Nicholson, K. L., Rodgers, M., Young‐McCaughan, S., Hall‐Clark, B. N., Fina, B. A., Dondanville, K. A., Cobos, B. A., et al. (2020) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.8116 · Cited by 28
“49 Second, this study did not include detailed assessment of potential barrierstothetreatment,suchasdetailedhomeworkcompliance assessment or tracking of adherence to treatment for comorbid sleep disorders such as sleep apnea.”
Pope, Z., & Gao, Z. (2020) · Journal of American College Health · doi:10.1080/07448481.2020.1726925 · Cited by 31
Wells, R. D., Guastaferro, K., Azuero, A., Rini, C., Hendricks, B. A., Dosse, C., Taylor, R., Williams, G. R., Engler, S., Smith, C., et al. (2021) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.11.017 · Cited by 28
Sample size justifications in Gait & Posture
McCrum, C. L., van Beek, J., Schumacher, C., Janssen, S., & Hooren, B. V. (2021) · Gait & Posture · doi:10.1016/j.gaitpost.2021.12.010 · Cited by 27
Metadata
- Authors
- Leon, A. C., Davis, L. L., & Kraemer, H. C.
- Affiliations
- Weill Cornell Medicine (US)
- University of Alabama (US)
- Stanford University (US)
- Corresponding author
- Leon, Andrew C.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2011-05
- Volume
- 45
- Issue
- 5
- Pages
- 626-629
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC3081994
- OpenAlex
- W2168434488
- Cited by
- 1,917
- References
- 17
- Citations by year
- 2026: 203 · 2025: 190 · 2024: 143 · 2023: 169 · 2022: 163 · 2021: 217 · 2020: 169 · 2019: 136 · 2018: 128 · 2017: 120 · 2016: 92 · 2015: 80 · 2014: 52 · 2013: 31 · 2012: 17
- FWCI
- 14.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pilot study, feasibility assessment, clinical trials, research design, sample size determination, effect size estimation, generalizability, recruitment and retention, intervention implementation, research funding review
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Feasibility Studies, Humans, Pilot Projects, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
