The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The role and interpretation of pilot studies in clinical research
Open accessAbstract
Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 5 No basis given · 2 No full text of the sources · 1 Sources not found
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“When event rates are reported and no adverse event is observed for a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
What the sources say
- Not matchedcites Jovanovic & Levy, 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Jovanovic et al., 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Basing a decision to proceed or terminate evaluation of a particular intervention on pilot data is perilous because there is a very good chance that the decision will be derived from false positive or false negative results.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each aspect of feasibility could be quite different from an uncontrolled study when intervention assignment is randomized and blinded, particularly if placebo is a distinct possibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these issues in mind, a pilot study can better address its goals if a control group is part of the design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A case series estimate tends to be steeped in optimism, particularly if an estimate of such magnitude is seldom, if ever, seen in full scale trials for psychiatric disorders.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(It is the population effect size, not the sample effect size, that an RCT is designed to detect with sufficient power.)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There has been a venerable tradition of using pilot studies to estimate between group effect sizes that, in turn, are used to inform the design of subsequent larger scale hypothesis testing studies. Despite it widespread use, it has been argued that the tradition is ill-founded. (Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot sample size is instead based on the pragmatics of recruitment and the necessities for examining feasibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With no inferential statistical tests, a pilot study will not provide p-values.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot study data generally should not be combined with data from the subsequent larger scale study. This is because it is quite likely that the methods will be modified after the pilot, even if minimally. Such changes in protocol risk adding additional, perhaps unknown, source of variation.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Are Pilot Data Included in the Larger Trial?
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Methods will likely be modified after a pilot study, even if minimally.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 13
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study.”
- Also stated in: § DISCUSSION · plain
“Pilot results inform feasibility, which in turn, is instructive in that it points to modifications needed in the planning and design of a larger efficacy trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The definition of clinically meaningful is not entirely empirically-based, but instead requires input from clinicians who treat the patient population of interest and perhaps from patients with the disorder.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These good clinical practices include the refinement of source documentation, informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring/oversight procedures, especially when multiple sites and investigators are engaged in the study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety, efficacy and effectiveness are not evaluated in a pilot.”
- Also stated in: § DISCUSSION · plain
“a pilot study is not a hypothesis testing study and therefore safety and efficacy are not evaluated.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot studies generate pilot data, their design need not be guided with the support of prior pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible, but highly unlikely, that the between group effect size (d) from a pilot study sample will provide a reasonable estimate of the population effect size (Δ), but that cannot be known based on the pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study can be critical in research staff training and provide experiences that strengthen and confirm competencies and skills required for the investigation to be conducted with accuracy and precision.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An alternative approach is to base sample size estimates for a full scale RCT on what is deemed a clinically meaningful effect.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > An Alternative to Using Pilot Data for Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study provides opportunity to develop consistent practices to enhance data integrity and the protection of human subjects.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We learn little if anything about the efficacy of an intervention with data from a small sample; yet, as discussed earlier, a great deal can be learned from a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This applies to effects sizes for any type of outcome, be it a severity rating, a response status, or survival status.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, with imprecision comes a vast disparity in sample size estimates and, if sample size determination for a subsequent study is based on an imprecise estimate, there is an enormous risk of underpowered or overpowered design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, pilot studies are a necessary first step in exploring novel interventions and novel applications of interventions – whether in a new patient population or with a novel delivery system (e.g., transdermal patch).”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although a pilot study will undoubtedly incorporate relevant outcome measures and can serve a vital role in treatment development, it is not, and should not, be considered a preliminary test of the intervention hypothesis.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The primary role of a pilot study is to examine the feasibility of a research endeavor.”
- Also stated in: § INTRODUCTION · plain
“The fundamental purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to ultimately be used in a larger scale study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This does not preclude the need for a theoretical rationale for the intervention or the methodology being proposed for a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a pilot study effect size is unduly large (i.e., a false positive result), subsequent trials will be designed with an inadequate number of participants to provide the statistical power needed to detect clinically meaningful effects and that would lead to negative trials.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, pilot studies provide an opportunity to implement and examine the feasibility of the adverse event reporting system.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is quite reasonable and expected that a pilot study is proposed with no pilot or other preliminary data supporting the proposal and that its proposed sample size is based on pragmatics such as patient flow and budgetary constraints.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention.”
- Also stated in: § INTRODUCTION · plain
“evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and/or implementation of the novel intervention.”
- Also stated in: § WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY · plain
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, and implementation of the novel intervention and each of these can be quantified”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are two fundamental reasons that hypothesis testing is not used in a pilot study: the limited state of knowledge about the methods or intervention in the patient population to be studied and the smaller proposed sample size.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a consequence, pilot studies can reduce the proportion of failed trials and allow research funds to be spent on projects for which feasibility has been demonstrated and quantified.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Well-conceived and implemented pilot studies will reduce the risk of several problems that are commonly faced in clinical trials. These include the inability to recruit the proposed sample size and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures or interventions, and the need to modify a protocol midway through a trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Estimates become more precise with larger sample sizes.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The true population effect size is unknown when a pilot study is designed.
“An essential challenge of therapeutic development is that the true population effect size is unknown at the time a pilot study is designed.”
The paper relies on this without support.
Methods will likely be modified after a pilot study, even if minimally.
“This is because it is quite likely that the methods will be modified after the pilot, even if minimally.”
The paper relies on this without support.
Pilot results presented to grant review committees tend to be selective, overly optimistic, and at times misrepresentational.
“Furthermore, pilot results that are presented to grant review committees tend to be selective, overly optimistic and, at times, misrepresentational.”
The paper relies on this without support.
Many pilot studies have far fewer than 18 participants per group.
“Many pilot studies have far fewer than 18 participants/group and therefore even greater imprecision.”
The paper relies on this without support.
It is not unusual for research grant applications and pilot study publications to convey optimism based on pilot data.
“Nevertheless, it is not unusual for research grant applications and pilot study publications to convey such optimism, particularly when based on pilot data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The planned sample size was 30 allowing for attrition. This study will inform the sample size for any future RCT (Leon et al., 2011).”
From A mixed methods pilot and feasibility open trial of internet-delivered cognitive behaviour therapy (iCanADAPT Advanced) for people with advanced cancer with depression and/or anxiety (2021) · cites this paper as Leon et al., 2011
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was a pilot study which like other pilot studies of MI did not have a power calculation. 31,32”
From A Pilot Randomized Controlled Trial to Assess the Impact of Motivational Interviewing on Initiating Behavioral Therapy for Migraine (2020) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data analyses in the context of a pilot study are not hypothesis-driven but serve to provide information regarding feasibility and acceptability of the intervention [55].”
From Acceptance and Commitment Therapy Wellness Program for Latine Adults Who Smoke and Have Psychological Distress: Protocol for a Feasibility Study (2023) · cites this paper as [55]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
From PandaMom – Feasibility and acceptability of an internet- and mobile-based intervention to enhance peripartum mental well-being and to prevent postpartum depression (2024) · cites this paper as Leon et al., 2011
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our use of effect sizes was, however, deemed appropriate for this pilot trial (55, 63).”
From EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU (2024) · cites this paper as [63]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work suggests early studies often test feasibility of methods (e.g., recruitment, randomization, retention, and assessment), not hypotheses [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
Citations
Cites
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · doi:10.1007/978-1-4757-2915-3 · Cited by 873
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · cited 1×
“(Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006) Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small samples.”
Self-citationWolfstein (1905) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-190502000-00009 · Cited by 346
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · cited 1×
“(Leon et al., 2006; Leon & Davis, 2009).”
Self-citationSixteen S ‐squared over D ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · cited 1×
“(N/group ≈ 16/d2; Lehr, 1992) The clinical interpretation of HAMD change of 1.2 to 3.0 would drive the choice among possible sample sizes in planning a study.”
BackgroundInfluentialJovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.1080/00031305.1997.10473947 · Cited by 179 · cited 1×
“…a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n. (Jovanovic & Levy, 1997; Jovanovic et al., 1997) For example, consider a study with N=15 receiving medication and no suicidal ideation was reported…”
BackgroundHarvey, I. M. (2009) · Public Health · doi:10.1016/j.puhe.2009.05.009 · Cited by 173
Jovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.2307/2685405 · Cited by 50
Cited by
Sorted by citations
Items 401–500 of 1,903
Gamification to Motivate the Unmotivated Smoker: The “Take a Break” Digital Health Intervention
Blok, A. C., Sadasivam, R. S., Amante, D. J., Kamberi, A., Flahive, J., Morley, J., Conigliaro, J. C., & Houston, T. K. (2019) · Games for Health Journal · doi:10.1089/g4h.2018.0076 · Cited by 21
“The purpose of feasibility testing is to evaluate the study design and collect process measures (such as engagement in the intervention), but not clinical outcomes.(14,15) Implementing any new system requires careful planning, and testing the game elements in a real-world setting is helpful in highlighting areas that…”
BackgroundJohansson, H., Freidle, M., Ekman, U., Schalling, E., Leavy, B., Svenningsson, P., Hagströmer, M., & Franzén, E. (2020) · Parkinson s Disease · doi:10.1155/2020/2410863 · Cited by 24
“Using an active control group also enables a more realistic exploration of recruitment and retention rates, as well as randomization and blinding processes [33].”
Guadagni, S., Clementi, M., Masedu, F., Fiorentini, G., Sarti, D., D, M. D. M., D, S. K. M., Papasotiriou, I., Apostolou, P., Aigner, K. R., et al. (2020) · International Journal of Molecular Sciences · doi:10.3390/ijms21134813 · Cited by 21
“The scope of this EOC small cohort pilot study was to provide a feasibility evaluation of our predictive oncotherapy CTC-based approach and was not designed to evaluate treatment safety, efficacy or effectiveness [48].”
MethodBraun, S. S., Roeser, R. W., & Mashburn, A. J. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00718-7 · Cited by 29
“assessing the feasibility of implementing the MBEB program under these novel conditions, a formal sample size calculation was not warranted, with the goal of recruiting as many teachers within the participating sites as possible [33].”
BackgroundRosenberg, A. R., Steiner, J. M., Lau, N., Fladeboe, K., Toprak, D., Gmuca, S., O’Donnell, M. B., Smith, K., Brown, C. E., & Yi‐Frazier, J. P. (2021) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2021.02.036 · Cited by 20
My Healthy Brain: a multimodal lifestyle program to promote brain health
Mace, R. A., Greenberg, J., Stauder, M. J., Reynolds, G., & Vranceanu, A. (2021) · Aging & Mental Health · doi:10.1080/13607863.2021.1904828 · Cited by 21
“…self-determination and subjective wellbeing measures to preface future mechanistic testing, 4) selecting appropriate analyses for pilot intervention studies (Leon et al., 2011), and 5) collecting informal feedback from participants, study clinicians, and multidisciplinary experts (Table 3).”
BackgroundMethodInfluentialOwnsworth, T., Chambers, S., Aitken, J. F., Foote, M., Pinkham, M. B., Gordon, L. G., Lock, G., Hanley, B., Gardner, T., Jones, S., et al. (2019) · European Journal of Cancer Care · doi:10.1111/ecc.13132 · Cited by 19
A Pilot Open Trial of a Digital Mindfulness-Based Intervention for Anorexia Nervosa
Sala, M., Levinson, C. A., Kober, H., & Roos, C. R. (2023) · Behavior Therapy · doi:10.1016/j.beth.2023.01.003 · Cited by 19
“First, this study was conducted with a very small sample size, and findings are therefore tentative, as small sample sizes provide less precise estimates of population effect sizes (Leon et al., 2011).”
Conroy, A. A., Hahn, J. A., Neilands, T. B., Darbes, L. A., Tebbetts, S., Mulauzi, N., Mkandawire, J., & Ssewamala, F. M. (2024) · AIDS and Behavior · doi:10.1007/s10461-024-04326-x · Cited by 22
Irvine, L. A., Crombie, I. K., Cunningham, K. B., Williams, B., Sniehotta, F. F., Norrie, J. D., Melson, A. J., Jones, C., Rice, P. M., Slane, P. W., et al. (2017) · Alcohol and Alcoholism · doi:10.1093/alcalc/agx067 · Cited by 20
“In keeping with guidance for feasibility studies, no statistical tests were carried out (Leon et al., 2011).”
BackgroundInfluentialPhotoVoice is a feasible method of program evaluation at a center serving adults with autism
Krutt, H., Dyer, L., Arora, A., Rollman, J. B., & Jozkowski, A. C. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.02.003 · Cited by 20
Brain fog and non-coeliac gluten sensitivity: Proof of concept brain MRI pilot study
Croall, I. D., Hoggard, N., Aziz, I., Hadjivassiliou, M., & Sanders, D. S. (2020) · PLoS ONE · doi:10.1371/journal.pone.0238283 · Cited by 20
“The purpose of pilot studies are to “examine the feasibility of an approach that is intended to be used in a larger scale study” [28], rather than being a full and formal analysis.”
BackgroundBarnes, S. M., Borges, L. M., Smith, G. P., Walser, R. D., Forster, J. E., & Bahraini, N. H. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.02.003 · Cited by 24
Wisting, L., Haugvik, S., Wennersberg, A. L., Hage, T. W., Stice, E. M., Olmsted, M. P., Ghaderi, A., Brunborg, C., Skrivarhaug, T., Dahl‐Jørgensen, K., et al. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23578 · Cited by 20
Saberi, P., Lisha, N. E., Erguera, X. A., Hudes, E. S., Johnson, M. O., Ruel, T., & Neilands, T. B. (2021) · JMIR Formative Research · doi:10.2196/26861 · Cited by 21
Weight‐neutral interventions in young people with high body mass index: A systematic review
Hoare, J. K., Lister, N. B., Garnett, S. P., Baur, L. A., & Jebeile, H. (2022) · Nutrition & Dietetics · doi:10.1111/1747-0080.12729 · Cited by 20
“The findings are limited in their reporting of cardiometabolic outcomes and the use of non-treatment control groups, and may also not be generalisable beyond the current, predominantly female and/or Hispanic/ Latino study populations.(66) Some caution is needed in inferring feasibility from the current evidence.”
BackgroundArnedt, J. T., Conroy, D. A., Stewart, H. J., Yeagley, E. E., Bowyer, G. E., Bohnert, K. M., & Ilgen, M. A. (2023) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2023.109835 · Cited by 22
Strauss, C., Rosten, C., Hayward, M., Lea, L., Forrester, E., & Bibby-Jones, A.-M. (2015) · Trials · doi:10.1186/s13063-015-0664-7 · Cited by 22
“A power calculation to determine sample size is not appropriate for this pilot study [23,24], as there is no intention to identify a statistically significant difference between the two treatment groups.”
BackgroundDesign and conduct of a large obstetric or neonatal randomized controlled trial
Tarnow‐Mordi, W., Cruz, M., & Morris, J. M. (2015) · Seminars in Fetal and Neonatal Medicine · doi:10.1016/j.siny.2015.10.001 · Cited by 19
Clayton, G. L., Smith, I. L., Higgins, J. P. T., Mihaylova, B., Thorpe, B., Cicero, R., Lokuge, K., Forman, J., Tierney, J. F., White, I. R., et al. (2017) · Trials · doi:10.1186/s13063-017-1955-y · Cited by 22
“review relied on pilot trials to inform these parameters [21, 22], while the INVEST results seem to suggest that relevant information will often be available from evidence syntheses.”
MethodSuicide bereavement: piloting a longitudinal study in Australia
Kõlves, K., & Leo, D. D. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019504 · Cited by 22
The role online review on mobile commerce adoption: an inclusive growth context
Pipitwanichakarn, T., & Wongtada, N. (2020) · Journal of Asia Business Studies · doi:10.1108/jabs-02-2019-0060 · Cited by 29
Onazi, M. M. A., Yurick, J. L., Harris, C., Nishimura, K., Suderman, K., Pituskin, E., Chua, N. S., & McNeely, M. L. (2020) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.10.028 · Cited by 21
Wilson, D. K., Alia, K. A., Kitzman, H. E., & Resnicow, K. (2014) · Childhood Obesity · doi:10.1089/chi.2013.0070 · Cited by 22
The added value of telemedicine services for physical rehabilitation
Jansen-Kosterink, S. M. (2014) · doi:10.3990/1.9789082319606 · Cited by 20
Pilot Trial Characteristics, Postpilot Design Modifications, and Feasibility of Full-Scale Trials
Ying, X., & Ehrhardt, S. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.33642 · Cited by 24
Antunes, M. D., Loures, F. C. N. D. R., de Souza, I. M. B., Cruz, A. T., Januário, P. D. O., Pinheiro, M. M. L. S., Schmitt, A. C. B., Frutos‐Bernal, E., Nogueras, A. M. M., & Marques, A. P. (2023) · Trials · doi:10.1186/s13063-023-07588-3 · Cited by 19
“The sample size was based on the assumptions of group educational interventions, where it is not necessary to formally calculate the sample size due to the characteristics of the intervention [27, 38, 78, 82–84].”
MethodAdjunctive yoga versus bibliotherapy for bipolar depression: A pilot randomized controlled trial
Weinstock, L. M., Broughton, M. K., Tezanos, K. M., Tremont, G. N., Gillette, T., & Uebelacker, L. A. (2016) · Mental health and physical activity · doi:10.1016/j.mhpa.2016.11.001 · Cited by 20
Greluș, A., Nica, D. V., Donath-Miklos, I., Belengeanu, V., Ioiart, I., & Popescu, C. (2017) · International Journal of Molecular Sciences · doi:10.3390/ijms18112465 · Cited by 19
“Yet, such experimental designs are commonly used in clinical pilot exploratory investigations [55,56] due to various reasons, such as cost, ethical, and patient availability [57].”
BackgroundThe impact of Snoezelen in pediatric complex continuing care: A pilot study
Koller, D., McPherson, A. C., Lockwood, I., Blain‐Moraes, S., & Nolan, J. (2018) · Journal of Pediatric Rehabilitation Medicine · doi:10.3233/prm-150373 · Cited by 20
“The small sample size of the present study is suitable for an exploratory investigation of this kind [44,45], but it also reflects the unique considerations of having children with complex health care needs involved in research.”
The family talk intervention in palliative care: a study protocol
Eklund, R., Kreicbergs, U. C., Alvariza, A., & Lövgren, M. (2018) · BMC Palliative Care · doi:10.1186/s12904-018-0290-8 · Cited by 20
“No power is calculated to estimate sample size because it is a small-scaled study and designed as a pilot with no aim to do any hypothesis testing [18, 19].”
BackgroundA pilot multisite study of patient navigation for pregnant women with opioid use disorder
Cochran, G., Smid, M. C., Krans, E. E., Bryan, M. A., Gordon, A. J., Lundahl, B., Silipigni, J., Haaland, B. A., & Tarter, R. E. (2019) · Contemporary Clinical Trials · doi:10.1016/j.cct.2019.105888 · Cited by 20
“Rather, our sample size is based on estimates of how many patients can be screened and consented within the study timeframe, an appropriate method for pilot studies [40].”
MethodLoeb, K. L., Weissman, R. S., Marcus, S. M., Pattanayak, C. W., Hail, L., Kung, K. C., Schron, D., Zucker, N. L., Grange, D. L., Lock, J., et al. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00985 · Cited by 19
“However, this pilot study aimed to inform the design and hypotheses of future, larger research trials (34) on early treatment strategies, rather than to draw definitive conclusions about treatment efficacy.”
BackgroundT'ai Chi for Chronic Low Back Pain in Older Adults: A Feasibility Trial
Sherman, K. J., Wellman, R., Hawkes, R. J., Phelan, E. A., Lee, T. E., & Turner, J. A. (2020) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2019.0438 · Cited by 18
“Sample sizes for feasibility studies need to be large enough to provide a high likelihood of surfacing any important problems that may exist.(41) Our sample size of 64 was chosen based on practical considerations to provide ample opportunity to identify problems with the study procedures, intervention protocols (inclu…”
BackgroundNizam, D. N. M., & Law, E. L. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102558 · Cited by 23
Greenberg, J., Singh, T., Iverson, G. L., Silverberg, N. D., Macklin, E. A., Parker, R. A., Giacino, J. T., Yeh, G. Y., & Vranceanu, A. (2020) · JMIR Research Protocols · doi:10.2196/25746 · Cited by 19
“In line with recommendations for analyses in pilot studies [69,70], we will refrain from conducting between-group analyses of efficacy.”
Introduction of a smartphone based behavioral intervention for migraine in the emergency department
Minen, M. T., Friedman, B. W., Adhikari, S., Corner, S. E., Powers, S. W., Seng, E. K., Grudzen, C. R., & Lipton, R. B. (2021) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2020.12.009 · Cited by 17
“On average, participants completed a PMR session 11±12 (1, 47) days within the 90-day study period for 10±6 (0, 22) minutes per day.”
BackgroundBerger, B., Jenetzky, E., Köblös, D., Stange, R., Baumann, A., Simstich, J., Michalsen, A., Schmelzer, K., & Martin, D. D. (2021) · Nutrition · doi:10.1016/j.nut.2021.111169 · Cited by 19
“This study provides initial data for safety rules that can be further evaluated in larger controlled trials [39].”
BackgroundNguyen, T. A., Tran, K., Esterman, A., Brijnath, B., Xiao, L. D., Schofield, P. E., Bhar, S. S., Wickramasinghe, N. S., Sinclair, R. G., Dang, T. H., et al. (2021) · JMIR Research Protocols · doi:10.2196/33572 · Cited by 22
“While pilot studies are not powered to examine effectiveness, this pilot RCT will provide data about acceptability, useability, and preliminary impact of the intervention, as well as the feasibility of the methods and procedures [18] for a fully powered RCT to test effectiveness.”
BackgroundMethodMobbs, C. A., Spittle, A. J., & Johnston, L. M. (2022) · Early Human Development · doi:10.1016/j.earlhumdev.2022.105551 · Cited by 20
Making the leap: From experimental psychopathology to clinical trials
Blackwell, S. E., & Woud, M. L. (2022) · Journal of Experimental Psychopathology · doi:10.1177/20438087221080075 · Cited by 19
“…an RCT designed to test the efficacy of your intervention you could also consider a formal feasibility or pilot trial4 (e.g. Lancaster et al., 2002; Leon et al., 2011); this would be a normal step before an efficacy trial5 within a standard psychological treatment development context, and again it…”
BackgroundMerle, J. L., Cook, C. R., Pullmann, M. D., Larson, M. F., Hamlin, C., Hugh, M. L., Brewer, S. K., Duong, M. T., Bose, M., & Lyon, A. R. (2022) · School Mental Health · doi:10.1007/s12310-022-09536-z · Cited by 23
“As is the norm for pilot studies (Leon et al., 2011), this study is underpowered when considering the complexity of the full analytic model and the multiple paths/tests.”
MethodO’Hara, K. L., Knowles, L. M., Guastaferro, K., & Lyon, A. R. (2022) · Implementation Research and Practice · doi:10.1177/26334895221131052 · Cited by 22
“…phase of DDBT, Test, involves small-scale pilot testing of the newly designed intervention components to ensure they are acceptable to end-users as well as feasible and appropriate for the setting before being formally tested for efficacy or effectiveness (Leon et al., 2011; Lyon et al., 2019).”
BackgroundWurz, A. J., McLaughlin, E., Hughes, K., Ellis, K. M., Chen, A., Cowley, L., Molina, H., Duchek, D., Eisele, M., & Culos-Reed, S. N. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01244-y · Cited by 20
“A pilot study can offer invaluable insights into the feasibility and acceptability of an intervention prior to expending resources on a full-scale study [27].”
BackgroundSalmoirago‐Blotcher, E., Druker, S., Meyer, F., Bock, B. C., Crawford, S. L., & Pbert, L. A. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.02.004 · Cited by 21
“Since pilot studies do not provide meaningful estimates of effect sizes for planning subsequent large randomized controlled trials due to the imprecision inherent in data from small samples [56,57], our proposed sample size of 30 students per condition/ school was based on practical consideration (costs and duration o…”
BackgroundRuggiero, K. J., Bunnell, B. E., Andrews, A. R., Davidson, T. M., Hanson, R. F., Danielson, C. K., Saunders, B. E., Soltis, K. E., Yarian, C., Chu, B. C., et al. (2015) · JMIR Research Protocols · doi:10.2196/resprot.4416 · Cited by 19
“This would allow us to identify and address barriers in recruitment, retention, fidelity assessment, and other procedures essential to successful conduct of a RCT.”
InfluentialPilot studies: Are they appropriately reported?
Kannan, S., & Gowri, S. (2015) · Perspectives in Clinical Research · doi:10.4103/2229-3485.167097 · Cited by 32
“One has to understand that a pilot study is not a hypothesis testing study where safety, efficacy or effectiveness can be assessed.[6] The pilot studies are underpowered in detecting any significant difference between the groups unless the sample size is based on an appropriate calculation.”
BackgroundLeffler, J. M., Junghans-Rutelonis, A. N., McTate, E., Geske, J. R., & Hughes, H. M. (2017) · Evidence-Based Practice in Child and Adolescent Mental Health · doi:10.1080/23794925.2017.1362329 · Cited by 19
“As this is not a hypothesis testing clinical trial designed to compare randomized treatment groups in order to draw an inference about effectiveness (Leon et al., 2011), no comparison or control group was utilized as part of the current study.”
MethodFord, J. D., Grasso, D. J., Levine, J., & Tennen, H. A. (2017) · Journal of Child & Adolescent Substance Abuse · doi:10.1080/1067828x.2017.1400484 · Cited by 18
Family-Centered Care During Constraint-Induced Therapy After Chronic Stroke: A Feasibility Study
Blanton, S., Scheibe, D. C., Rutledge, A. H., Regan, B., O’Sullivan, C. S., & Clark, P. C. (2018) · Rehabilitation Nursing · doi:10.1097/rnj.0000000000000197 · Cited by 19
Park, V. T., Ton, V., Tiêt, Q. Q., Vuong, Q. N., Yeo, G. W., & Gallagher-Thompson, D. E. (2019) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1016/j.trci.2019.05.006 · Cited by 20
““A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention” [15].”
MethodOrgeta, V., Tuijt, R., Leung, P., Verdaguer, E. S., Gould, R. L., Jones, R., & Livingston, G. (2019) · Journal of Alzheimer s Disease · doi:10.3233/jad-190696 · Cited by 19
Slip Buddy App for Weight Management: Randomized Feasibility Trial of a Dietary Lapse Tracking App
Pagoto, S., Tulu, B., Waring, M. E., Goetz, J. M., Bibeau, J., DiVito, J., Groshon, L. C., & Schroeder, M. W. (2021) · JMIR mhealth and uhealth · doi:10.2196/24249 · Cited by 21
“Statistical tests were not used to compare groups on weight loss because this pilot study was not powered for weight loss efficacy; thus, it is not appropriate to perform such tests, as discussed elsewhere [25].”
Beets, M. W., von Klinggraeff, L., Burkart, S. A., Jones, A., Ioannidis, J. P. A., Weaver, R. G., Okely, A. D., Lubans, D. R., van Sluijs, E. M. F., Jago, R., et al. (2021) · Obesity Reviews · doi:10.1111/obr.13369 · Cited by 18
““Behavioral intervention pilot studies” were defined as studies which test the feasibility of a behavioral intervention and/or provide evidence of a preliminary effect(s) in a hypothesized direction.(1,17,18) These studies are conducted separately from, and prior to, larger scale trials, with the results used to infor…”
ResultMobilization With Movement for Shoulder Dysfunction in Older Adults: A Pilot Trial
Lírio-Romero, C., Torres-Lacomba, M., Montoro, Y. C., Prieto‐Merino, D., da Costa, S. P., Marchante, M. J. V., & Pardo, G. B. (2015) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2015.03.001 · Cited by 19
EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU
Lichtenthal, W. G., Lief, L., Rogers, M., Russell, D., Viola, M., Winoker, H., Kakarala, S., Traube, C., Coats, T., Fadalla, C., et al. (2024) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2024.03.002 · Cited by 17
Baldo, J. V., Schendel, K., Lwi, S. J., Herron, T. J., Dempsey, D. G., Muir, J., Curran, B. C., Paulraj, S., Chok, J., & Cole, M. A. (2021) · Mindfulness · doi:10.1007/s12671-021-01751-0 · Cited by 19
King, C., Llewellyn, C. D., Shahmanesh, M., Abraham, C., Bailey, J., Burns, F. M., Clark, L., Copas, A. J., Howarth, A., Hughes, G., et al. (2019) · Health Technology Assessment · doi:10.3310/hta23120 · Cited by 18
Fuller, G. W., Keating, S., Goodacre, S. W., Herbert, E., Perkins, G. D., Rosser, A., Gunson, I. M., Miller, J., Ward, M., Bradburn, M. J., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035915 · Cited by 18
Beani, E., Menici, V., Cecchi, A., Cioni, M. L., Giampietri, M., Rizzi, R., Sgandurra, G., & Cioni, G. (2020) · Frontiers in Neurology · doi:10.3389/fneur.2020.601137 · Cited by 19
Kraemer, K. M., Litrownik, D., Moy, M. L., Wayne, P. M., Beach, D., Klings, E. S., Nieva, H. R., Pinheiro, A., Davis, R. B., & Yeh, G. Y. (2021) · COPD Journal of Chronic Obstructive Pulmonary Disease · doi:10.1080/15412555.2021.1928037 · Cited by 21
Contractor, A. A., Jin, L., & Weiss, N. H. (2022) · Journal of Psychotherapy Integration · doi:10.1037/int0000290 · Cited by 18
Pfledderer, C. D., von Klinggraeff, L., Burkart, S. A., Wolfenden, L., Ioannidis, J. P. A., & Beets, M. W. (2023) · Pilot and Feasibility Studies · doi:10.1186/s40814-023-01270-w · Cited by 19
“Pilot studies were defined as those studies which are conducted separately from and prior to a large-scale trial and are designed to test the feasibility of an intervention and/or provide evidence of preliminary effects before scale-up [3, 21, 28].”
BackgroundFord, K. L., Sawyer, M. B., Ghosh, S., Trottier, C. F., Disi, I. R., Easaw, J. C., Mulder, K. E., Koski, S. L., Starr, K. N. P., Bales, C. W., et al. (2024) · ESMO Open · doi:10.1016/j.esmoop.2024.103604 · Cited by 21
“33,44 Data comparisons were carried out with an alpha level of 5% but caution was exercised when interpreting analyses as this study was not powered for drawing statistical inference but rather was intended to inform the feasibility of the intervention and a larger-scale trial.”
Maheu, C., Lebel, S., Tomei, C., Singh, M. D., & Esplen, M. J. (2014) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2014.12.003 · Cited by 19
“One of the aims of the pilot study was to demonstrate participant acceptance and perceived benefits of a new intervention (Leon et al., 2011).”
BackgroundPersonalized Adaptive Control of Training Load in Cardio-Exergames—A Feasibility Study
Hoffmann, K., Sportwiss, D., Hardy, S., Wiemeyer, J., & Göbel, S. P. (2015) · Games for Health Journal · doi:10.1089/g4h.2014.0073 · Cited by 18
“14 Although there is no consensus concerning the application of inferential statistics in pilot and feasibility studies, 12,15 we used statistics to explore rather than test hypotheses because of the lack of knowledge addressing HR control in exergames.”
Sternheim, L. C., & Harrison, A. (2018) · Cogent Psychology · doi:10.1080/23311908.2018.1441594 · Cited by 24
Aji, A.-S. H., Awg-Manan, F., Abdullah, Y.-R., Kisut, R., Rahman, H. A., & Mumin, K. H. A. (2018) · Women and Birth · doi:10.1016/j.wombi.2018.11.005 · Cited by 22
Edwards, C., Garety, P. A., & Hardy, A. (2020) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2020.101564 · Cited by 18
Development of a mind body program for obese knee osteoarthritis patients with comorbid depression
Jacobs, C. A., Mace, R. A., Greenberg, J., Popok, P., Reichman, M., Lattermann, C. M., Burris, J. L., Macklin, E. A., & Vranceanu, A. (2021) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2021.100720 · Cited by 20
Khalifian, C., Leifker, F. R., Knopp, K. C., Wilks, C. R., Depp, C., Glynn, S., Bryan, C., & Morland, L. A. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23251 · Cited by 18
Teo, S.-H. J., Poh, X. W. W., Lee, T.-S. W., Guan, C., Cheung, Y. B., Fung, D. S. S., Zhang, H., Chin, Z. Y., Wang, C. C., Sung, M., et al. (2021) · Research in Autism · doi:10.1016/j.rasd.2021.101882 · Cited by 23
McDonald, T. A. M., Lalani, S., Chen, I., Cotton, C. M., MacDonald, L. L., Boursoulian, L. J., Wang, J., & Malow, B. A. (2022) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-022-05598-9 · Cited by 18
Huberty, J., Rowedder, L., Hekler, E. B., Adams, M. A., Hanigan, E., McClain, D. B., Balluff, M., Buman, M. P., & Bushar, J. A. (2015) · Translational Behavioral Medicine · doi:10.1007/s13142-015-0339-7 · Cited by 18
“Information gathered from this study will inform the design of a larger randomized control trial [42, 43].”
MethodMcDonnell, K. K., Hollen, P. J., Heath, J., & Andrews, J. O. (2015) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2015.08.006 · Cited by 18
“Strategies to enhance recruitment and retention were built into the design of this study (Leon et al., 2011).”
BackgroundMethodWalker, R., Aitken, L. M., Huxley, L., & Juttner, M. (2014) · Journal of Advanced Nursing · doi:10.1111/jan.12543 · Cited by 18
“The objective for this pilot study is to evaluate the feasibility of undertaking a RCT to assess effectiveness against pre-defined criteria.”
InfluentialWilliams, J., Stubbs, B., Gaughran, F., & Craig, T. (2016) · Trials · doi:10.1186/s13063-016-1660-2 · Cited by 19
“This number is based on recruiting a big-enough sample to assess the feasibility of the study and on the resources available [22].”
BackgroundFeasibility of Cognitive Training in Critically Ill Patients: A Pilot Study
Wassenaar, A., Rood, P. J. T., Boelen, D. H. E., Schoonhoven, L., Pickkers, P., & van den Boogaard, M. (2018) · American Journal of Critical Care · doi:10.4037/ajcc2018467 · Cited by 17
Bardus, M., Hamadeh, G. N., Hayek, B., & Kherfan, R. A. (2018) · JMIR Research Protocols · doi:10.2196/resprot.9793 · Cited by 19
“The estimation of preliminary efficacy will be based on the observed trends on data over time rather than traditional inferential statistics [93].”
BackgroundOnline Cognitive-Behavioral Group Intervention for Adolescents With Chronic Illness: A Pilot Study
Douma, M., Joosten, M. M. H., Scholten, L., Maurice‐Stam, H., & Grootenhuis, M. A. (2019) · Clinical Practice in Pediatric Psychology · doi:10.1037/cpp0000274 · Cited by 18
Dickson, K. S., Aarons, G. A., Anthony, L. G., Kenworthy, L. E., Crandal, B. R., Williams, K. N., & Brookman-Frazee, L. I. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00593-2 · Cited by 21
Bieleninik, Ł., Konieczna-Nowak, L., Knapik-Szweda, S., & Kwaśniok, J. (2020) · Nordic Journal of Music Therapy · doi:10.1080/08098131.2020.1781233 · Cited by 18
Metodología del estudio piloto
Gustavo Díaz, M. (2020) · Revista de Fomento Social · doi:10.4067/s0717-93082020000400172 · Cited by 17
Goodman, A. C., Ouellette, R. R., D’Agostino, E., Hansen, E., Lee, T. T. H., & Frazier, S. L. (2021) · Journal of Community Psychology · doi:10.1002/jcop.22587 · Cited by 20
Does improving indoor air quality lessen symptoms associated with chemical intolerance?
Perales, R. B., Palmer, R. F., Rincon, R., Viramontes, J. N., Walker, T., Jaén, C. R., & Miller, C. S. (2022) · Primary Health Care Research & Development · doi:10.1017/s1463423621000864 · Cited by 19
“However, pilot studies are performed prior to definitive trials to provide enough evidence of overall potential intervention benefits (Leon et al., 2011; Lee et al., 2014).”
BackgroundGatto, A. J., Elliott, T. J., Briganti, J. S., Stamper, M. J., Porter, N. D., Brown, A. M., Harden, S. M., Cooper, L. D., & Dunsmore, J. C. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.858370 · Cited by 24
“Developing prevention programs requires careful consideration of program delivery, engagement, and reception by the intended audience (33).”
BackgroundHarvie, D. S., Kelly, J., Kluver, J. P., Deen, M., Spitzer, E., & Coppieters, M. W. (2022) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2022.2129846 · Cited by 22
“Firstly, it is important to recognize that studies of small sample size are inherently imprecise, debate about the relevance of significance testing continues [37].”
BackgroundListiyandini, R. A., Andriani, A., Afsari, N., Krisnamurthi, P. B. U., Moulds, M. L., Mahoney, A., & Newby, J. M. (2024) · Mindfulness · doi:10.1007/s12671-024-02346-1 · Cited by 27
“Pilot studies, feasibility studies, and proof of concept studies are terms that have been used somewhat interchangeably in the literature and henceforth are referred to as “pilot stud-ies” (Leon et al., 2011).”
InfluentialConroy, A. A., Ruark, A. H., Mulauzi, N., Mkandawire, J., Darbes, L. A., Hahn, J. A., Neilands, T. B., Tebbetts, S., & Ssewamala, F. M. (2024) · Social Science & Medicine · doi:10.1016/j.socscimed.2024.117407 · Cited by 18
The Feasibility and Acceptability of “Arise”: An Online Substance Abuse Relapse Prevention Program
Sanchez, R. P., & Bartel, C. M. (2014) · Games for Health Journal · doi:10.1089/g4h.2014.0015 · Cited by 17
“23 We report here on the practicality of participant recruitment and retention, as well as the feasibility of administering the intervention and outcomes assessment components using this innovative methodology.”
How Should Surgical Residents Be Educated About Patient Safety: A Pilot Randomized Controlled Trial
Putnam, L. R., Pham, D. H., Ostovar-Kermani, T. G., et al. (2016) · Journal of surgical education · doi:10.1016/j.jsurg.2016.02.011 · Cited by 17
Keay, L. J., Saich, F., Clemson, L. M., Middlemiss, B. P. H. L., Johnson, B. A. J., Tumanik, H., Taylor, B. S. J., Munro, J., Borkowski, E., & Tinsley, F. (2017) · International Journal of Orientation & Mobility · doi:10.21307/ijom-2017-053 · Cited by 17
Chow, P. I., Showalter, S. L., Gerber, M. S., Kennedy, E. M., Brenin, D. R., Schroen, A. T., Mohr, D. C., Lattie, E. G., & Cohn, W. F. (2018) · JMIR Research Protocols · doi:10.2196/11452 · Cited by 16
Schlam, T. R., & Baker, T. B. (2019) · Games for Health Journal · doi:10.1089/g4h.2019.0030 · Cited by 18
Vicarious traumatisation in lawyers working with traumatised asylum seekers: a pilot study
Rønning, L., Blumberg, J., & Dammeyer, J. (2020) · Psychiatry Psychology and Law · doi:10.1080/13218719.2020.1742238 · Cited by 18
Crowe, M., Eggleston, K., Douglas, K. M., & Porter, R. (2020) · Bipolar Disorders · doi:10.1111/bdi.12971 · Cited by 18
Reneau, M. G. (2020) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2020.0071 · Cited by 17
“27 The author based sample size on the pragmatics of recruiting from the VA, patient flow, and budget constraints for purchasing equipment, and therefore, a benchmark of 10 study participants was established.”
Nameth, K., Brown, T., Bullock, K., Adler, S., Riva, G., Safer, D., & Runfola, C. (2021) · doi:10.20944/preprints202103.0152.v1 · Cited by 17
“Definitions of feasibility were based on the research literature [39].”
Westermann, K., Woud, M. L., Cwik, J. C., Graz, C., Nyhuis, P. W., Margraf, J., & Blackwell, S. E. (2021) · BJPsych Open · doi:10.1192/bjo.2021.1042 · Cited by 21
“In line with recommendations for pilot/feasibility studies, 16 the purpose of this study was not hypothesis testing, and thus we did not test the statistical significance of potential differences between treatment arms on the clinical outcome measures.”
BackgroundAschbrenner, K. A., Pepin, R. L., Mueser, K. T., Naslund, J. A., Rolin, S. A., Faber, M. J., & Bartels, S. J. (2014) · The International Journal of Psychiatry in Medicine · doi:10.2190/pm.48.2.e · Cited by 16
“The overall sample size (n = 41) is consistent with exploratory qualitative methods [38] and the intent and scope of a pilot study aimed at achieving preliminary descriptive findings [39].”
Metadata
- Authors
- Leon, A. C., Davis, L. L., & Kraemer, H. C.
- Affiliations
- Weill Cornell Medicine (US)
- University of Alabama (US)
- Stanford University (US)
- Corresponding author
- Leon, Andrew C.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2011-05
- Volume
- 45
- Issue
- 5
- Pages
- 626-629
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC3081994
- OpenAlex
- W2168434488
- Cited by
- 1,917
- References
- 17
- Citations by year
- 2026: 203 · 2025: 190 · 2024: 143 · 2023: 169 · 2022: 163 · 2021: 217 · 2020: 169 · 2019: 136 · 2018: 128 · 2017: 120 · 2016: 92 · 2015: 80 · 2014: 52 · 2013: 31 · 2012: 17
- FWCI
- 14.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pilot study, feasibility assessment, clinical trials, research design, sample size determination, effect size estimation, generalizability, recruitment and retention, intervention implementation, research funding review
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Feasibility Studies, Humans, Pilot Projects, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
