The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The role and interpretation of pilot studies in clinical research
Open accessAbstract
Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 5 No basis given · 2 No full text of the sources · 1 Sources not found
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“When event rates are reported and no adverse event is observed for a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
What the sources say
- Not matchedcites Jovanovic & Levy, 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Jovanovic et al., 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Basing a decision to proceed or terminate evaluation of a particular intervention on pilot data is perilous because there is a very good chance that the decision will be derived from false positive or false negative results.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each aspect of feasibility could be quite different from an uncontrolled study when intervention assignment is randomized and blinded, particularly if placebo is a distinct possibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these issues in mind, a pilot study can better address its goals if a control group is part of the design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A case series estimate tends to be steeped in optimism, particularly if an estimate of such magnitude is seldom, if ever, seen in full scale trials for psychiatric disorders.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(It is the population effect size, not the sample effect size, that an RCT is designed to detect with sufficient power.)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There has been a venerable tradition of using pilot studies to estimate between group effect sizes that, in turn, are used to inform the design of subsequent larger scale hypothesis testing studies. Despite it widespread use, it has been argued that the tradition is ill-founded. (Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot sample size is instead based on the pragmatics of recruitment and the necessities for examining feasibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With no inferential statistical tests, a pilot study will not provide p-values.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot study data generally should not be combined with data from the subsequent larger scale study. This is because it is quite likely that the methods will be modified after the pilot, even if minimally. Such changes in protocol risk adding additional, perhaps unknown, source of variation.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Are Pilot Data Included in the Larger Trial?
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Methods will likely be modified after a pilot study, even if minimally.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 13
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study.”
- Also stated in: § DISCUSSION · plain
“Pilot results inform feasibility, which in turn, is instructive in that it points to modifications needed in the planning and design of a larger efficacy trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The definition of clinically meaningful is not entirely empirically-based, but instead requires input from clinicians who treat the patient population of interest and perhaps from patients with the disorder.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These good clinical practices include the refinement of source documentation, informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring/oversight procedures, especially when multiple sites and investigators are engaged in the study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety, efficacy and effectiveness are not evaluated in a pilot.”
- Also stated in: § DISCUSSION · plain
“a pilot study is not a hypothesis testing study and therefore safety and efficacy are not evaluated.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot studies generate pilot data, their design need not be guided with the support of prior pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible, but highly unlikely, that the between group effect size (d) from a pilot study sample will provide a reasonable estimate of the population effect size (Δ), but that cannot be known based on the pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study can be critical in research staff training and provide experiences that strengthen and confirm competencies and skills required for the investigation to be conducted with accuracy and precision.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An alternative approach is to base sample size estimates for a full scale RCT on what is deemed a clinically meaningful effect.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > An Alternative to Using Pilot Data for Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study provides opportunity to develop consistent practices to enhance data integrity and the protection of human subjects.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We learn little if anything about the efficacy of an intervention with data from a small sample; yet, as discussed earlier, a great deal can be learned from a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This applies to effects sizes for any type of outcome, be it a severity rating, a response status, or survival status.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, with imprecision comes a vast disparity in sample size estimates and, if sample size determination for a subsequent study is based on an imprecise estimate, there is an enormous risk of underpowered or overpowered design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, pilot studies are a necessary first step in exploring novel interventions and novel applications of interventions – whether in a new patient population or with a novel delivery system (e.g., transdermal patch).”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although a pilot study will undoubtedly incorporate relevant outcome measures and can serve a vital role in treatment development, it is not, and should not, be considered a preliminary test of the intervention hypothesis.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The primary role of a pilot study is to examine the feasibility of a research endeavor.”
- Also stated in: § INTRODUCTION · plain
“The fundamental purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to ultimately be used in a larger scale study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This does not preclude the need for a theoretical rationale for the intervention or the methodology being proposed for a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a pilot study effect size is unduly large (i.e., a false positive result), subsequent trials will be designed with an inadequate number of participants to provide the statistical power needed to detect clinically meaningful effects and that would lead to negative trials.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, pilot studies provide an opportunity to implement and examine the feasibility of the adverse event reporting system.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is quite reasonable and expected that a pilot study is proposed with no pilot or other preliminary data supporting the proposal and that its proposed sample size is based on pragmatics such as patient flow and budgetary constraints.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention.”
- Also stated in: § INTRODUCTION · plain
“evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and/or implementation of the novel intervention.”
- Also stated in: § WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY · plain
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, and implementation of the novel intervention and each of these can be quantified”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are two fundamental reasons that hypothesis testing is not used in a pilot study: the limited state of knowledge about the methods or intervention in the patient population to be studied and the smaller proposed sample size.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a consequence, pilot studies can reduce the proportion of failed trials and allow research funds to be spent on projects for which feasibility has been demonstrated and quantified.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Well-conceived and implemented pilot studies will reduce the risk of several problems that are commonly faced in clinical trials. These include the inability to recruit the proposed sample size and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures or interventions, and the need to modify a protocol midway through a trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Estimates become more precise with larger sample sizes.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The true population effect size is unknown when a pilot study is designed.
“An essential challenge of therapeutic development is that the true population effect size is unknown at the time a pilot study is designed.”
The paper relies on this without support.
Methods will likely be modified after a pilot study, even if minimally.
“This is because it is quite likely that the methods will be modified after the pilot, even if minimally.”
The paper relies on this without support.
Pilot results presented to grant review committees tend to be selective, overly optimistic, and at times misrepresentational.
“Furthermore, pilot results that are presented to grant review committees tend to be selective, overly optimistic and, at times, misrepresentational.”
The paper relies on this without support.
Many pilot studies have far fewer than 18 participants per group.
“Many pilot studies have far fewer than 18 participants/group and therefore even greater imprecision.”
The paper relies on this without support.
It is not unusual for research grant applications and pilot study publications to convey optimism based on pilot data.
“Nevertheless, it is not unusual for research grant applications and pilot study publications to convey such optimism, particularly when based on pilot data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The planned sample size was 30 allowing for attrition. This study will inform the sample size for any future RCT (Leon et al., 2011).”
From A mixed methods pilot and feasibility open trial of internet-delivered cognitive behaviour therapy (iCanADAPT Advanced) for people with advanced cancer with depression and/or anxiety (2021) · cites this paper as Leon et al., 2011
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was a pilot study which like other pilot studies of MI did not have a power calculation. 31,32”
From A Pilot Randomized Controlled Trial to Assess the Impact of Motivational Interviewing on Initiating Behavioral Therapy for Migraine (2020) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data analyses in the context of a pilot study are not hypothesis-driven but serve to provide information regarding feasibility and acceptability of the intervention [55].”
From Acceptance and Commitment Therapy Wellness Program for Latine Adults Who Smoke and Have Psychological Distress: Protocol for a Feasibility Study (2023) · cites this paper as [55]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
From PandaMom – Feasibility and acceptability of an internet- and mobile-based intervention to enhance peripartum mental well-being and to prevent postpartum depression (2024) · cites this paper as Leon et al., 2011
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our use of effect sizes was, however, deemed appropriate for this pilot trial (55, 63).”
From EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU (2024) · cites this paper as [63]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work suggests early studies often test feasibility of methods (e.g., recruitment, randomization, retention, and assessment), not hypotheses [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
Citations
Cites
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · doi:10.1007/978-1-4757-2915-3 · Cited by 873
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · cited 1×
“(Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006) Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small samples.”
Self-citationWolfstein (1905) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-190502000-00009 · Cited by 346
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · cited 1×
“(Leon et al., 2006; Leon & Davis, 2009).”
Self-citationSixteen S ‐squared over D ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · cited 1×
“(N/group ≈ 16/d2; Lehr, 1992) The clinical interpretation of HAMD change of 1.2 to 3.0 would drive the choice among possible sample sizes in planning a study.”
BackgroundInfluentialJovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.1080/00031305.1997.10473947 · Cited by 179 · cited 1×
“…a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n. (Jovanovic & Levy, 1997; Jovanovic et al., 1997) For example, consider a study with N=15 receiving medication and no suicidal ideation was reported…”
BackgroundHarvey, I. M. (2009) · Public Health · doi:10.1016/j.puhe.2009.05.009 · Cited by 173
Jovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.2307/2685405 · Cited by 50
Cited by
Sorted by citations
Items 301–400 of 1,903
Internet Websites for Chest Pain Symptoms Demonstrate Highly Variable Content and Quality
Joury, A., Alshathri, M., Alkhunaizi, M., Jaleesah, N., & Pines, J. M. (2016) · Academic Emergency Medicine · doi:10.1111/acem.13039 · Cited by 27
Smith, S. W., Daunic, A. P., Algina, J. J., Pitts, D. L., Merrill, K. L., Cumming, M. M., & Allen, C. M. (2016) · Journal of Emotional and Behavioral Disorders · doi:10.1177/1063426616661702 · Cited by 28
A community pharmacy intervention for opioid medication misuse: A pilot randomized clinical trial
Cochran, G., Field, C. A., Karp, J. F., Seybert, A. L., Chen, Q., Ringwald, W. R., Hruschak, V., Chickering, S., Kincman, J., Merranko, A. J., et al. (2018) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2018.03.005 · Cited by 28
Logie, C. H., Lacombe‐Duncan, A., Persad, Y., Ferguson, T. B., Yehdego, D. M., Ryan, S., Forrester, M., Moses, C., & Guţă, A. (2019) · Transgender Health · doi:10.1089/trgh.2018.0040 · Cited by 29
Chermack, S. T., Bonar, E. E., Ilgen, M. A., Walton, M. A., Cunningham, R. M., Booth, B. M., & Blow, F. C. (2015) · Journal of Interpersonal Violence · doi:10.1177/0886260515586369 · Cited by 28
Peer navigation in African American breast cancer survivors
Mollica, M. A., Nemeth, L. S., Newman, S. D., Mueller, M., & Sterba, K. R. (2014) · Patient Related Outcome Measures · doi:10.2147/prom.s69744 · Cited by 26
The transition to self‐regulated learning for first‐year dental students: threshold concepts
Bowman, M. (2016) · European Journal Of Dental Education · doi:10.1111/eje.12193 · Cited by 27
Viaña, J. N. M., Bittlinger, M., & Gilbert, F. (2017) · Journal of Alzheimer s Disease · doi:10.3233/jad-161073 · Cited by 29
“Results of preliminary studies are used to plan the next stages of clinical trials [75]; however, they should not be used to justify efficacy and safety [75, 76] in a clinical setting.”
Cluxton‐Keller, F., Williams, M., Buteau, J., Donnelly, C. L., Stolte, P., Monroe-Cassel, M., & BRUCE, M. L. (2018) · JMIR Mental Health · doi:10.2196/11513 · Cited by 27
“Pilot studies are used to assess the feasibility of an intervention for a larger-scale study, and the use of pilot study effect size for sample size estimation can lead to type I and II errors [58].”
Braun, S. E., Dow, A. W., Loughan, A. R., Mladen, S. N., Crawford, M., Rybarczyk, B. D., & Kinser, P. A. (2020) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2020.102405 · Cited by 31
“Following guidelines for pilot feasibility trials and previously published studies adhering to these recommendations (Lancaster et al., 2004; Leon et al., 2011; Vranceanu et al., 2019), we measured the following domains of feasibility and acceptability using recommended metrics.”
MethodArain, M. A., Ahmad, A., Chiu, V., & Kembel, L. (2021) · BMC Geriatrics · doi:10.1186/s12877-020-01979-w · Cited by 29
Short-term effects of different PM2.5 ranges on daily all-cause mortality in Jinan, China
Ma, Z., Meng, X., Chen, C., Chao, B., Zhang, C., & Li, W. (2022) · Scientific Reports · doi:10.1038/s41598-022-09057-4 · Cited by 30
Chow, M. C. M., Kwok, S.-M., Luk, H. K. Y., Law, J. W. H., & Leung, B. P. K. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.06.003 · Cited by 27
“Treatment effects may be under- or overestimated because of the imprecision inherent in data from small samples (Sim and Lewis, 2012; Leon et al., 2011).”
BackgroundFrom “Hypercritics” to “Happy Campers”: Who Complains the Most in Fine Dining Restaurants?
Khalilzadeh, J., Ghahramani, L., & Tabari, S. (2017) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2017.1256802 · Cited by 27
Feasibility and Pilot Studies in Palliative Care Research: A Systematic Review
Jones, T. A., Olds, T. S., Currow, D. C., & Williams, M. T. (2017) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2017.02.015 · Cited by 27
“It has been suggested that pilot studies should not be hypothesis testing studies and therefore do not require sample size power calculations [4] and analyses should be largely descriptive [5].”
BackgroundMerrill, J. E., Boyle, H. K., Barnett, N. P., & Carey, K. B. (2017) · Addictive Behaviors · doi:10.1016/j.addbeh.2017.10.003 · Cited by 27
Lamprecht, A., & Padayachy, K. (2019) · Chiropractic & Manual Therapies · doi:10.1186/s12998-019-0238-y · Cited by 27
Luttrull, J. K., Sinclair, S. H., Elmann, S., & Glaser, B. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202097 · Cited by 27
“[64, 65] Comprehensive analysis is the province of prospective randomized clinical trials (RCTs).”
BackgroundInfluentialEvidence Use in Health Policy Making
Parkhurst, J. O., Ettelt, S., & Hawkins, B. (2018) · International series on public policy · doi:10.1007/978-3-319-93467-9 · Cited by 31
Cotter, E. W., Hornack, S. E., Fotang, J. P., Pettit, E., & Mirza, N. M. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00621-1 · Cited by 26
Occupational Stress and Its Effects on Nurses at a Health Facility in Ho Municipality, Ghana
Dartey, A. F., Tackie, V., Lotse, C. W., Dziwornu, E., Affrim, D., & Akosua, D. R. D. (2023) · SAGE Open Nursing · doi:10.1177/23779608231186044 · Cited by 26
Feasibility of a Trial on Improvisational Music Therapy for Children with Autism Spectrum Disorder
Geretsegger, M., Holck, U., Bieleninik, Ł., & Gold, C. (2016) · Journal of Music Therapy · doi:10.1093/jmt/thv038 · Cited by 29
Skills to enhance positivity in suicidal adolescents: results from a pilot randomized clinical trial
Yen, S., Ranney, M. L., Krek, M., Peters, J. R., Mereish, E., Tezanos, K. M., Kahler, C. W., Solomon, J., Beard, C., & Spirito, A. (2020) · The Journal of Positive Psychology · doi:10.1080/17439760.2019.1615105 · Cited by 25
Barroso, J. V., Madisetti, M., & Mueller, M. (2019) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2019.09.009 · Cited by 26
Activating primary care COPD patients with multi-morbidity through tailored self-management support
Ansari, S., Hosseinzadeh, H., Dennis, S. M., & Zwar, N. A. (2020) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-020-0171-5 · Cited by 28
Chu, K., Sidani, J. E., Matheny, S. J., Rothenberger, S. D., Miller, E. P., Valente, T. W., & Robertson, L. B. (2020) · Addictive Behaviors · doi:10.1016/j.addbeh.2020.106726 · Cited by 26
“This study adapted and combined modules most effective in reducing youth tobacco use, including counter-marketing techniques, developing confidence, refusal skills, self-regulation, and normative education (CATCH, 2019; Franks et al., 2015; Leon et al., 2011; McKenzie et al., 2001).”
MethodA Mindfulness-Based Physical Activity Intervention: A Randomized Pilot Study
Sala, M., Geary, B. A., & Baldwin, A. S. (2020) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000885 · Cited by 26
“, are less precise), with a large rate of false-positives (44).”
BackgroundWinger, J. G., Kelleher, S. A., Ramos, K., Check, D. K., Yu, J. A., Powell, V. D., Lerebours, R. C., Olsen, M. K., Keefe, F. J., Steinhauser, K. E., et al. (2023) · Psycho-Oncology · doi:10.1002/pon.6151 · Cited by 26
Working With Parents to Prevent Childhood Obesity: Protocol for a Primary Care-Based eHealth Study
Avis, J. L. S., Cave, A., Donaldson, S., Ellendt, C., Holt, N. L., Jelinski, S. E., Martz, P., Maximova, K., Padwal, R. S., Wild, T. C., et al. (2015) · JMIR Research Protocols · doi:10.2196/resprot.4147 · Cited by 29
“All of these elements are important to assess prior to determining the effectiveness of a newly developed intervention [25,47].”
Evaluation of a Novel Financial Navigator Pilot to Address Patient Concerns about Medical Care Costs
Banegas, M. P., Dickerson, J. F., Friedman, N. L., Mosen, D. M., Ender, A. X., Chang, T. R., Runge, T. A., & Hornbrook, M. C. (2019) · The Permanente Journal · doi:10.7812/tpp/18-084 · Cited by 26
Fealy, S., Chan, S., Wynne, O., Dowse, E., Ebert, L., Ho, R., Zhang, M. W. B., & Jones, D. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.13971 · Cited by 20
The SAFETY Investigators (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029054 · Cited by 25
“The rationale for transition of subject data from the pilot study to the definitive study has previously been discussed.(31) It is acceptable to pool the data if the study methods are not adjusted following the pilot study, and the research tools are standardised.”
Stevens, N. R., Miller, M. L., Pütz, A.-K., Padin, A. C., Adams, N., & Meyer, D. J. (2020) · Journal of Traumatic Stress · doi:10.1002/jts.22641 · Cited by 25
Rosenthal, J. L., Sauers‐Ford, H. S., Williams, J., Ranu, J., Tancredi, D. J., & Hoffman, K. R. (2021) · Academic Pediatrics · doi:10.1016/j.acap.2021.03.007 · Cited by 25
“26 We therefore stated explicit feasibility objectives with a priori success criteria.”
MethodWright, H., Martin, F., Clyne, W., Clark, C. C. T., Matouskova, G., McGillion, M. H., & Turner, A. (2021) · Journal of Medical Internet Research · doi:10.2196/28322 · Cited by 27
“As the focus of this paper was to assess the feasibility of an RCT, and not to evaluate the efficacy and acceptability of the HOPE program [48], we did not present participant feedback data here.”
BackgroundDoorley, J. D., Mace, R. A., Popok, P., Grunberg, V. A., Ragnhildstveit, A., & Vranceanu, A. (2021) · The Gerontologist · doi:10.1093/geront/gnab135 · Cited by 27
Salwen‐Deremer, J. K., Smith, M. T., Aschbrenner, K. A., Haskell, H. G., Speed, B. C., & Siegel, C. A. (2021) · BMJ Open Gastroenterology · doi:10.1136/bmjgast-2021-000805 · Cited by 26
“Based on widely cited guidance on the purpose of pilot feasibility studies, 41 a sample size of 10 was determined to be sufficient to investigate feasibility and acceptability of CBT-I, to develop protocols for recruitment and retention for a larger randomised controlled trial (RCT), and to…”
Mensah, I. K., & Mwakapesa, D. S. (2022) · Security and Communication Networks · doi:10.1155/2022/7373658 · Cited by 27
“Additionally, it has been indicated that the goal of a pilot study is to determine the feasibility of an approach that is intended to be applied in a larger-scale study, and therefore, pilot studies are not hypothesis testing studies [101, 104].”
BackgroundRetractedde Valle, M. K., & Wade, T. D. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23756 · Cited by 33
A personalized mobile app for physical activity: An experimental mixed-methods study
Tong, H. L., Quiroz, J. C., Kocaballi, A. B., Ijaz, K., Coiera, E. W., Chow, C. K., & Laranjo, L. (2022) · Digital Health · doi:10.1177/20552076221115017 · Cited by 33
“Given that our main aim is to evaluate the feasibility of the be.well app and its unique personalization approach (and not to test ef fi cacy/effectiveness), we followed published heuristics 40,41”
MethodCrable, E. L., Grogan, C. M., Purtle, J., Roesch, S. C., & Aarons, G. A. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00396-5 · Cited by 25
King, G. A., Kingsnorth, S., McPherson, A. C., Jones-Galley, K., Pinto, M., Fellin, M., Timbrell, N., & Savage, D. (2016) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2016.04.014 · Cited by 28
Hermann, B. A., Meyer, E. C., Schnurr, P. P., Batten, S. V., & Walser, R. D. (2016) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2016.07.001 · Cited by 26
Bhat, A., Reed, S., Mao, J., Vredevoogd, M., Russo, J., Unger, J. A., Rowles, R., & Unützer, J. (2017) · Journal of Psychosomatic Obstetrics and Gynaecology · doi:10.1080/0167482x.2017.1367381 · Cited by 20
“Effect sizes calculated from pilot studies can be imprecise [36], but can provide an”
MethodSlota, M. C., McLaughlin, M. K., Bradford, L., Langley, J., & Vittone, S. B. (2017) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2017.12.007 · Cited by 32
McCarty, C. A., Zatzick, D., Hoopes, T. R., Payne, K. M., Parrish, R. S., & Rivara, F. P. (2019) · Trials · doi:10.1186/s13063-019-3662-3 · Cited by 25
“ment effect estimates derived from these larger scale trials [95, 96].”
BackgroundMetodología del estudio piloto
Díaz, G. (2020) · Revista de Fomento Social · doi:10.4067/s0717-93082020000300100 · Cited by 25
Thomas, S. K., Fazakarley, L., Thomas, P., Brenton, S., Collyer, S., Perring, S., Scott, R., Galvin, K., & Hillier, C. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005172 · Cited by 25
“explore the feasibility of a full-scale formal evaluation of effectiveness.(90)”
BackgroundPhysical Activity Counseling for Children With Cystic Fibrosis
Moola, F. J., Garcia, E., Huynh, E., Henry, L. A., Penfound, S., Consunji‐Araneta, R., & Faulkner, G. (2017) · Respiratory Care · doi:10.4187/respcare.05009 · Cited by 25
Wootton, A. R., Legnitto, D., Gruber, V. A., Dawson-Rose, C. S., Neilands, T. B., Johnson, M. O., & Saberi, P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028522 · Cited by 23
Besoaín, F., Pérez-Navarro, A., Jacques‐Aviñó, C., Caylà, J. A., Barriga, N. A., & de Olalla, P. G. (2019) · JMIR mhealth and uhealth · doi:10.2196/14568 · Cited by 28
“This process has been divided in two steps: (1) a co-design methodology to enhance and address the functional requirements of the app by a group of potential users [15], and (2) a functional and a feasibility test has been performed to evaluate the app with a target group [25,26].”
MethodJunkins, A., Psaros, C., Ott, C., Azuero, A., Lambert, C. C., Cropsey, K. L., Savage, R., Haberer, J. E., Safren, S. A., & Kempf, M.-C. (2020) · Journal of Health Psychology · doi:10.1177/1359105320926526 · Cited by 24
“Because this was a pilot study, statistical inference was not conducted as the sample size was too small (Leon et al., 2011).”
BackgroundMartyn-Nemeth, P. A., Duffecy, J. L., Quinn, L., Steffen, A. D., Baron, K. G., Chapagai, S., Burke, L., & Reutrakul, S. (2022) · The Science of Diabetes Self-Management and Care · doi:10.1177/26350106221136495 · Cited by 25
“Sleep (by Actigraphy) Time in bed (min) 414-635 491 (53) Sleep duration (min) 307-470 399 (49) Sleep efficiency (%) 72-89 81 (6) Sleep variability (midsleep clocktime SD min.”
BackgroundKumbar, L. M., Soi, V., Adams, E., Deacon, C. B., Zidan, M., & Yee, J. (2017) · Hemodialysis International · doi:10.1111/hdi.12535 · Cited by 25
Khan, M. N., Dherani, M., Chiumento, A., Atif, N., Bristow, K., Sikander, S., & Rahman, A. (2017) · International Journal of Social Psychiatry · doi:10.1177/0020764017734001 · Cited by 25
“As this was a feasibility trial, the sample size was based on convenience and feasibility, with no power calculations carried out (Leon et al., 2011).”
BackgroundMacDougall, A. G., Price, E., Vandermeer, M. R. J., Lloyd, C., Bird, R., Sethi, R., Shanmugalingam, A., Carr, J., Anderson, K. K., & Norman, R. M. G. (2019) · Early Intervention in Psychiatry · doi:10.1111/eip.12753 · Cited by 24
Dunne, C. L., Fraser, J., & Gardner, G. E. (2013) · Midwifery · doi:10.1016/j.midw.2013.10.010 · Cited by 24
“A pilot study was conducted to assist with instrument testing and to forewarn the researcher about potential problems in the research process (Gardner et al., 2003; Leona et al., 2011).”
MethodIntegrative Couples Group Treatment for Emerging Adults With ADHD Symptoms
Wymbs, B. T., & Molina, B. S. G. (2015) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2014.06.008 · Cited by 24
Rico, T. M., dos Santos Machado, K., Fernandes, V. P., Madruga, S. W., Noguez, P. T., Barcelos, C. R. G., Santin, M. M., Petrarca, C. R., & Dumith, S. C. (2017) · Journal of Medical Systems · doi:10.1007/s10916-017-0831-3 · Cited by 21
“However, the purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study [43].”
BackgroundMindfulness-Based Intervention For Nurses In AIDS Care In China: A Pilot Study
Pan, C., Wang, H., Chen, M., Cai, Y., Xiao, C., Tang, Q., & Koniak-Griffin, D. (2019) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s223036 · Cited by 30
“Well designed and implemented pilot studies decrease the probability of problems that may arise in clinical trials.(37) The data from this study will enhance understanding about strategies that may be successful in implementing a mindfulness intervention with nurses caring for PLWH within a busy and stressful work env…”
BackgroundA feasibility study of a mobile app to treat insomnia
Aji, M., Glozier, N., Bartlett, D. J., Peters, D., Calvo, R. A., Zheng, Y., Grunstein, R. R., & Gordon, C. J. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibaa019 · Cited by 22
Otte, C., Chae, W. R., Nowacki, J., Kaczmarczyk, M., Piber, D., Roepke, S., Märschenz, S., Lischewski, S., Schmidt, S. H., Ettrich, B., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040119 · Cited by 23
Vance, D. E., Fazeli, P. L., Azuero, A., Wadley, V. G., Raper, J. L., & Ball, K. (2021) · AIDS and Behavior · doi:10.1007/s10461-021-03230-y · Cited by 23
A feasibility study of a peer-facilitated physical activity intervention in methadone maintenance
Abrantes, A. M., Noppen, D. V., Bailey, G. L., Uebelacker, L. A., Buman, M. P., & Stein, M. D. (2021) · Mental health and physical activity · doi:10.1016/j.mhpa.2021.100419 · Cited by 23
Gex, K. S., Mun, E., Barnett, N. P., McDevitt‐Murphy, M. E., Ruggiero, K. J., Thurston, I. B., Olin, C. C., Voss, A. T., Withers, A. J., & Murphy, J. G. (2022) · Psychology of Addictive Behaviors · doi:10.1037/adb0000838 · Cited by 24
Taylor, K. M., Bradley, J., & Cella, M. (2022) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12395 · Cited by 22
“As this study was not intended to detect effects through inferential statistical tests, power analysis was not required (Leon et al., 2011).”
Fioratti, I., Miyamoto, G. C., Fandim, J. V., Ribeiro, C. P. P., Batista, G. D., Freitas, G. E., Palomo, A. S., dos Reis, F. J. J., Costa, L. O. P., Maher, C. G., et al. (2022) · JMIR Formative Research · doi:10.2196/35743 · Cited by 23
“One important step before conducting a large clinical trial is to test the feasibility of the process involved in the main study, such as recruitment rates and the resources needed [18-20].”
BackgroundSolution Focused Financial Therapy: A Brief Report of a Pilot Study
Archuleta, K. L., Burr, E. A., Carlson, M. B., et al. (2015) · Journal of Financial Therapy · doi:10.4148/1944-9771.1081 · Cited by 23
“Results do indicate that a larger scale study is warranted (Leon, Davis, & Kraemer, 2011).”
Cook, P. F., Carrington, J. M., Schmiege, S. J., Starr, W., & Reeder, B. (2015) · Patient Preference and Adherence · doi:10.2147/ppa.s88222 · Cited by 23
“Although pilot studies do not necessarily produce reliable estimates of an intervention’s effects, they can be useful in identifying its more or less useful aspects and can guide revisions to make the intervention more efficacious in future full-scale RCTs.(22)”
Crave-Out: A Distraction/Motivation Mobile Game to Assist in Smoking Cessation
DeLaughter, K. L., Sadasivam, R. S., Kamberi, A., English, T. M., Seward, G., Chan, S. W., Volkman, J. E., Amante, D. J., & Houston, T. K. (2016) · JMIR Serious Games · doi:10.2196/games.4566 · Cited by 26
“Importantly, our study was designed to provide estimates of effect on cravings to guide future studies, and was not designed to have definitive results on this measure [13-17].”
Sebire, S. J., Edwards, M. J., Fox, K. R., Davies, B., Banfield, K., Wood, L., & Jago, R. (2016) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.2015-0217 · Cited by 23
Bossink, L. W. M., van der Putten, A. A., Waninge, A., et al. (2017) · Clinical Rehabilitation · doi:10.1177/0269215516687347 · Cited by 23
“0 (18, 100) QOL-PMD – Physical wellbeing (0-100) T0 17 62.”
Kim, M., Lee, H., Kiang, P. N., & Allison, J. J. (2019) · Health Education Research · doi:10.1093/her/cyz022 · Cited by 23
“Given the pilot nature of this study [38, 39], our”
BackgroundMace, R. A., Gates, M. V., Popok, P., Kulich, R. J., Quiroz, Y. T., & Vranceanu, A. (2020) · The Gerontologist · doi:10.1093/geront/gnaa179 · Cited by 25
“Nevertheless, signals of overall improvement indicate that both groups fit the target population and that the measures were sensitive to changes in outcomes (Leon et al., 2011).”
BackgroundThe Food Allergy Parent Mentoring Program: A Pilot Intervention
Ramos, A. M., Cooke, F., Miller, E., & Herbert, L. (2021) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsab019 · Cited by 23
Bibi, A., Margraf, J., & Blackwell, S. E. (2020) · Journal of Experimental Psychopathology · doi:10.1177/2043808720918030 · Cited by 26
“As a pilot rather than hypothesis-testing study, the sample size was determined by pragmatic considerations rather than to provide a certain level of statistical power (Leon et al., 2011).”
BackgroundMethodInfluentialCuijpers, A. C. M., Linskens, F. G., Bongers, B. C., Stassen, L. P. S., Lubbers, T., & van Meeteren, N. L. U. (2022) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2022.04.012 · Cited by 22
Mijnster, T., Boersma, G. J., Meijer, E., & Lancel, M. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12071070 · Cited by 24
Allison, J. J., Nguyen, H. L., Ha, D. A., Chiriboga, G., Ly, H. N., Tran, H. T. D., Phan, N. T., Nguyen, V. C., Kim, M., & Goldberg, R. J. (2016) · Trials · doi:10.1186/s13063-015-1147-6 · Cited by 22
“Leon, Davis, and Kraemer state that “power analyses should not be presented in an application for a pilot study that does not propose inferential results” [41].”
BackgroundMethodO’Connell, K., Pierlot, C. M., O’Shea, H., et al. (2016) · Journal of Biomedical Materials Research Part B Applied Biomaterials · doi:10.1002/jbm.b.33694 · Cited by 22
Schmutte, T. J., Davidson, L., & O’Connell, M. J. (2017) · Psychiatric Quarterly · doi:10.1007/s11126-017-9516-9 · Cited by 22
“Although consistent with the scope of a feasibility study [72], the uncontrolled, within-subjects design of this study provides no comparison group to address other threats to internal and external validity, such as maturation, Hawthorne and Rosenthal effects, or other factors not associated with the intervention that…”
ResultTelehealth intervention for Latina breast cancer survivors: A pilot
Meneses, K., Gisiger-Camata, S., Benz, R. L., Raju, D. A., Bail, J. R., Benitez, T. J., Pekmezi, D., & McNees, P. (2018) · Women s Health · doi:10.1177/1745506518778721 · Cited by 22
Bittlinger, M., & Müller, S. (2018) · BMC Medical Ethics · doi:10.1186/s12910-018-0275-4 · Cited by 23
“The proper purpose of first-in-human studies with very small sample size (n < 10) is feasibility [86, 94].”
BackgroundVoils, C. I., Adler, R., Strawbridge, E. M., Grubber, J. M., Allen, K. D., Olsen, M. K., McVay, M. A., Raghavan, S., Raffa, S. D., & Funk, L. M. (2020) · Health Psychology · doi:10.1037/hea0000835 · Cited by 26
“We did not calculate inferential statistics because statistics based on small sample sizes are unlikely to reflect populationbased estimates (Kraemer, Mintz, Noda, Tinklenberg, & Yesavage, 2006; Leon et al., 2011).”
BackgroundBeani, E., Menici, V., Sicola, E., Ferrari, A., Feys, H., Klingels, K., Mailleux, L., Boyd, R. N., Cioni, G., & Sgandurra, G. (2023) · European Journal of Physical and Rehabilitation Medicine · doi:10.23736/s1973-9087.23.07945-5 · Cited by 21
“…paradigm in the clinical environment. the tele-upcat has been developed to provide aot training directly at home. prior to examining effectiveness, 17, 18 a feasibility study was performed, 19 with positive results. in fact, the tele-upcat has already been assessed, in detail, for the…”
BackgroundUgwu, O. P.-C., Okon, M. B., Alum, E. U., Ugwu, C. N., Anyanwu, G. E., Basajja, M., Ogenyi, F. C., Eze, V. H. U., Anyanwu, C. N., Ezeonwumelu, J. O. C., et al. (2025) · Medicine · doi:10.1097/md.0000000000043542 · Cited by 29
“[148] These studies often include fewer than 100 people and lack the statistical power to prove a convincing therapeutic benefit of gut microbiota modification.”
Yao, D., Lu, C., Wen, Z.-H., et al. (2016) · Trials · doi:10.1186/s13063-016-1272-x · Cited by 21
“Since it was a pilot study instead of a hypothesis test, it is inappropriate to make any inferences or state findings of efficacy [14].”
BackgroundCarlyle, M., Rockliff, H., Edwards, R., Ene, C. G., Karl, A., Marsh, B., Hartley, L., & Morgan, C. J. A. (2019) · Substance Abuse Research and Treatment · doi:10.1177/1178221819836726 · Cited by 22
Leach, H. J., Potter, K., & Hidde, M. C. (2019) · Journal of Physical Activity and Health · doi:10.1123/jpah.2018-0667 · Cited by 21
Eaton, C. K., Comer, M., Pruette, C. S., Psoter, K. J., & Riekert, K. A. (2020) · Journal of Medical Internet Research · doi:10.2196/19861 · Cited by 22
“Given that this was a pilot study, results of our pilot RCT should be interpreted with caution and understood as an important step in refining our new intervention and informing whether further testing is warranted, rather than evidence of its efficacy [32-34].”
ResultSocial and Demographic Predictors of Consumers’ Word of Mouth Engagement in Czechia
Mladenović, D., Bruni, R., & Kalia, P. (2020) · Journal of International Consumer Marketing · doi:10.1080/08961530.2020.1800547 · Cited by 23
Minen, M. T., Schaubhut, K. B., & Morio, K. (2020) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2020.102489 · Cited by 21
Targeting executive function for weight loss in adults with overweight or obesity
Eichen, D. M., Pasquale, E. K., Twamley, E. W., & Boutelle, K. N. (2021) · Physiology & Behavior · doi:10.1016/j.physbeh.2021.113540 · Cited by 24
“NEXT demonstrated feasibility and acceptability, meeting the recommendations of what is to be explored in a pilot study [129].”
BackgroundMaster, H., Coronado, R. A., Whitaker, S., Block, S., Vanston, S. W., Pennings, J. S., Gupta, R., Robinette, P. E., Stephens, B. F., Abtahi, A. M., et al. (2023) · Physical Therapy · doi:10.1093/ptj/pzad096 · Cited by 24
Runfola, C. D., Zucker, N. L., Holle, A. F. V., Mazzeo, S. E., Hodges, E. A., Perrin, E. M., Bentley, M. E., Ulman, T. F., Hoffman, E. R., Forsberg, S., et al. (2013) · International Journal of Eating Disorders · doi:10.1002/eat.22178 · Cited by 21
Conducting Pilot and Feasibility Studies
Cope, D. (2015) · Oncology nursing forum · doi:10.1188/15.onf.196-197 · Cited by 39
Nguyen, H. L., Allison, J. J., Ha, D. A., Chiriboga, G., Ly, H. N., Tran, H. T. D., Nguyen, C. K., Dang, D. M., Phan, N. T., Nguyen, V. C., et al. (2017) · Pilot and Feasibility Studies · doi:10.1186/s40814-017-0136-9 · Cited by 21
“Leon, Davis, and Kraemer state that “power analyses should not be presented in an application for a pilot study that does not propose inferential results” [19].”
BackgroundMethodAcceptance and Commitment Therapy for Anger Dysregulation with Military Veterans: A Pilot Study
Donahue, J. J., Santanello, A. P., Marsiglio, M. C., & Male, L. M. V. (2017) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-017-9361-z · Cited by 22
“However, the pilot nature of this study should be considered when interpreting any inferential statistics, as results may be prone to both Type I and Type II error (Leon et al. 2011).”
BackgroundDeBiasse, M. A., Bowen, D. J., Quatromoni, P. A., Quinn, E. K., & Quintiliani, L. M. (2017) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2017.08.011 · Cited by 21
Nguyen, T. A., Nguyen, H., Pham, T., Nguyen, T. H., & Hinton, L. (2018) · Medicine · doi:10.1097/md.0000000000012553 · Cited by 22
Metadata
- Authors
- Leon, A. C., Davis, L. L., & Kraemer, H. C.
- Affiliations
- Weill Cornell Medicine (US)
- University of Alabama (US)
- Stanford University (US)
- Corresponding author
- Leon, Andrew C.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2011-05
- Volume
- 45
- Issue
- 5
- Pages
- 626-629
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC3081994
- OpenAlex
- W2168434488
- Cited by
- 1,917
- References
- 17
- Citations by year
- 2026: 203 · 2025: 190 · 2024: 143 · 2023: 169 · 2022: 163 · 2021: 217 · 2020: 169 · 2019: 136 · 2018: 128 · 2017: 120 · 2016: 92 · 2015: 80 · 2014: 52 · 2013: 31 · 2012: 17
- FWCI
- 14.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pilot study, feasibility assessment, clinical trials, research design, sample size determination, effect size estimation, generalizability, recruitment and retention, intervention implementation, research funding review
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Feasibility Studies, Humans, Pilot Projects, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
