The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The role and interpretation of pilot studies in clinical research
Open accessAbstract
Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
40 Own interpretation · 5 No basis given · 2 No full text of the sources · 1 Sources not found
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“When event rates are reported and no adverse event is observed for a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
What the sources say
- Not matchedcites Jovanovic & Levy, 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Jovanovic et al., 1997
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Basing a decision to proceed or terminate evaluation of a particular intervention on pilot data is perilous because there is a very good chance that the decision will be derived from false positive or false negative results.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each aspect of feasibility could be quite different from an uncontrolled study when intervention assignment is randomized and blinded, particularly if placebo is a distinct possibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these issues in mind, a pilot study can better address its goals if a control group is part of the design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Rationale for a Control or Comparison Group in a Pilot
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A case series estimate tends to be steeped in optimism, particularly if an estimate of such magnitude is seldom, if ever, seen in full scale trials for psychiatric disorders.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(It is the population effect size, not the sample effect size, that an RCT is designed to detect with sufficient power.)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“There has been a venerable tradition of using pilot studies to estimate between group effect sizes that, in turn, are used to inform the design of subsequent larger scale hypothesis testing studies. Despite it widespread use, it has been argued that the tradition is ill-founded. (Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006)”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot sample size is instead based on the pragmatics of recruitment and the necessities for examining feasibility.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With no inferential statistical tests, a pilot study will not provide p-values.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot study data generally should not be combined with data from the subsequent larger scale study. This is because it is quite likely that the methods will be modified after the pilot, even if minimally. Such changes in protocol risk adding additional, perhaps unknown, source of variation.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Are Pilot Data Included in the Larger Trial?
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Methods will likely be modified after a pilot study, even if minimally.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 13
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study.”
- Also stated in: § DISCUSSION · plain
“Pilot results inform feasibility, which in turn, is instructive in that it points to modifications needed in the planning and design of a larger efficacy trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The definition of clinically meaningful is not entirely empirically-based, but instead requires input from clinicians who treat the patient population of interest and perhaps from patients with the disorder.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These good clinical practices include the refinement of source documentation, informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring/oversight procedures, especially when multiple sites and investigators are engaged in the study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety, efficacy and effectiveness are not evaluated in a pilot.”
- Also stated in: § DISCUSSION · plain
“a pilot study is not a hypothesis testing study and therefore safety and efficacy are not evaluated.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § DISCUSSION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pilot studies generate pilot data, their design need not be guided with the support of prior pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible, but highly unlikely, that the between group effect size (d) from a pilot study sample will provide a reasonable estimate of the population effect size (Δ), but that cannot be known based on the pilot data.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study can be critical in research staff training and provide experiences that strengthen and confirm competencies and skills required for the investigation to be conducted with accuracy and precision.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An alternative approach is to base sample size estimates for a full scale RCT on what is deemed a clinically meaningful effect.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > An Alternative to Using Pilot Data for Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot study provides opportunity to develop consistent practices to enhance data integrity and the protection of human subjects.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY > Good Clinical Practices
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We learn little if anything about the efficacy of an intervention with data from a small sample; yet, as discussed earlier, a great deal can be learned from a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This applies to effects sizes for any type of outcome, be it a severity rating, a response status, or survival status.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, with imprecision comes a vast disparity in sample size estimates and, if sample size determination for a subsequent study is based on an imprecise estimate, there is an enormous risk of underpowered or overpowered design.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Hypothetical Example
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, pilot studies are a necessary first step in exploring novel interventions and novel applications of interventions – whether in a new patient population or with a novel delivery system (e.g., transdermal patch).”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although a pilot study will undoubtedly incorporate relevant outcome measures and can serve a vital role in treatment development, it is not, and should not, be considered a preliminary test of the intervention hypothesis.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The primary role of a pilot study is to examine the feasibility of a research endeavor.”
- Also stated in: § INTRODUCTION · plain
“The fundamental purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to ultimately be used in a larger scale study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This does not preclude the need for a theoretical rationale for the intervention or the methodology being proposed for a pilot study.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a pilot study effect size is unduly large (i.e., a false positive result), subsequent trials will be designed with an inadequate number of participants to provide the statistical power needed to detect clinically meaningful effects and that would lead to negative trials.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, pilot studies provide an opportunity to implement and examine the feasibility of the adverse event reporting system.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Tolerability and Preliminary Safety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is quite reasonable and expected that a pilot study is proposed with no pilot or other preliminary data supporting the proposal and that its proposed sample size is based on pragmatics such as patient flow and budgetary constraints.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > SAMPLE SIZE DETERMINATION IN DESIGNING A PILOT STUDY > Pilot Data for a Pilot Study
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention.”
- Also stated in: § INTRODUCTION · plain
“evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and/or implementation of the novel intervention.”
- Also stated in: § WHAT A PILOT STUDY CAN DO: ASSESS FEASIBILITY · plain
“A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, and implementation of the novel intervention and each of these can be quantified”
§ The Role and Interpretation of Pilot Studies in Clinical Research > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § INTRODUCTION · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are two fundamental reasons that hypothesis testing is not used in a pilot study: the limited state of knowledge about the methods or intervention in the patient population to be studied and the smaller proposed sample size.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a consequence, pilot studies can reduce the proportion of failed trials and allow research funds to be spent on projects for which feasibility has been demonstrated and quantified.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Well-conceived and implemented pilot studies will reduce the risk of several problems that are commonly faced in clinical trials. These include the inability to recruit the proposed sample size and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures or interventions, and the need to modify a protocol midway through a trial.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > DISCUSSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Estimates become more precise with larger sample sizes.”
§ The Role and Interpretation of Pilot Studies in Clinical Research > EXCEEDING THE LIMITATIONS OF A PILOT STUDY: WHAT PILOT STUDIES CANNOT DO > Pilot Study Effect Sizes and Sample Size Determination
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The true population effect size is unknown when a pilot study is designed.
“An essential challenge of therapeutic development is that the true population effect size is unknown at the time a pilot study is designed.”
The paper relies on this without support.
Methods will likely be modified after a pilot study, even if minimally.
“This is because it is quite likely that the methods will be modified after the pilot, even if minimally.”
The paper relies on this without support.
Pilot results presented to grant review committees tend to be selective, overly optimistic, and at times misrepresentational.
“Furthermore, pilot results that are presented to grant review committees tend to be selective, overly optimistic and, at times, misrepresentational.”
The paper relies on this without support.
Many pilot studies have far fewer than 18 participants per group.
“Many pilot studies have far fewer than 18 participants/group and therefore even greater imprecision.”
The paper relies on this without support.
It is not unusual for research grant applications and pilot study publications to convey optimism based on pilot data.
“Nevertheless, it is not unusual for research grant applications and pilot study publications to convey such optimism, particularly when based on pilot data.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The planned sample size was 30 allowing for attrition. This study will inform the sample size for any future RCT (Leon et al., 2011).”
From A mixed methods pilot and feasibility open trial of internet-delivered cognitive behaviour therapy (iCanADAPT Advanced) for people with advanced cancer with depression and/or anxiety (2021) · cites this paper as Leon et al., 2011
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was a pilot study which like other pilot studies of MI did not have a power calculation. 31,32”
From A Pilot Randomized Controlled Trial to Assess the Impact of Motivational Interviewing on Initiating Behavioral Therapy for Migraine (2020) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data analyses in the context of a pilot study are not hypothesis-driven but serve to provide information regarding feasibility and acceptability of the intervention [55].”
From Acceptance and Commitment Therapy Wellness Program for Latine Adults Who Smoke and Have Psychological Distress: Protocol for a Feasibility Study (2023) · cites this paper as [55]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies.”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [2, 20, 21, 23].”
From Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back! (2021) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Since the focus was on feasibility and acceptability of the intervention and we did no hypothesis testing, no power calculation was required (Leon et al., 2011).”
From PandaMom – Feasibility and acceptability of an internet- and mobile-based intervention to enhance peripartum mental well-being and to prevent postpartum depression (2024) · cites this paper as Leon et al., 2011
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our use of effect sizes was, however, deemed appropriate for this pilot trial (55, 63).”
From EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU (2024) · cites this paper as [63]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work suggests early studies often test feasibility of methods (e.g., recruitment, randomization, retention, and assessment), not hypotheses [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [48]
Citations
Cites
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · doi:10.1007/978-1-4757-2915-3 · Cited by 873
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · cited 1×
“(Friedman, Furberg and DeMets, 1998; Kraemer et al., 2006) Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small samples.”
Self-citationWolfstein (1905) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-190502000-00009 · Cited by 346
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · cited 1×
“(Leon et al., 2006; Leon & Davis, 2009).”
Self-citationSixteen S ‐squared over D ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · cited 1×
“(N/group ≈ 16/d2; Lehr, 1992) The clinical interpretation of HAMD change of 1.2 to 3.0 would drive the choice among possible sample sizes in planning a study.”
BackgroundInfluentialJovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.1080/00031305.1997.10473947 · Cited by 179 · cited 1×
“…a particular category, the rule of three should be applied to estimate the upper bound of the 95% CI, where the upper bound is approximately 3/n. (Jovanovic & Levy, 1997; Jovanovic et al., 1997) For example, consider a study with N=15 receiving medication and no suicidal ideation was reported…”
BackgroundHarvey, I. M. (2009) · Public Health · doi:10.1016/j.puhe.2009.05.009 · Cited by 173
Jovanovic, B. D., & Levy, P. S. (1997) · The American Statistician · doi:10.2307/2685405 · Cited by 50
Cited by
Sorted by citations
Items 601–700 of 1,903
Yao, D., Lu, C., Wen, Z.-H., et al. (2021) · Annals of Palliative Medicine · doi:10.21037/apm-20-2575 · Cited by 13
“So, it is inappropriate to make any inferences or state findings of efficacy, as this was a pilot study, as opposed to a hypothesis test (20).”
ResultFeasibility of a Manualized Mindful Yoga Intervention for Patients With Chronic Mood Disorders
Vollbehr, N. K., Hoenders, R. H. J., Bartels‐Velthuis, A. A., & Ostafin, B. D. (2021) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000539 · Cited by 12
Flores, E. J., Neil, J. M., Tiersma, K. M., Pappano, C. R., Mulligan, C. K., Alphen, M. U. V., Park, E. R., & Irwin, K. E. (2021) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2021.07.010 · Cited by 13
Evaluating a Therapeutic Powered Mobility Camp for Children with Severe Cerebral Palsy
Rosenberg, L., Maeir, A., & Gilboa, Y. (2021) · Canadian Journal of Occupational Therapy · doi:10.1177/00084174211034938 · Cited by 13
“…full study, this is nevertheless permissible when a study uses validated tools and is primarily concerned with methodological aspects, especially when the sample is small (van Teijlingen & Hundley, 2002) and when the protocol changes are minimal and clearly explained (Leon et al., 2011).”
BackgroundMethodWorking memory training for adult glioma patients: a proof-of-concept study
Braun, S. E., Aslanzadeh, F. J., Lanoye, A. M., Fountain‐Zaragoza, S., Malkin, M. G., & Loughan, A. R. (2021) · Journal of Neuro-Oncology · doi:10.1007/s11060-021-03839-y · Cited by 13
Littman, A. J., Bratman, G. N., Lehavot, K., Engel, C. C., Fortney, J. C., Peterson, A. C., Jones, A., Klassen, C., Brandon, J., & Frumkin, H. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051885 · Cited by 19
Implementation of a youth violence prevention programme in primary care
Roche, J., Philyaw-Kotov, M. L., Sigel, E., Eisman, A. B., Wernette, G. T., Resnicow, K., Carter, P. M., Cunningham, R. M., & Walton, M. A. (2021) · Injury Prevention · doi:10.1136/injuryprev-2021-044293 · Cited by 12
Seah, B., Espnes, G. A., Hong, W. T., & Wang, W. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03605-3 · Cited by 15
“As this was a pilot study, the sample size was determined based on the pragmatics of recruitment and essentials for examining feasibility [46], adequate to pilot two rounds of intervention.”
MethodWisting, L., Haugvik, S., Wennersberg, A. L., Hage, T. W., Stice, E. M., Olmsted, M. P., Ghaderi, A., Brunborg, C., Skrivarhaug, T., Dahl‐Jørgensen, K., et al. (2024) · Eating Disorders · doi:10.1080/10640266.2024.2331391 · Cited by 13
“Though pilot studies like this cannot determine a program’s causal effects (Leon et al., 2011), they are valuable in informing the potential for a future RCT (Eldridge et al., 2016).”
Pilot Randomized Controlled Trial of a Syndemics Intervention with HIV-Positive, Cocaine-Using Women
Jemison, D., Jackson, S., Oni, O., Cats-Baril, D., Thomas-Smith, S., Batchelder, A., Rodriguez, A. E., Dilworth, S. E., Metsch, L. R., Jones, D. L., et al. (2019) · AIDS and Behavior · doi:10.1007/s10461-019-02625-2 · Cited by 13
Kapoor, H., Nisiewicz, M. J., Jayavarapu, R., et al. (2020) · Journal of Clinical Imaging Science · doi:10.25259/jcis_173_2019 · Cited by 13
“[5] Although thermal ablation with microwave has been theoretically shown to confer benefits such as overcoming problems of tissue charring and vascular heat sink,[6,7] no definitive evidence supports the superiority of microwave over radiofrequency ablation (RFA).[8,9] Importantly, the abovementioned practical and th…”
BackgroundFox, K., Gans, K. M., McCurdy, K., Risica, P. M., Jennings, E., Gorin, A. A., Papandonatos, G. D., & Tovar, A. (2020) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2020.100583 · Cited by 14
“are often too small for robust estimates [137,138].”
BackgroundShallwani, S. M., Towers, A., Newman, A., Salvador, S., Yung, A., Gilbert, L., Gotlieb, W. H., Zeng, X. Z., & Thomas, D. (2021) · Current Oncology · doi:10.3390/curroncol28010048 · Cited by 12
“They allow us to explore feasibility outcomes and to examine the potential effectiveness of novel interventions [31,32].”
BackgroundBurnette, C. B., Davies, A., & Mazzeo, S. E. (2021) · Eating Disorders · doi:10.1080/10640266.2021.1877886 · Cited by 13
“Thus, it is vital to collect feasibility and acceptability data prior to developing larger-scale randomized-controlled trials (RCTs; Leon et al., 2011).”
BackgroundJimenez, D. E., Aschbrenner, K. A., Burrows, K., Pratt, S. I., Alegria, M. M., & Bartels, S. J. (2014) · Journal of Latina/o Psychology · doi:10.1037/lat0000020 · Cited by 13
Tailored communications for obesity prevention in pediatric primary care: a feasibility study
Wright, J. A., Whiteley, J. A., Watson, B. L., Gorin, S. S., & Hayman, L. L. (2017) · Health Education Research · doi:10.1093/her/cyx063 · Cited by 15
Nielsen, S. S., Skou, S. T., Larsen, A. E., Polianskis, R., Pawlak, W. Z., Vægter, H. B., Søndergaard, J., & Christensen, J. R. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-060920 · Cited by 12
The impact of livestreaming e-commerce on consumer purchase intention in China
Sima, H., Wu, S., Yan, Z. J., Luo, N., & Chen, Y. (2025) · Journal of Consumer Marketing · doi:10.1108/jcm-02-2024-6564 · Cited by 20
Liljehult, J., Mølsted, S., Møller, T., Overgaard, D., Adamsen, L., Jarden, M., & Christensen, T. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00583-4 · Cited by 12
Developing Signs of Safety: A Deaf-Accessible Counseling Toolkit for Trauma and Addiction
Anderson, M. L., Glickman, N., Craig, K. S. W., Crane, A. K. S., Wilkins, A. M., & Najavits, L. (2021) · Clinical Psychology and Psychotherapy · doi:10.1002/cpp.2596 · Cited by 7
“Our sample of 15 participants was based on the limited scope of a stage IA pilot study (Leon et al., 2011) and was underpowered to detect a large effect using inferential statistics (e.g., t test, GEE modelling).”
MethodPembroke, M., Bradley, J. A., Mueller, M., Mollica, M. A., & Nemeth, L. S. (2021) · Oncology nursing forum · doi:10.1188/21.onf.279-290 · Cited by 12
“A power analysis was not conducted because of the core focus of feasibility studies and the answering of the question, “Can this intervention be carried out?” prior to further investment of time and resources (Bowen et al., 2009; Leon et al., 2011; Tickle-Degnen, 2013).”
Engaging primary care patients with existing online tools for weight loss: A pilot trial
McVay, M. A., Cooper, K. B., Donahue, M. L., Seoane, M. C., Shah, N. R., Webb, F. J., Perri, M. G., & Jake‐Schoffman, D. E. (2022) · Obesity Science & Practice · doi:10.1002/osp4.592 · Cited by 12
Fernandes, C.-S. F., Deng, Y., Tran, A. H., Hieftje, K. D., Pendergrass, T. M., Taylor, C. K., & Fiellin, L. E. (2022) · Games for Health Journal · doi:10.1089/g4h.2021.0118 · Cited by 13
“While the findings are promising, they must be interpreted in the context of a small sample size 50 and that the study was conducted in one school setting.”
Harris‐Hayes, M., Zorn, P., Steger-May, K., Burgess, M. M., DeMargel, R. D., Kuebler, S., Clohisy, J. C. F., & Haroutounian, S. (2023) · Physical Therapy · doi:10.1093/ptj/pzad111 · Cited by 12
“Using common principles associated with joint mobilization, we developed a standardized joint mobilization protocol that allows for individualization of treatment parameters within an RCT.”
MethodInfluentialUse of Relational Agents to Improve Family Communication in Type 1 Diabetes: Methods
Thompson, D. I., Cullen, K. W., REDONDO, M. J., & Anderson, B. J. (2016) · JMIR Research Protocols · doi:10.2196/resprot.5817 · Cited by 12
“Feasibility studies are designed to contribute to a well-informed main trial [36-38] and are the first step in intervention development [37-39].”
Milbury, K., Tsao, A. S., Liao, Z. X., Owns, A., Engle, R., Gonzalez, E. A., Bruera, E. D., & Cohen, L. G. (2018) · Pilot and Feasibility Studies · doi:10.1186/s40814-018-0231-6 · Cited by 13
Determining Sample Size Using Fast and Slow Thinking
Dattalo, P. (2018) · Journal of Social Service Research · doi:10.1080/01488376.2018.1436632 · Cited by 12
de Andrade, G. N., Umeda, I. I. K., Fuchs, A., et al. (2021) · Clinics · doi:10.6061/clinics/2021/e2550 · Cited by 12
“We did not estimate the effect size because of the inherent data imprecision from small samples (31).”
BackgroundLarsen, B. A., Greenstadt, E., Olesen, B., Osuna, L., Godino, J., Marcus, B. H., DUNSIGER, S. I., Meyer, D. M., & Zive, M. (2022) · Trials · doi:10.1186/s13063-022-06105-2 · Cited by 12
“Consistent with expert opinion [55, 56], we integrated”
Gray, H. G., Tchao, D., Lewis-Fung, S. E., Pardini, S., Harris, L. R., & Appel, L. (2022) · JMIR Research Protocols · doi:10.2196/41523 · Cited by 11
Saeidnia, H. R., Kozak, M., Ausloos, M., Lund, B. D., Ghorbi, A., & Mohammadzadeh, Z. (2023) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2023.101310 · Cited by 13
Ngan, H. Y., Chong, Y. Y., Loo, K. M., & Chien, W. T. (2023) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2023.09.002 · Cited by 14
Esber, A. L., Moyo, P., Munjoma, M. W., Francis, S. A., van de Wijgert, J. H. H. M., Chipato, T., & Turner, A. N. (2014) · Sexually Transmitted Infections · doi:10.1136/sextrans-2014-051764 · Cited by 12
Acceptability of a Social Support Intervention for Re-entering Prisoners
Pettus‐Davis, C., Howard, M. O., Murugan, V., Roberts-Lewis, A. C., Scheyett, A., Botnick, C., & Vance, M. M. (2015) · Journal of the Society for Social Work and Research · doi:10.1086/680340 · Cited by 14
“individual format), discuss appropriateness of targeted recipient group characteristics, examine the extent of satisfaction with the intervention approach, and highlight factors that facilitate recipients’ completion of the intervention (Leon et al., 2011; Moore, Carter, Nietert, & Stewart, 2011; Rounsaville et al., 2…”
BackgroundEllis‐Hill, C., Gracey, F., Thomas, S., Lamont‐Robinson, C., Thomas, P., Marques, E. M. R., Grant, M., Nunn, S., Cant, R., Galvin, K., et al. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008888 · Cited by 12
“Statistical analysis This is a feasibility study and so the aim is not to provide a fully powered hypothesis test of the effectiveness of the intervention.(75) Instead, analyses will focus on estimating key feasibility study parameters.”
BackgroundHealth behavior change benefits: Perspectives of Latinos with serious mental illness
Jimenez, D. E., Burrows, K., Aschbrenner, K. A., Barré, L. K., Pratt, S. I., Alegria, M. M., & Bartels, S. J. (2016) · Transcultural Psychiatry · doi:10.1177/1363461516632388 · Cited by 12
Giorgi, G., Ferrarello, F., Merlo, F., et al. (2016) · Journal of Geriatric Physical Therapy · doi:10.1519/jpt.0000000000000113 · Cited by 12
Aschbrenner, K. A., Patten, C. A., & Brunette, M. F. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx033 · Cited by 12
“Third, this pilot study was not designed to test an efficacy hypothesis; rather, it was designed as an initial step to explore the feasibility and potential benefits of a novel intervention [31, 32].”
Meyer, A.-L., & Leboeuf‐Yde, C. (2018) · Chiropractic & Manual Therapies · doi:10.1186/s12998-018-0198-7 · Cited by 12
The Sequential Scale-Up of an Evidence-Based Intervention: A Case Study
Thomas, J., Cook, T. D., Klein, A., Starkey, P., & DeFlorio, L. (2018) · Evaluation Review · doi:10.1177/0193841x18786818 · Cited by 13
Muhanna, A. M., & Floyd, M. J. (2018) · Radiography · doi:10.1016/j.radi.2018.10.003 · Cited by 14
Zhang, X., Li, H., Lv, Y., Zhu, Z., Shen, X., Lu, Q., Wang, W., Wang, Z., Jiang, Z., Yang, L., et al. (2019) · Frontiers in Neurology · doi:10.3389/fneur.2019.01062 · Cited by 12
“The initial sample size of our study was estimated based on the available studies and the pragmatics of recruitment and the necessities for examining feasibility (26).”
MethodAcademic Perspectives and Approaches to Social Media Use in Higher Education: A Pilot Study.
Sutherland, K. E., Terton, U., Davis, C., et al. (2020) · International journal on teaching and learning in higher education · Cited by 12
Yoga’s impact on risk and protective factors for disordered eating: a pilot prevention trial
Pacanowski, C. R., Diers, L., Crosby, R., MacKenzie, M. J., & Neumark‐Sztainer, D. (2020) · Eating Disorders · doi:10.1080/10640266.2020.1763110 · Cited by 11
Promoting Active Aging: Lessons Learned in an Implementation Pilot in Primary Care
Callahan, K. E., Willard, J. J., Foley, K. L., Miller, M. E., Houston, D. K., Kritchevsky, S. B., Williamson, J. D., Applegate, W. B., Girma, F., Whitehead, S. E., et al. (2020) · Journal of the American Geriatrics Society · doi:10.1111/jgs.16838 · Cited by 12
Zisman‐Ilani, Y., Hurford, I. M., Bowen, A., Salzer, M. S., & Thomas, E. C. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00757-0 · Cited by 13
“Given that this study’s purposes were to assess feasibility of intervention delivery and research procedures and to collect preliminary data about the intervention potential effectiveness and related patient- and intervention-level factors, the sample size was based on pragmatics rather than on power and is consistent…”
ResultMajor, D. H., Grotle, M., Littlewood, C., Brox, J. I., Matre, D., Gallet, H. V., & Røe, Y. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00767-6 · Cited by 14
Zhou, Y., & Wade, T. D. (2021) · Journal of Behavioral and Cognitive Therapy · doi:10.1016/j.jbct.2020.11.005 · Cited by 13
Cramer, H. (2021) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2021.0197 · Cited by 12
“6 However, the ABSENCE of adverse events in a pilot has no meaning whatsoever, since this may simply be due to a lack of power.”
Levine, M. S., Fox, N. L., Thompson, B., Taylor, W., Darlington, A. C., Hoeden, J. V. D., Emmett, E. A., & Rutten, W. (1986) · PubMed · Cited by 12
Open science practices for eating disorders research
Burke, N. L., Frank, G., Hilbert, A., Hildebrandt, T., Klump, K. L., Thomas, J. J., Wade, T. D., Walsh, B. T., Wang, S. B., & Weissman, R. S. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23607 · Cited by 12
Makanjuola, A., Lynch, M., Hartfiel, N., Cuthbert, A., Wheeler, H. T., & Edwards, R. T. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191710658 · Cited by 12
“Social Return on Investment Methodology The National Institute for Health and Care Excellence (NICE) advocates the use of both cost–benefit analysis (CBA) and cost–utility analysis (CUA) for evaluating public health interventions [14].”
BackgroundApelin-17 to diagnose idiopathic pulmonary arterial hypertension: A biomarker study
Foris, V., Kovács, G., Avian, A., Bálint, Z., Douschan, P., Ghanim, B., Klepetko, W., Olschewski, A., & Olschewski, H. (2023) · Frontiers in Physiology · doi:10.3389/fphys.2022.986295 · Cited by 12
“This adaptive design allowed us to include a sufficient number of patients in a reasonable period of time (Leon et al., 2011; Whitehead et al., 2016).”
MethodOpen pilot trial of an interactive digital application for campus sexual violence prevention
Orchowski, L. M., Zinzow, H. M., Thompson, M. P., & Wood, S. (2023) · Journal of Community Psychology · doi:10.1002/jcop.22985 · Cited by 13
Preliminary Efficacy of Let’s Talk Tech: Technology Use Planning for Dementia Care Dyads
Berridge, C. W., Turner, N. R., Liu, L., Fredriksen‐Goldsen, K. I., Lyons, K. S., Demiris, G., Kaye, J. A., & Lober, W. B. (2023) · Innovation in Aging · doi:10.1093/geroni/igad018 · Cited by 16
“The goal of the pilot study was to test the preliminary efficacy of a novel, self-administered intervention and identify appropriate primary outcome measures from preliminary testing (Lancaster, 2015; Leon et al., 2011).”
Herrera, Y. M., Vadiveloo, M. K., Blau, S., Oaks, B. M., Quashie, N. T., & Tovar, A. (2024) · Appetite · doi:10.1016/j.appet.2024.107647 · Cited by 11
Pressman, A. R., Law, H., Stahl, R., Scott, A., Jacobson, A. S., Dean, L., Sudat, S. E. K., Obillo, A., & Avins, A. L. (2019) · Trials · doi:10.1186/s13063-019-3355-y · Cited by 11
Effects of Physical Activity on Neurophysiological and Gene Expression Profiles in Chronic Back Pain
Kim, K., Ramesh, D., Perry, M., Bernier, K. M., Young, E. E., Walsh, S. J., & Starkweather, A. R. (2019) · Nursing Research · doi:10.1097/nnr.0000000000000400 · Cited by 14
Waring, M. E., Libby, B. A., Simas, T. A. M., Bracken, M. L., Oleski, J. L., Herrera, V., Wang, J., & Pagoto, S. (2019) · JMIR Research Protocols · doi:10.2196/15530 · Cited by 11
“Thus, as recommended [81,83], we will calculate sample size requirements for the subsequent full-scale noninferiority trial based on a clinically meaningful noninferiority margin for percent weight loss and variance in weight loss observed in previous adequately powered trials of postpartum weight…”
InfluentialMichelson, K. N., Charleston, E., Aniciete, D. Y., Sorce, L. R., Fragen, P., Persell, S. D., Ciolino, J. D., Clayman, M. L., Rychlik, K. L., Jones, V., et al. (2020) · American Journal of Critical Care · doi:10.4037/ajcc2020478 · Cited by 12
Accessible Virtual Arts Recreation for Wellbeing Promotion in Long-Term Care Residents
Murphy, K. J., Swaminathan, S., Howard, E., Altschuler, A., Rogan, J., Beauchet, O., Dupuis, K., Galea, L. A. M., Hogan, D. B., Lingum, N. R., et al. (2020) · Journal of Applied Gerontology · doi:10.1177/0733464820967195 · Cited by 13
Taney, K. G., Smith, M. M., Cummings, N. P., & Lozano, A. J. (2021) · Frontiers in Veterinary Science · doi:10.3389/fvets.2021.762842 · Cited by 12
“We recognize the limitations of effect size estimates from studies with small samples, including imprecise and biased estimates (55); however, clinically meaningful effects in this sample have been described and can be used to inform larger studies that may improve the knowledge base and further the goal of reducing s…”
BackgroundA Novel Movement-Evoked Pain Provocation Test for Older Adults With Persistent Low Back Pain
Simon, C. B., Hicks, G. E., Pieper, C. F., Kraus, V. B., Keefe, F. J., & Colon-Emeric, C. S. (2023) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000001101 · Cited by 11
Ng, L. C., Miller, A. N., Bowers, G., Cheng, Y., Brigham, R., Tai, M. H., Smith, A. M., Mueser, K. T., Fortuna, L. R., & Coles, M. S. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104310 · Cited by 10
“With a small underpowered sample size, the within-group change estimates are highly imprecise and do not allow for accurate, unbiased, inferential statistics (Leon et al., 2011; Sim, 2019; Thabane et al., 2010).”
BackgroundLiu, T., Martinez-Torres, K. A., Mazzone, J., Camarata, S., & Lense, M. D. (2023) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-023-06053-z · Cited by 11
Determinants of Employee Engagement in Hotel Industry in Malaysia. A Theoretical Model
Zainol, S. S. B., Hussin, S. B. M., & Othman, M. S. B. (2016) · International Journal of Academic Research in Accounting Finance and Management Sciences · doi:10.6007/ijarafms/v6-i3/2146 · Cited by 12
Bunnell, B. E., Davidson, T. M., Hamblen, J. L., Cook, D. L., Grubaugh, A. L., Lozano, B. E., Tuerk, P. W., & Ruggiero, K. J. (2017) · Pilot and Feasibility Studies · doi:10.1186/s40814-017-0121-3 · Cited by 11
Crane, S. M., Backus, L. R., Stockman, B., Carpenter, J. S., Lin, L., & Haase, J. E. (2017) · Journal of Pediatric Oncology Nursing · doi:10.1177/1043454217723863 · Cited by 11
Effect of remifentanil on postoperative nausea and vomiting: a randomized pilot study
Watanabe, T., Moriya, K., Tsubokawa, N., et al. (2018) · Journal of Anesthesia · doi:10.1007/s00540-018-2550-4 · Cited by 11
de Assis, G. A., Brandão, A. F., Corrêa, A. G. D., & Castellano, G. (2019) · Journal on Interactive Systems · doi:10.5753/jis.2019.717 · Cited by 11
“the risk of several problems that are commonly faced in clinical trials [39].”
BackgroundMayo‐Wilson, L. J., Coleman, J., Timbo, F., Ssewamala, F. M., Linnemayr, S., Yi, G. T., Kang, B., Johnson, M. W., Yenokyan, G., Dodge, B., et al. (2020) · AIDS and Behavior · doi:10.1007/s10461-020-02931-0 · Cited by 11
The emotional dance with depression: A longitudinal investigation of OULA® for depression in women
Hellem, T., Sung, Y., Ferguson, H., & Hildreth, L. A. (2020) · Journal of Bodywork and Movement Therapies · doi:10.1016/j.jbmt.2020.05.007 · Cited by 11
Chu, K., Matheny, S. J., Furek, A., Sidani, J. E., Radio, S., Miller, E. P., Valente, T. W., & Robertson, L. B. (2021) · Trials · doi:10.1186/s13063-020-04990-z · Cited by 11
Femi-Ajao, O. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18042024 · Cited by 11
Sarris, J. J., Byrne, G. J., Oliver, G., Cribb, L., Blair‐West, S., Castle, D. J., Dean, O. M., Camfield, D., Brakoulias, V., Bousman, C., et al. (2021) · CNS Spectrums · doi:10.1017/s1092852921000638 · Cited by 12
“Firstly, it was an openlabel design (though as noted previously placebo response rates may be lower in OCD than comparable disorders, and such pilot studies are crucial to determine feasibility for future larger RCT designs).(42,71) Another limitation was that while the study included people with OCD who were on “trea…”
ResultLysne, P. E., Palit, S., Moraís, C. A., DeMonte, L. C., Lakdawala, M., Sibille, K. T., & Bartley, E. J. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00923-y · Cited by 13
“Because of the potential for high variability and imprecision in outcomes in pilot samples [89, 90, 92], differences across baseline and post-intervention measures will be analyzed using descriptive statistics with means, standard deviations, and ranges reported.”
MethodGroup radical openness: A feasibility study
Egan, R., Long, E., McElvaney, J., & Booth, R. (2021) · Counselling and Psychotherapy Research · doi:10.1002/capr.12480 · Cited by 13
Briggs, S. M., Lipoff, J. B., & Collier, S. M. (2022) · JMIR Dermatology · doi:10.2196/33833 · Cited by 11
“We did not perform statistical hypothesis testing, in accordance with best practices [10].”
MethodVilardaga, J. P., Kelleher, S. A., Diachina, A. K., Riley, J. A., & Somers, T. J. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01121-0 · Cited by 13
“Secondary outcome analyses were conducted to examine possible indicators of improvement and given the small sample size should be interpreted with caution [47].”
BackgroundRandomized controlled trials in neurosurgery
Takroni, R., Sharma, S., Reddy, K. K. V., Zagzoog, N., Aljoghaiman, M. S., Alotaibi, M. B., & Farrokhyar, F. (2022) · Surgical Neurology International · doi:10.25259/sni_1032_2021 · Cited by 11
Zatzick, D. (2012) · Depression and Anxiety · doi:10.1002/da.21912 · Cited by 11
“We envisioned that such measures could early on in the intervention development process inform feasibility assessments targeting breadth of applicability metrics for populations of injured trauma survivors.[14] Breadth of applicability indices had the potential to compliment assessments of intervention treatment effec…”
BackgroundIndrayanti, I., Salsabila, A. K., Amrita, V., Alhaddad, M. M., Saskia, A. B., & Ramadhani, D. P. (2025) · SSM - Mental Health · doi:10.1016/j.ssmmh.2025.100504 · Cited by 15
Menici, V., Antonelli, C., Beani, E., Mattiola, A., Giampietri, M., Martini, G., Rizzi, R., Cecchi, A., Cioni, M. L., Cioni, G., et al. (2021) · Frontiers in Pediatrics · doi:10.3389/fped.2021.673956 · Cited by 10
Luberto, C. M., Wang, A., Li, R., Pagliaro, J. A., Park, E. R., & Bhatt, A. (2022) · International Journal of Cardiology Congenital Heart Disease · doi:10.1016/j.ijcchd.2022.100324 · Cited by 10
McClure, J. B., Heffner, J. L., Krakauer, C. A., Mun, S., Klasnja, P., & Catz, S. L. (2023) · JMIR mhealth and uhealth · doi:10.2196/46155 · Cited by 10
“Consistent with the purpose of a pilot trial [21], the goal of this work is to provide proof of concept for the app’s feasibility and acceptability.”
MethodPeterson, C. M., Grange, D. L., Mara, C. A., Modi, A. C., Offenbacker, E. J., Zegarac, M. C., Stevens, K. S., Matthews, A. L., Mitan, L. P., & Shaffer, A. (2024) · International Journal of Eating Disorders · doi:10.1002/eat.24149 · Cited by 10
Frank, H. E., Guzman, L. E., Ayalasomayajula, S., Albanese, A. M., Dunklee, B., Harvey, M., Bouchard, K. L., Vadiveloo, M. K., Yaroch, A. L., Scott, K., et al. (2024) · Pilot and Feasibility Studies · doi:10.1186/s40814-024-01467-7 · Cited by 10
“We did not conduct power analyses for this study given recommendations that power analyses are not appropriate for pilot studies that do not propose inferential tests [57].”
MethodThe State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines
Oppenlaender, J., Abbas, T., & Gadiraju, U. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641023 · Cited by 11
“However, the estimation of the sample size required for the main trial needs to be performed cautiously since a crowd pilot study only provides the estimated value of standardized effect size [123].”
BackgroundLong, J. M., Yoder, A. D., Woodcock, L., et al. (2024) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2024.07.007 · Cited by 10
Carlon, H. A., Hurlocker, M. C., Hoeppner, B. B., & Witkiewitz, K. A. (2025) · Addiction · doi:10.1111/add.70019 · Cited by 10
Lee, J. L. C., Wong, A. Y. L., Ng, P. H. F., Fu, S. N., Fong, K. N. K., Cheng, A. S. K., Lee, K. N. K., Sun, R., Zhang, H. Y., & Xiao, R. (2025) · JMIR mhealth and uhealth · doi:10.2196/69259 · Cited by 11
“To provide insights into the magnitude and direction of effects, results are presented alongside 95% CIs [58,59].”
Mollica, M. A., & Mitchell, A. (2013) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2013.12.163 · Cited by 10
Getting Research on Games for Health Funded
Baranowski, T., Blumberg, F. C., Gao, Z., Kato, P. M., Kok, G., Lu, A. S., Lyons, E., Morrill, B. A., Peng, W., Prins, P. J. M., et al. (2016) · Games for Health Journal · doi:10.1089/g4h.2016.0106 · Cited by 10
Stevens, M. L., Lin, C. C., Hancock, M. J., Wisbey-Roth, T., Latimer, J., & Maher, C. G. (2017) · Physiotherapy · doi:10.1016/j.physio.2017.08.006 · Cited by 10
Teh, R. C., Visvanathan, R., Ranasinghe, D. C., & Wilson, A. (2017) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000129 · Cited by 10
“Sample sizes and withdrawal rates within this pilot study, were influenced by the pragmatics of recruitment and the need to assess study feasibility.(51) In addition, items developed for survey data collection (based on interviews with five focus group participants) may not have been representative of all relevant iss…”
BackgroundLee, J., Chang, S. H., & Jolin, J. (2020) · Journal of Motor Learning and Development · doi:10.1123/jmld.2020-0017 · Cited by 12
Vaidya, R., VISINTAINER, P. F., & Singh, R. (2021) · Journal of Perinatology · doi:10.1038/s41372-021-01113-7 · Cited by 10
“As the study was designed as a pilot, no formal tests of hypotheses were conducted [18].”
The Multiphase Optimization Strategy (MOST) in Child Maltreatment Prevention Research
Guastaferro, K., Strayhorn, J. C., & Collins, L. M. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-021-02062-7 · Cited by 11
Nicotera, N., & Viggiano, E. (2021) · Journal of Addictions & Offender Counseling · doi:10.1002/jaoc.12095 · Cited by 15
Feasibility of RESP-FIT: Technology-Enhanced Self-Management Intervention for Adults with COPD
Miller, S., Teufel, R. J., Nichols, M. G., Davenport, P. W., Mueller, M., Silverman, E., Madisetti, M., Pittman, M., Kelechi, T. J., & Strange, C. (2021) · International Journal of COPD · doi:10.2147/copd.s326675 · Cited by 10
“As this was a feasibility pilot study, sample size was determined for pragmatic reasons rather than power.(32) Study sample demographic and clinical characteristics, as well as the feasibility outcomes, were analyzed with descriptive statistics using frequencies, proportions and measures of central tendency (means, me…”
Metadata
- Authors
- Leon, A. C., Davis, L. L., & Kraemer, H. C.
- Affiliations
- Weill Cornell Medicine (US)
- University of Alabama (US)
- Stanford University (US)
- Corresponding author
- Leon, Andrew C.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2011-05
- Volume
- 45
- Issue
- 5
- Pages
- 626-629
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC3081994
- OpenAlex
- W2168434488
- Cited by
- 1,917
- References
- 17
- Citations by year
- 2026: 203 · 2025: 190 · 2024: 143 · 2023: 169 · 2022: 163 · 2021: 217 · 2020: 169 · 2019: 136 · 2018: 128 · 2017: 120 · 2016: 92 · 2015: 80 · 2014: 52 · 2013: 31 · 2012: 17
- FWCI
- 14.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pilot study, feasibility assessment, clinical trials, research design, sample size determination, effect size estimation, generalizability, recruitment and retention, intervention implementation, research funding review
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Feasibility Studies, Humans, Pilot Projects, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
