The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Initial Scale Development: Sample Size for Pilot Studies
Abstract
Pilot studies are often recommended by scholars and consultants to address a variety of issues, including preliminary scale or instrument development. Specific concerns such as item difficulty, item discrimination, internal consistency, response rates, and parameter estimation in general are all relevant. Unfortunately, there is little discussion in the extant literature of how to determine appropriate sample sizes for these types of pilot studies. This article investigates the choice of sample size for pilot studies from a perspective particularly related to instrument development. Specific recommendations are made for researchers regarding how many participants they should use in a pilot study for initial scale development.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Johanson & Brooks (2010) did recommend a sample size of 12 participants per arm for pilot trials.”
From A Feasibility and Pilot Randomised Dismantling Trial of the Efficacy of Self-As-Context During Acceptance and Commitment Therapy. (2021) · cites this paper as Johanson & Brooks (2010)
In the citing paper
“As the current study was below the average (Billingham etal., 2013) and below the recommended (Johanson & Brooks, 2010) sample size”
From A Feasibility and Pilot Randomised Dismantling Trial of the Efficacy of Self-As-Context During Acceptance and Commitment Therapy. (2021) · cites this paper as Johanson & Brooks, 2010
Citations
Cites
Sample size of 12 per group rule of thumb for a pilot study
Julious, S. A. (2005) · Pharmaceutical Statistics · doi:10.1002/pst.185 · Cited by 3,068
Considerations in determining sample size for pilot studies
Hertzog, M. A. (2008) · Research in Nursing & Health · doi:10.1002/nur.20247 · Cited by 2,337
Handbook in research and evaluation
(1971) · Behavior Therapy · doi:10.1016/s0005-7894(71)80129-6 · Cited by 1,488
Bootstrapping: A Nonparametric Approach to Statistical Inference.
Aa, Mooney, C. Z., & Duval, R. D. (1994) · Journal of the American Statistical Association · doi:10.2307/2290969 · Cited by 1,365
Mooney, C., & Duval, R. (1993) · doi:10.4135/9781412983532 · Cited by 893
Statistical Inference for Coefficient Alpha
Feldt, L. S., Woodruff, D. J., & Salih, F. A. (1987) · Applied Psychological Measurement · doi:10.1177/014662168701100107 · Cited by 526
van Belle, G. (2002) · Wiley-Interscience eBooks · Cited by 461
Nelson, L. S. (2003) · Journal of Quality Technology · doi:10.1080/00224065.2003.11980200 · Cited by 422
WHAT SAMPLE SIZE is "ENOUGH" in INTERNET SURVEY RESEARCH?
Hill, R. L., & Hamilton, W. P. (1998) · Cited by 359
By Design: Planning Research on Higher Education.
Centra, J. A., Light, R. L., Singer, J. D., & Willett, J. B. (1991) · The Journal of Higher Education · doi:10.2307/1982292 · Cited by 266
Educational and Psychological Measurement
Finch, W. H., & French, B. F. (2018) · Routledge eBooks · doi:10.4324/9781315650951 · Cited by 239
Hedges, L., & Vevea, J. L. (1996) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986021004299 · Cited by 223
Confidence Intervals for Effect Sizes
Fan, X., & Thompson, B. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401614001 · Cited by 219
Confidence Intervals About Score Reliability Coefficients, Please: An EPM Guidelines Editorial
Fan, X., & Thompson, B. (2001) · Educational and Psychological Measurement · doi:10.1177/00131640121971365 · Cited by 191
Alpha's standard error (ASE): an accurate and precise confidence interval estimate.
Duhachek, A., & Iacobucci, D. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.5.792 · Cited by 176
Light, R. J., Singer, J. D., & Willett, J. B. (1990) · Harvard University Press eBooks · doi:10.4159/9780674040267 · Cited by 148
Elements of Research in Nursing
Davitz, L. J., Treece, E. W., & Treece, J. W. (1974) · AJN American Journal of Nursing · doi:10.2307/3469658 · Cited by 123
Elements of Research in Nursing
Collins, M. A. (1983) · AORN Journal · doi:10.1016/s0001-2092(07)70018-2 · Cited by 91
Elements of Research in Nursing (3rd ed.)
Ventura, M. R. (1983) · Cancer Nursing · doi:10.1097/00002820-198302000-00012 · Cited by 17
Research works printed but not linked: 3
- Crocker, L., 1986, Classical and modern test theory
- Hill, R., 1998, Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 6
- Lowry, R., 2008, VassarStats: Website for statistical calculation
Cited by
Sorted by citations
A Review of Key Likert Scale Development Advances: 1995–2019
Jebb, A. T., Ng, V., & Tay, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.637547 · Cited by 1,114
A simple formula for the calculation of sample size in pilot studies
Viechtbauer, W., Smits, L., Kotz, D., Budé, L., Spigt, M., Serroyen, J., & Crutzen, R. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.014 · Cited by 612
“These considerations have led to various guidelines for choosing an appropriate sample size for a pilot study, such as 12 participants per group [3], values in the range of 10 to 40 participants per group depending on the parameter of interest [4,5], at least 9% of the main trial’s sample size [6], or at least 50 part…”
BackgroundLewis, M., Bromley, K., Sutton, C. J., McCray, G., Myers, H. L., & Lancaster, G. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00770-x · Cited by 330
Importance–performance analysis in tourism: A framework for researchers
Lai, I. K. W., & Hitchcock, M. J. (2014) · Tourism Management · doi:10.1016/j.tourman.2014.11.008 · Cited by 326
Sample size determination for conducting a pilot study to assess reliability of a questionnaire
Bujang, M. A., Omar, E. D., Foo, D. H. P., & Hon, Y. K. (2024) · Restorative Dentistry & Endodontics · doi:10.5395/rde.2024.49.e3 · Cited by 349
Metadata
- Authors
- Johanson, G. A., & Brooks, G. P.
- Affiliations
- Ohio University (US)
- Corresponding author
- Johanson, George A.
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2009-12-18
- Volume
- 70
- Issue
- 3
- Pages
- 394-400
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- OpenAlex
- W1984294110
- Cited by
- 1,012
- References
- 19
- Citations by year
- 2026: 134 · 2025: 115 · 2024: 110 · 2023: 99 · 2022: 89 · 2021: 104 · 2020: 83 · 2019: 67 · 2018: 49 · 2017: 34 · 2016: 33 · 2015: 29 · 2014: 25 · 2013: 19 · 2012: 13
- FWCI
- 9.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational and Psychological Measurement (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- pilot study, sample size determination, scale development, instrument development, item difficulty, item discrimination, internal consistency, response rate, parameter estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
