Cited by 1,012Psychometric Methodologies and TestingHealth Education and ValidationSurvey Methodology and Nonresponse

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Initial Scale Development: Sample Size for Pilot Studies

Johanson, George A., Brooks, Gordon P.·2010·Educational and Psychological Measurement

Open this paper in CoChat — free account

Abstract

Pilot studies are often recommended by scholars and consultants to address a variety of issues, including preliminary scale or instrument development. Specific concerns such as item difficulty, item discrimination, internal consistency, response rates, and parameter estimation in general are all relevant. Unfortunately, there is little discussion in the extant literature of how to determine appropriate sample sizes for these types of pilot studies. This article investigates the choice of sample size for pilot studies from a perspective particularly related to instrument development. Specific recommendations are made for researchers regarding how many participants they should use in a pilot study for initial scale development.

Integrity signals

Registry

Crossref · 10.1177/0013164409355692

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Sample size of 12 per group rule of thumb for a pilot study
Julious, S. A. (2005) · Pharmaceutical Statistics · doi:10.1002/pst.185 · Cited by 3,068
Considerations in determining sample size for pilot studies
Hertzog, M. A. (2008) · Research in Nursing & Health · doi:10.1002/nur.20247 · Cited by 2,337
Handbook in research and evaluation
(1971) · Behavior Therapy · doi:10.1016/s0005-7894(71)80129-6 · Cited by 1,488
Bootstrapping: A Nonparametric Approach to Statistical Inference.
Aa, Mooney, C. Z., & Duval, R. D. (1994) · Journal of the American Statistical Association · doi:10.2307/2290969 · Cited by 1,365
Bootstrapping
Mooney, C., & Duval, R. (1993) · doi:10.4135/9781412983532 · Cited by 893
Statistical Inference for Coefficient Alpha
Feldt, L. S., Woodruff, D. J., & Salih, F. A. (1987) · Applied Psychological Measurement · doi:10.1177/014662168701100107 · Cited by 526
Statistical rules of thumb
van Belle, G. (2002) · Wiley-Interscience eBooks · Cited by 461
Statistical Rules of Thumb
Nelson, L. S. (2003) · Journal of Quality Technology · doi:10.1080/00224065.2003.11980200 · Cited by 422
WHAT SAMPLE SIZE is "ENOUGH" in INTERNET SURVEY RESEARCH?
Hill, R. L., & Hamilton, W. P. (1998) · Cited by 359
By Design: Planning Research on Higher Education.
Centra, J. A., Light, R. L., Singer, J. D., & Willett, J. B. (1991) · The Journal of Higher Education · doi:10.2307/1982292 · Cited by 266

Show all 19 references

Research works printed but not linked: 3

Crocker, L., 1986, Classical and modern test theory
Hill, R., 1998, Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 6
Lowry, R., 2008, VassarStats: Website for statistical calculation

Sorted by citations

A Review of Key Likert Scale Development Advances: 1995–2019
Jebb, A. T., Ng, V., & Tay, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.637547 · Cited by 1,114
A simple formula for the calculation of sample size in pilot studies
Viechtbauer, W., Smits, L., Kotz, D., Budé, L., Spigt, M., Serroyen, J., & Crutzen, R. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.014 · Cited by 612
“These considerations have led to various guidelines for choosing an appropriate sample size for a pilot study, such as 12 participants per group [3], values in the range of 10 to 40 participants per group depending on the parameter of interest [4,5], at least 9% of the main trial’s sample size [6], or at least 50 part…”
Background
Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back!
Lewis, M., Bromley, K., Sutton, C. J., McCray, G., Myers, H. L., & Lancaster, G. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00770-x · Cited by 330
Importance–performance analysis in tourism: A framework for researchers
Lai, I. K. W., & Hitchcock, M. J. (2014) · Tourism Management · doi:10.1016/j.tourman.2014.11.008 · Cited by 326
Sample size determination for conducting a pilot study to assess reliability of a questionnaire
Bujang, M. A., Omar, E. D., Foo, D. H. P., & Hon, Y. K. (2024) · Restorative Dentistry & Endodontics · doi:10.5395/rde.2024.49.e3 · Cited by 349
Roles of Innovation Leadership on Using Big Data Analytics to Establish Resilient Healthcare Supply Chains to Combat the COVID-19 Pandemic: A Multimethodological Study
Bag, S., Gupta, S., Choi, T., & Kumar, A. (2021) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2021.3101590 · Cited by 214
“It was determined that 30 representative participants would be a reasonable minimum recommendation for a pilot study [76], [77].”
Background
Understanding the importance that consumers attach to social media sharing (ISMS): Scale development and validation
Dedeoğlu, B. B., Taheri, B., Okumuş, F., & Gannon, M. J. (2019) · Tourism Management · doi:10.1016/j.tourman.2019.103954 · Cited by 201
A Step-by-Step Process on Sample Size Determination for Medical Research
Bujang, M. A. (2021) · Malaysian Journal of Medical Sciences · doi:10.21315/mjms2021.28.2.2 · Cited by 177
Microplastic particles in human blood and their association with coagulation markers
Lee, D., Jung, J., Park, S.-A., Lee, Y., Kim, J., Han, C., Kim, H., Lee, J. H., & Hong, Y.-C. (2024) · Scientific Reports · doi:10.1038/s41598-024-81931-9 · Cited by 166
“The sample size was determined based on Johanson and Brooks’ recommendation of 30 participants for a preliminary study, with a predicted dropout rate of 20% 27 .”
21st-century digital skills instrument aimed at working professionals: Conceptual development and empirical validation
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2018) · Telematics and Informatics · doi:10.1016/j.tele.2018.08.006 · Cited by 137
Measuring what matters in healthcare: a practical guide to psychometric principles and instrument development
Swan, K., Speyer, R., Scharitzer, M., Farneti, D., Brown, T., Woisard, V., & Cordier, R. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1225850 · Cited by 157
“The first draft should be tested in a small sample of the target population (24–36 respondents are suggested for this phase) (Johanson and Brooks, 2010) and then refined with due consideration of the psychometric performance and user feedback on the instrument’s implementation and interaction…”
Background
How Positive and Negative Electronic Word of Mouth (Ewom) Affects Customers’ Intention to Use New Online Channels: A Dual Stage Multi Group-Sem and Ann Analysis
Alnoor, A., Tiberius, V., Atiyah, A. G., Khaw, K. W., Teh, S. Y., Chew, X., & Abbas, S. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2125610 · Cited by 145
“According to Johanson and Brooks (2010), a sample of 23 respondents is adequate.”
Background
Is Coefficient Alpha Robust to Non-Normal Data?
Sheng, Y., & Sheng, Z. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00034 · Cited by 120
“ORIGINAL RESEARCH ARTICLE published: 15 February 2012 doi: 10.3389/fpsyg.2012.00034”
BackgroundInfluential
The adoption of operational environmental sustainability approaches in the Thai manufacturing sector
Piyathanavong, V., Garza‐Reyes, J. A., Kumar, P. V., Guzmán, G. M., & Mangla, S. K. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.02.093 · Cited by 119
The development and psychometric testing of a theory‐based instrument to evaluate nurses’ perception of clinical reasoning competence
Liou, S., Liu, H., Tsai, H., Tsai, Y., Lin, Y., Chang, C., & Cheng, C. (2016) · Journal of Advanced Nursing · doi:10.1111/jan.12831 · Cited by 104
Development of the State Optimism Measure
Millstein, R. A., Chung, W., Hoeppner, B. B., Boehm, J. K., Legler, S. R., Mastromauro, C. A., & Huffman, J. C. (2019) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2019.04.002 · Cited by 119
“[58–61]), and absolute number of subjects, which can range from 30 to > 500 [60, 62, 63].”
Background
Developing and Testing the Transformative Travel Experience Scale (TTES)
Soulard, J., McGehee, N. G., & Knollenberg, W. (2020) · Journal of Travel Research · doi:10.1177/0047287520919511 · Cited by 106
Development and Psychometric Testing of the Nurse Practitioner Primary Care Organizational Climate Questionnaire
Poghosyan, L., Nannini, A., Finkelstein, S. R., Mason, E. J., & Shaffer, J. A. (2013) · Nursing Research · doi:10.1097/nnr.0b013e3182a131d2 · Cited by 96
“The final sample consisted of 81 NPs, which was well above the suggested minimum recommendation of 30 participants for the initial scale development (Johanson & Brooks, 2010).”
Background
Measuring employee perception of performance management system effectiveness
Sharma, N. P., Sharma, T., & Agarwal, M. N. (2016) · Employee Relations · doi:10.1108/er-01-2015-0006 · Cited by 96
Factors related to teachers' attitudes towards the inclusive education of students with severe intellectual disabilities in Riyadh, Saudi
Alquraini, T. (2012) · Journal of Research in Special Educational Needs · doi:10.1111/j.1471-3802.2012.01248.x · Cited by 90
The effect of celebrity endorsement on consumer behavior: Case of the Lebanese jewelry industry
Hani, S., Azouri, M., & Azouri, A. (2018) · Arab Economic and Business Journal · doi:10.1016/j.aebj.2018.11.002 · Cited by 89
Physicians’ attitudes and knowledge concerning antibiotic prescription and resistance: questionnaire development and reliability
Rodrigues, A. T., Ferreira, M., Roque, F., Falcão, A. C., Ramalheira, E., Figueiras, A., & Herdeiro, M. T. (2015) · BMC Infectious Diseases · doi:10.1186/s12879-015-1332-y · Cited by 82
“Even so, 30 participants in each group of physicians would seem to be a reasonable number for a pilot study where the purpose is preliminary survey or scale development [45].”
Background
Asia Pacific Management and Business Application
(2016) · Asia Pacific Management and Business Application · doi:10.21776/ub.apmba · Cited by 82
Determinants of Attitude towards Islamic Financing among Halal-certified Micro and SMEs: A Preliminary Investigation
Jaffar, M. A., & Musa, R. (2014) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2014.04.017 · Cited by 80
Development and Validation of Digital Health Technology Literacy Assessment Questionnaire
Yoon, J., Lee, M., Ahn, J. S., Oh, D., Shin, S.-Y., Chang, Y. J., & Cho, J. (2022) · Journal of Medical Systems · doi:10.1007/s10916-022-01800-8 · Cited by 74
“of participants (N) to variables (p) set traditionally to 5:1 (5 participants per item) [26].”
Background
Factors determining employee career success in the Chinese hotel industry: A perspective of Job-Demand Resources theory
Lei, C., Hossain, M. S., Mostafiz, M. I., & Khalifa, G. S. A. (2021) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2021.07.001 · Cited by 88
“This present study was conducted with 50 hotel employee as a pilot study sample (Johanson & Brooks, 2010).”
Method
The effect of virtual reality experience quality on destination visit intention and virtual reality travel intention
Yersüren, S., & Özel, Ç. H. (2023) · Journal of Hospitality and Tourism Technology · doi:10.1108/jhtt-02-2023-0046 · Cited by 74
Healing the neurophysiological roots of trauma: a controlled study examining loreta z-score neurofeedback and HRV biofeedback for chronic PTSD
Bell, A. N., Moss, D., & Kallmeyer, R. J. (2019) · NeuroRegulation · doi:10.15540/nr.6.2.54 · Cited by 65
“In such cases, authors have proposed the most appropriate sample for this early stage of research should be 10–30 participants per group (Johanson & Brooks, 2010; Julious, 2005).”
Background
Measuring Athletic Mental Energy (AME): Instrument Development and Validation
Lu, F. J.-H., Gill, D. L., Yang, C. M. C., Lee, P.-F., Chiu, Y., Hsu, Y., & Kuan, G. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02363 · Cited by 78
Systematic development and adjustment of the German version of the Supportive and Palliative Care Indicators Tool (SPICT-DE)
Afshar, K., Feichtner, A., Boyd, K. J., Murray, S. A., Jünger, S., Wiese, B., Schneider, N., & Müller-Mundt, G. (2018) · BMC Palliative Care · doi:10.1186/s12904-018-0283-7 · Cited by 63
“According to Johanson and Brooks a sample size of about 30 participants is considered adequate for pilot studies aiming at scale or tool development [25].”
BackgroundResult
The role of nostalgia-evoking stimuli at nostalgia-themed restaurants in explaining benefits, consumption value and post-purchase behavioral intention
Gu, Q., Li, M., & Kim, S. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.102955 · Cited by 61
HAPPIER WORKING FOR THEMSELVES?
Massey, B. L., & Elmore, C. (2011) · Journalism Practice · doi:10.1080/17512786.2011.579780 · Cited by 61
Development and psychometric testing of a Clinical Reasoning Evaluation Simulation Tool (CREST) for assessing nursing students' abilities to recognize and respond to clinical deterioration
Liaw, S. Y., Rashasegaran, A., Wong, L. F., Deneen, C. C., Cooper, S., Levett-Jones, T., Goh, H. S., & Ignacio, J. (2018) · Nurse Education Today · doi:10.1016/j.nedt.2017.12.009 · Cited by 65
Baclofen for stroke patients with persistent hiccups: a randomized, double-blind, placebo-controlled trial
Zhang, C., Zhang, R., Zhang, S., Xu, M., & Zhang, S. (2014) · Trials · doi:10.1186/1745-6215-15-295 · Cited by 60
Pilot Study of the International Parkinson and Movement Disorder Society‐sponsored Non‐motor Rating Scale (MDS‐NMS)
Martínez‐Martín, P., Schrag, A. E., Weintraub, D. A., Rizos, A., Rodríguez‐Blázquez, C., Chaudhuri, К. R., & on behalf of the IPMDS Non Motor PD Study Group (2019) · Movement Disorders Clinical Practice · doi:10.1002/mdc3.12728 · Cited by 62
Impact of strategic and operational risk management practices on firm performance: An empirical investigation
Singh, N. P., & Hong, P. C. (2020) · European Management Journal · doi:10.1016/j.emj.2020.03.003 · Cited by 67
Exoscore: A Design Tool to Evaluate Factors Associated With Technology Acceptance of Soft Lower Limb Exosuits by Older Adults
Shore, L., Power, V., Hartigan, B., Schülein, S., Graf, E. S., de Eyto, A., & O'Sullivan, L. W. (2019) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819868122 · Cited by 59
Exploring the corporate university phenomenon: Development and implementation of a comprehensive survey
Abel, A. L., & Li, J. (2012) · Human Resource Development Quarterly · doi:10.1002/hrdq.21122 · Cited by 54
Nation, Face, and Identity: An Initial Investigation of National Face in East Asia
Chen, R., & Hwang, K. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01557 · Cited by 57
“Front.”
Development of the Pregnancy and Motherhood Evaluation Questionnaire (PMEQ) for evaluating and measuring the impact of physical disability on pregnancy and the management of motherhood: a pilot study
Panuccio, F., Berardi, A., Márquez, M. A., Messina, M. P., Valente, D., Tofani, M., & Galeoto, G. (2020) · Disability and Rehabilitation · doi:10.1080/09638288.2020.1802520 · Cited by 53
“given a criterion of maximum information with minimum cost, 30 representative participants from the population of interest is a reasonable minimum recommendation for a pilot study where the purpose is preliminary survey or scale development [35].”
Background
Effects of peer tutoring on middle school students’ mathematics self-concepts
Miravet, L. M., & Alegre, F. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231410 · Cited by 76
“Furthermore, although as indicated above Krejcie and Morgan (1970) [73] and Johanson and Brooks (2009) [74] state that 368 were necessary, they referred to the case in which a random sampling is viable.”
Background
Development of a Feasible Implementation Fidelity Protocol Within a Complex Physical Therapy–Led Self-Management Intervention
Toomey, E. C., Matthews, J. J., Guérin, S., & Hurley, D. A. (2016) · Physical Therapy · doi:10.2522/ptj.20150446 · Cited by 54
Portuguese Physical Literacy Assessment Questionnaire (PPLA-Q) for adolescents (15–18 years) from grades 10–12: development, content validation and pilot testing
Mota, J., Martins, J., & Onofre, M. (2021) · BMC Public Health · doi:10.1186/s12889-021-12230-5 · Cited by 58
“Pilot testing Pilot testing, or pretesting constitutes an opportunity to (1) test the application of items in development to a representative sample of target population [90]; (2) gather feasibility evidence to plan a larger scale study [91]; and (3) gather data for preliminary item analysis and estimates of reliabili…”
Background
Assessing self-regulated learning: The case of vocabulary learning through information and communication technologies
Kızıl, A. Ş., & Savran, Z. (2018) · Computer Assisted Language Learning · doi:10.1080/09588221.2018.1428201 · Cited by 50
“Given the fact that a reasonable minimum recommendation for the pilot studies of scale development is 30 participants (Johanson & Brooks, 2010), the sample size of the study could be considered appropriate.”
Background
Students' mindset to adopt AI chatbots for effectiveness of online learning in higher education
Rahman, M. K., Ismail, N. A., Hossain, D., & Hossen, M. S. (2025) · Future Business Journal · doi:10.1186/s43093-025-00459-0 · Cited by 84
“This method is crucial in instrument refinement and improving the precision and effectiveness of the instrument for the primary study [50].”
The interplay among green absorptive capacity, green entrepreneurial, and learning orientations and their effect on triple bottom line performance
Marzouk, J., & Ebrashi, R. E. (2023) · Business Strategy and the Environment · doi:10.1002/bse.3588 · Cited by 54
The Evolution of a Robust and Reliable Brand Experience Scale in the Malaysian Context: An Empirical Evidence
AL-Fakhri, I., & Alabdullah, T. T. Y. (2021) · Business Ethics and Leadership · doi:10.21272/bel.5(4).59-67.2021 · Cited by 47
Net valence analysis of iris recognition technology-based FinTech
Al‐Debei, M. M., Hujran, O., & Al‐Adwan, A. S. (2024) · Financial Innovation · doi:10.1186/s40854-023-00509-y · Cited by 48
“In addition, a pre-test of the questionnaire was conducted with 50 banking customers to evaluate the wording, com - pleteness, and order of the questions (Johanson and Brooks 2010).”
Method
Do Consumers Acculturated to Global Consumer Culture Buy More Impulsively? The Moderating Role of Attitudes towards and Beliefs about Advertising
Czarnecka, B., & Schivinski, B. (2019) · Journal of Global Marketing · doi:10.1080/08911762.2019.1600094 · Cited by 47
Developing a Scale to Measure Students’ Attitudes toward Science
Akkuş, A. (2019) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.548516 · Cited by 45
“In the method, to develop a scale, based on suggestions from the literature, the guidelines have been determined (Brinkman, 2009; Deshpande, 2004; Hinkin, 1998; Hinkin et. al., 1997; Johanson & Brooks, 2010; Ajzen, 2005b; Francis et. al., 2004; Cabrera-Nguyen, 2010; Hof, 2012).”
BackgroundMethod
Talent management practices on employee performance among academic staff of Malaysian private universities: employee engagement as a mediator
Abdullahi, M. S., Raman, K., & Solarin, S. A. (2020) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-08-2020-0283 · Cited by 69
Talent management practices on employee performance: a mediating role of employee engagement in institution of higher learning: quantitative analysis
Abdullahi, M. S., Adeiza, A., Abdelfattah, F., Fatma, M., Fawehinmi, O. O., & Aigbogun, O. (2022) · Industrial and Commercial Training · doi:10.1108/ict-10-2021-0075 · Cited by 44
Do customers’ perceptions of Islamic banking services predict satisfaction and word of mouth? Evidence from Islamic banks in Bangladesh
Rahman, M. K., Hoque, M. N., Yusuf, S. N. S., Yusoff, M. N. H., & Begum, F. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280108 · Cited by 60
Current and future options for opioid use disorder: a survey assessing real-world opinion of service users on novel therapies including depot formulations of buprenorphine
Gilman, M., Li, L., Hudson, K., Lumley, T., Myers, G., Corte, C., & Littlewood, R. (2018) · Patient Preference and Adherence · doi:10.2147/ppa.s180641 · Cited by 42
Determinants of 3Rs behaviour in plastic usage: A study among Malaysians
T’ing, L. C., Moorthy, K., Mei, C. Y., Foo, P. Y., Ying, W. Z., Khong, C. W., Chern, G. Z., & Lin, T. Z. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e05805 · Cited by 44
The perception of success in the halal market: developing a halal entrepreneurship success scale
Salaheldeen, M., Battour, M., Nazri, M. A., Bustamam, U. S. A., & Hashim, A. J. C. M. (2022) · Journal of Islamic marketing · doi:10.1108/jima-10-2021-0341 · Cited by 49
Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines
Graziotin, D., Lenberg, P., FELDT, R. H., & Wagner, S. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3469888 · Cited by 59
Effect of organizational culture on employee performance: A mediating role of employee engagement in Malaysia educational sector
Abdullahi, M. S., Raman, K., & Solarin, S. A. (2021) · Siti Hasmah Digital Library-MMU Institutiona Repository (Multimedia University) · doi:10.22034/ijsom.2021.3.1 · Cited by 41
Quality assurance and halal control points for the food industry
Lau, A. N., Jamaludin, M. H., & Soon, J. M. (2016) · Nutrition & Food Science · doi:10.1108/nfs-03-2016-0026 · Cited by 40
“A sample of 30 participants from the population of interest 66 is a reasonable minimum recommendation for a pilot study where the purpose is 67 preliminary survey or scale development (Johanson and Brooks, 2009).”
Background
Evaluating the Status of SMEs in Jordan with Respect to Industry 4.0: A Pilot Study
Shqair, M. I., & Altarazi, S. (2022) · Logistics · doi:10.3390/logistics6040069 · Cited by 45
“For instance, Saunders and Tosey [44] recommend a minimum of 10 respondents, Aaker and Keppel [45] recommend 15 to 25 respondents, Taylor et al. [46] recommend 15 to 30 respondents, Johanson and Brooks [47] recommend that 30 respondents’ sample is a reasonable count for a pilot study.”
Background
Metoclopramide for patients with intractable hiccups: a multicentre, randomised, controlled pilot study
Wang, T., & Wang, D. (2014) · Internal Medicine Journal · doi:10.1111/imj.12542 · Cited by 39
Improved Functional Performance in Individuals With Dementia After a Moderate-Intensity Home-Based Exercise Program: A Randomized Controlled Trial
Dawson, N. T., Judge, K. S., & Gerhart, H. D. (2017) · Journal of Geriatric Physical Therapy · doi:10.1519/jpt.0000000000000128 · Cited by 39
Fostering firm resilience through organizational ambidexterity capability and resource availability: amid the COVID-19 outbreak
Gayed, S., & Ebrashi, R. E. (2022) · International journal of organizational analysis · doi:10.1108/ijoa-09-2021-2977 · Cited by 42
Impact of Service Expectation, Experiential Quality, and Perceived Value on Hotel Customer Satisfaction
Saut, M., & Bie, S. (2022) · Journal of Quality Assurance in Hospitality & Tourism · doi:10.1080/1528008x.2022.2141414 · Cited by 59
“Additionally, to validate the questionnaire’s quality, a piloting study was conducted with 32 respondents, as recommended by Johanson and Brooks (2013), to ensure the internal quality of items used to measure each construct based on reliability tests.”
Method
Moderate intensity aerobic exercise may enhance neuroplasticity of the contralesional hemisphere after stroke: a randomised controlled study
Hill, G., Johnson, F., Uy, J., Serrada, I., Benyamin, B., van den Berg, M., & Hordacre, B. G. (2023) · Scientific Reports · doi:10.1038/s41598-023-40902-2 · Cited by 38
“A total sample size of 30 participants has been consistently documented as sufficient for a two-armed pilot study 21 .”
Background
Validity and Reliability Learning Transfer Item Using Rasch Measurement Model
Yasin, R. M., Yunus, F. A. N., Rus, R. C., Ahmad, A., & Rahim, M. B. (2015) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2015.08.143 · Cited by 53
“Whereas Johanson and Brooks (2010) suggest the minimum number of 30 people for a pilot study in which the aim is to study early or development scale.”
Background
The reliability of the German version of the Richards Campbell Sleep Questionnaire
Krotsetis, S., Richards, K. C., Behncke, A., & Köpke, S. (2017) · Nursing in Critical Care · doi:10.1111/nicc.12275 · Cited by 38
The Intimate Partner Violence Stigma Scale: Initial Development and Validation
Crowe, A., Overstreet, N. M., & Murray, C. E. (2019) · Journal of Interpersonal Violence · doi:10.1177/0886260519834095 · Cited by 37
Using modified grounded theory for conducting systematic research study on sustainable project management field
Alnsour, M. (2022) · MethodsX · doi:10.1016/j.mex.2022.101897 · Cited by 36
A systematic review of digital citizenship empirical studies for practitioners
Shi, G., Chan, K. K., & Lin, X. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11383-z · Cited by 35
“Johanson and Brooks (2010) advised that a small sample size was suitable for a pilot study.”
Background
Critical success factors of environmental management accounting practices: findings from Malaysian manufacturing industry
Fuzi, N. M., Habidin, N. F., Janudin, S. E., & Ong, S. Y. Y. (2018) · Measuring Business Excellence · doi:10.1108/mbe-03-2018-0015 · Cited by 35
Measuring within-day cognitive performance using the experience sampling method: A pilot study in a healthy population
Verhagen, S. J. W., Daniëls, N. E. M., Bartels, S. L., Tans, S., Borkelmans, K. W. H., de Vugt, M. E., & Delespaul, P. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226409 · Cited by 44
Exploring the influence of workplace supports and relationships on safe medication practice: A pilot study of Australian graduate nurses
Sahay, A., Hutchinson, M., & East, L. (2015) · Nurse Education Today · doi:10.1016/j.nedt.2015.01.012 · Cited by 37
The Influence of Accounting Information Systems, Knowledge Management Capabilities, and Innovation on Organizational Performance in Iraqi SMEs
Kareem, H. M., Aziz, K. A., Maelah, R., Yunus, Y. M., Alsheikh, A. H., & Alsheikh, W. H. (2021) · International Journal of Knowledge Management · doi:10.4018/ijkm.2021040104 · Cited by 34
The Role of Accounting Information System and Knowledge Management to Enhancing Organizational Performance in Iraqi SMEs
Kareem, H. M., Dauwed, M., Meri, A., Jarrar, M., Al‐Bsheish, M., & Aldujaili, A. A. (2021) · Sustainability · doi:10.3390/su132212706 · Cited by 34
Employee engagement as a mediating variable on the relationship between employee relation practice and employee performance in a developing economy
Abdullahi, M. S., Raman, K., Solarin, S. A., & Adeiza, A. (2021) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-06-2021-0222 · Cited by 32
The Death Literacy Index: translation, cultural adaptation, and validation of the Chinese version
Che, S. L., Li, X., Zhu, M., & Ng, W. I. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1140475 · Cited by 32
“A minimum sample size of 30 from the population of interest is generally recommended for a pilot study (36).”
Background
Operational practices for integrating lean and industry 4.0 – a dynamic capabilities perspective
Komkowski, T., Sony, M., Antony, J., Lizarelli, F. L., Garza‐Reyes, J. A., & Tortorella, G. L. (2024) · International Journal of Production Research · doi:10.1080/00207543.2024.2381127 · Cited by 31
Electroacupuncture at Jing-jiaji points for neck pain caused by cervical spondylosis: a study protocol for a randomized controlled pilot trial
Sun, Z., Yue, J., & Zhang, Q. (2013) · Trials · doi:10.1186/1745-6215-14-360 · Cited by 31
“Due to the short duration, lasting four weeks, the desired sample size for this pilot study is 36 patients, with 18 for each group, assuming a drop-out rate of 20%, which is the minimum sample size necessary to evaluate the effectiveness of EA [33].”
Background
Identifying the Barriers to Games and Simulations in Education
Justice, L. J., & Ritzhaupt, A. D. (2015) · Journal of Educational Technology Systems · doi:10.1177/0047239515588161 · Cited by 32
“…the information for which it was designed, determining the teacher perceived barriers to the use of games and simulations in formal education, the survey instrument was distributed to a large number of educators so that enough responses could be acquired and measured (Johanson & Brooks, 2010).”
Method
Effects of Using Immersive Virtual Reality for Science Education on Learning Outcomes: A Randomized Controlled Pilot Study
Xie, T., Li, Y., & Tang, Y. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3263587 · Cited by 30
“Johanson et al. [56], in 2010 recommended that samples as small as 10–15 per group would be sufﬁcient for a feasibility study.”
Background
Spore powder of Ganoderma lucidum for the treatment of Alzheimer disease
Wang, G., Wang, L.-H., Wang, C., & Qin, L.-H. (2018) · Medicine · doi:10.1097/md.0000000000010636 · Cited by 30
Factors influencing cruise vacations: the impact of online reviews and environmental friendliness
Bahja, F., Çobanoğlu, C., Berezina, K., & Lusby, C. (2018) · Tourism Review · doi:10.1108/tr-12-2017-0207 · Cited by 29
Social Media Influencers’ Traits and Purchase Intention: A Moderated Mediation Effect of Attitude Towards Brand Credibility and Brand Familiarity
Kareem, S. A., & Venugopal, P. (2023) · FIIB Business Review · doi:10.1177/23197145231162257 · Cited by 37
Yoga and Pilates compared to pelvic floor muscle training for urinary incontinence in elderly women: A randomised controlled pilot trial
Kannan, P., Hsu, W. H., Suen, W. T., Chan, L. M., Assor, A., & Ho, C. M. (2021) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2021.101502 · Cited by 52
Protocol for development and validation of instruments to measure women’s empowerment in urban sanitation across countries in South Asia and Sub-Saharan Africa: the Agency, Resources and Institutional Structures for Sanitation-related Empowerment (ARISE) scales
Sinharoy, S. S., Conrad, A., Patrick, M., McManus, S., & Caruso, B. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-053104 · Cited by 28
The development of a spelling assessment tool informed by Triple Word Form Theory
Daffern, T., Mackenzie, N., & Hemmings, B. (2015) · The Australian Journal of Language and Literacy · doi:10.1007/bf03651958 · Cited by 27
Pre‐operative self‐efficacy education vs. usual care for patients undergoing joint replacement surgery: a pilot randomised controlled trial
Cooke, M. L., Walker, R., Aitken, L. M., Freeman, A., Pavey, S., & Cantrill, R. M. (2015) · Scandinavian Journal of Caring Sciences · doi:10.1111/scs.12223 · Cited by 27
Older Adults Adoption, Use and Diffusion of E-Government Services in Saudi Arabia, Hail City: A Quantitative Study
Alfalah, A. A., Choudrie, J., & Spencer, N. H. (2017) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2017.357 · Cited by 27
“Using the reasoning that responses between 100 and 200 are considered to be sufficient for a pilot study intending to undertake a comprehensive item analysis [37], it was felt that 257 replies are suitable for this study.”
Background
Customer perception of CRM implementation in banking context: Scale development and validation
Dubey, N. K., & Sangle, P. (2019) · Journal of Advances in Management Research · doi:10.1108/jamr-12-2017-0118 · Cited by 32
Safe Place collage protocol: Art making for managing traumatic stress
Tripp, T., Potash, J. S., & Brancheau, D. (2019) · Journal of Trauma & Dissociation · doi:10.1080/15299732.2019.1597813 · Cited by 26
Guided internet-based transdiagnostic intervention for Indonesian university students with symptoms of anxiety and depression: A pilot study protocol
Rahmadiana, M., Karyotaki, E., Passchier, J., Cuijpers, P., van Ballegooijen, W., Wimbarti, S., & Riper, H. (2019) · Internet Interventions · doi:10.1016/j.invent.2018.11.002 · Cited by 25
Views of Indonesian consumer towards medical tourism experience in Malaysia
Saragih, H. S., & Jonathan, P. (2019) · Journal of Asia Business Studies · doi:10.1108/jabs-04-2018-0135 · Cited by 25
Developing a methodology for assessing livability potential: An evidence from a metropolitan urban agglomeration (MUA) in Kolkata, India
Paul, A. (2020) · Habitat International · doi:10.1016/j.habitatint.2020.102263 · Cited by 25
Development of the Teaching Interpersonal Skills for Telehealth Checklist
Henry, B. W., Billingsly, D., Block, D. E., & Ehrmann, J. (2021) · Evaluation & the Health Professions · doi:10.1177/0163278721992831 · Cited by 25
Industry 4.0 Readiness of Technology Companies: A Pilot Study from Malaysia
Soomro, M. A., Hanafiah, M. H., Abdullah, N. L., Ali, M. H., & Jusoh, M. S. (2021) · Administrative Sciences · doi:10.3390/admsci11020056 · Cited by 25
Assessing the challenges of e-learning in Malaysia during the pandemic of Covid-19 using the geo-spatial approach
Jafar, A., Dollah, R., Sakke, N., Mapa, M. T., Hua, A. K., Eboy, O. V., Joko, E. P., Hassan, D., & Hung, C. V. (2022) · Scientific Reports · doi:10.1038/s41598-022-22360-4 · Cited by 25
“To verify the instrument’s validity and reliability can be delivered and utilized in excellent condition, a pilot study with 50 respondents should be conducted beforehand 27,28 .”
Method
Digital capabilities in emerging market firms: Construct development, scale validation, and implications for SMEs
Maycotte-Felkel, S., Álvarez-Risco, A., Garcia-Valenzuela, E., & Kuljis, M. (2025) · Journal of Open Innovation Technology Market and Complexity · doi:10.1016/j.joitmc.2025.100513 · Cited by 24
An implementation of remote laboratory for secondary science education
Tho, S. W., & Yeung, Y. Y. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12273 · Cited by 24
Readiness and Challenges of E-Learning during the COVID-19 Pandemic Era: A Space Analysis in Peninsular Malaysia
Jafar, A., Dollah, R., Mittal, P., Idris, A., Kim, J. E., Abdullah, M. S., Joko, E. P., TEJUDDIN, D. N. A., Sakke, N., Zakaria, N. S., et al. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20020905 · Cited by 24
“The minimal sample size for pilot research for testing reliability using Cronbach’s alpha is thirty respondents [45] or more [46].”
Method

Authors: Johanson, G. A., & Brooks, G. P.
Affiliations: Ohio University (US)
Corresponding author: Johanson, George A.
ORCID: Johanson, George A.
Brooks, Gordon P.
Venue: Educational and Psychological Measurement
Publisher: SAGE Publications
Published: 2009-12-18
Volume: 70
Issue: 3
Pages: 394-400
ISSN: 0013-1644 (ISSN-L)
1552-3888
DOI: 10.1177/0013164409355692
OpenAlex: W1984294110
Cited by: 1,012
References: 19
Citations by year: 2026: 134 · 2025: 115 · 2024: 110 · 2023: 99 · 2022: 89 · 2021: 104 · 2020: 83 · 2019: 67 · 2018: 49 · 2017: 34 · 2016: 33 · 2015: 29 · 2014: 25 · 2013: 19 · 2012: 13
FWCI: 9.92
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational and Psychological Measurement (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Health Sciences › Nursing › Research and Theory › Health Education and Validation
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Keywords: pilot study, sample size determination, scale development, instrument development, item difficulty, item discrimination, internal consistency, response rate, parameter estimation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Initial Scale Development: Sample Size for Pilot Studies

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks