The DOI matches its Crossref record; 8 of the 9 references it cites were found in a registry or an index.
Beyond the five-user assumption: Benefits of increased sample sizes in usability testing
Open accessAbstract
It is widely assumed that 5 participants suffice for usability testing. In this study, 60 users were tested and random sets of 5 or more were sampled from the whole, to demonstrate the risks of using only 5 participants and the benefits of using more. Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%. With 10 users, the lowest percentage of problems revealed by any one set was increased to 80%, and with 20 users, to 95%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Purposive sampling was used to select 10 individuals 43 with at least 50% of the following characteristics: 65þ years old, individuals of diverse race/ethnicity, male, education less than college, and taking at least 3 medications.”
From A chatbot for hypertension self-management support: user-centered design, development, and usability testing (2023) · cites this paper as [43]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior research suggests that our sample size is adequate for usability testing, 43”
From A chatbot for hypertension self-management support: user-centered design, development, and usability testing (2023) · cites this paper as [43]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Ten participants were chosen at each stage of the recruitment process—the mean percentage of problems identified in usability studies rises from 85% to 95% when the number of participants increases from 5 to 10 [36].”
From A user-centred design framework for mHealth (2020) · cites this paper as [36]
The cited source says what the claim attributes to it.
In the citing paper
“Our sample size was chosen in line with prior work around formative testing for usability [19,43]: our goal is to explore the frst,”
From Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023) · cites this paper as [19]
In the citing paper
“This may be reasonable, given that simulations suggest that 10 users will identify a minimum of 80 % of the problems with the technology during usability testing, and 20 users will identify 95 % of the problems [42].”
From Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers (2015) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1993) · Cited by 9,879
(1993) · Elsevier eBooks · doi:10.1016/c2009-0-21512-1 · Cited by 5,552
Managing the Risks of Organizational Accidents
REASON, J. T. (2016) · Routledge eBooks · doi:10.4324/9781315543543 · Cited by 4,523
A mathematical model of the finding of usability problems
Nielsen, J., & Landauer, T. K. (1993) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93 · doi:10.1145/169059.169166 · Cited by 1,961
Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?
Virzi, R. A. (1992) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089203400407 · Cited by 1,269
Spool, J. M., & Schroeder, W. (2001) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/634067.634236 · Cited by 162
Why and when five test users aren’t enough
Woolrych, A., & Cockton, G. (2001) · Cited by 99
How many users does it take to change a Web site?
Hudson, W. (2001) · ACM SIGCHI Bulletin · doi:10.1145/967222.967230 · Cited by 9
Flagged first, then by citations
1 of the 9 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Grosvenor, L. (1999).Software usability: Challenging the myths and assumptions in an emerging field. Unpublished master’s thesis, University of Texas, Austin.
- Wentworth, R. J. (1996, March).Group chairman’s factual report (American Airlines Flight 965). Washington, DC: National Transportation Safety Board, Office of Aviation Safety.
Cited by
Sorted by citations
Items 801–900 of 1,000
Evaluating StackGenVis with a Comparative User Study
Chatzimparmpas, A., Park, V., & Kerren, A. (2022) · IEEE Pacific Visualization Symposium · doi:10.1109/pacificvis53943.2022.00025 · Cited by 1
Multimodal Wearable System for Motor Rehabilitation: Usability and Acceptability
Perego, P., Sironi, R., Scagnoli, M., Terraroli, M., Standoli, C. E., & Andreoni, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08645-8_50 · Cited by 1
Utility Evaluation of Cross-document Information Extraction
Ji, H., Chen, Z., Feldman, J., González-Pardo, A., Grishman, R., & Upadhyay, V. K. (2010) · North American Chapter of the Association for Computational Linguistics · Cited by 1
Foresti, G. L., Martinel, N., Micheloni, C., & Vernier, M. (2013) · doi:10.1002/9781118577516.ch17 · Cited by 1
Scaling Usability in Terms of Requirements: A Method for Evaluating User Interfaces
Fracker, M. L. (2010) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193121005400607 · Cited by 1
Nurses' use of iPads for clinical documentation : a usability study
Ho, W. Y. (2014) · cIRcle (University of British Columbia) · doi:10.14288/1.0165730 · Cited by 1
Grunwell, D. (2017) · Queensland University of Technology · doi:10.5204/thesis.eprints.103323 · Cited by 1
Designer-driven Procedural Game Content Generation using Multi-agent Evolutionary Computation
Kruse, J. P. (2019) · Tuwhera (Auckland University of Technology) · Cited by 1
Improving IoT device transparency by means of privacy labels
Railean, A. (2022) · doi:10.53846/goediss-9225 · Cited by 1
Bertuzzi, V., Semonella, M., Castelnuovo, G., Andersson, G., & Pietrabissa, G. (2023) · doi:10.2196/preprints.46692 · Cited by 1
Sidani, A., Martins, J. P., Soeiro, A. A. V., & Simões, M. L. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4827024 · Cited by 1
Infrastructure recovery curve estimation using Gaussian process regression on expert elicited data
Cao, Q. D., Miles, S. B., & Choe, Y. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.108054
Nash, E. (2005) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120504902401
Using Stressors in Usability Tests
Janneck, M., & Balin, M. (2013) · International Journal of Sociotechnology and Knowledge Development · doi:10.4018/ijskd.2013070102
An Exploratory Look at Online Instruction Delivery Across Electronic Devices
Labriola, J. T., McCarthy, M., & Obi, C. (2014) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/2666216.2666229
When a User Interface is Good Enough: User Ratings in UI Design
Fracker, M. L., Heck, M., & Goeschel, G. (2010) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193121005400611
A Phone Learning Model for Enhancing Productivity of Visually Impaired Civil Servants
Ambrose, A. A., & Azeta, V. I. (2012)
Martin, J. P., Miller, M. K., & Gipson, K. G. (2011)
Linking Context to Evaluation in the Design of Safety Critical Interfaces
Feary, M., Billman, D. O., Chen, X., Howes, A., Lewis, R. L., Sherry, L., & Singh, S. P. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39232-0_22
Cunningham, L., Kirkscey, R., Reynolds-Dyk, A., Small, N., Tran, C., & Tucker, V. M. (2015) · Journal of Usability Studies archive
Usability of Mobile Internet Interface: Full Browsing Navigation and Menu Tasks
최준호, JUNGKWON, K., & 정동훈 (2009) · Journal of Digital Design · doi:10.17280/jdd.2009.9.3.005
How to Successfully Execute Remote Online User Research
Santiago, E., & Resnick, M. L. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571230
How to Categorize Users from a Design Point of View?
Lorentzen, L. (2016) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-41962-6_52
Using OpenGL State History for Graphics Debugging
Dyk, B. V. (2013) · Digital Library (University of West Bohemia)
The Design and Validation of Virtual Trailblazing and Guidance Interfaces for the VTrail System
Iaboni, D. (2010) · UWSpace (University of Waterloo)
User Experience of Access Points: Eye-tracking, Metadata, and Usability Testing
Krahmer, A. (2016) · ThinkTech (Texas Tech University)
Accuracy of oral HIV self-tests in Kenya
Kurth, A., Siika, A. M., Were, E. O., Yebei, V. N., Emonyi, W., Sidle, J. E., Cleland, C. M., Macharia, S. M., Sang, E., & Chhun, N. (2019) · doi:10.7910/dvn/t7afgb
Usability evaluation of a locomotor therapy device considering different strategies
Langthaler, S., Lassnig, A., Baumgärtner, C., & Schröttner, J. (2016) · Current Directions in Biomedical Engineering · doi:10.1515/cdbme-2016-0018
Faculty of Science & Technology
Tran, T. (2013)
Narasimha, S. (2016) · TigerPrints (Clemson University)
Performance and Usability Analysis of Proof-of-proximity (PoP) Framework
Malkani, Y. A., Keerio, A., Mahesar, A. W., & Dahri, K. M. (2013) · SINDH UNIVERSITY RESEARCH JOURNAL -SCIENCE SERIES
Aircraft Automation Policy Implications for Aviation Safety
Blum, S. (2017) · Digital Collections of Colorado (Colorado State University)
Environment-aware freeform displays: An evaluation in a tabletop setting
Cotting, D., Kostka, J., Adelsberger, R., & Groß, M. H. (2011) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-006787713
Nascimento, L. S., Margolis, I., Ahram, T., Martins, L. B., & Correia, W. F. M. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-60492-3_25
The Bee's Knees or Spines of a Spider: What Makes an 'Insect' Interesting?
Shipley, N. J. (2017) · TigerPrints (Clemson University)
The Development of an Academic Library’s Mobile Website
Tidal, J. (2017) · CUNY Academic Works (City University of New York)
Heyman, S. (2017) · KTH Publication Database DiVA (KTH Royal Institute of Technology)
Exploring the Optimal Number of Trials and Participants for Touchpad Testing Using Fitts-based Tasks
Avera, A., Kelling, N., Harper, C. A., Burks, R., & Bowman, H. E. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601574
Empirical Evaluation of Open Government Data Visualisations
Ornig, E., Faichney, J., & Stantić, B. (2017) · Griffith Research Online
A Virtual Environment for Accessible Desktop Navigation
Dickinson, M., Clawson, K., & Horsman, G. (2017) · Sunderland Repository (University of Sunderland)
Visual based finger interactions for mobile phones
Kerr, S. (2010)
Yu, J. N., Li, J. I. C., Ng, E., & Custodio, B. P. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-94373-2_9
Cosquer, M., & Deuff, D. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-96071-5_197
User Acceptance of Employer-Sponsored Personal Health Records
Warnakulasuriya, M. N. F. (2018) · Queensland University of Technology · doi:10.5204/thesis.eprints.121762
Ramadan, A. A. B. (2017) · doi:10.32469/10355/66718
Broky, D. B., Daniels, T. J., & Lemke, M. R. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-816163-0.00011-6
Ριζοπούλου, Ε. Ε., & Σοφός, Α. (2019) · Ανοικτή Εκπαίδευση το περιοδικό για την Ανοικτή και εξ Αποστάσεως Εκπαίδευση και την Εκπαιδευτική Τεχνολογία · doi:10.12681/jode.16131
The use of novel technology to deliver falls prevention exercise to older adults
Howes, S. C. (2018) · Ulster University Research Portal (Ulster University)
da Costa, J. B. D., Seruffo, M. C. D. R., Eliasquevici, M. K., & Souza, D. D. S. (2016) · doi:10.5753/semish.2016.9518
이예림 (2017) · Seoul National University Open Repository (Seoul National University)
Usability Testing of a Digital Competence Assessment and Certification System
Sobodić, A., & Balaban, I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50513-4_15
Fighting Voice Spam with a Virtual Assistant PrototypePreprint
Pandit, S., Liu, J., Roberto Perdisci, A., & Ahamad, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03554
Investigating the usefulness of a micro high performance computing system as an educational tool
Mwasaga, N. M., & Joy, M. S. (2020) · International Conference on Intelligent and Innovative Computing Applications · doi:10.1145/3415088.3415105
The role of user session on the heuristic evaluation
Alqurni, J. S. (2019)
Intelligent management of virtualised computer based workloads and systems
Oakes, J. (2020) · NECTAR - Northampton Electronic Collection of Thesis and Research (University of Northampton)
Demırkol, D., & Hanci, A. K. (2020) · Istanbul University Press eBooks · doi:10.26650/b/et07.2020.012.09
Brum, R. C. D. S., Fernandes, T. J., Xavier, J. R., Maia, M. D. L. S., Camacho, L. A. B., Sampaio, F. C., Ferreira, I. P., & Pardini, M. I. D. M. C. (2021) · Revista Ibero-Americana de Humanidades, Ciências e Educação · doi:10.51891/rease.v7i3.750
A Tool for Collaborative Anatomical Dissection
Zorzal, E. R., Sousa, M., Mendes, D., Paulo, S. F., Rodrigues, P., Jorge, J. A., & Lopes, D. S. (2021) · Human-computer interaction series · doi:10.1007/978-3-030-61905-3_3
Pfister, P. B. (2021) · doi:10.26481/dis.20210617pb
Akın, M. (2021) · Journal of Business Research - Turk · doi:10.20491/isarder.2021.1117
Eilu, E. (2021) · Springer eBooks · doi:10.1007/978-3-030-61089-0_5
Ürün-Müşteri Memnuniyeti-Kullanıcı Deneyimi Açısından Erzincan Kış Turizminin Değerlendirilmesi
Akın, M., & DEMİR, E. (2021) · Uluslararası Türk Dünyası Turizm Araştırmaları Dergisi · doi:10.37847/tdtad.894672
Analyzing and Prioritizing Usability Issues in Games
Rehman, U., Abbasi, A. Z., Shah, M. U., Idrees, A., Ilahi, H., & Hlavacs, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77277-2_9
Standardization of Light Emitting Diode Feedback on Metered Solar Home Systems
Koech, V., & Mbogo, C. (2020) · International Journal of Professional Practice · doi:10.71274/ijpp.v8i1.64
Ün-Duygu-Akıcılık (ÜDA) Modelinde Marka Değerlendirme: GSM Operatörleri Üzerine Bir Çalışma
Akın, M. (2021) · Girişimcilik İnovasyon ve Pazarlama Araştırmaları Dergisi · doi:10.31006/gipad.979005
Akın, M. (2021) · Journal of Communication Science Researches · doi:10.7456/100201100/003
Price, A. I., Chu, L. F., Lin, Y. L., Levin, A. S., Tumetto, F., de Almeida, R. M. A., de Almeida, A. P. S. S., Ciofi-Silva, C. L., Fontana, L., de Oliveira, N. A., et al. (2022) · doi:10.2196/preprints.36549
Zentrum für Ergonomie und Medizintechnik, F. M., König, S., Hinricher, N., Zentrum für Ergonomie und Medizintechnik, F. M., Backhaus, C., & Zentrum für Ergonomie und Medizintechnik, F. M. (2021) · ASU Arbeitsmedizin Sozialmedizin Umweltmedizin · doi:10.17147/asu-2110-8503
Deuff, D., & Cosquer, M. (2013) · doi:10.1002/9781118574829.biblio
Logeswaran, A., Chong, Y. J., & Edmunds, M. R. (2020) · doi:10.2196/preprints.18695
Razek, A. R. A., Raban, M., Hengels, A., van Husen, C., Elstermann, M., Häfner, V., Häfner, P., & Imran, S. (2020) · Springer eBooks · doi:10.1007/978-3-662-60732-9_2
Fallon, C., Lemire, M., Dumont, D., Parent, É., Figueroa, E., Cummings, I., Brousseau, J., Paquet, N., Plante, S., & Witteman, H. O. (2020) · doi:10.31232/osf.io/btjh5
Fanchin, M., & Gamberini, P. P. A. L. (2020) · IADIS International Conferences Interfaces and Human Computer Interaction; Game and Entertainment Technologies; and Computer Graphics, Visualization, Computer Vision and Image Processing (IHCI/GET/CGVCVIP) · doi:10.33965/ihci_get2020_202010l005
Manjrekar, O., Biswas, R., Agrawal, A., Barandwal, N., Gupta, A., Chaudhari, P., Mishra, S., Anavardekar, K., Fernandes, V., & Arya, K. (2022) · New Techno Humanities · doi:10.1016/j.techum.2022.05.002
Girişimcilik ve Sosyal Medya: Sosyal Medya Sezgisellik Değerlendirmesi ve Sınıf Projesi
Akın, M., & TOKAY, S. B. (2022) · Girişimcilik İnovasyon ve Pazarlama Araştırmaları Dergisi · doi:10.31006/gipad.1076083
Innovative Interaction Software for Remote Assistance ApplicationPreprint
Sumithra, T. V., Ragha, L. R., Desai, R., & Vaishya, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4159662
Impact of Gender as a Confounding Factor in Perceived Usability: A Family of Observational StudiesPreprint
Cachero, C., Meliá, S., & Luján‐Mora, S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4170551
Akın, M. (2022) · İzmir İktisat Dergisi · doi:10.24988/ije.1047552
Marley, J., Larsson, C., Piccininic, F., Howes, S. C., Casoni, E., Ekvall-Hansson, E., & McDonough, S. (2022) · Research Square · doi:10.21203/rs.3.rs-2185414/v1
Prabandani, A. D., Tolle, H., & Rokhmawati, R. I. (2022) · Jurnal Teknologi Informasi dan Ilmu Komputer · doi:10.25126/jtiik.2022976760
Implementasi Virtual Tour JTE UNRI Menggunakan Kombinasi Foto Panorama Dan Rekayasa 3D
Susilo, E., Andhi, R. R., Ujang, F., & Anhar, A. (2023) · Infotek Jurnal Informatika dan Teknologi · doi:10.29408/jit.v6i1.7456
Muna, M. F., Trapsilawati, F., Ushada, M., & Alsana, S. (2023) · AIP conference proceedings · doi:10.1063/5.0116124
Aziza, N. L., Surrayensih, S., Wijayanti, T., & Sumari, A. D. W. (2023) · Compiler · doi:10.28989/compiler.v12i1.1612
EVALUASI DAN PERBAIKAN UI/UX WEBSITE INVITEES MENGGUNAKAN METODE HUMAN CENTERED DESIGN
Ticoalu, G. B., Musdar, I. A., & Abdul Munir, S. (2023) · KHARISMA Tech · doi:10.55645/kharismatech.v18i1.287
Research on Chinese Font Size of Automobile Central Control Interface Driving
Shao, J., Yang, Z., Li, Y., Li, X., Huang, Y., & Deng, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35908-8_15
Analisis Kualitas Interaksi Pengunjung Dengan Website Bali Vulcano Jeep: Pendekatan Kuesioner UX
Dewi, N. W. J. K., & Antara, I. G. M. Y. (2023) · Jurnal Ilmiah Telsinas Elektro Sipil dan Teknik Informasi · doi:10.38043/telsinas.v6i1.4471
Metadata
- Authors
- Faulkner, L.
- Affiliations
- Applied Research Laboratories, The University of Texas at Austin (US)
- Corresponding author
- Faulkner, Laura
- Venue
- Behavior Research Methods, Instruments, & Computers
- Published
- 2003-08
- Volume
- 35
- Issue
- 3
- Pages
- 379-383
- ISSN
- 0743-3808 (ISSN-L)
- 1532-5970
- OpenAlex
- W2088028234
- Semantic Scholar
- 7369703e67a6812b4476fc95f8085c4d20ebe2ff
- Cited by
- 1,053
- References
- 9
- Citations by year
- 2026: 95 · 2025: 83 · 2024: 98 · 2023: 83 · 2022: 83 · 2021: 92 · 2020: 81 · 2019: 59 · 2018: 61 · 2017: 51 · 2016: 46 · 2015: 39 · 2014: 33 · 2013: 33 · 2012: 25
- FWCI
- 8.34
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Behavior Research Methods, Instruments, & Computers (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Persona Design and Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- usability testing, sample size, number of participants, random sampling, problem identification
- MeSH terms
- Consumer Behavior, Data Collection, Humans, Product Surveillance, Postmarketing, Software, Task Performance and Analysis, User-Computer Interface, Sample Size, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
