Number of people required for usability evaluation: the 10±2 rule
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Introduction Usability evaluation is essential to make sure that software products newly released are easy to use, efficient, and effective to reach goals, and satisfactory to users. For example, when a software company wants to develop and sell a new product, the company needs to evaluate usability of the new product before launching it at a market to avoid the possibility that the new product may contain usability problems, which span from cosmetic problems to severe functional problems. Three widely used methods for usability evaluation are Think Aloud (TA), Heuristic Evaluation (HE) and Cognitive Walkthrough (CW). TA method is commonly employed with a lab-based user testing, while there are variants of TA methods, including thinking out aloud at user's workplace instead of at labs. What we discuss here is the TA method that is combined with a lab-based user testing, in which test users use products while simultaneously and continuously thinking out aloud, and experimenters record users' behaviors and verbal protocols in the laboratory. HE is a usability inspection method, in which a small number of evaluators find usability problems in a user interface design by examining an interface and judging its compliance with well-known usability principles, called heuristics. CW is a theory-based method, in which evaluators evaluate every step necessary to perform a scenario-based task, and look for usability problems that would interfere with learning by exploration. These three methods have their own advantages and disadvantages. For instance, TA method provides good qualitative data from a small number of test users, but laboratory environment may influence test user's behaviors. HE is a cheap, fast and easy-to-use method, while it often finds too specific and low-priority usability problems, including even not real problems. CW helps find mismatches between users' and designers' conceptualization of a task, but it needs extensive knowledge of cognitive psychology and technical details to apply. However, even though these advantages and disadvantages show overall characteristics of three major usability evaluation methods, we cannot compare them quantitatively and see their efficiency clearly. Because one of reasons why so-called discounted methods, such as HE and CW, were developed is to save costs of usability evaluation, cost-related criteria for comparing usability evaluation are meaningful to usability practitioners as well as usability researchers. One of the most disputable issues related to cost of usability evaluation is sample size. That is, how many users or evaluators are needed to achieve a targeted usability evaluation performance, for example, 80% of overall discovery rate? The sample size of usability evaluation is known to depend on an estimate of problem discovery rate across participants. The overall discovery rate is a common quantitative measure that is used to show the effectiveness of a specific usability evaluation method in most of usability evaluation studies. It is also called overall detection rate or thoroughness measure, which is the ratio of 'the sum of unique usability problems detected by all experiment participants' against 'the number of usability problems that exist in the evaluated systems', ranging between 0 and 1. The overall discovery rates were reported more than any other criterion measure in the usability evaluation experiments and also a key component for projecting required sample size for usability evaluation study. Thus, how many test users or evaluators participate in the usability evaluation is a critical issue, considering its cost-effectiveness.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, a sample size of 10±2 is sufficient for discovering 80 per cent of usability issues [55].”
From A user-centred design framework for mHealth (2020) · cites this paper as [55]
Citations
Cites
Heuristic evaluation of user interfaces
Nielsen, J., & Molich, R. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97281 · Cited by 3,533
Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?
Virzi, R. A. (1992) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089203400407 · Cited by 1,190
Estimating the number of subjects needed for a thinking aloud test
Nielsen, J. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1065 · Cited by 476
Sample sizes for usability studies: Additional considerations
Lewis, J. R. (1994) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089403600215 · Cited by 341
What is gained and lost when using evaluation methods other than empirical testing
Desurvire, H., Kondziela, J. M., & Atwood, M. E. (1993) · Cited by 107
Relaxing the homogeneity assumption in usability testing
Caulton, D. A. (2001) · Behaviour & Information Technology · doi:10.1080/01449290010020648 · Cited by 90
Law, L.-C., & Hvannberg, E. Þ. (2002) · Nordic Conference on Human-Computer Interaction · doi:10.1145/572020.572030 · Cited by 87
Evaluation of Procedures for Adjusting Problem-Discovery Rates Estimated From Small Samples
Lewis, J. R. (2001) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1304_06 · Cited by 72
Analysis of combinatorial user effect in international usability tests
Law, E. L., & Hvannberg, E. Þ. (2004) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985694 · Cited by 63
Novice heuristic evaluations of a complex interface
Slavković, A., & Cross, K. (1999) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/632716.632902 · Cited by 31
Cited by
Items 301–400 of 563
M&E-NetPay: A Micropayment System for Mobile and Electronic Commerce
Huang, X., Bao, J., Dai, X., Singh, E., Huang, W., & Huang, C. (2016) · Symmetry · doi:10.3390/sym8080074 · Cited by 4
Usability evaluation in practice: Selecting the appropriate method
Hayat, H., & Lock, R. (2020) · Figshare · Cited by 4
Embalagens de medicamentos: diretrizes para o desenvolvimento
Lucio, C. D. C. (2013) · Americanae (AECID Library) · Cited by 4
Ouwerkerk, M., Eijssen, I. C. J. M., van der Linden, M. M. W., Wijnands, I. M., Dorssers, F., Rietberg, M. B., Beckerman, H., & de Groot, V. (2022) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2022.01.168 · Cited by 3
Schraagen, J. M. (2024) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003410379-22 · Cited by 3
User Perspectives and Usability Insights in a Self-Service Portal
Matloobtalab, M., & Ferati, M. (2025) · International Journal of Web Portals · doi:10.4018/ijwp.372057 · Cited by 3
MultiUX : A Human-AI Collaborative Tool to Facilitate Multiple Usability Test Video Analyses
Shen, L., Shi, Q., Kuang, E., Qiu, L., Zhou, S., Hui, P., & Fan, M. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2605526 · Cited by 3
Wei, Y., Lin, Z., Ou, B., Wu, J., Chen, G., Xu, G., Lin, X., Zheng, J., & Xiao, H. (2026) · Asia-Pacific Journal of Oncology Nursing · doi:10.1016/j.apjon.2026.100852 · Cited by 3
Grubišić, A., Šarić‐Grgić, I., Gašpar, A., & Žitko, B. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70071 · Cited by 3
Investigating learning with an interactive tutorial: A mixed-methods strategy
de Villiers, R., & Becker, D. (2016) · Innovations in Education and Teaching International · doi:10.1080/14703297.2016.1266959 · Cited by 3
Usability Evaluation of an EHR to Improve Physician Performance
Clarke, M. A., Belden, J. L., & Kim, M. S. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-50838-8_33 · Cited by 3
Liu, H., Chen, T., Yam, L. Y.-C., Wong, E. L., Bennell, K. L., Yeoh, E., Cheung, M. T., & Or, C. K. (2025) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813251357891 · Cited by 3
Usability Assessment of a Philippine Commuting Navigation Application
Toledo, J., Salgado, A. C., Gozum, C. C., & Custodio, B. P. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-51828-8_38 · Cited by 3
Meirose, J., & Lian, B. E. (2019) · Journal of Electronic Resources in Medical Libraries · doi:10.1080/15424065.2019.1590171 · Cited by 3
Treadstone: A Social-Media-Inspired Platform for Multi-Agent Collaborative Data AnalysisPreprint
Lee, H., Cho, S., Benjamin, W., Lee, C., Jeon, H., Jeong, D., Shin, S., Ko, S., & Elmqvist, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.19774 · Cited by 3
A User-Oriented Test environment based on User-Interface evaluation graphical controls
Charfi, S., Trabelsi, A., Ezzedine, H., & Kolski, C. (2013) · IFAC Proceedings Volumes · doi:10.3182/20130811-5-us-2037.00060 · Cited by 3
Charfi, S. (2013) · theses.fr (ABES) · doi:10.70675/05a38df8za87dz4548z8828zf3bec0c4a9f6 · Cited by 3
Walk before you run: using heuristic evaluation to assess a training tool prototype
Monasor, M. J., Noll, J., Vizcaíno, A., Piattini, M. G., & Beecham, S. (2014) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2601248.2601271 · Cited by 3
Skolimowski, K., Rola, B., Tokarski, P. I., Rybak, K., Łazaruk, K., Plechawska–Wójcik, M., & Dzieńkowski, M. (2026) · Advances in Science and Technology – Research Journal · doi:10.12913/22998624/210532 · Cited by 3
Impact analysis of enterprise resource planning post-implementation modifications
Parhizkar, M. (2016) · City Research Online (City University London) · Cited by 3
FRVT Part 4A: MORPH - utility of 1:N face recognition for morph detection
Ngan, M. L., Grother, P. J., & Hanaoka, K. (2022) · doi:10.6028/nist.ir.8430 · Cited by 3
Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation
Kim, T., Shin, J., Kim, Y.-H., Bae, S., & Kim, S. (2024) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2024.nlp4convai-1.3 · Cited by 3
Faujdar, V. (. (2019) · Research Repository (Delft University of Technology) · Cited by 3
Using Design Thinking to break social barriers: An experience report with former inmates
Canedo, E. D., Cançado, E. C. R., Mota, A. P. B., Bandeira, I. N., Costa, P. H. T., Lima, F., Amaral, L. H. V., & Bonifácio, R. (2024) · Journal of Software Evolution and Process · doi:10.1002/smr.2648 · Cited by 3
Holzhüter, M., Betke, H., Fröhlich, A., & Klafft, M. (2024) · Proceedings of the ... International ISCRAM Conference · doi:10.59297/09pere02 · Cited by 3
Sorour, A. S., & Atkins, A. S. (2024) · Cloud Computing and Data Science · doi:10.37256/ccds.6120255631 · Cited by 3
Interactive TV user interfaces: how fast is too fast?
Golja, M., Stojmenova, E., & Humar, I. (2013) · Multimedia Tools and Applications · doi:10.1007/s11042-013-1365-4 · Cited by 3
Bancin, R. F., & Rachmaniah, M. (2022) · Jurnal Ilmu Komputer dan Agri-Informatika · doi:10.29244/jika.9.1.1-12 · Cited by 3
Toribio-Guzmán, J. M., García‐Holgado, A., Soto‐Pérez, F., García‐Peñalvo, F. J., & Franco, M. (2016) · Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'21) · doi:10.1145/3012430.3012568 · Cited by 3
Comulada, W. S., Swendeman, D. T., Ho, Y. X., Streck, J. M., Lewis, D.-M., Rezai, R., Warren, D., Pachas, G. N., Chandler, M., Jackson, J. L., et al. (2026) · JMIR Formative Research · doi:10.2196/84683 · Cited by 2
Rupere, T., & Jakovljević, M. (2020) · Cited by 2
Celleri, M., Klajner, F., Cremades, F. C., Garay, C. J., Etchevers, M. J., & Osma, J. (2025) · Journal of Clinical Psychology · doi:10.1002/jclp.70037 · Cited by 2
Using Think-Aloud Protocol in Immersive VR Evaluations
Zhang, X., & Simeone, A. L. (2023) · Human-computer interaction series · doi:10.1007/978-3-031-05804-2_8 · Cited by 2
Technology Acceptance Model for Enhanced Shopping Experience Through Online Recommendation Agent
Lališová, D., Karpavičė, J., & Tambo, T. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-49215-0_41 · Cited by 2
Pengaruh Aspek Pedagogis dan Interaksi Sistem pada Penilaian Usabilitas Sistem E-Learning
Permatasari, I., & Hutabarat, P. M. (2022) · Generic · doi:10.18495/generic.v14i2.129 · Cited by 2
Implementing ALiS: Towards a Reference Architecture for Augmented Living Spaces
Boehmer, M., Damarowsky, J., & Kühnel, S. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.394 · Cited by 2
From Web-Based to Mobile: Experiences of Developing a Personally Controlled Health Management System
Biederer, M., Arguel, A., Liu, J. L., & Lau, A. Y. S. (2014) · eJournal of health informatics · Cited by 2
Evaluating the Usability and Engagement of a Gamified, Desktop, Virtual Art Appreciation Module
Li, S., Nasri, N. M., & Norman, H. (2025) · Multimodal Technologies and Interaction · doi:10.3390/mti9060053 · Cited by 2
Exploring Mixed-Interaction Mode in a Virtual Cockpit: Controller and Hand Gesture Integration
Lee, Y., Connor, A. M., & Marks, S. (2025) · Virtual Worlds · doi:10.3390/virtualworlds4020028 · Cited by 2
Creating Greater Synergy Between HCI Academia and Practice
Nah, F. F., Galletta, D. F., Knight, M. M., Lewis, J. R., Pruitt, J., Salvendy, G., Sheng, H., & Wichansky, A. M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20895-4_68 · Cited by 2
Self-citationPanzavolta, A., Caffarra, P., Vita, D. D., Dodich, A., Fonti, C., L’Abbate, F., Lavorgna, L., Laganà, V., Marra, C., Papagno, C., et al. (2025) · Neurological Sciences · doi:10.1007/s10072-025-08542-8 · Cited by 2
Sauro, J., & Lewis, J. R. (2012) · Elsevier eBooks · doi:10.1016/b978-0-12-384968-7.00006-0 · Cited by 2
Seidel, A., Weidle, F., & Börner, C. (2021) · International Conference of the Immersive Learning Research Network (iLRN) · doi:10.23919/ilrn52045.2021.9459389 · Cited by 2
Łazaruk, K., Tokarski, P. I., Rybak, K., Plechawska–Wójcik, M., & Dzieńkowski, M. (2025) · IEEE Access · doi:10.1109/access.2025.3614018 · Cited by 2
The Impact of Usability on Patient Safety in Long-Term Care
Abujarad, F., Swierenga, S. J., Dennis, T. A., & Post, L. A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20895-4_21 · Cited by 2
(2020) · Elsevier eBooks · doi:10.1016/b978-0-12-824426-5.00019-2 · Cited by 2
Dafalla, T. D. M. (2013) · UVic’s Research and Learning Repository (University of Victoria) · Cited by 2
Testing usability metrics with underserved populations
Hallonbacka, M. (2024) · Aaltodoc (Aalto University) · Cited by 2
Generating a domain-specific inspection method through an adaptive framework
Alrobaea, R. (2015) · UEA Digital Repository (University of East Anglia) · Cited by 2
Brant-Ribeiro, T., & Cattelan, R. G. (2017) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2016.24.3.86 · Cited by 2
Towards Lean EHR Usability Heuristics for Behavioral Health Providers
Boehler, J. (2020) · ResearchWorks at the University of Washington (University of Washington) · Cited by 2
Comparative critiquing and example-based approach for learning client-server design
Jamal, N. A. A., & Ali, N. M. (2017) · IEEE Conference on e-Learning, e-Management and e-Services (IC3e) · doi:10.1109/ic3e.2017.8409234 · Cited by 2
Chatbot Based Public Sensoring to improve Situational Awareness
Betke, H., Peitzsch, S., Boldt, J., Reimann, D., & Kox, T. (2024) · Proceedings of the ... International ISCRAM Conference · doi:10.59297/h3r7bp59 · Cited by 2
Quality Assessment of an Expert System: An Instrument of Regular Feedback from Users
Begier, B. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-19968-4_10 · Cited by 2
Development and Human-Centered Evaluation: A Motion-Based Mixed-Reality Flight Simulator
Zintl, M., Kimura, S., & Holzapfel, F. (2026) · Journal of Aerospace Information Systems · doi:10.2514/1.i011738 · Cited by 2
Zintl, M., Liedtke, S., Janetzko, D., Kimura, S., Schmidt-Moll, C., Faber, T., Holzapfel, F., Klinker, G., & Bengler, K. J. (2024) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc62030.2024.10749170 · Cited by 2
Begier, B. (2013) · International Journal of Intelligent Information and Database Systems · doi:10.1504/ijiids.2013.053828 · Cited by 2
Working with Users to Ensure Quality of Innovative Software Product despite Uncertainties
Begier, B. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-20039-7_34 · Cited by 2
Multimodal E-Commerce: A Usability and Social Presence Investigation
Aborokbah, M. M. (2014) · DMU Open Research Archive (De Montfort University) · Cited by 2
How can usability contribute to user experience? A study in the domain of e-commerce
Moczarny, I., De Villiers, M. R. (., & Van Biljon, J. (2012) · Unisa Institutional Repository (University of South Africa) · Cited by 2
Teaching Educators About Acquired Brain Injury: A Usability Study Evaluating the Teach-ABI Module
Saly, L. (2021) · TSpace (University of Toronto) · Cited by 2
Huang, E. (2010) · Tuwhera (Auckland University of Technology) · Cited by 2
Verhoeven, R., Kooi, E. M. W., Obermann-Borst, S. A., Geurtzen, R., Labrie, N., Verhagen, E. A. A., & Hulscher, J. B. F. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-087939 · Cited by 2
Varmazyar, M., Homayoun, A. A., Karbasi, F., & Garosi, E. (2026) · Scientific Reports · doi:10.1038/s41598-026-60297-0 · Cited by 2
Usability evaluation of detectable warning surfaces in Quebec City (Canada): an exploratory study
Routhier, F., Lettre, J., Fiset, D., & Morales, E. (2019) · Disability and Rehabilitation · doi:10.1080/09638288.2019.1655103 · Cited by 2
Development and user testing study of MozzHub: a bipartite network-based dengue hotspot detector
Labadin, J., Hong, B. H., Tiong, W. K., Gill, B. S., Perera, D., Rigit, A. R. H., Singh, S., Tan, C. V., Ghazali, S. M., Jelip, J., et al. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14120-3 · Cited by 2
Public engagement with biomedical research through location-sensitive technology
Naylor, R., Elliott, K. A., Gray, K. M., & Wadley, G. (2011) · International Conference on Communities and Technologies (C&T) · doi:10.1145/2103354.2103379 · Cited by 2
Qualitative Findings from Usability Testing of an E-learning Tutorial
de Villiers, R., & Becker, D. (2012) · E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education · Cited by 2
Learning with an e-learning tutorial: Cognition and the value of errors
de Villiers, R., Becker, D., & Adebesin, T. F. (2012) · Unisa Institutional Repository (University of South Africa) · Cited by 2
A Usability Testing on the Smart Device-based AAC Application
김정연, & Si-won, K. (2016) · 특수교육교과교육연구 · doi:10.24005/seci.2016.9.2.59 · Cited by 2
Septiandi, Y., & Suzianti, A. (2019) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3369985.3369995 · Cited by 2
Framework for improved capture of usability requirements through usage-centred design
Alotaibi, K. J. (2015) · ePrints Soton (University of Southampton) · Cited by 2
Aplicación de la experiencia de usuario accesible en entornos web de recursos educativos abiertos
Navarrete, R. D. L. C. B. (2017) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 2
Exergaming : its Social Aspects
Mahat, S. (2019) · Theseus (Ammattikorkeakoulujen) · Cited by 2
Punchoojit, L., & Hongwarittorrn, N. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2551571 · Cited by 1
Böhmer, M., Kühnel, S., & Damarowsky, J. (2024) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-5237-3.ch016 · Cited by 1
Böhmer, M., Kühnel, S., Damarowsky, J., & Brendel, B. (2025) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/3757308.3757311 · Cited by 1
Lee, C. S., Osop, H., Goh, D. H., Chia, D. C. Y., & Nguyen, C. T. M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31048-4_29 · Cited by 1
Hartmann, E. M., Küper, A., Swoboda, J., Beckmann, C. L., Lodde, G. C., Livingstone, E., Schadendorf, D., & Sachweh, S. (2026) · JMIR Cancer · doi:10.2196/79294 · Cited by 1
Iterative user-centered evaluation of a context-driven clinical dashboard for melanoma treatment
Hartmann, E. M., Borys, K., Idrissi-Yaghir, A., Lodde, G. C., Livingstone, E., Schadendorf, D., Nensa, F., Brieden, T., & Sachweh, S. (2026) · Scientific Reports · doi:10.1038/s41598-026-74167-2 · Cited by 1
Online structured education for people newly diagnosed with type 2 diabetes
Poduval, S. (2019) · UCL Discovery (University College London) · Cited by 1
AIBlockAgent: Design and Development of an AI Agent to Support Block-Based Programming Learning
Choa, H., & Koo, D. (2026) · The Journal of Korean Association of Computer Education · doi:10.32431/kace.2026.29.3.011 · Cited by 1
Böhmer, M., Kühnel, S., & Damarowsky, J. (2026) · Decision Support Systems · doi:10.1016/j.dss.2026.114652 · Cited by 1
Lindner, S., & Kühnel, S. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.221 · Cited by 1
Damarowsky, J., Kühnel, S., Seyffarth, T., & Sackmann, S. (2022) · HMD Praxis der Wirtschaftsinformatik · doi:10.1365/s40702-021-00831-y · Cited by 1
Health Literacy–Based Heart Failure Self-care (H2Lit) Application: Development and Usability Testing
Cajita, M., Doorenbos, A. Z., Vuckovic, K. M., Tintle, N., & Dunn, S. L. (2025) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000001169 · Cited by 1
Erdemli, M. (2018) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2018-12886 · Cited by 1
Usability Evaluation of a Smart Textile-Based Cushion System for Ubiquitous Breathing Exercises
Fernandes, S., Vázquez, C. G., Ramos, A. O., Jahanbakhsh, N., Hafid, A., & Vega-Barbas, M. (2026) · Sensors · doi:10.3390/s26154933 · Cited by 1
Improving Student Mental Health Through Health Objectives in a Mobile App
Vinogradov, M., Chang, M., Lin, F. O., & Yan, Y. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-29548-5_8 · Cited by 1
Rogers, I., Grice-Jackson, T., Ford, E. M., Howat, J., Salimkumar, R., Frere‐Smith, K., O’Connor, N., Bastiaens, H., & van Marwijk, H. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000395 · Cited by 1
Development of minimum data set and electronic registry for hemodialysis patients management
Karami, M., Nabovati, E., & Mirpanahi, N. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02914-y · Cited by 1
Nazari, M., Emami, H., Rabiei, R., Rabiee, H. R., Salari, A., & Sadr, H. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109490 · Cited by 1
Correlation of affiliate performance against web evaluation metrics
Miehling, M. J. (2014) · Edinburgh Napier Research Repository (Edinburgh Napier University) · doi:10.17869/enu.177070 · Cited by 1
Kibria, M. G., Kucirka, L. M., Stuebe, A. M., & Mostafa, J. (2026) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi69079.2026.00230 · Cited by 1
Usability testing and redesign of a mobile application for real estate
Avdela, F., & Parisi, S. (2023) · International Conference of the ACM Greek SIGCHI Chapter (CHIGreece) · doi:10.1145/3609987.3609996 · Cited by 1
Mauri, S. A., Fonseca, D., Herrero-Martín, J., & Caro-Via, S. (2026) · Applied Sciences · doi:10.3390/app16094493 · Cited by 1
Herrera-Murillo, D. J., Power, P. A., Lopez-Pellicer, F. J., Baldassarri, S., & Nogueras-Iso, J. (2026) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2026.3682070 · Cited by 1
Usability Testing of a Digital Competence Assessment and Certification System
Sobodić, A., & Balaban, I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50513-4_15 · Cited by 1
Robertson, M. M., Parker, T. R., Henning, R. A., Nobrega, S., Gilmer, D. O., Morocho, C., Calvo, A. J., Cavallari, J. M., & Brennan, M. (2026) · Work · doi:10.1177/10519815261433950 · Cited by 1
Kwint, E., Marsman, M., Veen, A. V. ‘., Hauptmann, H., & Brinkkemper, S. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.06.352 · Cited by 1
Metadata
- Authors
- Hwang, W., & Salvendy, G.
- Affiliations
- Soongsil University (KR)
- Purdue University West Lafayette (US)
- ORCID
- Hwang, Wonil
- Venue
- Communications of the ACM
- Published
- 2010-05
- Volume
- 53
- Issue
- 5
- Pages
- 130-133
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2016022053
- Cited by
- 410
- References
- 11
- Citations by year
- 2026: 40 · 2025: 36 · 2024: 26 · 2023: 43 · 2022: 27 · 2021: 21 · 2020: 29 · 2019: 30 · 2018: 22 · 2017: 24 · 2016: 23 · 2015: 15 · 2014: 24 · 2013: 17 · 2012: 17
- FWCI
- 31.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Communications of the ACM (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- usability evaluation, sample size, think-aloud, heuristic evaluation, cognitive walkthrough, detection rate, evaluation costs, user interface design, verbal protocols, user testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.