Using usability heuristics to evaluate patient safety of medical devices
Open accessAbstract
Objective To modify the traditional heuristic evaluation method of assessing software usability so that it can be applied to medical devices and used to evaluate the patient safety of those devices through the identification and assessment of usability problems.Design Heuristic evaluation, a usability inspection method commonly used for software usability evaluation, was modified and extended for medical devices. The modified method was used to evaluate and compare the patient safety of two 1-channel volumetric infusion pumps.Results The modified heuristic evaluation method was successfully applied to medical devices. One hundred and ninety-two heuristic violations were categorized for 89 usability problems identified for Pump 1, and 121 heuristic violations were categorized for the 52 usability problems identified for Pump 2. Pump 1 had more usability problems with high severity ratings than Pump 2. In general, Pump 1 was found to have more usability issues that are likely to induce more medical errors.Conclusions Heuristic evaluation, when modified for medical devices, is a useful, efficient, and low cost method for evaluating patient safety features of medical devices through the identification of usability problems and their severities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The survey instrument was based on the surveys developed by Zhang et al. and Davis et al., with modifications to match the research topic and eliminate irrelevant items.
“The survey instrument was based on the surveys developed by Zhang et al. [29] and Davis et al. [36], with modifications to match the research topic and eliminate irrelevant items.”
From Determinants of user acceptance of a specific social platform for older adults: An empirical examination of user interface characteristics and behavioral intention (2017) · cites this paper as [29]
Section 3.1 presents 14 usability heuristics, and Section 4 describes evaluators applying them to infusion pumps. A search for a survey or questionnaire instrument found no passage in our copy saying Zhang et al. developed a survey, so the attributed basis for the citing paper’s survey cannot be verified. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1993) · Cited by 9,879 · see citing works
(1993) · Elsevier eBooks · doi:10.1016/c2009-0-21512-1 · Cited by 5,533 · see citing works
The Psychology of Human-Computer Interaction
Card, S. K., Moran, T. P., & Newell, A. (2018) · CRC Press eBooks · doi:10.1201/9780203736166 · Cited by 4,103 · see citing works
Heuristic evaluation of user interfaces
Nielsen, J., & Molich, R. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97281 · Cited by 3,533 · see citing works
Nielsen, J. (1995) · Conference companion on Human factors in computing systems - CHI '95 · doi:10.1145/223355.223730 · Cited by 2,620 · see citing works
The psychology of human-computer interaction
Rogers, Y. (1984) · Applied Ergonomics · doi:10.1016/0003-6870(84)90205-9 · Cited by 1,970 · see citing works
(1987) · Andalas University Repository (Andalas University) · Cited by 1,584 · see citing works
Brown, A. B., & Patterson, D. A. (2001) · Cited by 1,396 · see citing works
Finding usability problems through heuristic evaluation
Nielsen, J. (1992) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/142750.142834 · Cited by 1,066 · see citing works
Preventable Anesthesia Mishaps
Cooper, J. B., Newbower, R. S., Long, C. D., & McPeek, B. (1978) · Anesthesiology · doi:10.1097/00000542-197812000-00004 · Cited by 929 · see citing works
(1992) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 451 · see citing works
Suess, A. P. (1997) · Journal of interactive instruction development · Cited by 443 · see citing works
Applying Human Factors to the Design of Medical Equipment: Patient-Controlled Analgesia
Lin, L., Isla, R., Doniz, K., Harkness, H., Vicente, K. J., & Doyle, D. J. (1998) · Journal of Clinical Monitoring and Computing · doi:10.1023/a:1009928203196 · Cited by 141 · see citing works
Using the Keystroke-Level Model to Estimate Execution Times
Kieras, D. E. (2003) · Cited by 139 · see citing works
Obradovich, J. H., & Woods, D. D. (1996) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872096778827251 · Cited by 124 · see citing works
Designing human-centered distributed information systems
Zhang, J., Patel, V. L., Johnson, K. A., & Smith, J. W. (2002) · IEEE Intelligent Systems · doi:10.1109/mis.2002.1039831 · Cited by 121 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Cognitive and usability engineering methods for the evaluation of clinical information systems.
Kushniruk, A. W., & Patel, V. L. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.01.003 · Cited by 760 · see citing works
Self-citationJaspers, M. W. M. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.10.002 · Cited by 649 · see citing works
TURF: Toward a unified framework of EHR usability
Zhang, J., & Walji, M. F. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.08.005 · Cited by 301 · see citing works
Self-citationShah, S. G. S., Robinson, I., & AlShawi, S. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309990328 · Cited by 237 · see citing works
Comprehensive Analysis of a Medication Dosing Error Related to CPOE: Table 1
Horsky, J., Kuperman, G. J., & Patel, V. L. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1740 · Cited by 197 · see citing works
Self-citationJohnsen, H. M., Fossum, M., Vivekananda-Schmidt, P., Fruhling, A., & Slettebø, Å. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.06.014 · Cited by 141 · see citing works
Salman, H. M., Wan Ahmad, W. F., & Sulaiman, S. (2018) · IEEE Access · doi:10.1109/access.2018.2827358 · Cited by 134 · see citing works
Rundo, L., Pirrone, R., Vitabile, S., Sala, E., & Gambino, O. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103479 · Cited by 113 · see citing works
Applying heuristic evaluation to improve the usability of a telemedicine system
Tang, Z., Johnson, T. R., Tindall, R. D., & Zhang, J. (2006) · Telemedicine and e-Health · doi:10.1089/tmj.2006.12.24 · Cited by 102 · see citing works
Self-citationHeuristic evaluation of paper-based Web pages: a simplified inspection usability methodology
Allen, M., Currie, L. M., Bakken, S., Patel, V. L., & Cimino, J. J. (2006) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.10.004 · Cited by 95 · see citing works
Self-citationDunn Lopez, K., Chin, C.-L., Leitão Azevedo, R. F., Kaushik, V., Roy, B., Schuh, W., Banks, K., Sousa, V., & Morrow, D. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103359 · Cited by 62 · see citing works
Tsai, T.-H., Chang, H.-T., Chen, Y.-J., & Chang, Y.-S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180102 · Cited by 59 · see citing works
Applying usability heuristics to radiotherapy systems
Chan, A. J., Islam, M. K., Rosewall, T., Jaffray, D. A., Easty, A. C., & Cafazzo, J. A. (2012) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2011.05.077 · Cited by 58 · see citing works
- Authors
- Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L., & Kubose, T.
- Affiliations
- The University of Texas Health Science Center at Houston (US)
- Columbia University (US)
- Rice University (US)
- Corresponding author
- Zhang, Jiajie
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2003-02
- Volume
- 36
- Issue
- 1-2
- Pages
- 23-30
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W2123051583
- Cited by
- 591
- References
- 16
- Citations by year
- 2026: 25 · 2025: 21 · 2024: 25 · 2023: 37 · 2022: 15 · 2021: 37 · 2020: 42 · 2019: 39 · 2018: 38 · 2017: 35 · 2016: 33 · 2015: 29 · 2014: 28 · 2013: 35 · 2012: 17
- FWCI
- 18.76
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (P01 HS 11544, HS 11544)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Physical Sciences › Engineering › Biomedical Engineering › Intravenous Infusion Technology and Safety
- Keywords
- heuristic evaluation, usability, medical devices, patient safety, infusion pumps, medical errors, usability problems, human factors
- MeSH terms
- Algorithms, Decision Support Techniques, Equipment Failure, Equipment Safety, Infusion Pumps, Man-Machine Systems, Quality Assurance, Health Care, Software Validation, User-Computer Interface, Safety Management, Medical Errors, Equipment Failure Analysis, Benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
