Review of health information technology usability study methodologies
Open accessThe DOI matches its Crossref record; 77 of the 78 references it cites were found in a registry or an index.
Abstract
Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user-task interaction, n=42; stage 2, system-task interaction, n=310; stage 3, user-task-system interaction, n=69; stage 4, user-task-system-environment interaction, n=54; and stage 5, user-task-system-environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Usability challenges present a major barrier to health information technology adoption 29”
From A chatbot for hypertension self-management support: user-centered design, development, and usability testing (2023) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
(1993) · Cited by 9,879
Nielsen, J. (1995) · Conference companion on Human factors in computing systems - CHI '95 · doi:10.1145/223355.223730 · Cited by 2,620
IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use
Lewis, J. R. (1995) · International Journal of Human-Computer Interaction · doi:10.1080/10447319509526110 · Cited by 2,429
Development of an instrument measuring user satisfaction of the human-computer interface
Chin, J. P., Diehl, V. A., & Norman, L. K. (1988) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '88 · doi:10.1145/57167.57203 · Cited by 1,609
Participatory Design: Principles and Practices
(2017) · CRC Press eBooks · doi:10.1201/9780203744338 · Cited by 1,500
A Practical Guide to Usability Testing
Dumas, J. F., & Redish, J. C. (1993) · Cited by 1,335
Poissant, L. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1700 · Cited by 901
Biomedical Informatics: Computer Applications in Health Care and Biomedicine
(2024) · Biomedical Informatics: Computer Applications in Health Care and Biomedicine · doi:10.33029/9704-8204-9-bin-2024-1-744 · Cited by 777
Cognitive and usability engineering methods for the evaluation of clinical information systems.
Kushniruk, A. W., & Patel, V. L. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.01.003 · Cited by 760
Research works printed but not linked: 3
- 1993, Handbook Of Usability Testing
- 2008, User-centered approaches for eliciting functional requirements and designing user interfaces: a nursing documentation case study, J Biomed Inform
- 1982, Using the “Think Aloud” Method in Cognitive Interface Design
Cited by
Yardley, L., Morrison, L., Bradbury, K., & Muller, I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4055 · Cited by 1,442
Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., Payne, T. H., Rosenbloom, S. T., Weaver, C., & Zhang, J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001458 · Cited by 618
Mohr, D. C., Schueller, S. M., Montague, E., Burns, M. N., & Rashidi, P. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3077 · Cited by 510
Mohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432
Zahabi, M., Kaber, D. B., & Swangnetr, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815576827 · Cited by 240
Metadata
- Authors
- Yen, P.-Y., & Bakken, S.
- Affiliations
- The Ohio State University (US)
- Corresponding author
- Yen, Po-Yin
- ORCID
- Yen, Po-Yin
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-08-09
- Volume
- 19
- Issue
- 3
- Pages
- 413-422
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3341772
- OpenAlex
- W2121431887
- Semantic Scholar
- 9e355b8a884bed1c649b029c0ca2e6fdbeb6f0b6
- Cited by
- 390
- References
- 78
- Citations by year
- 2026: 17 · 2025: 29 · 2024: 26 · 2023: 31 · 2022: 31 · 2021: 36 · 2020: 40 · 2019: 23 · 2018: 35 · 2017: 32 · 2016: 16 · 2015: 29 · 2014: 17 · 2013: 22 · 2012: 2
- FWCI
- 2.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institute of Nursing Research (P30NR010677)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Decision Sciences › Information Systems and Management › Technology Adoption and User Behaviour
- Keywords
- health information technology, usability evaluation, usability model, system development life cycle, task-based assessment, decision support system, methodological recommendations
- MeSH terms
- Humans, Medical Informatics, Models, Theoretical, Research Design, Technology Assessment, Biomedical, User-Computer Interface
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Review of health information technology usability study methodologies. https://app.cochat.ai/papers/10.1136/amiajnl-2010-000020
- The paper
- Yen, P.-Y., & Bakken, S. (2012). Review of health information technology usability study methodologies. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2010-000020
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.