CoChat

Review of health information technology usability study methodologies

Open access
Yen, Po-Yin, Bakken, Suzanne · 2012 · Journal of the American Medical Informatics Association

The DOI matches its Crossref record; 77 of the 78 references it cites were found in a registry or an index.

Abstract

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user-task interaction, n=42; stage 2, system-task interaction, n=310; stage 3, user-task-system interaction, n=69; stage 4, user-task-system-environment interaction, n=54; and stage 5, user-task-system-environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/amiajnl-2010-000020

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Data management flowchart. In stage 2, system validation was performed by sensitivity and specificity testing, receiver operating characteristic curve, or observer variation and can be identified at title and abstract level with MeSH search. We identified 27 articles from 346 full-text articles and 283 articles at the title and abstract level.
Figure 1 Data management flowchart. In stage 2, system validation was performed by sensitivity and specificity testing, receiver operating characteristic curve, or observer variation and can be identified at title and abstract level with MeSH search. We identified 27 articles from 346 full-text articles and 283 articles at the title and abstract level.CC BY-NC · PubMed Central
Figure 2 A stratified view of health information technology evaluation. Level 1 targets system specification to understand user-task interaction for system development. Level 2 examines task performance to assess system validation and human–computer interaction. Level 3 aims to incorporate environmental factors to identify work processes and system impacts in real settings. Task/expectation complexity, user variances, and organizational support are factors that influence the use of the system, but are not problems of the system itself, and need to be differentiated from system-related issues.
Figure 2 A stratified view of health information technology evaluation. Level 1 targets system specification to understand user-task interaction for system development. Level 2 examines task performance to assess system validation and human–computer interaction. Level 3 aims to incorporate environmental factors to identify work processes and system impacts in real settings. Task/expectation complexity, user variances, and organizational support are factors that influence the use of the system, but are not problems of the system itself, and need to be differentiated from system-related issues.CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 77 references

Research works printed but not linked: 3

  1. 1993, Handbook Of Usability Testing
  2. 2008, User-centered approaches for eliciting functional requirements and designing user interfaces: a nursing documentation case study, J Biomed Inform
  3. 1982, Using the “Think Aloud” Method in Cognitive Interface Design

Cited by

See all 457

Metadata

Authors
Yen, P.-Y., & Bakken, S.
Affiliations
The Ohio State University (US)
Corresponding author
Yen, Po-Yin
Venue
Journal of the American Medical Informatics Association
Published
2011-08-09
Volume
19
Issue
3
Pages
413-422
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W2121431887
Cited by
390
References
78
Citations by year
2026: 17 · 2025: 29 · 2024: 26 · 2023: 31 · 2022: 31 · 2021: 36 · 2020: 40 · 2019: 23 · 2018: 35 · 2017: 32 · 2016: 16 · 2015: 29 · 2014: 17 · 2013: 22 · 2012: 2
FWCI
2.03
Type
journal article
Language
en
Open access
Licence
cc-by-nc-nd
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
National Institute of Nursing Research (P30NR010677)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Social Sciences › Decision Sciences › Information Systems and Management › Technology Adoption and User Behaviour
Keywords
health information technology, usability evaluation, usability model, system development life cycle, task-based assessment, decision support system, methodological recommendations
MeSH terms
Humans, Medical Informatics, Models, Theoretical, Research Design, Technology Assessment, Biomedical, User-Computer Interface

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Review of health information technology usability study methodologies. https://app.cochat.ai/papers/10.1136/amiajnl-2010-000020
The paper
Yen, P.-Y., & Bakken, S. (2012). Review of health information technology usability study methodologies. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2010-000020

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.