Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Computerized clinical reminders (CRs) were designed to reduce clinicians' reliance on their memory and to present evidence-based guidelines at point of care. However, the literature indicates that CR adoption and effectiveness has been variable. We examined the impact of four design modifications to CR software on learnability, efficiency, usability, and workload for intake nursing personnel in an outpatient clinic setting. These modifications were included in a redesign primarily to address barriers to effective CR use identified during a previous field study. DESIGN: In a simulation experiment, 16 nurses used prototypes of the current and redesigned system in a within-subject comparison for five simulated patient encounters. Prior to the experimental session, participants completed an exploration session, where "learnability" of the current and redesigned systems was assessed. MEASUREMENTS: Time, performance, and survey data were analyzed in conjunction with semi-structured debrief interview data. RESULTS: The redesign was found to significantly increase learnability for first-time users as measured by time to complete the first CR, efficiency as measured by task completion time for two of five patient scenarios, usability as determined by all three groupings of questions taken from a commonly used survey instrument, and two of six workload subscales of the NASA Task Load Index (TLX) survey: mental workload and frustration. CONCLUSION: Modest design modifications to existing CR software positively impacted variables that likely would increase the willingness for first-time nursing personnel to adopt and consistently use CRs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
The quality of health care delivered to adults in the United States
McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa022615 · Cited by 5,079
Self-citationGarg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use
Lewis, J. R. (1995) · International Journal of Human-Computer Interaction · doi:10.1080/10447319509526110 · Cited by 2,429
Using usability heuristics to evaluate patient safety of medical devices
Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L., & Kubose, T. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00060-1 · Cited by 591
A framework for analyzing the cognitive complexity of computer-assisted clinical ordering
Horsky, J., Kaufman, D. R., Oppenheim, M. I., & Patel, V. L. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00062-5 · Cited by 137 · cited 1×
“such as usability testing, cognitive task analysis,”
Human Factors Barriers to the Effective Use of Ten HIV Clinical Reminders
Patterson, E. S., Nguyen, A. D., Halloran, J. P., & Asch, S. M. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1364 · Cited by 115
Self-citation
Cited by
Review of health information technology usability study methodologies
Yen, P.-Y., & Bakken, S. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2010-000020 · Cited by 390
Saleem, J. J., Russ, A. L., Sanderson, P., Johnson, T. R., Zhang, J., & Sittig, D. F. (2009) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638638 · Cited by 72
“It has been used in several domains to assess workload in complex environments, including for the evaluation of CISs, such as a pulmonary display [121] and computerized clinical reminders [122].”
MethodSelf-citationClarke, M. A., Belden, J. L., & Kim, M. S. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12277 · Cited by 30
Metadata
- Authors
- Saleem, J. J., Patterson, E. S., Militello, L., Anders, S., Falciglia, M., Wissman, J. A., Roth, E. M., & Asch, S. M.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-09-01
- Volume
- 14
- Issue
- 5
- Pages
- 632-640
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2158100229
- Cited by
- 90
- References
- 7
- FWCI
- 9.75
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses. https://app.cochat.ai/papers/10.1197/jamia.m2163
- The paper
- Saleem, J. J., Patterson, E. S., Militello, L., Anders, S., Falciglia, M., Wissman, J. A., Roth, E. M., & Asch, S. M. (2007). Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses. Journal of the American Medical Informatics Association. https://doi.org/10.1197/jamia.m2163
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.