A usability and safety analysis of electronic health records: a multi-center study
Open accessAbstract
To characterize the variability in usability and safety of EHRs from two vendors across four healthcare systems (2 Epic and 2 Cerner). Twelve to 15 emergency medicine physicians participated from each site and completed six clinical scenarios. Keystroke, mouse click, and video data were collected. From the six scenarios, two diagnostic imaging, laboratory, and medication tasks were analyzed. There was wide variability in task completion time, clicks, and error rates. For certain tasks, there were an average of a nine-fold difference in time and eight-fold difference in clicks. Error rates varied by task (X-ray 16.7% to 25%, MRI: 0 to 10%, Lactate: 0% to 14.3%, Tylenol: 0 to 30%; Taper: 16.7% to 50%). The variability in time, clicks, and error rates highlights the need for improved implementation optimization. EHR implementation, in addition to vendor design and development, is critical to usable and safe products.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Widespread adoption of electronic health records has increased clinical documentation workload, contributing to stress and clinician burnout.
“The widespread adoption of electronic health records has expanded clinical documentation workload, directly contributing to increasing stress and clinician burnout7–9.”
From Adapted large language models can outperform medical experts in clinical text summarization (2024) · cites this paper as [8]
The Background and significance section says EHR usability problems create burdensome workflows and inefficiencies that contribute to clinician frustration. It does not say that widespread EHR adoption increased clinical documentation workload or directly caused increasing stress and burnout. The Discussion examines task time, clicks, and errors, not those broader outcomes; page numbers are unavailable in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredCurrent variability in EHR usability across health care systems and vendors is wide, with certain tasks having an average of a 9-fold difference in time and 8-fold difference in clicks between different EHRs.
“The current variability in EHR usability across health care systems and vendors has been shown to be wide, with certain tasks having an average of a 9-fold difference in time and 8-fold difference in clicks between different EHRs.42”
From The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians (2020) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., Payne, T. H., Rosenbloom, S. T., Weaver, C., & Zhang, J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001458 · Cited by 618 · see citing works
Health IT and Patient Safety: Building Safer Systems for Better Care
Chou, D. (2012) · JAMA · doi:10.1001/jama.308.21.2282-a · Cited by 496 · see citing works
Electronic Health Record Usability Issues and Potential Contribution to Patient Harm
Howe, J. L., Adams, K. T., Hettinger, A. Z., & Ratwani, R. M. (2018) · JAMA · doi:10.1001/jama.2018.1171 · Cited by 281 · see citing works
Self-citationMixed Results In The Safety Performance Of Computerized Physician Order Entry
Metzger, J. B., Welebob, E. M., Bates, D. W., Lipsitz, S. R., & Classen, D. C. (2010) · Health Affairs · doi:10.1377/hlthaff.2010.0160 · Cited by 166 · see citing works
Sample size in usability studies
Schmettow, M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133824 · Cited by 123 · see citing works
Meeks, D. W., Takian, A., Sittig, D. F., Singh, H., & Barber, N. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001762 · Cited by 91 · see citing works
Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015) · JAMA · doi:10.1001/jama.2015.8372 · Cited by 91 · see citing works
Self-citationFairbanks, T., Savage, E., Adams, K., Wittie, M., Boone, E., Hayden, A., Barnes, J., Hettinger, Z., Gettinger, A., & Ratwani, R. M. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2016-06-r-0105 · Cited by 76 · see citing works
Self-citationKoppel, R., & Kreda, D. A. (2010) · Studies in health technology and informatics · doi:10.3233/978-1-60750-569-3-7 · Cited by 41 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Department of Health and Human Services-Office of the National Coor- dinator for Health Information Technology. HIT Standards Committee and HIT Policy Committee Implementation and Usability Hearing. Washington D.C: the United States Department of Health and Human Services; 2013: 1-87.
Cited by
Sorted by citations
Adapted large language models can outperform medical experts in clinical text summarization
Van Veen, D., Van Uden, C., Blankemeier, L., Delbrouck, J.-B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E. P., Seehofnerová, A., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02855-5 · Cited by 891 · see citing works
Melnick, E. R., Dyrbye, L. N., Sinsky, C. A., Trockel, M., West, C. P., Nedelec, L., Tutty, M. A., & Shanafelt, T. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2019.09.024 · Cited by 413 · see citing works
“42 Therefore, the unacceptably low scores with wide variation reported here are likely representative of the current state of EHR usability; namely, that usability is generally poor but that there is a wide range of EHR usability across systems, specialties, and practice settings.”
Human factors and usability for health information technology: old and new challenges
Carayon, P., & Hoonakker, P. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677907 · Cited by 179 · see citing works
Identifying electronic health record usability and safety challenges in pediatric settings
Ratwani, R. M., Savage, E., Will, A., Fong, A., Karavite, D., Muthu, N., Rivera, A. J., Gibson, C., Asmonga, D., Moscovitch, B., et al. (2018) · Health Affairs · doi:10.1377/hlthaff.2018.0699 · Cited by 163 · see citing works
Self-citationClinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts
Veen, D. V., Uden, C. V., Blankemeier, L., Delbrouck, J.-B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E. P., Seehofnerova, A., et al. (2023) · doi:10.21203/rs.3.rs-3483777/v1 · Cited by 892 · see citing works
“The widespread adoption of electronic health records (EHR) has expanded clinical documentation workload, directly contributing to increasing stress and clinician burnout [26, 32, 63].”
BackgroundThe complex case of EHRs: examining the factors impacting the EHR user experience
Tutty, M. A., Carlasare, L. E., Lloyd, S., & Sinsky, C. A. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz021 · Cited by 94 · see citing works
Moy, A. J., Hobensack, M., Marshall, K., Vawdrey, D. K., Kim, E. Y., Cato, K. D., & Rossetti, S. C. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad038 · Cited by 98 · see citing works
Varma, N., Cygankiewicz, I., Turakhia, M. P., Heidbuchel, H., Hu, Y.-F., Chen, L. Y., Couderc, J.-P., Cronin, E. M., Estep, J. D., Grieten, L., et al. (2021) · Circulation: Arrhythmia and Electrophysiology · doi:10.1161/circep.120.009204 · Cited by 74 · see citing works
Digital Health COVID-19 Impact Assessment: Lessons Learned and Compelling Needs
Microsoft, Lee, P., Abernethy, A., Verily, Shaywitz, D., HealthTech, A., Gundlapalli, A., Prevention, U. S. C. F. D. C. A., Weinstein, J., Healthcare, M., et al. (2022) · NAM Perspectives · doi:10.31478/202201c · Cited by 49 · see citing works
Ray, J. M., Ratwani, R. M., Sinsky, C. A., Frankel, R. M., Friedberg, M. W., Powsner, S. M., Rosenthal, D. I., Wachter, R. M., & Melnick, E. R. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz098 · Cited by 34 · see citing works
Self-citationImproving Electronic Health Record Usability and Safety Requires Transparency
Ratwani, R. M., Hodgkins, M., & Bates, D. W. (2018) · JAMA · doi:10.1001/jama.2018.14079 · Cited by 36 · see citing works
Self-citation
- Authors
- Ratwani, R. M., Savage, E., Will, A., Arnold, R., Khairat, S., Miller, K., Fairbanks, R. J., Hodgkins, M., & Hettinger, A. Z.
- Affiliations
- Georgetown University (US)
- MedStar Health (US)
- Drexel University (US)
- University of North Carolina at Chapel Hill (US)
- American Medical Association (US)
- Corresponding author
- Ratwani, Raj M
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-07-02
- Volume
- 25
- Issue
- 9
- Pages
- 1197-1201
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7646875
- OpenAlex
- W2810283546
- Cited by
- 180
- References
- 9
- Citations by year
- 2026: 29 · 2025: 18 · 2024: 15 · 2023: 21 · 2022: 18 · 2021: 32 · 2020: 19 · 2019: 22 · 2018: 4 · 2012: 1
- FWCI
- 24.28
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- American Medical Association
- MedStar Health
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- electronic health records, usability, patient safety, epic, Cerner, emergency physicians, diagnostic imaging, laboratory test ordering, medication administration, EHR implementation, human-computer interaction
- MeSH terms
- Diagnostic Imaging, Humans, Task Performance and Analysis, User-Computer Interface, Medical Records Systems, Computerized, Medical Errors, Medical Order Entry Systems, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
