CoChat
Sign Up
Cited by 105Electronic Health Records SystemsNursing Diagnosis and DocumentationMedical Coding and Health Information

Studying Workflow and Workarounds in Electronic Health Record–Supported Work to Improve Health System Performance

Open access
Zheng, Kai, Ratwani, Raj M., Adler-Milstein, Julia·2020·Annals of Internal Medicine
DOI 10.7326/m19-0871PMC PMC8061456
Open this paper in CoChat — free account

Abstract

Clinical workflow is the enactment of a series of steps to perform a clinical activity. The transition from paper to electronic health records (EHRs) over the past decade has been characterized by profound challenges supporting clinical workflow, impeding frontline clinicians' ability to deliver safe, efficient, and effective care. In response, there has been substantial effort to study clinical workflow as well as workarounds-exceptions to routine workflow-in order to identify opportunities for improvement. This article describes predominant methods of studying workflow and workarounds and provides examples of the applications of these methods along with the resulting insights. Challenges to studying workflow and workarounds are described, and recommendations for how to approach such studies are given. Although there is not yet a set of standard approaches, this article helps advance workflow research that ultimately serves to inform how to coevolve the design of EHR systems and organizational decisions about processes, roles, and responsibilities in order to support clinical workflow that more consistently delivers on the potential benefits of a digitized health care system.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7326/m19-0871

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing approaches for evaluating the impact of CDS on provider EHR-use activities and clinical workflow heavily rely on manual data collected through surveys, qualitative interviews, direct observation, and audio/video recording.

    “Existing approaches for evaluating the impact of CDS on provider EHR-use activities and clinical workflow heavily rely on manual data collected through surveys, qualitative interviews, direct observation, and audio/video recording.10,13,16–19”

    From Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling (2025) · cites this paper as [19]

    In “Approaches to Studying Workflow and Workarounds” (page unavailable), the paper says qualitative observations, including video ethnography, and interviews are the most commonly used approaches; it also describes observer-led time-motion studies. It does not establish the same pattern specifically for CDS-impact evaluations or identify audio recordings as a common data source, and it describes EHR log analysis as another popular method. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Audit logs capture an ordered sequence of a clinician's activities within the EHR and provide a record of clinical work.

    “Audit logs capture an ordered sequence of a clinician's activities within the EHR and provide a record of clinical work”

    From A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions (2026) · cites this paper as [5]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 34 · Cited by 105

Research works printed but not linked: 34

  1. [1]Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide. Journal of the American Medical Informatics Association. 2017; 24(6): 1142– 8. 29016973 PMC7651985
  2. [2]Jamoom EW, Yang N, Hing E. Adoption of Certified Electronic Health Record Systems and Electronic Information Sharing in Physician Offices: United States, 2013 and 2014. 2016. 26828707
  3. [4]Rosenbaum L Transitional chaos or enduring harm? The EHR and the disruption of medicine. New England Journal of Medicine. 2015; 373(17): 1585– 8. 26488690
  4. [7]Ratwani RM, Savage E, Will A, Fong A, Karavite D, Muthu N, Identifying Electronic Health Record Usability And Safety Challenges In Pediatric Settings. Health affairs 2018; 37(11): 1752– 9. 30395517
  5. [8]Zheng K, Abraham J, Novak L, Reynolds T, Gettinger A. A survey of the literature on unintended consequences associated with health information technology: 2014–2015. Yearbook of medical informatics. 2016; 25(01): 13– 29. PMC5171546 27830227
  6. [9]Bloomrosen M, Starren J, Lorenzi NM, Ash JS, Patel VL, Shortliffe EH. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. Journal of the American Medical Informatics Association. 2011; 18(1): 82– 90. 21169620 PMC3005876
  7. [10]Cain C, Haque S. Organizational workflow and its impact on work quality. 2008. 21328740
  8. [11]National Research Council. Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions. Washington DC; 2009. 20662117
  9. [12]Kobayashi M, Fussell SR, Xiao Y, Seagull FJ. Work coordination, workflow, and workarounds in a medical context. CHI’05 extended abstracts on Human factors in computing systems, 2005. ACM: 1561– 4.
  10. [13]Halbesleben JR, Wakefield DS, Wakefield BJ. Work-arounds in health care settings: literature review and research agenda. Health care management review. 2008; 33(1): 2– 12. 18091439
  11. [14]Ash JS, Gorman PN, Lavelle M, Payne TH, Massaro TA, Frantz GL, A cross-site qualitative study of physician order entry. Journal of the American Medical Informatics Association. 2003; 10(2): 188– 200. 12595408 PMC150372
  12. [15]Powers EM, Shiffman RN, Melnick ER, Hickner A, Sharifi M. Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review. Journal of the American Medical Informatics Association. 2018; 25(11): 1556– 66. 30239810 PMC6915824
  13. [16]Zheng K, Hanauer DA, Padman R, Johnson MP Jr, Hussain AA, Ye W, Handling Anticipated Exceptions in Clinical Care: Investigating the Benefits and Consequences of Providing “Exit Strategies” in Computerized Clinical Information Systems. Journal of the American Medical Informatics Association. 2011; 18: 883– 9. 21676941 PMC3197991
  14. [17]Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review. Journal of the American medical informatics Association. 2009; 16(4): 539– 49. 19390113 PMC2705258
  15. [18]Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results. Health Affairs. 2011; 30(3): 464– 71. 21383365
  16. [20]Zheng K, Haftel HM, Hirschl RB, O’reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. Journal of the American Medical Informatics Association. 2010; 17(4): 454– 61. 20595314 PMC2995654
  17. [21]Niazkhani Z, Pirnejad H, van der Sijs H, Aarts J. Evaluating the medication process in the context of CPOE use: the significance of working around the system. International journal of medical informatics. 2011; 80(7): 490– 506. 21555237
  18. [22]Schwarz N, Oyserman D. Asking questions about behavior: Cognition, communication, and questionnaire construction. The American Journal of Evaluation. 2001; 22(2): 127– 60.
  19. [23]Finkler SA, Knickman JR, Hendrickson G, Lipkin M Jr, Thompson WG. A comparison of work-sampling and time-and-motion techniques for studies in health services research. Health services research. 1993; 28(5): 577. 8270422 PMC1069965
  20. [24]Burke TA, McKee JR, Wilson HC, Donahue RM, Batenhorst AS, Pathak DS. A comparison of time-and-motion and self-reporting methods of work measurement. Journal of Nursing Administration. 2000; 30(3): 118– 25. 10725940
  21. [25]Zheng K, Guo MH, Hanauer DA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. Journal of the american medical informatics Association. 2011; 18(5): 704– 10. 21527407 PMC3168304
  22. [26]Lopetegui M, Yen P-Y, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about? Journal of biomedical informatics. 2014; 49: 292– 9. 24607863 PMC4058370
  23. [27]Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. Journal of the American Medical Informatics Association. 2009; 16(2): 228– 37. 19074301 PMC2649313
  24. [29]Hirsch AG, Jones JB, Lerch VR, Tang X, Berger A, Clark DN, The electronic health record audit file: the patient is waiting. Journal of the American Medical Informatics Association. 2017; 24(e1): e28– e34. 27375293 PMC7651927
  25. [31]Furniss SK, Burton MM, Grando A, Larson DW, Kaufman DR. Integrating Process Mining and Cognitive Analysis to Study EHR Workflow. AMIA Annual Symposium Proceedings, Nov 12–16, 2016. American Medical Informatics Association (Chicago, IL): 580. PMC5333235 28269854
  26. [32]Calvitti A, Hochheiser H, Ashfaq S, Bell K, Chen Y, El Kareh R, Physician activity during outpatient visits and subjective workload. Journal of Biomedical Informatics. 2017; 69: 135– 49. 28323114
  27. [33]Kannampallil T, Li Z, Zhang M, Cohen T, Robinson DJ, Franklin A, Making sense: sensor-based investigation of clinician activities in complex critical care environments. Journal of Biomedical Informatics. 2011; 44(3): 441– 54. 21345380
  28. [34]Zheng K, Hanauer DA, Weibel N, Agha Z. Computational Ethnography: Automated and Unobtrusive Means for Collecting Data In Situ for Human–Computer Interaction Evaluation Studies. Cognitive Informatics for Biomedicine: Springer; 2015: 111– 40.
  29. [35]Ratwani RM, Savage E, Will A, Arnold R, Khairat S, Miller K, A usability and safety analysis of electronic health records: a multi-center study. Journal of the American Medical Informatics Association. 2018; 25(9): 1197– 201. 29982549 PMC7646875
  30. [36]Everson J, Kocher KE, Adler-Milstein J. Health information exchange associated with improved emergency department care through faster accessing of patient information from outside organizations. Journal of the American Medical Informatics Association. 2016. PMC7651934 27521368
  31. [37]Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. Journal of the American Medical Informatics Association. 2010; 17(3): 265– 73. 20442143 PMC2995718
  32. [39]Pizziferri L, Kittler AF, Volk LA, Honour MM, Gupta S, Wang S, Primary care physician time utilization before and after implementation of an electronic health record: A time-motion study. Journal of Biomedical Informatics. 2005; 38(3): 176– 88. 15896691
  33. [40]Lopetegui M, Yen P-Y, Lai AM, Embi PJ, Payne PR. Time Capture Tool (TimeCaT): development of a comprehensive application to support data capture for Time Motion Studies. AMIA Annual Symposium Proceedings, 2012. American Medical Informatics Association: 596. 23304332 PMC3540552
  34. [41]Westbrook JI, Ampt A. Design, application and testing of the Work Observation Method by Activity Timing (WOMBAT) to measure clinicians’ patterns of work and communication. international journal of medical informatics. 2009; 78: S25– S33. 18951838

Authors
Zheng, K., Ratwani, R. M., & Adler-Milstein, J.
Affiliations
  • University of California, Irvine (US)
  • MedStar Health (US)
  • School of Medicine, University of California, San Francisco, San Francisco, California (J.A.)
ORCID
Venue
Annals of Internal Medicine
Publisher
American College of Physicians
Published
2020-06-02
Volume
172
Issue
11_Supplement
Pages
S116-S122
ISSN
  • 0003-4819 (ISSN-L)
  • 1539-3704
DOI
10.7326/m19-0871
PMC
PMC8061456
OpenAlex
W3028609544
Cited by
105
References
34
Citations by year
2026: 26 · 2025: 21 · 2024: 15 · 2023: 15 · 2022: 13 · 2021: 11 · 2020: 4
FWCI
1.64
Type
journal article
Language
en
Open access
Locations
  • Annals of Internal Medicine (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • eScholarship (California Digital Library) (submittedVersion)
Funders
  • National Institutes of Health (UL1TR001414)
  • National Center for Advancing Translational Sciences
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
Keywords
clinical workflow, electronic health records, workarounds, health system performance, EHR design, clinical process improvement
MeSH terms
Delivery of Health Care, Humans, Workflow, Electronic Health Records, Quality Improvement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review