CoChat

Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning

Open access
Chen, Bob, Alrifai, Wael, Gao, Cheng, Jones, Barrett, Novak, Laurie, Lorenzi, Nancy, France, Daniel, Malin, Bradley, Chen, You · 2021 · Journal of the American Medical Informatics Association

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Abstract

OBJECTIVE: The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics. MATERIALS AND METHODS: We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit. RESULTS: We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness. CONCLUSIONS: The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamia/ocaa338

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 28 references

Research works printed but not linked: 4

  1. 10. Monsen CB, Singh A, Fellner J, Jackson SL. Measuring provider efficiency in Epic: a preliminary mixed-methods exploration of the provider efficiency profile. J Gen Intern Med 2018; 33: S253.
  2. 20. Satop € a € a V, Albrecht J, Irwin D, Raghavan B. Finding a 'kneedle' in a haystack: Detecting knee points in system behavior. In: Proceedings of the 31st International Conference on Distributed Computing Systems Workshops ; 2011: 166-71.
  3. 21. Ramos J. Using TF-IDF to determine word relevance in document queries. In: Proceedings of the 20th International Conference on Machine Learning ; 2003: 133-42.
  4. 25. Rozinat A, van der Aalst WM. Decision mining in ProM. In: Proceedings of the International Conference on Business Management ; 2006; 4102: 420-5.

Cited by

See all 40

Metadata

Authors
Chen, B., Alrifai, W., Gao, C., Jones, B., Novak, L., Lorenzi, N., France, D., Malin, B., & Chen, Y.
Affiliations
  • Vanderbilt University (US)
  • Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
  • Vanderbilt University Medical Center (US)
Corresponding author
  • Chen, Bob
  • Chen, You
Venue
Journal of the American Medical Informatics Association
Published
2021-02-12
Volume
28
Issue
6
Pages
1168-1177
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W3129040096
Cited by
39
References
28
Citations by year
2026: 8 · 2025: 6 · 2024: 5 · 2023: 4 · 2022: 9 · 2021: 7
FWCI
4.38
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Library of Medicine of the National Institutes of Health (R01LM012854, T32LM012412)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
clinician workload, process mining, neonatal intensive care unit nurses, task completion time, event sequences
MeSH terms
Unsupervised Machine Learning, Humans, Infant, Newborn, Intensive Care Units, Neonatal, Workload, Workflow, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.