Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics. MATERIALS AND METHODS: We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit. RESULTS: We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness. CONCLUSIONS: The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, much of the past work in HCI, or in studies using audit logs 35-37 have not made meaningful attempts to determine whether human action sequences have inherent underlying sequential structure, similar to a language.”
From A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions (2026) · cites this paper as [36]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Most of these studies have relied on embedding short-run sequences or individual actions within a sequence, without accounting for the temporal progression or dependencies within action sequences.”
From A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions (2026) · cites this paper as [36]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635 · cited 1×
“23 For unsupervised clustering, we used the Leiden community detection algo-rithm 24 to learn session clusters from the low-dimensional representation of the data generated by t-SNE; each detected cluster of sessions is considered a task in this study.”
MethodRuxton, G. D. (2006) · Behavioral Ecology · doi:10.1093/beheco/ark016 · Cited by 1,988
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W.-J., Sinsky, C. A., & Gilchrist, V. J. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2121 · Cited by 972 · cited 2×
“8,11,12 16 One example study leveraged audit logs to identify 15 tasks, including clerical (eg, assigning Current Procedural Terminology and International Classification of Diseases–Tenth Revision codes), medical care (eg, reviewing an encounter note), and inbox tasks (eg, developing a letter to a…”
BackgroundBelkina, A. C., Ciccolella, C. O., Anno, R., Halpert, R., Spidlen, J., & Snyder-Cappione, J. E. (2019) · Nature Communications · doi:10.1038/s41467-019-13055-y · Cited by 625
Face recognition using kernel principal component analysis
Kim, K. I., Jung, K., & Kim, H. J. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.991133 · Cited by 515
Electronic medical records and physician stress in primary care: results from the MEMO Study
Babbott, S., Manwell, L. B., Brown, R., Montague, E., Williams, E., Schwartz, M., Hess, E., & Linzer, M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001875 · Cited by 366
Association of Electronic Health Record Design and Use Factors With Clinician Stress and Burnout
Kroth, P. J., Morioka-Douglas, N., Veres, S., Babbott, S., Poplau, S., Qeadan, F., Parshall, C., Corrigan, K., & Linzer, M. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.9609 · Cited by 333
Overhage, J. M., & McCallie, D. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3684 · Cited by 238
Robertson, S. L., Robinson, M. D., & Reid, A. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-16-00123.1 · Cited by 170
Research works printed but not linked: 1
- Monsen, 2018, Measuring provider efficiency in Epic: a preliminary mixed-methods exploration of the provider efficiency profile, J Gen Intern Med, 33, S253
Cited by
Chen, Y., Lehmann, C. U., & Malin, B. (2024) · Journal of Medical Internet Research · doi:10.2196/60258 · Cited by 110
Self-citationThe use of machine learning and artificial intelligence within pediatric critical care
Shah, N. A., Arshad, A., Mazer, M. B., Carroll, C. L., Shein, S. L., & Remy, K. E. (2022) · Pediatric Research · doi:10.1038/s41390-022-02380-6 · Cited by 76
Process mining applications in the healthcare domain: A comprehensive review
Guzzo, A., Rullo, A., & Vocaturo, E. (2021) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1442 · Cited by 66
Using electronic health record audit log data for research: insights from early efforts
Kannampallil, T., & Adler‐Milstein, J. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac173 · Cited by 59
Rule, A., Melnick, E. R., & Apathy, N. C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac177 · Cited by 58
Metadata
- Authors
- Chen, B., Alrifai, W., Gao, C., Jones, B., Novak, L., Lorenzi, N., France, D., Malin, B., & Chen, Y.
- Affiliations
- Vanderbilt University (US)
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
- Vanderbilt University Medical Center (US)
- Corresponding author
- Chen, Bob
- Chen, You
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-02-12
- Volume
- 28
- Issue
- 6
- Pages
- 1168-1177
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC8200270
- OpenAlex
- W3129040096
- Cited by
- 39
- References
- 28
- Citations by year
- 2026: 8 · 2025: 6 · 2024: 5 · 2023: 4 · 2022: 9 · 2021: 7
- FWCI
- 4.38
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Library of Medicine of the National Institutes of Health (R01LM012854, T32LM012412)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- clinician workload, process mining, neonatal intensive care unit nurses, task completion time, event sequences
- MeSH terms
- Unsupervised Machine Learning, Humans, Infant, Newborn, Intensive Care Units, Neonatal, Workload, Workflow, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning. https://app.cochat.ai/papers/10.1093/jamia/ocaa338
- The paper
- Chen, B., Alrifai, W., Gao, C., Jones, B., Novak, L., Lorenzi, N., France, D., Malin, B., & Chen, Y. (2021). Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaa338
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.