Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics. MATERIALS AND METHODS: We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit. RESULTS: We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness. CONCLUSIONS: The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, much of the past work in HCI, or in studies using audit logs 35-37 have not made meaningful attempts to determine whether human action sequences have inherent underlying sequential structure, similar to a language.”
From A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions (2026) · cites this paper as [36]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Most of these studies have relied on embedding short-run sequences or individual actions within a sequence, without accounting for the temporal progression or dependencies within action sequences.”
From A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions (2026) · cites this paper as [36]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635 · cited 1×
“23 For unsupervised clustering, we used the Leiden community detection algo-rithm 24 to learn session clusters from the low-dimensional representation of the data generated by t-SNE; each detected cluster of sessions is considered a task in this study.”
MethodRuxton, G. D. (2006) · Behavioral Ecology · doi:10.1093/beheco/ark016 · Cited by 1,988
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W.-J., Sinsky, C. A., & Gilchrist, V. J. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2121 · Cited by 972 · cited 2×
“8,11,12 16 One example study leveraged audit logs to identify 15 tasks, including clerical (eg, assigning Current Procedural Terminology and International Classification of Diseases–Tenth Revision codes), medical care (eg, reviewing an encounter note), and inbox tasks (eg, developing a letter to a…”
BackgroundBelkina, A. C., Ciccolella, C. O., Anno, R., Halpert, R., Spidlen, J., & Snyder-Cappione, J. E. (2019) · Nature Communications · doi:10.1038/s41467-019-13055-y · Cited by 625
Face recognition using kernel principal component analysis
Kim, K. I., Jung, K., & Kim, H. J. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.991133 · Cited by 515
Electronic medical records and physician stress in primary care: results from the MEMO Study
Babbott, S., Manwell, L. B., Brown, R., Montague, E., Williams, E., Schwartz, M., Hess, E., & Linzer, M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001875 · Cited by 366
Association of Electronic Health Record Design and Use Factors With Clinician Stress and Burnout
Kroth, P. J., Morioka-Douglas, N., Veres, S., Babbott, S., Poplau, S., Qeadan, F., Parshall, C., Corrigan, K., & Linzer, M. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.9609 · Cited by 333
Overhage, J. M., & McCallie, D. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3684 · Cited by 238
Robertson, S. L., Robinson, M. D., & Reid, A. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-16-00123.1 · Cited by 170
Research works printed but not linked: 4
- 10. Monsen CB, Singh A, Fellner J, Jackson SL. Measuring provider efficiency in Epic: a preliminary mixed-methods exploration of the provider efficiency profile. J Gen Intern Med 2018; 33: S253.
- 20. Satop € a € a V, Albrecht J, Irwin D, Raghavan B. Finding a 'kneedle' in a haystack: Detecting knee points in system behavior. In: Proceedings of the 31st International Conference on Distributed Computing Systems Workshops ; 2011: 166-71.
- 21. Ramos J. Using TF-IDF to determine word relevance in document queries. In: Proceedings of the 20th International Conference on Machine Learning ; 2003: 133-42.
- 25. Rozinat A, van der Aalst WM. Decision mining in ProM. In: Proceedings of the International Conference on Business Management ; 2006; 4102: 420-5.
Cited by
Chen, Y., Lehmann, C. U., & Malin, B. (2024) · Journal of Medical Internet Research · doi:10.2196/60258 · Cited by 111
Self-citationThe use of machine learning and artificial intelligence within pediatric critical care
Shah, N., Arshad, A., Mazer, M. B., Carroll, C. L., Shein, S. L., & Remy, K. E. (2023) · Pediatric Research · doi:10.1038/s41390-022-02380-6 · Cited by 63
Process mining applications in the healthcare domain: A comprehensive review
Guzzo, A., Rullo, A., & Vocaturo, E. (2021) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1442 · Cited by 66
Using electronic health record audit log data for research: insights from early efforts
Kannampallil, T., & Adler-Milstein, J. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac173 · Cited by 49
Rule, A., Melnick, E. R., & Apathy, N. C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac177 · Cited by 58
Moy, A. J., Aaron, L., Cato, K., Schwartz, J., Elias, J. E., Trepp, R., & Rossetti, S. C. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1736625 · Cited by 42
Predicting next-day discharge via electronic health record access logs
Zhang, X., Yan, C., Malin, B. A., Patel, M. B., & Chen, Y. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab211 · Cited by 38
Self-citationKyambade, M., & Namatovu, A. (2025) · Leadership in health services · doi:10.1108/lhs-02-2025-0025 · Cited by 37
Team is brain: leveraging EHR audit log data for new insights into acute care processes
Rose, C. C., Thombley, R. L., Noshad, M., Lu, Y., Clancy, H., Schlessinger, D., Li, R., Liu, V. X., Chen, J. H., & Adler‐Milstein, J. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac201 · Cited by 31
Liu, H., Lou, S. S., Warner, B. C., Harford, D. R., Kannampallil, T., & Lu, C. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539056 · Cited by 28
Yan, C., Zhang, X., Yang, Y., Kang, K., Were, M. C., Embí, P. J., Patel, M. B., Malin, B., Kho, A., & Chen, Y. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.36383 · Cited by 27
Self-citationDagli, M. M., Rajesh, A., Asaad, M., & Butler, C. E. (2021) · The American Surgeon · doi:10.1177/00031348211065101 · Cited by 26
Measuring the cognitive effort associated with task switching in routine EHR-based tasks
Bartek, B., Lou, S. S., & Kannampallil, T. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104349 · Cited by 24
Lou, S. S., Liu, H., Harford, D., Lu, C., & Kannampallil, T. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac239 · Cited by 13
Measuring and Maximizing Undivided Attention in the Context of Electronic Health Records
Chen, Y., Adler‐Milstein, J., & Sinsky, C. A. (2022) · Applied Clinical Informatics · doi:10.1055/a-1892-1437 · Cited by 13
Self-citationEmerging Domains for Measuring Health Care Delivery With Electronic Health Record Metadata
Tawfik, D. S., Rule, A., Alexanian, A., Cross, D. A., Holmgren, A. J., Lou, S. S., Hinz, E. M., Rose, C. C., Viswanadham, R. V. N., Mishuris, R. G., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/64721 · Cited by 12
Chang, H., Liang, S.-X., Li, Z.-Y., Li, R.-N., Zhu, H.-B., He, M., Cui, H., He, J., Li, Y., Wang, J.-R., et al. (2025) · Journal of Translational Medicine · doi:10.1186/s12967-025-06670-5 · Cited by 11
Pachamanova, D. A., Glover, W. J., Li, Z., Docktor, M., & Gujral, N. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab185 · Cited by 11
Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study
Li, P., Chen, B., Rhodes, E., Slagle, J. M., Alrifai, M. W., France, D. J., & Chen, Y. (2021) · JMIR Medical Informatics · doi:10.2196/28998 · Cited by 10
Self-citationZhang, X., Yan, C., Yang, Y., Li, Z., Feng, Y., Malin, B., & Chen, Y. (2024) · medRxiv · doi:10.1101/2024.09.12.24313594 · Cited by 8
Self-citationReimagining clinical AI: from clickstreams to clinical insights with EHR use metadata
Yan, C., Zhang, X., Kannampallil, T., Adler‐Milstein, J., & Chen, Y. (2025) · npj Health Systems · doi:10.1038/s44401-025-00040-5 · Cited by 7
Self-citationArtificial Intelligence in Intelligent Healthcare Systems–Opportunities and Challenges
Petreska, A., & Ristevski, B. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-60840-7_17 · Cited by 6
Zhang, X., Kang, K., Yan, C., Feng, Y., Vandekar, S., Yu, D., Rosenbloom, S. T., Samuels, J. M., Srivastava, G., Williams, D. B., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/56573 · Cited by 6
Self-citationChen, J., Cutrona, S. L., Dharod, A., Moses, A., Bridges, A., Ostasiewski, B., Foley, K. L., Houston, T. K., iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control, Chen, J., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf050 · Cited by 5
Inferring EHR Utilization Workflows through Audit Logs.
Zhang, X., Zhao, Y., Yan, C., Derr, T., & Chen, Y. (2022) · PubMed Central · Cited by 5
Self-citationKim, S., Lou, S. S., Cobb, A. D., Jha, S., & Kannampallil, T. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag163 · Cited by 3
Multi-Perspective Process Mining Interfaces for HL7 AuditEvent Repositories: XES and OCEL
Pointner, A., Krauss, O., Erhard, A., Schuler, A., & Helm, E. (2023) · Studies in health technology and informatics · doi:10.3233/shti230034 · Cited by 3
Progress toward a science of learning systems for healthcare
Henry, S. B. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab104 · Cited by 3
Using Electronic Health Record Metadata to Understand Clinician Work and Behavior
Holmgren, A. J., Steitz, B. D., Lou, S. S., & Apathy, N. C. (2025) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-82971-0_15 · Cited by 2
Santana, C., Mukherjee, C., Quazi, A., Menezes, R., Filkov, V., Sigdel, D., Choi, H., Ebong, I., Sirish, P., Anderson, N., et al. (2026) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2026.111599 · Cited by 1
Kim, S., Lou, S., Cobb, A., Jha, S., & Kannampallil, T. (2026) · medRxiv · doi:10.64898/2026.06.24.26356449
Coombe, T., Burrows, L., Milne‐Ives, M., & Meinert, E. (2023) · doi:10.2196/preprints.49164
Learning from Non-Routine Events and Teamwork in Intensive Care Units: Challenges and Opportunities
Gong, Y., & Chen, Y. (2024) · Studies in health technology and informatics · doi:10.3233/shti230980
Self-citationExploring Task Patterns in EHR Workflows Using Action Sequence Embedding and Graph-Based Analysis
Kim, S., Kannampallil, T., Wick, B. D., Holmgren, A. J., Thombley, R. L., & Lou, S. S. (2025) · Studies in health technology and informatics · doi:10.3233/shti251045
Li, H., Khanna, V. V., Apathy, N. C., Holmgren, A. J., Loza, A. J., & Melnick, E. R. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag018
Yenidoğan Yoğun Bakım Ünitesinde Yapay Zeka Uygulamaları ve Hemşirelik Açısından Değerlendirilmesi
Kutman, G. (2026) · Balkan Sağlık Bilimleri Dergisi · doi:10.61830/balkansbd.1844500
Lu, S., Zhu, Y., Lu, Z., Wu, C., Pan, G., & Wang, S. (2026) · Research Square · doi:10.21203/rs.3.rs-9202808/v1
Morales, D., Gutierrez, A., Navarro, L., & Ríos, P. (2026) · Journal of Health Informatics and Digital Systems · doi:10.68159/y753802699
Zhou, X., Liu, L., Husain, Z., Zhou, X., Lakha, S. F., & Montague, E. N. H. (2026) · doi:10.2196/preprints.114892
MEDILIVE: A Platform for Patient Log Tracking and Healthcare Analysis
Manigandan, S. K., Kannan, B., Krishna Kumar, E., Madhavan, M., Malleswari, M., & Ramya, D. J. (2026) · International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI) · doi:10.1109/icoici69824.2026.11723609
Metadata
- Authors
- Chen, B., Alrifai, W., Gao, C., Jones, B., Novak, L., Lorenzi, N., France, D., Malin, B., & Chen, Y.
- Affiliations
- Vanderbilt University (US)
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
- Vanderbilt University Medical Center (US)
- Corresponding author
- Chen, Bob
- Chen, You
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-02-12
- Volume
- 28
- Issue
- 6
- Pages
- 1168-1177
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC8200270
- OpenAlex
- W3129040096
- Cited by
- 39
- References
- 28
- Citations by year
- 2026: 8 · 2025: 6 · 2024: 5 · 2023: 4 · 2022: 9 · 2021: 7
- FWCI
- 4.38
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Library of Medicine of the National Institutes of Health (R01LM012854, T32LM012412)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- clinician workload, process mining, neonatal intensive care unit nurses, task completion time, event sequences
- MeSH terms
- Unsupervised Machine Learning, Humans, Infant, Newborn, Intensive Care Units, Neonatal, Workload, Workflow, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.