Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers’ perceived control over their workload in the EHR
Open accessThe DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
Objective: To understand if providers who had additional electronic health record (EHR) training improved their satisfaction, decreased personal EHR-use time, and decreased turnaround time on tasks. Materials and Methods: This pre-post study with no controls evaluated the impact of a supplemental EHR training program on a group of academic and community practice clinicians that previously had go-live group EHR training and 20 months experience using this EHR on self-reported data, calculated EHR time, and vendor-reported metrics. Results: Providers self-reported significant improvements in their knowledge of efficiency tools in the EHR after training and doubled (significant) their preference list entries (mean pre = 38.1 [65.88], post = 63.5 [90.47], P < .01). Of the 7 EHR satisfaction variables, only 1 self-reported variable significantly improved after training: Control over my workload in the EHR (mean pre = 2.7 [0.96], post = 3.0 [1.04], P < .01). There was no significant decrease in their calculated EHR usage outside of clinic (mean pre = 0.39 [0.77] to post = 0.37 [0.48], P = .73). No significant difference was seen in turnaround time for patient calls (mean pre = 2.3 [2.06] days, post = 1.9 [1.76] days, P = .08) and results (mean before = 4.0 [2.79] days, after = 3.2 [2.33] days, P = .03). Discussion: Multiple sources of data provide a holistic view of the provider experience in the EHR. This study suggests that individualized EHR training can improve the knowledge of EHR tools and satisfaction with their perceived control of EHR workload, however this did not translate into less Clinician Logged-In Outside Clinic (CLOC) time, a calculated metric, nor quicker turnaround on in box tasks. CLOC time emerged as a potential less-costly surrogate metric for provider satisfaction in EHR work than surveying clinicians. Further study is required to understand the cost-benefit of various interventions to decrease CLOC time. Conclusions: This supplemental EHR training session, 20 months post go-live, where most participants elected to receive 2 or fewer sessions did significantly improve provider satisfaction with perceived control over their workload in the EHR, but it was not effective in decreasing EHR-use time outside of clinic. CLOC time, a calculated metric, could be a practical trackable surrogate for provider satisfaction (inverse correlation) with after-hours time spent in the EHR. Further study into interventions that decrease CLOC time and improve turnaround time to respond to inbox tasks are suggested next steps.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
From Triple to Quadruple Aim: Care of the Patient Requires Care of the Provider
Bodenheimer, T., & Sinsky, C. (2014) · The Annals of Family Medicine · doi:10.1370/afm.1713 · Cited by 3,462
Shanafelt, T. D., Boone, S., Tan, L., Dyrbye, L. N., Sotile, W., Satele, D., West, C. P., Sloan, J., & Oreskovich, M. R. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.3199 · Cited by 3,439
Shanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J., & West, C. P. (2015) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2015.08.023 · Cited by 2,885
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494 · cited 3×
“Future studies should also evaluate how much training on EHR tool usage can significantly impact efficiency in the inbox work, as primary care clinicians are now spending up to 50% of their time on these tasks (responding to patient messages, prescription refills, and results review).(12,20,27,31,34) For example, in o…”
BackgroundResultBuntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011) · Health Affairs · doi:10.1377/hlthaff.2011.0178 · Cited by 1,311
Shanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P. (2016) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2016.05.007 · Cited by 1,086
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W.-J., Sinsky, C. A., & Gilchrist, V. J. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2121 · Cited by 972 · cited 3×
“Future studies should also evaluate how much training on EHR tool usage can significantly impact efficiency in the inbox work, as primary care clinicians are now spending up to 50% of their time on these tasks (responding to patient messages, prescription refills, and results review).(12,20,27,31,34) For example, in o…”
BackgroundResultFriedberg, M., Chen, P., Van Busum, K., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D., Brook, R., et al. (2014) · RAND Health Quarterly · doi:10.7249/rhq-03-04-01 · Cited by 707
Addressing Physician Burnout: The Way Forward
Shanafelt, T. D., Dyrbye, L. N., & West, C. P. (2017) · JAMA · doi:10.1001/jama.2017.0076 · Cited by 447
Physician Burnout in the Electronic Health Record Era: Are We Ignoring the Real Cause?
Downing, N. L., Bates, D. W., & Longhurst, C. A. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0139 · Cited by 418
Research works printed but not linked: 2
- Nechyporenko T, McKibbon A. Review of different training approaches to facilitate successful adoption of EMR systems by physicians in hospital settings. MeRC (McMaster eBusiness Research Centre) Working Paper Series; 2017 http://hdl.handle.net/11375/20846 Accessed February 18, 2019
- Babbott S, Manwell L, Brown R, et al Electronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inf Assoc 2014; 21 (e1): e100– 6. PMC3957395 24005796 · DOI · Paper page
Cited by
Adler-Milstein, J., Zhao, W., Willard-Grace, R., Knox, M., & Grumbach, K. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz220 · Cited by 356
Moy, A. J., Schwartz, J. M., Chen, R., Sadri, S., Lucas, E., Cato, K. D., & Rossetti, S. C. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa325 · Cited by 284
“Likewise, DiAngi et al proposed the “calculated EHR time outside of clinic” (CLOC) metric for ambulatory settings to measure after clinic hours using EHR usage logs and were able to correlate their findings with selfreported time spent in the EHR after clinic hours.(22) The HHS Report—Strategy on Reducing Burden Relat…”
BackgroundMethodInfluentialMetrics for assessing physician activity using electronic health record log data
Sinsky, C. A., Rule, A., Cohen, G., Arndt, B. G., Shanafelt, T. D., Sharp, C. D., Baxter, S. L., Tai-Seale, M., Yan, S., Chen, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz223 · Cited by 162
“While EHR log data has shown promise in measuring time spent on these clinical activities, the use of EHR log data to further understand the clinical environment is a nascent science.(27,28) All certified EHRs in the United States are required to maintain EHR logs, tracking who accesses which patient records at what t…”
MethodMeasuring Documentation Burden in Healthcare
Murad, M. H., Stelling, B. E. V., West, C. P., Hasan, B., Simha, S., Saadi, S. M., Firwana, M., Viola, K. E., Prokop, L. J., Nayfeh, T., et al. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-08956-8 · Cited by 90
Jose, A., Tortorella, G. L., Vassolo, R. S., Kumar, M., & Cawley, A. F. M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010478 · Cited by 85
Tawfik, D. S., Sinha, A., Bayati, M., Adair, K. C., Shanafelt, T. D., Sexton, J. B., & Profit, J. (2021) · Journal of Medical Internet Research · doi:10.2196/26817 · Cited by 78
Virtanen, L., Kaihlanen, A., Laukka, E., Gluschkoff, K., & Heponiemi, T. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104432 · Cited by 72
Documentation dynamics: note composition, burden, and physician efficiency
Apathy, N. C., Rotenstein, L., Bates, D. W., & Holmgren, A. J. (2023) · Health Services Research · doi:10.1111/1475-6773.14097 · Cited by 70
Predicting physician burnout using clinical activity logs: Model performance and lessons learned
Lou, S. S., Liu, H., Warner, B. C., Harford, D., Lu, C., & Kannampallil, T. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104015 · Cited by 49
Using electronic health record audit log data for research: insights from early efforts
Kannampallil, T., & Adler-Milstein, J. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac173 · Cited by 49
Rule, A., Melnick, E. R., & Apathy, N. C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac177 · Cited by 58
Health professionals’ perceptions of electronic health records system: a mixed method study in Ghana
Mensah, N. K., Adzakpah, G., Kissi, J., Abdulai, K., Taylor-Abdulai, H. B., Johnson, S. B., Opoku, C., Hallo, C., & Boadu, R. O. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02672-3 · Cited by 58
Nguyen, O. T., Shah, S., Gartland, A., Parekh, A., Turner, K. L., Feldman, S. S., & Merlo, L. J. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa289 · Cited by 52
Nurse workarounds in the electronic health record: An integrative review
Fraczkowski, D., Matson, J., & Lopez, K. D. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa050 · Cited by 51
Temporal Associations Between EHR-Derived Workload, Burnout, and Errors: a Prospective Cohort Study
Lou, S. S., Lew, D., Harford, D. R., Lu, C., Evanoff, B. A., Duncan, J. G., & Kannampallil, T. (2022) · Journal of General Internal Medicine · doi:10.1007/s11606-022-07620-3 · Cited by 46
Interventions to Reduce Electronic Health Record-Related Burnout: A Systematic Review
Kang, C., & Sarkar, I. N. (2023) · Applied Clinical Informatics · doi:10.1055/a-2203-3787 · Cited by 41
Ferry, A. M., Davis, M. J., Rumprecht, E., Nigro, A., Desai, P. K., & Hollier, L. H. (2021) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000003651 · Cited by 36
Physician Burnout and Timing of Electronic Health Record Use
Micek, M. A., Arndt, B., Tuan, W.-J., Trowbridge, E., Dean, S. M., Lochner, J., Sampene, E., & Pandhi, N. (2020) · ACI Open · doi:10.1055/s-0039-3401815 · Cited by 28
Humphrey-Murto, S. M., Makus, D., Moore, S., Duffy, K. W., Maniate, J. M., Scowcroft, K., Buba, M., & Rangel, J. C. (2022) · Medical Education · doi:10.1111/medu.14944 · Cited by 32
Kannampallil, T., Abraham, J., Lou, S. S., & Payne, P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa305 · Cited by 30
Measuring Electronic Health Record Use in the Pediatric ICU Using Audit-Logs and Screen Recordings
Sinha, A., Stevens, L. A., Su, F., Pageler, N. M., & Tawfik, D. S. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1733851 · Cited by 26
Self-citationMusa, S., Dergaa, I., Yasin, R. A. S., & Singh, R. (2023) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s414200 · Cited by 25
Esmaeilzadeh, P., & Mirzaei, T. (2021) · JMIR Medical Informatics · doi:10.2196/28497 · Cited by 25
The role of organizations in shaping physician use of electronic health records
Cross, D. A., Holmgren, A. J., & Apathy, N. C. (2023) · Health Services Research · doi:10.1111/1475-6773.14203 · Cited by 22
English, E. F., Holmstrom, H., Kwan, B. W., Suresh, K., Rotholz, S., Lin, C., & Sieja, A. (2022) · Applied Clinical Informatics · doi:10.1055/s-0041-1740482 · Cited by 19
Quantified electronic health record (EHR) use by academic surgeons
Cox, M. L., Risoli, T., Peskoe, S. B., Turner, D. A., & Migaly, J. (2021) · Surgery · doi:10.1016/j.surg.2020.12.009 · Cited by 18
Khairat, S., Zalla, L. C., Gartland-Gray, A., & Seashore, C. J. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.689646 · Cited by 18
Sieja, A., Whittington, M. D., Patterson, V., Markley, K., Holmstrom, H., Rotholz, S., Gonzalez, C., Carpenter, M. S., & Lin, C. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab073 · Cited by 16
Smith, D. C., Thumm, E. B., Anderson, J., Kissler, K., Reed, S. M., Centi, S. M., Staley, A. W., Hernandez, T. L., & Barton, A. J. (2024) · Journal of Midwifery & Women's Health · doi:10.1111/jmwh.13601 · Cited by 15
Impact of Changes in EHR Use during COVID-19 on Physician Trainee Mental Health
Holzer, K. J., Lou, S. S., Goss, C. W., Strickland, J. R., Evanoff, B., Duncan, J. G., & Kannampallil, T. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1731000 · Cited by 15
Sinha, A., Shanafelt, T. D., Trockel, M., Wang, H., & Sharp, C. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1731678 · Cited by 14
Diaz‐Garelli, F., Strowd, R. E., III, Ahmed, T., Lycan, T. W., Daley, S., Wells, B. J., & Topaloĝlu, Ü. (2021) · JCO Clinical Cancer Informatics · doi:10.1200/cci.20.00174 · Cited by 11
Kim, S., Warner, B. C., Lew, D., Lou, S. S., & Kannampallil, T. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae171 · Cited by 8
Lynch, K. A., Baron, S. W., Rikin, S., Kanevsky, J., Kelly, C. B., Carrozzi, G., Wey, G., & Yang, K. (2023) · Quality Management in Health Care · doi:10.1097/qmh.0000000000000417 · Cited by 7
Lourie, E. M., Stevens, L. A., & Webber, E. C. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa056 · Cited by 7
Self-citationAlsolamy, M. Q. (2021) · International Journal of Customer Relationship Marketing and Management · doi:10.4018/ijcrmm.2021040103 · Cited by 6
Fong, A., Iscoe, M. S., Sinsky, C. A., Haimovich, A. D., Williams, B., O’Connell, R. T., Goldstein, R. L., & Melnick, E. R. (2022) · JMIR Medical Informatics · doi:10.2196/34954 · Cited by 6
Graph theoretic visualization of patient and health worker messaging in the EHR
Haq, M. Z. U., Hornback, A. M., Harzand, A., Gutman, D. A., Gallaher, B., Schoenberg, E. D., Zhu, Y., Wang, M. D., & Anderson, B. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1422208 · Cited by 5
Chen, J., Chi, W. N., Ravichandran, U., Solomonides, A. E., Trimark, J., Patel, S., McNulty, B., Shah, N. S., & Brown, S. (2024) · Applied Clinical Informatics · doi:10.1055/s-0044-1786368 · Cited by 4
Secure messaging telehealth billing in the digital age: moving beyond time-based metrics
Ko, D., Tachinardi, U., & Warm, E. J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae250 · Cited by 4
Sega, A., Bossan, A., Abrams, M. P., Cendan, V., Gartland, A., Nguyen, D., & Simms-Cendan, J. S. (2021) · Journal of Student-Run Clinics · doi:10.59586/jsrc.v7i1.228 · Cited by 4
Boggs, Z. H., Frazier, H., & Kon, R. H. (2025) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-24-00496.1 · Cited by 3
Sharp, C., & Stevens, L. A. (2019) · Springer eBooks · doi:10.1007/978-3-319-42135-3_6 · Cited by 3
Self-citationImproving Health Informatics Competencies Using an Open-source Educational EHR
Purkayastha, S., Fulton, C. R., & Ratna, S. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00112 · Cited by 2
Towards Lean EHR Usability Heuristics for Behavioral Health Providers
Boehler, J. (2020) · ResearchWorks at the University of Washington (University of Washington) · Cited by 2
Taking a Team Approach: Keep Up with the EHR with a Training and Optimization Program
McEntee, R., Hitt, J. R., & Sieja, A. (2024) · Applied Clinical Informatics · doi:10.1055/a-2437-0185 · Cited by 1
Howland, M., Lockhart, Z., & Jariwala, S. P. (2026) · Applied Clinical Informatics · doi:10.1055/a-2882-9924 · Cited by 1
The Impact of Electronic Health Records (EHRs) on Healthcare Accounting and Financial Management
Shekar Miryala, D. S. G., Gupta, S., Garg, M., Fatima, H., & Sharma, S. (2024) · Frontiers in Health Informatics · Cited by 1
Physicians’ Perceptions of Facilitators and Barriers in Electronic Health Record Education
Martin, A. (2020) · ScholarWorks (Walden University)
Effectiveness of Electronic Documentation: A Case Report
Raymond, C., Zapf, S. A., & Hudgins, E. (2020) · The Open Journal of Occupational Therapy · doi:10.15453/2168-6408.1722
Ioerger, P., & Smith, T. R. (2024) · Kansas Journal of Medicine · doi:10.17161/kjm.vol17.21681
Aryeetey, E. O., & Garg, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-19981-2_11
Quantifying and Visualizing Emergency Physician Workflow: Observational Time-Motion Study
Henreid, A. J., Ioannides, K. L. H., Pevnick, J. M., Cohen, T., Torbati, S. S., Nuckols, T. K., & Berdahl, C. T. (2026) · JMIR Medical Informatics · doi:10.2196/85983
Boggs, Z. H., Frazier, H., Martindale, J. R., & Kon, R. H. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-026-10720-z
Assessing the Impact of E-Health on Healthcare Provider Workload and Job Satisfaction in Ghana
Aboagye, S., & Kusi, A. Y. (2026) · Ghana Journal of Nursing and Midwifery. · doi:10.69600/gjnmid/2026.v3i3.26-49
Olomi, G. A., Manongi, R. N., Yeates, K., Makasi, C. E., Katiti, V. W., Woodworth, S., Khashan, A. S., & Mmbaga, B. T. (2026) · JMIR Formative Research · doi:10.2196/90400
Metadata
- Authors
- DiAngi, Y. T., Stevens, L. A., Halpern – Felsher, B., Pageler, N. M., & Lee, T. C.
- Affiliations
- Sutter Health (US)
- Stanford Health Care (US)
- Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA
- Corresponding author
- DiAngi, Yumi T
- Venue
- JAMIA Open
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-03-21
- Volume
- 2
- Issue
- 2
- Pages
- 222-230
- ISSN
- 2574-2531 (ISSN-L)
- PMC
- PMC6952029
- OpenAlex
- W2954996172
- Cited by
- 52
- References
- 33
- Citations by year
- 2026: 4 · 2025: 1 · 2024: 8 · 2023: 6 · 2022: 9 · 2021: 13 · 2020: 8 · 2019: 3
- FWCI
- 8.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- JAMIA Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Keywords
- electronic health records, EHR training, provider satisfaction, turnaround time, provider workload, EHR adoption
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.