Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers’ perceived control over their workload in the EHR
Open accessThe DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
Objective: To understand if providers who had additional electronic health record (EHR) training improved their satisfaction, decreased personal EHR-use time, and decreased turnaround time on tasks. Materials and Methods: This pre-post study with no controls evaluated the impact of a supplemental EHR training program on a group of academic and community practice clinicians that previously had go-live group EHR training and 20 months experience using this EHR on self-reported data, calculated EHR time, and vendor-reported metrics. Results: Providers self-reported significant improvements in their knowledge of efficiency tools in the EHR after training and doubled (significant) their preference list entries (mean pre = 38.1 [65.88], post = 63.5 [90.47], P < .01). Of the 7 EHR satisfaction variables, only 1 self-reported variable significantly improved after training: Control over my workload in the EHR (mean pre = 2.7 [0.96], post = 3.0 [1.04], P < .01). There was no significant decrease in their calculated EHR usage outside of clinic (mean pre = 0.39 [0.77] to post = 0.37 [0.48], P = .73). No significant difference was seen in turnaround time for patient calls (mean pre = 2.3 [2.06] days, post = 1.9 [1.76] days, P = .08) and results (mean before = 4.0 [2.79] days, after = 3.2 [2.33] days, P = .03). Discussion: Multiple sources of data provide a holistic view of the provider experience in the EHR. This study suggests that individualized EHR training can improve the knowledge of EHR tools and satisfaction with their perceived control of EHR workload, however this did not translate into less Clinician Logged-In Outside Clinic (CLOC) time, a calculated metric, nor quicker turnaround on in box tasks. CLOC time emerged as a potential less-costly surrogate metric for provider satisfaction in EHR work than surveying clinicians. Further study is required to understand the cost-benefit of various interventions to decrease CLOC time. Conclusions: This supplemental EHR training session, 20 months post go-live, where most participants elected to receive 2 or fewer sessions did significantly improve provider satisfaction with perceived control over their workload in the EHR, but it was not effective in decreasing EHR-use time outside of clinic. CLOC time, a calculated metric, could be a practical trackable surrogate for provider satisfaction (inverse correlation) with after-hours time spent in the EHR. Further study into interventions that decrease CLOC time and improve turnaround time to respond to inbox tasks are suggested next steps.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
From Triple to Quadruple Aim: Care of the Patient Requires Care of the Provider
Bodenheimer, T., & Sinsky, C. (2014) · The Annals of Family Medicine · doi:10.1370/afm.1713 · Cited by 3,462
Shanafelt, T. D., Boone, S., Tan, L., Dyrbye, L. N., Sotile, W., Satele, D., West, C. P., Sloan, J., & Oreskovich, M. R. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.3199 · Cited by 3,439
Shanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J., & West, C. P. (2015) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2015.08.023 · Cited by 2,885
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494 · cited 3×
“Future studies should also evaluate how much training on EHR tool usage can significantly impact efficiency in the inbox work, as primary care clinicians are now spending up to 50% of their time on these tasks (responding to patient messages, prescription refills, and results review).(12,20,27,31,34) For example, in o…”
BackgroundResultBuntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011) · Health Affairs · doi:10.1377/hlthaff.2011.0178 · Cited by 1,311
Shanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P. (2016) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2016.05.007 · Cited by 1,086
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W.-J., Sinsky, C. A., & Gilchrist, V. J. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2121 · Cited by 972 · cited 3×
“Future studies should also evaluate how much training on EHR tool usage can significantly impact efficiency in the inbox work, as primary care clinicians are now spending up to 50% of their time on these tasks (responding to patient messages, prescription refills, and results review).(12,20,27,31,34) For example, in o…”
BackgroundResultFriedberg, M., Chen, P., Van Busum, K., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D., Brook, R., et al. (2014) · RAND Health Quarterly · doi:10.7249/rhq-03-04-01 · Cited by 707
Addressing Physician Burnout: The Way Forward
Shanafelt, T. D., Dyrbye, L. N., & West, C. P. (2017) · JAMA · doi:10.1001/jama.2017.0076 · Cited by 447
Physician Burnout in the Electronic Health Record Era: Are We Ignoring the Real Cause?
Downing, N. L., Bates, D. W., & Longhurst, C. A. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0139 · Cited by 418
In Search of Joy in Practice: A Report of 23 High-Functioning Primary Care Practices
Sinsky, C. A., Willard-Grace, R., Schutzbank, A. M., Sinsky, T. A., Margolius, D., & Bodenheimer, T. (2013) · The Annals of Family Medicine · doi:10.1370/afm.1531 · Cited by 415
Tai-Seale, M., Olson, C. W., Li, J., Chan, A. S., Morikawa, C., Durbin, M., Wang, W., & Luft, H. S. (2017) · Health Affairs · doi:10.1377/hlthaff.2016.0811 · Cited by 379 · cited 2×
“Future studies should also evaluate how much training on EHR tool usage can significantly impact efficiency in the inbox work, as primary care clinicians are now spending up to 50% of their time on these tasks (responding to patient messages, prescription refills, and results review).(12,20,27,31,34) For example, in o…”
BackgroundResultElectronic medical records and physician stress in primary care: results from the MEMO Study
Babbott, S., Manwell, L. B., Brown, R., Montague, E., Williams, E., Schwartz, M., Hess, E., & Linzer, M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001875 · Cited by 366
Clinical Information Technologies and Inpatient Outcomes: A Multiple Hospital Study
Amarasingham, R., Plantinga, L., Diener-West, M., Gaskin, D. J., & Powe, N. R. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2008.520 · Cited by 354
Wolfstadt, J. I., Gurwitz, J. H., Field, T. S., Lee, M., Kalkar, S., Wu, W., & Rochon, P. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0504-5 · Cited by 235
Factors Affecting Physician Professional Satisfaction
Friedberg, M., Van Busum, K., Chen, P., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D., Brook, R., et al. (2013) · RAND Corporation eBooks · doi:10.7249/rb9740 · Cited by 216
Health Care Providers' Comfort With and Barriers to Care of Transgender Youth
Vance, S. R., Halpern-Felsher, B. L., & Rosenthal, S. M. (2015) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2014.11.002 · Cited by 188
Self-citationRobertson, S. L., Robinson, M. D., & Reid, A. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-16-00123.1 · Cited by 170 · cited 2×
“Our study also showed correlations between EHR experience and measured EHR time, which is in line with previous work demonstrating that providers spending more than 6 h per week in the EHR after clinic hours were 3 times more likely to report burnout.(7)”
BackgroundResultA Time-Motion Study of Primary Care Physicians’ Work in the Electronic Health Record Era
Young, R. A., Burge, S. K., Kumar, K. A., Wilson, J. M., & Ortiz, D. F. (2018) · Family Medicine · doi:10.22454/fammed.2018.184803 · Cited by 159
Meaningful Use of Electronic Health Records: Experiences From the Field and Future Opportunities
Slight, S. P., Berner, E. S., Galanter, W., Huff, S., Lambert, B. L., Lannon, C., Lehmann, C. U., McCourt, B. J., McNamara, M., Menachemi, N., et al. (2015) · JMIR Medical Informatics · doi:10.2196/medinform.4457 · Cited by 68
Secondary use of electronic health record data for clinical workflow analysis
Hribar, M. R., Read-Brown, S., Goldstein, I. H., Reznick, L. G., Lombardi, L., Parikh, M., Chamberlain, W., & Chiang, M. F. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx098 · Cited by 64
Training providers: beyond the basics of electronic health records
Bredfeldt, C. E., Awad, E. B., Joseph, K., & Snyder, M. H. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-503 · Cited by 58
Dastagir, M. T., Chin, H. L., McNamara, M., Poteraj, K., Battaglini, S., & Alstot, L. (2012) · PubMed Central · Cited by 52
Pérez, E. R. F., Winters, J. L., & Gajic, O. (2007) · American Journal of Hematology · doi:10.1002/ajh.20888 · Cited by 49
Successful Physician Training Program for Large Scale EMR Implementation
Stevens, L. A., Mailes, E. S., Goad, B. A., Longhurst, C. A., & Pantaleoni, J. L. (2015) · Applied Clinical Informatics · doi:10.4338/aci-2014-09-cr-0076 · Cited by 47
Self-citationPollak, V. E., & Lorch, J. A. (2007) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-7-38 · Cited by 40
Kirshner, M. C., Salomon, H., & Chin, H. L. (2004) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2003.11.001 · Cited by 29
Designing An Individualized EHR Learning Plan For Providers
DiAngi, Y. T., Schremp, J. D., Martorana, M. J., Miller, R. E., Lee, T. C., Pageler, N. M., & Stevens, L. A. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2017-04-0054 · Cited by 29
Self-citationContraception for Adolescents and Young Adults in the Inpatient Setting: The Providers’ Perspective
Goldstein, R. L., Carlson, J. L., & Halpern‐Felsher, B. L. (2018) · Hospital Pediatrics · doi:10.1542/hpeds.2017-0167 · Cited by 26
Self-citationPatient–Provider Communication: Does Electronic Messaging Reduce Incoming Telephone Calls?
Dexter, E. N., Fields, S., Rdesinski, R. E., Sachdeva, B., Yamashita, D., & Marino, M. (2016) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2016.05.150371 · Cited by 23
Novel Metrics for Improving Professional Fulfillment
DiAngi, Y. T., Lee, T. C., Sinsky, C. A., Bohman, B. D., & Sharp, C. D. (2017) · Annals of Internal Medicine · doi:10.7326/m17-0658 · Cited by 19
Self-citationThe Value of Clinical Teachers for EMR Implementations and Conversions
Pantaleoni, J. L., Longhurst, C., & Stevens, L. A. (2015) · Applied Clinical Informatics · doi:10.4338/aci-2014-09-ie-0075 · Cited by 15
Self-citation
Research works printed but not linked: 1
- Babbott, 2014, Electronic medical records and physician stress in primary care: results from the MEMO Study, J Am Med Inf Assoc, 21, e100, 10.1136/amiajnl-2013-001875 · DOI · Paper page
Cited by
Adler-Milstein, J., Zhao, W., Willard-Grace, R., Knox, M., & Grumbach, K. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz220 · Cited by 356
Moy, A. J., Schwartz, J. M., Chen, R., Sadri, S., Lucas, E., Cato, K. D., & Rossetti, S. C. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa325 · Cited by 284
“Likewise, DiAngi et al proposed the “calculated EHR time outside of clinic” (CLOC) metric for ambulatory settings to measure after clinic hours using EHR usage logs and were able to correlate their findings with selfreported time spent in the EHR after clinic hours.(22) The HHS Report—Strategy on Reducing Burden Relat…”
BackgroundMethodInfluentialMetrics for assessing physician activity using electronic health record log data
Sinsky, C. A., Rule, A., Cohen, G., Arndt, B. G., Shanafelt, T. D., Sharp, C. D., Baxter, S. L., Tai-Seale, M., Yan, S., Chen, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz223 · Cited by 162
“While EHR log data has shown promise in measuring time spent on these clinical activities, the use of EHR log data to further understand the clinical environment is a nascent science.(27,28) All certified EHRs in the United States are required to maintain EHR logs, tracking who accesses which patient records at what t…”
MethodMeasuring Documentation Burden in Healthcare
Murad, M. H., Stelling, B. E. V., West, C. P., Hasan, B., Simha, S., Saadi, S. M., Firwana, M., Viola, K. E., Prokop, L. J., Nayfeh, T., et al. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-08956-8 · Cited by 89
Jose, A., Tortorella, G. L., Vassolo, R. S., Kumar, M., & Cawley, A. F. M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010478 · Cited by 85
Metadata
- Authors
- DiAngi, Y. T., Stevens, L. A., Halpern – Felsher, B., Pageler, N. M., & Lee, T. C.
- Affiliations
- Sutter Health (US)
- Stanford Health Care (US)
- Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA
- Corresponding author
- DiAngi, Yumi T
- Venue
- JAMIA Open
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-03-21
- Volume
- 2
- Issue
- 2
- Pages
- 222-230
- ISSN
- 2574-2531 (ISSN-L)
- PMC
- PMC6952029
- OpenAlex
- W2954996172
- Cited by
- 52
- References
- 33
- Citations by year
- 2026: 4 · 2025: 1 · 2024: 8 · 2023: 6 · 2022: 9 · 2021: 13 · 2020: 8 · 2019: 3
- FWCI
- 8.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- JAMIA Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Keywords
- electronic health records, EHR training, provider satisfaction, turnaround time, provider workload, EHR adoption
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.