Explaining Variability in Electronic Health Record Effort in Primary Care Ambulatory Encounters
Open accessAbstract
Background Electronic health record (EHR) user interface event logs are fast providing another perspective on the value and efficiency EHR technology brings to health care. Analysis of these detailed usage data has demonstrated their potential to identify EHR and clinical process design factors related to user efficiency, satisfaction, and burnout. Objective This study aimed to analyze the event log data across 26 different health systems to determine the variability of use of a single vendor's EHR based on four event log metrics, at the individual, practice group, and health system levels. Methods We obtained de-identified event log data recorded from June 1, 2018, to May 31, 2019, from 26 health systems' primary care physicians. We estimated the variability in total Active EHR Time, Documentation Time, Chart Review Time, and Ordering Time across health systems, practice groups, and individual physicians. Results In total, 5,444 physicians (Family Medicine: 3,042 and Internal Medicine: 2,422) provided care in a total of 2,285 different practices nested in 26 health systems. Health systems explain 1.29, 3.55, 3.45, and 3.30% of the total variability in Active Time, Documentation Time, Chart Review Time, and Ordering Time, respectively. Practice-level variability was estimated to be 7.96, 13.52, 8.39, and 5.57%, respectively, and individual physicians explained the largest proportion of the variability for those same outcomes 17.09, 27.49, 17.51, and 19.75%, respectively. Conclusion The most variable physician EHR usage patterns occurs at the individual physician level and decreases as you move up to the practice and health system levels. This suggests that interventions to improve individual users' EHR usage efficiency may have the most potential impact compared with those directed at health system or practice levels.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Clinical documentation can be variable in quality.
“Clinical documentation can be variable in quality19,20”
From A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
From Triple to Quadruple Aim: Care of the Patient Requires Care of the Provider
Bodenheimer, T., & Sinsky, C. (2014) · The Annals of Family Medicine · doi:10.1370/afm.1713 · Cited by 3,462 · see citing works
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W.-J., Sinsky, C. A., & Gilchrist, V. J. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2121 · Cited by 972 · see citing works
Time allocation in primary care office visits
Tai‐Seale, M., McGuire, T. G., & Zhang, W. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00689.x · Cited by 472 · see citing works
Tai-Seale, M., Olson, C. W., Li, J., Chan, A. S., Morikawa, C., Durbin, M., Wang, W., & Luft, H. S. (2017) · Health Affairs · doi:10.1377/hlthaff.2016.0811 · Cited by 379 · see citing works
Standardization as a Mechanism to Improve Safety in Health Care
Rozich, J. D., Howard, R. J., Justeson, J. M., Macken, P. D., Lindsay, M. E., & Resar, R. K. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30001-8 · Cited by 294 · see citing works
Physician Time Spent Using the Electronic Health Record During Outpatient Encounters
Overhage, J. M., & McCallie, D. P. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3684 · Cited by 249 · see citing works
Self-citationTai‐Seale, M., Dillon, E. C., Yan, Y., Nordgren, R., Steinberg, R. L., Nauenberg, T., Lee, T. C., Meehan, A. E., Li, J., Chan, A. S., et al. (2019) · Health Affairs · doi:10.1377/hlthaff.2018.05509 · Cited by 225 · see citing works
Zheng, K., Guo, M. H., & Hanauer, D. A. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000083 · Cited by 194 · see citing works
Metrics for assessing physician activity using electronic health record log data
Sinsky, C. A., Rule, A., Cohen, G., Arndt, B. G., Shanafelt, T. D., Sharp, C. D., Baxter, S. L., Tai-Seale, M., Yan, S., Chen, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz223 · Cited by 162 · see citing works
EHR audit logs: A new goldmine for health services research?
Adler‐Milstein, J., Adelman, J. S., Tai‐Seale, M., Patel, V. L., & Dymek, C. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103343 · Cited by 133 · see citing works
The performance impact of business process standardization
Muenstermann, B., Eckhardt, A., & Weitzel, T. (2010) · Business Process Management Journal · doi:10.1108/14637151011017930 · Cited by 129 · see citing works
Cohen, G. R., Friedman, C. P., Ryan, A. M., Richardson, C. R., & Adler-Milstein, J. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05025-3 · Cited by 117 · see citing works
A comparison of a multistate inpatient EHR database to the HCUP Nationwide Inpatient Sample
DeShazo, J. P., & Hoffman, M. A. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-1025-7 · Cited by 116 · see citing works
Standardizing Care Processes and Improving Quality Using Pathways and Continuous Quality Improvement
Lavelle, J. M., Schast, A. P., & Keren, R. (2015) · Current Treatment Options in Pediatrics · doi:10.1007/s40746-015-0026-4 · Cited by 95 · see citing works
The complex case of EHRs: examining the factors impacting the EHR user experience
Tutty, M. A., Carlasare, L. E., Lloyd, S., & Sinsky, C. A. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz021 · Cited by 94 · see citing works
Two‐Minute Mental Health Care for Elderly Patients: Inside Primary Care Visits
Tai‐Seale, M., McGuire, T. G., Colenda, C. C., Rosen, D., & Cook, M. A. (2007) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2007.01467.x · Cited by 81 · see citing works
Standardize to Excellence: Improving the Quality and Safety of Care with Clinical Pathways
Kurtin, P. S., & Stucky, E. R. (2009) · Pediatric Clinics of North America · doi:10.1016/j.pcl.2009.05.005 · Cited by 81 · see citing works
Local Investment in Training Drives Electronic Health Record User Satisfaction
Longhurst, C., Davis, T., Maneker, A., Eschenroeder, H., Dunscombe, R., Reynolds, G., Clay, B., Moran, T., Graham, D., Dean, S., et al. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1688753 · Cited by 76 · see citing works
Melnick, E. R., Ong, S. Y., Fong, A., Socrates, V., Ratwani, R. M., Nath, B., Simonov, M., Salgia, A. D. T., Williams, B., Marchalik, D., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab011 · Cited by 67 · see citing works
Rule, A., Melnick, E. R., & Apathy, N. C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac177 · Cited by 58 · see citing works
Measures of electronic health record use in outpatient settings across vendors
Baxter, S. L., Apathy, N. C., Cross, D. A., Sinsky, C. A., & Hribar, M. R. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa266 · Cited by 57 · see citing works
Prevention of unplanned extubations in neonates through process standardization
Fontánez-Nieves, T. D., Frost, M. S., Anday, E. K., et al. (2016) · Journal of Perinatology · doi:10.1038/jp.2015.219 · Cited by 40 · see citing works
Jaulin, F., Lopes, T., & Martin, F. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.07.002 · Cited by 37 · see citing works
25 × 5 Symposium to reduce documentation burden: report-out and call for action
Hobensack, M., Levy, D. R., Cato, K., Detmer, D. E., Johnson, K. B., Williamson, J., Murphy, J., Moy, A., Withall, J., Lee, R., et al. (2022) · Applied Clinical Informatics · doi:10.1055/s-0042-1746169 · Cited by 37 · see citing works
Moy, A. J., Aaron, L., Cato, K., Schwartz, J., Elias, J. E., Trepp, R., & Rossetti, S. C. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1736625 · Cited by 33 · see citing works
Rotenstein, L. S., Holmgren, A. J., Downing, N. L., Longhurst, C. A., & Bates, D. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.16375 · Cited by 31 · see citing works
Hron, J. D., & Lourie, E. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz222 · Cited by 27 · see citing works
Pediatrician Electronic Health Record Time Use for Outpatient Encounters
Overhage, J. M., & Johnson, K. B. (2020) · PEDIATRICS · doi:10.1542/peds.2019-4017 · Cited by 22 · see citing works
Self-citationMeasuring time clinicians spend using EHRs in the inpatient setting: a national, mixed-methods study
Cohen, G. R., Boi, J., Johnson, C., Brown, L., & Patel, V. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab042 · Cited by 22 · see citing works
The role of organizations in shaping physician use of electronic health records
Cross, D. A., Holmgren, A. J., & Apathy, N. C. (2023) · Health Services Research · doi:10.1111/1475-6773.14203 · Cited by 21 · see citing works
A Typology of Business Process Standardization Strategies
Goel, K., Bandara, W., & Gable, G. G. (2021) · Business & Information Systems Engineering · doi:10.1007/s12599-021-00693-0 · Cited by 20 · see citing works
Reflections on the Documentation Burden Reduction AMIA Plenary Session through the Lens of 25 × 5
Levy, D. R., Sloss, E. A., Chartash, D., Corley, S. T., Mishuris, R. G., Rosenbloom, S. T., & Tiase, V. L. (2022) · Applied Clinical Informatics · doi:10.1055/a-1976-2052 · Cited by 13 · see citing works
Evaluating Electronic Health Record Limitations and Time Expenditure in a German Medical Center
de Hoop, T., & Neumuth, T. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1739519 · Cited by 11 · see citing works
Téoule, P., Birgin, E., Mertens, C., et al. (2020) · Cancers · doi:10.3390/cancers12020434 · Cited by 9 · see citing works
Implementation of a Standardized Approach to Improve the Pediatric Discharge Medication Process
Philips, K., Zhou, R., Lee, D. S., et al. (2021) · PEDIATRICS · doi:10.1542/peds.2019-2711 · Cited by 9 · see citing works
Automation of Balanced Nested Design; NeDPy
Obinna, M. T., Ogoke, U. P., & Nduka, E. C. (2020) · International journal of statistics and applications · Cited by 2 · see citing works
Heinen, Y., Wolff, G., Klein, K., et al. (2022) · The Journal of invasive cardiology/The journal of invasive cardiology · doi:10.25270/jic/22.00065 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Melnyk B. National academy of medicine's action collaborative on clinician well-being and resilience: a solution-focused strategy is designed to curtail the burnout epidemic. Am Nurse Today 2019; 14(04):61-64
Cited by
Sorted by citations
A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation
Asgari, E., Montaña-Brown, N., Dubois, M., Khalil, S., Balloch, J., Yeung, J. A., & Pimenta, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01670-7 · Cited by 423 · see citing works
- Authors
- Overhage, J. M., Qeadan, F., Choi, E. H. E., Vos, D., & Kroth, P. J.
- Affiliations
- The Overhage Group, Zionsville, Indiana, United States
- Loyola University Chicago (US)
- University of New Mexico (US)
- Western Michigan University (US)
- Corresponding author
- Kroth, Philip J.
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2024-03
- Volume
- 15
- Issue
- 02
- Pages
- 212-219
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC10954376
- OpenAlex
- W4394759276
- Cited by
- 13
- References
- 37
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 2
- FWCI
- 3.96
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Applied Clinical Informatics (publishedVersion)
- PubMed Central (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
- Keywords
- electronic health records, primary care, documentation time, inter-physician variability, general practice, internal medicine
- MeSH terms
- Burnout, Professional, Documentation, Humans, Physicians, Primary Health Care, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
