How is the electronic health record being used? Use of EHR data to assess physician-level variability in technology use
Open accessAbstract
BACKGROUND: Studies of the effects of electronic health records (EHRs) have had mixed findings, which may be attributable to unmeasured confounders such as individual variability in use of EHR features. OBJECTIVE: To capture physician-level variations in use of EHR features, associations with other predictors, and usage intensity over time. METHODS: Retrospective cohort study of primary care providers eligible for meaningful use at a network of federally qualified health centers, using commercial EHR data from January 2010 through June 2013, a period during which the organization was preparing for and in the early stages of meaningful use. RESULTS: Data were analyzed for 112 physicians and nurse practitioners, consisting of 430,803 encounters with 99,649 patients. EHR usage metrics were developed to capture how providers accessed and added to patient data (eg, problem list updates), used clinical decision support (eg, responses to alerts), communicated (eg, printing after-visit summaries), and used panel management options (eg, viewed panel reports). Provider-level variability was high: for example, the annual average proportion of encounters with problem lists updated ranged from 5% to 60% per provider. Some metrics were associated with provider, patient, or encounter characteristics. For example, problem list updates were more likely for new patients than established ones, and alert acceptance was negatively correlated with alert frequency. CONCLUSIONS: Providers using the same EHR developed personalized patterns of use of EHR features. We conclude that physician-level usage of EHR features may be a valuable additional predictor in research on the effects of EHRs on healthcare quality and costs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies using raw audit logs fit into three broad categories: manual mapping, identification of networks, application of automatic methods.
“Studies using raw audit logs fit into three broad categories:”
From Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling (2025) · cites this paper as [77]
The Metric development section describes five categories of EHR-use metrics, not three categories of studies using raw audit logs. The Introduction and Discussion discuss ways to assess EHR use, but the searched passages do not classify audit-log studies as manual mapping, network identification, and automatic methods (searched: audit logs, manual mapping, networks, automatic methods). (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · see citing works
Kuperman, G. J., Teich, J. M., Tanasijevic, M. J., Ma'Luf, N., Rittenberg, E., Jha, A., Fiskio, J., Winkelman, J., & Bates, D. W. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.0060512 · Cited by 252 · see citing works
Overrides of medication-related clinical decision support alerts in outpatients
Nanji, K. C., Slight, S. P., Seger, D. L., Cho, I., Fiskio, J. M., Redden, L. M., Volk, L. A., & Bates, D. W. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001813 · Cited by 219 · see citing works
Del Beccaro, M. A., Jeffries, H. E., Eisenberg, M. A., & Harry, E. D. (2006) · Pediatrics · doi:10.1542/peds.2006-0367 · Cited by 181 · see citing works
Ash, J. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040102 · Cited by 174 · see citing works
Miller, R. A., Waitman, L. R., Chen, S., & Rosenbloom, S. T. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.08.009 · Cited by 144 · see citing works
Flagged first, then by citations
Research works printed but not linked: 26
- Blumenthal D, Tavenner M. The "meaningful use" regulation for electronic health records. N Engl J Med 2010;363:501-4.
- Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409-16.
- Chaudhry B, Jerome W, Shinyi W, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742-52.
- Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116:1506-12.
- Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293:1197-203.
- Upperman JS, Staley P, Friend K, et al. The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital. J Pediatr Surg 2005;40:57-9.
- Poon EG, Wright A, Simon SR, et al. Relationship between use of electronic health record features and health care quality: results of a statewide survey. Med Care 2010;48:203-9.
- Amarasingham R, Plantinga L, Diener-West M, et al. Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med 2009;169:108-14.
- Ancker JS, Kern LM, Abramson E, et al. The triangle model for evaluating the effect of health information technology on healthcare quality and safety. JAMIA 2011;18:749-53.
- Berg M, Aarts J, van der Lei J. ICT in health care: sociotechnical approaches. Methods Inf Med 2003;42:297-301.
- Amarasingham R, Diener-West M, Weiner M, et al. Clinical information technology capabilities in four U.S. hospitals: testing a new structural performance measure. Med Care 2006;44:216-24.
- Makam AN, Lanham HJ, Batchelor K, et al. Use and satisfaction with key functions of a common commercial electronic health record: a survey of primary care providers. BMC Med Inform Decis Mak 2013;13:86.
- Lanham HJ, Sittig DF, Leykum LK, et al. Understanding differences in electronic health record (EHR) use: linking individual physicians' perceptions of uncertainty and EHR use patterns in ambulatory care. J Am Med Inform Assoc 2014;21:73-81.
- Tamblyn R, Reidel K, Patel V. Physicians' response to computerised alerts for psychotropic drugs in older persons: a multilevel analysis of the associated alert, patient and physician characteristics. BMJ Open 2012;2:e001384.
- van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006;13:138-47.
- Eslami S, Abu-Hanna A, de Keizer NF. Evaluation of outpatient computerized physician medication order entry systems: a systematic review. J Am Med Inform Assoc. 2007;14:400-6.
- Covell D, Uman G, Manning P. Information needs in office practice: are they being met? Ann Intern Med 1985;103:596-9.
- Hartung DM, Hunt J, Siemienczuk J, et al. Clinical implications of an accurate problem list on heart failure treatment. J Gen Intern Med 2005;20:143-7.
- Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10:523-30.
- Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14:29-40.
- Kaushal R, Kern LM, Barron Y, et al. Electronic prescribing improves medication safety in community-based office practices. J Gen Intern Med 2010;25:530-6.
- Stead WW, Lin HS., eds. Computational technology for effective health care: immediate steps and strategic directions. Washington, DC: National Academies Press, 2009.
- Weiner J, Starfield B, Steinwachs D, et al. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care 1991;29:452-72.
- Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res 2013;14:e162.
- Kesselheim AS, Cresswell K, Phansalkar S, et al. Clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation. Health Aff (Millwood) 2011;30:2310-7.
- DeLone WH, McLean ER. The Delone and McLean model of information systems success: a 10-year update. J Manag Info Syst 2003;19:9-30.
Cited by
Sorted by citations
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
“Additional details from this study are published elsewhere [29, 30].”
BackgroundSelf-citationMeunier, P.-Y., Raynaud, C., Guimaraes, E., Gueyffier, F., & Letrilliart, L. (2023) · The Annals of Family Medicine · doi:10.1370/afm.2908 · Cited by 149 · see citing works
Cohen, G. R., Friedman, C. P., Ryan, A. M., Richardson, C. R., & Adler-Milstein, J. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05025-3 · Cited by 117 · see citing works
Pranaat, R., Mohan, V., O'Reilly, M., Hirsh, M., McGrath, K., Scholl, G., Woodcock, D., & Gold, J. A. (2017) · JMIR Medical Informatics · doi:10.2196/medinform.7883 · Cited by 26 · see citing works
Chen, J., Cutrona, S. L., Dharod, A., Moses, A., Bridges, A., Ostasiewski, B., Foley, K. L., Houston, T. K., iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control, Chen, J., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf050 · Cited by 3 · see citing works
- Authors
- Ancker, J. S., Kern, L. M., Edwards, A., Nosal, S., Stein, D. M., Hauser, D., Kaushal, R., & with the HITEC Investigators
- Affiliations
- Weill Cornell Medicine (US)
- Institute for Family Health (US)
- Corresponding author
- Ancker, Jessica S
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-11
- Volume
- 21
- Issue
- 6
- Pages
- 1001-1008
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC4215048
- OpenAlex
- W2102704809
- Cited by
- 91
- References
- 6
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 3 · 2023: 3 · 2022: 7 · 2021: 13 · 2020: 9 · 2019: 13 · 2018: 5 · 2017: 12 · 2016: 7 · 2015: 10
- FWCI
- 17.67
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- New York State Department of Health (C025877)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
- Keywords
- electronic health records, meaningful use, clinical decision support alerts, after-visit summary, panel management, primary care providers, nurse practitioners, federally qualified health centers, unmeasured confounding
- MeSH terms
- Adult, Decision Making, Computer-Assisted, Female, Humans, Male, Nurse Practitioners, Retrospective Studies, United States, Electronic Health Records, Physicians, Primary Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
