Notifications Received by Primary Care Practitioners in Electronic Health Records: A Taxonomy and Time Analysis
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
BACKGROUND Asynchronous electronic health record (EHR)-based alerts used to notify practitioners via an inbox-like format rather than through synchronous computer "pop-up" messages are understudied. Our objective was to create an asynchronous alert taxonomy and measure the impact of different alert types on practitioner workload. METHODS We quantified and categorized asynchronous alerts according to the information they conveyed and conducted a time-motion analysis to assess practitioner workload. We reviewed alert information transmitted to all 47 primary care practitioners (PCPs) at a large, tertiary care Veterans Affairs facility over 4 evenly spaced 28-day periods. An interdisciplinary team used content analysis to categorize alerts according to their conveyed information. We then created an alert taxonomy and used it to calculate the mean number of alerts of each type PCPs received each day. We conducted a time-motion study of 26 PCPs while they processed their alerts. We used these data to estimate the uninterrupted time practitioners spend processing alerts each day. RESULTS We extracted 295,792 asynchronously generated alerts and created a taxonomy of 33 alert types categorized under 6 major categories: Test Results, Referrals, Note-Based Communication, Order Status, Patient Status Changes, and Incomplete Task Reminders. PCPs received a mean of 56.4 alerts/day containing new information. Based on 749 observed alert processing episodes, practitioners spent an estimated…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Factors such as increased structured documentation requirements, computerized physician order entry (CPOE), inbox management, patient portals, and a redistribution of tasks previously performed by clinical staff to clinicians has led to more work that is not direct face time with patients.”
From Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations (2017) · cites this paper as [11]
Citations
Cites
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617
Graneheim, U. H., & Lundman, B. (2004) · Nurse Education Today · doi:10.1016/j.nedt.2003.10.001 · Cited by 20,772
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · cited 1×
“” Practitioners often override warning messages from synchronous alerts when they are presented with an abundance of incorrect, redundant, confusing,(25) or clinically insignifiant information.(26) Asynchronous alert messages also must be succinct, intuitive, and lack confusing abbreviations.”
BackgroundPoon, E. G., Gandhi, T. K., Sequist, T. D., Murff, H. J., Karson, A. S., & Bates, D. W. (2004) · Archives of Internal Medicine · doi:10.1001/archinte.164.20.2223 · Cited by 301
Singh, H., Thomas, E. J., Mani, S., Sittig, D., Arora, H., Espadas, D., Khan, M. M., & Petersen, L. A. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.263 · Cited by 212 · cited 2×
“We previously found that about 7% of abnormal laboratory and imaging test result alerts were missed in EHR systems.(8,9) Information overload from too any alerts might be a potential reason, given that practiioners must sort through a large number of alerts daily.”
BackgroundSelf-citationSingh, H., Thomas, E. J., Sittig, D. F., Wilson, L., Espadas, D., Khan, M. M., & Petersen, L. A. (2010) · The American Journal of Medicine · doi:10.1016/j.amjmed.2009.07.027 · Cited by 160 · cited 2×
“We previously found that about 7% of abnormal laboratory and imaging test result alerts were missed in EHR systems.(8,9) Information overload from too any alerts might be a potential reason, given that practiioners must sort through a large number of alerts daily.”
BackgroundSelf-citationUnderstanding diagnostic errors in medicine: a lesson from aviation
Singh, H. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2005.016444 · Cited by 156 · cited 1×
“Providing certain additional patientspecific information in the alert text, such as a prior laboratory value in the case of an abnormal laboratory result, would permit better situational awareness and improve efficiency.(28) Third, during our time-motion study, interruptions were common and were reportedly more freque…”
BackgroundSelf-citationDesign and implementation of a comprehensive outpatient Results Manager
Poon, E. G., Wang, S. J., Gandhi, T. K., Bates, D. W., & Kuperman, G. J. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00061-3 · Cited by 111 · cited 1×
“Asynchronous alerts often convey aluable and actionable information etween traditional face-to-face paient-practitioner encounters.(6) Unlike synchronous alerts, asynchronous alerts require practitioners to actively access their EHR’s inbox application to read alerts.”
BackgroundUnderstanding the management of electronic test result notifications in the outpatient setting
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Esquivel, A., Singh, S., & Singh, H. (2011) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-11-22 · Cited by 68
Self-citationProvider management strategies of abnormal test result alerts: a cognitive task analysis
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Espadas, D., Davis, T., & Singh, H. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3200 · Cited by 65 · cited 3×
“Prior research has shown that practitioners may not manage this influx of electronic information as efficiently.(10,20-23) For instance, practitioners might spend a substantial amount of time sifting through electronic records to locate information.”
BackgroundSelf-citationWahls, T. (2007) · Journal of Ambulatory Care Management · doi:10.1097/01.jac.0000290402.89284.a9 · Cited by 21
Research works printed but not linked: 21
- Coiera, 1998, Communication behaviours in a hospital setting: an observational study, BMJ, 316, 673, 10.1136/bmj.316.7132.673 · DOI · Paper page
- Magnus, 2002, GPs' views on computerized drug interaction alerts: questionnaire survey, J Clin Pharm Ther, 27, 377, 10.1046/j.1365-2710.2002.00434.x · DOI · Paper page
- Weingart, 2003, Physicians' decisions to override computerized drug alerts in primary care, Arch Intern Med, 163, 2625, 10.1001/archinte.163.21.2625 · DOI · Paper page
- Taylor, 2004, Reasons for physician non-adherence to electronic drug alerts, Stud Health Technol Inform, 11, 1101
- Wahls, 2007, The frequency of missed test results and associated treatment delays in a highly computerized health system, BMC Fam Pract, 8, 32, 10.1186/1471-2296-8-32 · DOI · Paper page
- Fava, 2007, Information overload, the patient and the clinician, Psychother Psychosom, 76, 1, 10.1159/000096358 · DOI
- Wright, 2011, Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems, J Am Med Inform Assoc, 18, 232, 10.1136/amiajnl-2011-000113 · DOI · Paper page
- Bratt, 1999, A comparison of four approaches for measuring clinician time use, Health Policy Plan, 14, 374, 10.1093/heapol/14.4.374 · DOI
- Finkler, 1993, A comparison of work-sampling and time-and-motion techniques for studies in health services research, Health Serv Res, 28, 577
- Schumacher, 2010, Commentary: electronic health records and human performance, J Oncol Pract, 6, 125, 10.1200/JOP.091098 · DOI
- Linder, 2006, Barriers to electronic health record use during patient visits, AMIA Annu Symp Proc, 499
- Simon, 2008, Electronic health records: which practices have them, and how are clinicians using them?, J Eval Clin Pract, 14, 43, 10.1111/j.1365-2753.2007.00787.x · DOI · Paper page
- Pizziferri, 2005, Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study, J Biomed Inform, 38, 176, 10.1016/j.jbi.2004.11.009 · DOI · Paper page
- Weingart, 2009, Clinicians' assessments of electronic medication safety alerts in ambulatory care, Arch Intern Med, 169, 1627, 10.1001/archinternmed.2009.300 · DOI · Paper page
- Biron, 2009, Work interruptions and their contribution to medication administration errors: an evidence review, Worldviews Evid Based Nurs, 6, 70, 10.1111/j.1741-6787.2009.00151.x · DOI · Paper page
- Doerr, 2010, Between-visit workload in primary care, J Gen Intern Med, 25, 1289, 10.1007/s11606-010-1470-2 · DOI · Paper page
- Farber, 2007, How much time do physicians spend providing care outside of office visits?, Ann Intern Med, 147, 693, 10.7326/0003-4819-147-10-200711200-00005 · DOI · Paper page
- Ammenwerth, 2009, The time needed for clinical documentation versus direct patient care, Methods Inf Med, 48, 84, 10.3414/ME0569 · DOI · Paper page
- Margolius, 2010, Transforming primary care: from past practice to the practice of the future, Health Aff (Millwood), 29, 779, 10.1377/hlthaff.2010.0045 · DOI · Paper page
- Sittig, 2009, Eight rights of safe electronic health record use, JAMA, 302, 1111, 10.1001/jama.2009.1311 · DOI · Paper page
- Singh, 2010, Ten strategies to improve management of Abnormal Test Result Alerts in the Electronic Health Record, J Patient Saf, 6, 121, 10.1097/PTS.0b013e3181ddf652 · DOI · Paper page
Cited by
From Triple to Quadruple Aim: Care of the Patient Requires Care of the Provider
Bodenheimer, T., & Sinsky, C. (2014) · The Annals of Family Medicine · doi:10.1370/afm.1713 · Cited by 3,462
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W.-J., Sinsky, C. A., & Gilchrist, V. J. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2121 · Cited by 972
Young-Onset Colorectal Cancer: Is It Time to Pay Attention?
You, Y. N. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.602 · Cited by 273
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269
Metadata
- Authors
- Murphy, D. R., Reis, B., Sittig, D. F., & Singh, H.
- Venue
- The American Journal of Medicine
- Publisher
- Elsevier BV
- Published
- 2012-02
- Volume
- 125
- Issue
- 2
- Pages
- 209.e1-209.e7
- ISSN
- 0002-9343
- Cited by
- 89
- References
- 13
- Language
- en
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.