Perceived Electronic Health Record Usability as a Predictor of Task Load and Burnout Among US Physicians: Mediation Analysis
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Background: Electronic health record (EHR) usability and physician task load both contribute to physician professional burnout. The association between perceived EHR usability and workload has not previously been studied at a national level. Better understanding these interactions could give further information as to the drivers of extraneous task load. Objective: This study aimed to determine the relationship between physician-perceived EHR usability and workload by specialty and evaluate for associations with professional burnout. Methods: A secondary analysis of a cross-sectional survey of US physicians from all specialties was conducted from October 2017 to March 2018. Among the 1250 physicians invited to respond to the subsurvey analyzed here, 848 (67.8%) completed it. EHR usability was assessed with the System Usability Scale (SUS; range: 0-100). Provider task load (PTL) was assessed using the mental demand, physical demand, temporal demand, and effort required subscales of the National Aeronautics and Space Administration Task Load Index (range: 0-400). Burnout was measured using the Maslach Burnout Inventory. Results: The mean scores were 46.1 (SD 22.1) for SUS and 262.5 (SD 71.7) for PTL. On multivariable analysis adjusting for age, gender, relationship status, medical specialty, practice setting, hours worked per week, and number of nights on call per week, physician-rated EHR usability was associated with PTL, with each 1-point increase in SUS score (indicating more favorable) associated with a 0.57-point decrease in PTL score (P<.001). On mediation analysis, higher SUS score was associated with lower PTL score, which was associated with lower odds of burnout. Conclusions: A strong association was observed between EHR usability and workload among US physicians, with more favorable usability associated with less workload. Both outcomes were associated with the odds of burnout, with task load acting as a mediator between EHR usability and burnout. Improving EHR usability while decreasing task load has the potential to allow practicing physicians more working memory for medical decision making and patient communication.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · doi:10.1007/978-1-4419-8126-4 · Cited by 5,531
An Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · cited 1×
“Consistent with convention [34,36], the language of the SUS was modified such that references to “the system” being evaluated were changed to “my EHR.””
MethodSweller, J. (2011) · Psychology of Learning and Motivation · doi:10.1016/b978-0-12-387691-1.00002-8 · Cited by 4,856
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 10×
“Using the mental demand, physical demand, temporal demand, and effort required subscales of the National Aeronautics and Space Administration Task Load Index (NASA-TLX), a national survey of 4622 physicians conducted by our team identified a strong association between provider task load (PTL) and burnout but did not a…”
BackgroundMethodInfluentialShanafelt, T. D., Boone, S., Tan, L., Dyrbye, L. N., Sotile, W., Satele, D., West, C. P., Sloan, J., & Oreskovich, M. R. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.3199 · Cited by 3,439 · cited 1×
“Given this finding and the growing literature on the association of gender on physician burnout [32,50,51], future work should explore the interactions between these variables and further evaluate how EHR usability and task load vary by personal and professional characteristics.”
BackgroundSelf-citationA meta-analytic examination of the correlates of the three dimensions of job burnout
Lee, R. T., & Ashforth, B. E. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.2.123 · Cited by 3,262 · cited 4×
“The rationale for exclusion of the frustration and performance NASA-TLX items from our measurement of PTL is that on principal component analysis with oblimin rotation and Kaiser normalization to determine underlying patterns between the 6 items of the NASA-TLX and the emotional exhaustion and depersonalization scales…”
MethodInfluentialShanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J., & West, C. P. (2015) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2015.08.023 · Cited by 2,885
Self-citationBurnout and self-reported patient care in an internal medicine residency program
Shanafelt, T. D., Bradley, K. A., Wipf, J. E., & Back, A. L. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-5-200203050-00008 · Cited by 2,165 · cited 1×
“Respondents with a high score on the emotional exhaustion (≥27) or depersonalization subscale (≥10) were considered to have at least 1 symptom of burnout [41,45-47].”
BackgroundSelf-citationThe Factor Structure of the System Usability Scale
Lewis, J. R., & Sauro, J. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-02806-9_12 · Cited by 1,278
Cognitive load theory, educational research, and instructional design: some food for thought
de Jong, T. (2010) · Instructional Science · doi:10.1007/s11251-009-9110-0 · Cited by 1,180 · cited 1×
“Specific areas to target could include consolidating the display of related information, reducing redundancy of information, and increasing standardization [63,64].”
BackgroundShanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P. (2016) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2016.05.007 · Cited by 1,086 · cited 3×
“Partially due to excessive time spent on EHR activities, EHR usability is a specific source of physician dissatisfaction and stress [7,14-16].”
BackgroundSelf-citationUnderstanding Self-Report Bias in Organizational Behavior Research
Donaldson, S. I., & Grant-Vallone, E. J. (2002) · Journal of Business and Psychology · doi:10.1023/a:1019637632584 · Cited by 1,068 · cited 1×
“Third, self-reported data is subject to limitations [55,56].”
BackgroundShanafelt, T. D., West, C. P., Sinsky, C., Trockel, M., Tutty, M., Satele, D. V., Carlasare, L. E., & Dyrbye, L. N. (2019) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2018.10.023 · Cited by 914
Self-citationThomas, N. K. (2004) · JAMA · doi:10.1001/jama.292.23.2880 · Cited by 750
Friedberg, M., Chen, P., Van Busum, K., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D., Brook, R., et al. (2014) · RAND Health Quarterly · doi:10.7249/rhq-03-04-01 · Cited by 707
Physician stress and burnout: the impact of health information technology
Gardner, R. L., Cooper, E., Haskell, J., Harris, D. A., Poplau, S., Kroth, P. J., & Linzer, M. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy145 · Cited by 566
DeLeeuw, K. E., & Mayer, R. E. (2008) · Journal of Educational Psychology · doi:10.1037/0022-0663.100.1.223 · Cited by 558
Rosen, I. M., Gimotty, P. A., Shea, J. A., & Bellini, L. M. (2006) · Academic Medicine · doi:10.1097/00001888-200601000-00020 · Cited by 495 · cited 1×
“Respondents with a high score on the emotional exhaustion (≥27) or depersonalization subscale (≥10) were considered to have at least 1 symptom of burnout [41,45-47].”
BackgroundPhysician Burnout in the Electronic Health Record Era: Are We Ignoring the Real Cause?
Downing, N. L., Bates, D. W., & Longhurst, C. A. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0139 · Cited by 418 · cited 3×
“In addition to usability, the clerical burden associated with documentation, order entry, inbox management, and other EHR administrative tasks (eg, prior authorizations, documentation of care consistent with quality measures) that are not necessarily intrinsic to the practice of medicine also contributes to excessive…”
BackgroundMelnick, E. R., Dyrbye, L. N., Sinsky, C. A., Trockel, M., West, C. P., Nedelec, L., Tutty, M. A., & Shanafelt, T. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2019.09.024 · Cited by 413 · cited 6×
“If that is the case, the association of clerical burden with task load and burnout could be stronger than that of EHR usability [7,16].”
BackgroundResultInfluentialSelf-citationElectronic medical records and physician stress in primary care: results from the MEMO Study
Babbott, S., Manwell, L. B., Brown, R., Montague, E., Williams, E., Schwartz, M., Hess, E., & Linzer, M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001875 · Cited by 366
Self-report bias and underreporting of depression on the BDI-II
Hunt, M., Auriemma, J., & Cashaw, A. C. A. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8001_10 · Cited by 301 · cited 1×
“Third, self-reported data is subject to limitations [55,56].”
BackgroundElectronic Health Record Usability Issues and Potential Contribution to Patient Harm
Howe, J. L., Adams, K. T., Hettinger, A. Z., & Ratwani, R. M. (2018) · JAMA · doi:10.1001/jama.2018.1171 · Cited by 281 · cited 1×
“Current EHR usability challenges have resulted in systems that many find unnecessarily complex and prone to error, thereby increasing physician cognitive load and resulting in technological errors that can sometimes reach the patient [10-13].”
BackgroundZahabi, M., Kaber, D. B., & Swangnetr, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815576827 · Cited by 240
Olson, K., Sinsky, C., Rinne, S. T., Long, T., Vender, R., Mukherjee, S., Bennick, M., & Linzer, M. (2019) · Stress and Health · doi:10.1002/smi.2849 · Cited by 202
Self-citationWeigl, M., Müller, A., Vincent, C., Angerer, P., & Sevdalis, N. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000188 · Cited by 190
A usability and safety analysis of electronic health records: a multi-center study
Ratwani, R. M., Savage, E., Will, A., Arnold, R., Khairat, S., Miller, K., Fairbanks, R. J., Hodgkins, M., & Hettinger, A. Z. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy088 · Cited by 180 · cited 3×
“For example, one dimension of usability is error tolerance, a system’s ability to prevent or recover from an error; due to poor usability, EHR users may not be aware of errors they make in the EHR [11,12].”
BackgroundIdentifying electronic health record usability and safety challenges in pediatric settings
Ratwani, R. M., Savage, E., Will, A., Fong, A., Karavite, D., Muthu, N., Rivera, A. J., Gibson, C., Asmonga, D., Moscovitch, B., et al. (2018) · Health Affairs · doi:10.1377/hlthaff.2018.0699 · Cited by 163
Metrics for assessing physician activity using electronic health record log data
Sinsky, C. A., Rule, A., Cohen, G., Arndt, B. G., Shanafelt, T. D., Sharp, C. D., Baxter, S. L., Tai-Seale, M., Yan, S., Chen, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz223 · Cited by 162 · cited 1×
“Specific areas to target could include consolidating the display of related information, reducing redundancy of information, and increasing standardization [63,64].”
BackgroundSelf-citationMazur, L. M., Mosaly, P. R., Moore, C., & Marks, L. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.1709 · Cited by 127 · cited 2×
“Our findings are consistent with multiple smaller studies of task load in both simulated [29,30] and real world settings [58,59] suggesting that more usable EHR interfaces are associated with lower cognitive load.”
BackgroundResultThe complex case of EHRs: examining the factors impacting the EHR user experience
Tutty, M. A., Carlasare, L. E., Lloyd, S., & Sinsky, C. A. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz021 · Cited by 94 · cited 2×
“In addition to usability, the clerical burden associated with documentation, order entry, inbox management, and other EHR administrative tasks (eg, prior authorizations, documentation of care consistent with quality measures) that are not necessarily intrinsic to the practice of medicine also contributes to excessive…”
BackgroundSelf-citationPollack, A. H., & Pratt, W. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.19301 · Cited by 53 · cited 2×
“Our findings are consistent with multiple smaller studies of task load in both simulated [29,30] and real world settings [58,59] suggesting that more usable EHR interfaces are associated with lower cognitive load.”
BackgroundResultAriza, F., Kalra, D., & Potts, H. W. (2015) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v22i4.85 · Cited by 34
Ray, J. M., Ratwani, R. M., Sinsky, C. A., Frankel, R. M., Friedberg, M. W., Powsner, S. M., Rosenthal, D. I., Wachter, R. M., & Melnick, E. R. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz098 · Cited by 34 · cited 1×
“…into consideration to design EHR interfaces and workflows that meet users’needs and are free from unnecessary extraneous cognitive load, such that the EHR becomes part of a physician work environment more closely resembling a “manageable cockpit” that is streamlined, ergonomic, and safe [10,61,62].”
BackgroundSelf-citationShould US doctors embrace electronic health records?
(2017) · BMJ · doi:10.1136/bmj.j242 · Cited by 18 · cited 4×
“Current EHR usability challenges have resulted in systems that many find unnecessarily complex and prone to error, thereby increasing physician cognitive load and resulting in technological errors that can sometimes reach the patient [10-13].”
BackgroundInfluentialSelf-citation
Cited by
Shanafelt, T. D., West, C. P., Sinsky, C., Trockel, M., Tutty, M., Wang, H., Carlasare, L. E., & Dyrbye, L. N. (2022) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2021.11.021 · Cited by 434
Self-citationArtificial Intelligence–Generated Draft Replies to Patient Inbox Messages
Garcia, P., Ma, S. P., Shah, S., Smith, M., Jeong, Y., Devon-Sand, A., Tai-Seale, M., Takazawa, K., Clutter, D., Vogt, K., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.3201 · Cited by 266
Self-citationLi, C., Parpia, C., Sriharan, A., & Keefe, D. T. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-060865 · Cited by 106
Wu, Y., Wu, M., Wang, C., Lin, J., Liu, J., & Liu, S. (2024) · JMIR Medical Informatics · doi:10.2196/54811 · Cited by 87
Documentation dynamics: note composition, burden, and physician efficiency
Apathy, N. C., Rotenstein, L., Bates, D. W., & Holmgren, A. J. (2023) · Health Services Research · doi:10.1111/1475-6773.14097 · Cited by 70
Metadata
- Authors
- Melnick, E. R., Harry, E., Sinsky, C. A., Dyrbye, L. N., Wang, H., Trockel, M. T., West, C. P., & Shanafelt, T.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-12-22
- Volume
- 22
- Issue
- 12
- Pages
- e23382
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W3112632867
- Cited by
- 100
- References
- 36
- FWCI
- 10.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.