CoChat
Sign Up
Cited by 72Electronic Health Records SystemsUsability and User Interface DesignInformation Systems Theories and Implementation

Current Challenges and Opportunities for Better Integration of Human Factors Research with Development of Clinical Information Systems

Open access
Saleem, J. J., Russ, A. L., Sanderson, P., Johnson, T. R., Zhang, J., Sittig, D. F.·2009·Yearbook of Medical Informatics
DOI 10.1055/s-0038-1638638
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVES: Clinical information system (CIS) developers and implementers have begun to look to other scientific disciplines for new methods, tools, and techniques to help them better understand clinicians and their organizational structures, clinical work environments, capabilities of clinical information and communications technology, and the way these structures and processes interact. The goal of this article is to help CIS researchers, developers, implementers, and evaluators better understand the methods, tools, techniques, and literature of the field of human factors. METHODS: We developed a framework that explains how six key human factors topics relate to the design, implementation, and evaluation of CISs. RESULTS: Using this framework we discuss the following six topics: 1) informatics and patient safety; 2) user interface design and evaluation; 3) workflow and task analysis; 4) clinical decision making and decision support; 5) distributed cognition; and 6) mental workload and situation awareness. CONCLUSIONS: Integrating the methods, tools, and lessons learned from each of these six areas of human factors research early in CIS design and incorporating them iteratively during development can improve user performance, user satisfaction, and integration into clinical workflow. Ultimately, this approach will improve clinical information systems and healthcare delivery.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1055/s-0038-1638638

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Involvement of human factors engineering, cognitive engineering, interaction design, psychology, sociology, anthropology, etc, in all phases of HIT design and implementation could substantially improve HIT usability, efficiency, safety, and user satisfaction.

    “The involvement of human factors engineering, cognitive engineering, interaction design, psychology, sociology, anthropology, etc, in all phases of HIT design and implementation will not be a panacea, but could substantially improve HIT usability, efficiency, safety, and user satisfaction. 80”

    From Health information technology: fallacies and sober realities (2010) · cites this paper as [80]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 140 · Cited by 72

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 62

  1. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med 2007;167(13):1400-5.
  2. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized phy- sician order entry systems in facilitating medica- tion errors. JAMA 2005;293(10):1197-203.
  3. Cochran GL, Jones KJ, Brockman J, Skinner A, Hicks RW. Errors prevented by and associated with bar-code medication administration systems. Jt Comm J Qual Patient Saf. 2007 May;33(5):293- 301, 245.
  4. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Sys- tematic Review. J Am Med Inform Assoc 2008;15(5):585-600.
  5. Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med 2003;348(25):2526-34.
  6. Zhang J, Patel VL, Johnson TR, Shortliffe EH: A cognitive taxonomy of medical errors. J Biomed Inform 2004;37(3):193-204.
  7. Hartzband P, Groopman J. Off the record-avoid- ing the pitfalls of going electronic. N Engl J Med 2008;358(16):1656-8.
  8. Weir CR, Hurdle JF, Felgar MA, Hoffman JM, Roth B, Nebeker JR. Direct text entry in electronic progress notes. An evaluation of input errors. Meth- ods Inf Med 2003;42(1):61-7.
  9. Glassman PA, Belperio P, Simon B, Lanto A, Lee M. Exposure to automated drug alerts over time: effects on clinicians' knowledge and perceptions. Med Care 2006;44(3):250-6.
  10. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006;13(2):138-47.
  11. Sarter NB, Woods DD, Billings C.E. Automation Surprises. In Handbook of Human Factors and Er- gonomics. Edited by G.Salvendy: Wiley; 1997.
  12. Niazkhani Z. Evaluating the impact of CPOE sys- tems on medical workflow: a mixed method study. Stud Health Technol Inform 2008;136:881-2.
  13. Tucker AL, Edmondson AC. Why Hospitals Don't Learn from Failures: Organizational and Psycho- logical Dynamics that Inhibit System Change. California Management Review 2003;45(2):55-72.
  14. Render ML. Research and redesign are safer than warnings and rules. Crit Care Med 2004; 32(4):1074-5.
  15. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37(1):56-76.
  16. Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform 2003;36(1-2):23-30.
  17. Kirwan B, Ainsworth LK: A Guide to Task Analy- sis. Bristol, PA: Taylor & Francis, Inc.; 1992.
  18. Lane R, Stanton NA, Harrison D. Applying hierar- chical task analysis to medication administration errors. Appl Ergon 2006;37(5):669-79.
  19. DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using health care Failure Mode and Effect Analy- sis: the VA National Center for Patient Safety's prospective risk analysis system. Jt Comm J Qual Improv 2002;28(5):248-67, 209.
  20. Gosbee J, Anderson T. Human factors engineering design demonstrations can enlighten your RCA team. Qual Saf Health Care 2003;12(2):119-21.
  21. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, et al. Unexpected in- creased mortality after implementation of a com- mercially sold computerized physician order entry system. Pediatrics 2005;116(6):1506-12.
  22. Littlejohns P, Wyatt JC, Garvican L. Evaluating com- puterised health information systems: hard lessons still to be learnt. BMJ 2003; 326(7394):860-3.
  23. Schulman J, Kuperman GJ, Kharbanda A, Kaushal R. Discovering how to think about a hospital pa- tient information system by struggling to evaluate it: a committee's journal. J Am Med Inform Assoc 2007;14(5):537-41.
  24. Myers B. Challenges of HCI design and imple- mentation. Interactions 1994;1(1):73-83.
  25. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and impor- tance of unintended consequences related to com- puterized provider order entry. J Am Med Inform Assoc 2007;14(4):415-23.
  26. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care-an interactive sociotechnical analysis. J Am Med Inform Assoc 2007;14(5):542-9.
  27. Runyon B, Handler T, Hieb B, Shaffer V, Jonathan E, Lovelock JD, et al. Hype cycle for healthcare provider applications and systems. In: Gartner In- dustry Report ID Number G00148329 edition. Stamford, CT: Gartner Inc.; 2007.
  28. Preece J, Sharp H, Rogers Y. Interaction Design; Beyond human-computer interaction (2nd edition). New York: Wiley; 2007.
  29. Pilemalm S, Timpka T. Third generation participa- tory design in health informatics-making user par- ticipation applicable to large-scale information sys- tem projects. J Biomed Inform 2008;41(2):327-39.
  30. McGowan JJ, Cusack CM, Poon EG. Formative evaluation: a critical component in EHR implemen- tation. J Am Med Inform Assoc 2008;15(3):297-301.
  31. Westbrook JI, Braithwaite J, Georgiou A, Ampt A, Creswick N, Coiera E, et al. Multimethod evalua- tion of information and communication technolo- gies in health in the context of wicked problems and sociotechnical theory. J Am Med Inform Assoc 2007;14(6):746-55.
  32. Karsh BT. Beyond usability: designing effective tech- nology implementation systems to promote patient safety. Qual Saf Health Care 2004;13(5):388-94.
  33. Liljegren E. Usability in a medical technology con- text assessment of methods for usability evaluation of medical equipment. International Journal of In- dustrial Ergonomics 2006;36:345-52.
  34. Jaspers MW. A comparison of usability methods for testing interactive health technologies: method- ological aspects and empirical evidence. Int J Med Inform 2009;78(5):340-53.
  35. Peute LW, Jaspers MW. The significance of a usabil- ity evaluation of an emerging laboratory order entry system. Int J Med Inform 2007;76(2-3):157-68.
  36. Bernonville S, Kolski C, Leroy N, Beuscart-Zephir MC. Integrating the SE and HCI models in the human factors engineering cycle for re-engineering Computerized Physician Order Entry systems for medications: Basic principles illustrated by a case study. Int J Med Inform. In press 2008.
  37. Johnson CM, Johnson TR, Zhang J. A user-cen- tered framework for redesigning health care inter- faces. J Biomed Inform 2005;38(1):75-87.
  38. Benyon D, Turner P, Turner S. Designing interac- tive systems: People, activities, contexts. Harlow, UK: Addison-Wesley; 2005.
  39. Reid PP, Compton D, Grossman JH, Fanjiang G. Committee on Engineering and the Health Care System. Building a better delivery system: A new engineering/health care partnership. In: Washing- ton, D.C.: National Academies Press; 2005.
  40. Sharp A, McDermott P. Workflow Modeling: Tools for Process Improvement and Application Develop- ment, 2nd Edition. Artech House Publishers; 2008.
  41. Hackos JTP, Redish JC. User and Task Analysis for Interface Design. 1st ed. Wiley; 1998.
  42. Booch G, Rumbaugh J, Jacobson I. Unified Mod- eling Language User Guide, The 2nd Edition. Addison-Wesley Professional; 2005.
  43. Butler KA, Zhang J, Esposito C, Bahrami A, Hebron R, Kieras D. Work-centered design: a case study of a mixed-initiative scheduler. In: Proceedings of the SIGCHI conference on Human factors in comput- ing systems, San Jose, CA; 2007. p. 747-56.
  44. Wickens CD, Lee JD, Liu Y, Gordon Becker SE. An Introduction to Human Factors Engineering. 2nd Ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2004.
  45. Barnett GO, Cimino JJ, Hupp JA, Hoffer EP. DXplain. An evolving diagnostic decision-support system. JAMA 1987;258(1):67-74.
  46. Friedman CP, Elstein AS, Wolf FM, Murphy GC, Franz TM, Heckerling PS, et al. Enhancement of clinicians' diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. JAMA 1999:282(19):1851-6.
  47. Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF Jr, et al. A computer- assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998;338(4):232-8.
  48. Gardner RM, Cannon GH, Morris AH, Olsen KR, Price WG. Computerized Blood Gas Interpretation and Reporting System. Computer 1975;8(1):39-45.
  49. Sittig DF, Pace NL, Gardner RM, Beck E, Morris AH. Implementation of a computerized patient ad- vice system using the HELP clinical information system. Comput Biomed Res 1989;22(5):474-87.
  50. Bates DW, Pappius E, Kuperman GJ, Sittig D, Burstin H, Fairchild D, et al. Using information systems to measure and improve quality. Int J Med Inform 1999;53(2-3):115-24.
  51. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, et al. The impact of computerized phy- sician order entry on medication error prevention. J Am Med Inform Assoc 1999;6(4):313-21.
  52. Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the frequency of errors in medicine using information technol- ogy. J Am Med Inform Assoc 2001;8(4):299-308.
  53. Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, et al. Improving resi- dents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA 2000;284(11):1411-6.
  54. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of com- puterized clinical decision support systems on prac- titioner performance and patient outcomes: a system- atic review. JAMA 2005;293(10):1223-38.
  55. Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc 2005;12(4):431-7.
  56. Shea S, DuMouchel W, Bahamonde L. A meta- analysis of 16 randomized controlled trials to evalu- ate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc 1996;3(6):399-409.
  57. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national ac- tion on clinical decision support. J Am Med Inform Assoc 2007;14(2):141-5.
  58. Snyder-Halpern R. Indicators of organizational readi- ness for clinical information technology/systems innovation: a Delphi study. Int J Med Inform 2001;63(3):179-204.
  59. Hollan J, Hutchins E, Kirsh D. Distributed cogni- tion: toward a new foundation for human-computer interaction. ACM Transactions on Computer-Hu- man Interaction 2000;7(2):174-96.
  60. Hutchins E. Cognition in the wild. Cambridge, MA: MIT Press; 1995.
  61. Hutchins E. How a cockpit remembers its speed. Cognitive Science 1995;19:265-88.
  62. Norman DA. Cognitive artifacts. In Designing inter- action: Psychology at the human-computer inter- face. New York: Cambridge University Press; 1991.

Authors
Saleem, J. J., Russ, A. L., Sanderson, P., Johnson, T. R., Zhang, J., & Sittig, D. F.
Affiliations
  • Regenstrief Institute (US)
  • Health Services Research & Development (US)
  • The University of Queensland (AU)
  • The University of Texas Health Science Center at Houston (US)
  • Memorial Hermann (US)
Corresponding author
Sittig, D. F.
ORCID
Venue
Yearbook of Medical Informatics
Publisher
Georg Thieme Verlag KG
Published
2009-08
Volume
18
Issue
01
Pages
48-58
ISSN
  • 0943-4747 (ISSN-L)
  • 2364-0502
DOI
10.1055/s-0038-1638638
OpenAlex
W1493558849
Cited by
72
References
140
Citations by year
2026: 5 · 2025: 3 · 2024: 3 · 2023: 1 · 2022: 3 · 2021: 4 · 2019: 7 · 2018: 6 · 2017: 3 · 2016: 4 · 2015: 2 · 2014: 4 · 2013: 10 · 2012: 5
FWCI
11.96
Type
journal article
Language
en
Open access
Locations
  • Yearbook of Medical Informatics (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • U.S. Department of Veterans Affairs
  • U.S. National Library of Medicine (R56LM006942)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
Keywords
human factors, clinical information systems, patient safety, user interface design, workflow analysis, clinical decision support, distributed cognition, mental workload, situational awareness, clinical workflow integration, iterative design, system evaluation
MeSH terms
Decision Making, Computer-Assisted, Humans, Ergonomics, Information Systems, Task Performance and Analysis, User-Computer Interface, Patient Safety

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review