CoChat
Sign Up
Cited by 114Artificial Intelligence in Healthcare and EducationHuman-Automation Interaction and SafetyElectronic Health Records Systems

Research Trends in Artificial Intelligence Applications in Human Factors Health Care: Mapping Review

Open access
Asan, Onur, Choudhury, Avishek·2021·JMIR Human Factors
DOI 10.2196/28236PMC PMC8277302

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Despite advancements in artificial intelligence (AI) to develop prediction and classification models, little research has been devoted to real-world translations with a user-centered design approach. AI development studies in the health care context have often ignored two critical factors of ecological validity and human cognition, creating challenges at the interface with clinicians and the clinical environment. Objective: The aim of this literature review was to investigate the contributions made by major human factors communities in health care AI applications. This review also discusses emerging research gaps, and provides future research directions to facilitate a safer and user-centered integration of AI into the clinical workflow. Methods: We performed an extensive mapping review to capture all relevant articles published within the last 10 years in the major human factors journals and conference proceedings listed in the “Human Factors and Ergonomics” category of the Scopus Master List. In each published volume, we searched for studies reporting qualitative or quantitative findings in the context of AI in health care. Studies are discussed based on the key principles such as evaluating workload, usability, trust in technology, perception, and user-centered design. Results: Forty-eight articles were included in the final review. Most of the studies emphasized user perception, the usability of AI-based devices or technologies, cognitive workload, and user’s trust in AI. The review revealed a nascent but growing body of literature focusing on augmenting health care AI; however, little effort has been made to ensure ecological validity with user-centered design approaches. Moreover, few studies (n=5 against clinical/baseline standards, n=5 against clinicians) compared their AI models against a standard measure. Conclusions: Human factors researchers should actively be part of efforts in AI design and implementation, as well as dynamic assessments of AI systems’ effects on interaction, workflow, and patient outcomes. An AI system is part of a greater sociotechnical system. Investigators with human factors and ergonomics expertise are essential when defining the dynamic interaction of AI within each element, process, and result of the work system.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/28236

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Usability of and satisfaction with AI tools are generally high.

    “researchers are starting to pay attention to end users [35] and are finding that usability of and satisfaction with AI tools are generally high [35-37].”

    From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [35]

    The Results’ Summary of Included Studies reports that most patients found one AI symptom checker easy to use and that 19 of 20 children found a chatbot easy to use. These examples support positive usability findings, but the review does not establish that usability and satisfaction with AI tools are generally high across the studies it covers; its Perception, Usability, Workload, and Trust section also describes usability barriers. No page numbers were provided in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Most of these studies used quantitative methods and examined tools that have already been developed.

    “Most of these studies used quantitative methods and examined tools that have already been developed [35-37].”

    From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [35]

    The Results, Summary of Included Studies (page unavailable), says that most of the 48 studies evaluated AI from the user perspective, which supports the claim that they examined tools. It does not establish that most used quantitative methods or that the tools had already been developed; it also describes studies focused on algorithm development. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Our respondents asked for steps that could be taken in response to predictions and wanted to know that those actions would lead to better health, echoing the sentiment found in other studies.

    “Although usability and accuracy were deemed important, our respondents asked for steps that could be taken in response to predictions and wanted to know that those actions would lead to better health, echoing the sentiment found in other studies [35].”

    From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [35]

    In Results > Summary of Included Studies (page unavailable), the review reports that tailored recommendations were important to stakeholders designing a risk-prediction tool and that some patients lacked information about self-management. This supports the desire for steps to take, but our copy does not report that users wanted assurance those steps would lead to better health. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

  • Our respondents regarded the technology with skepticism, similar to others.

    “Similar to others, they also regarded the technology with skepticism [35,41,42].”

    From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [35]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 94

  1. [1]Wang P On defining artificial intelligence J Artif Gen Intell 2019 10 2 1 37 10.2478/jagi-2019-0002 · DOI · Paper page
  2. [2]Leão C Gonçalves P Cepeda T Botelho L Silva C Study of the knowledge and impact of artificial intelligence on an academic community 2018 9 25 International Conference on Intelligent Systems (IS) 2018 Funchal 891 895 10.1109/is.2018.8710477 · DOI
  3. [3]McCarthy J Hayes P Meltzer B Michie D Some philosophical problems from the standpoint of artificial intelligence Machine Intelligence 4 1981 Edinburgh, UK Edinburgh University Press 463 502
  4. [5]Helm JM Swiergosz AM Haeberle HS Karnuta JM Schaffer JL Krebs VE Spitzer AI Ramkumar PN Machine learning and artificial intelligence: definitions, applications, and future directions Curr Rev Musculoskelet Med 2020 2 13 1 69 76 10.1007/s12178-020-09600-8 31983042 31983042 PMC7083992 · DOI
  5. [6]Asan O Bayrak AE Choudhury A Artificial intelligence and human trust in healthcare: focus on clinicians J Med Internet Res 2020 6 19 22 6 e15154 10.2196/15154 32558657 32558657 PMC7334754 · DOI · Paper page
  6. [7]Choudhury A Asan O Role of artificial intelligence in patient safety outcomes: systematic literature review JMIR Med Inform 2020 7 24 8 7 e18599 10.2196/18599 32706688 32706688 PMC7414411 · DOI · Paper page
  7. [8]Ardila D Kiraly AP Bharadwaj S Choi B Reicher JJ Peng L Tse D Etemadi M Ye W Corrado G Naidich DP Shetty S End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography Nat Med 2019 6 25 6 954 961 10.1038/s41591-019-0447-x 31110349 31110349 · DOI · Paper page
  8. [9]Obermeyer Z Emanuel EJ Predicting the future - big data, machine learning, and clinical medicine N Engl J Med 2016 9 29 375 13 1216 1219 10.1056/NEJMp1606181 27682033 27682033 PMC5070532 · DOI · Paper page
  9. [10]Esteva A Kuprel B Novoa RA Ko J Swetter SM Blau HM Thrun S Dermatologist-level classification of skin cancer with deep neural networks Nature 2017 2 02 542 7639 115 118 10.1038/nature21056 28117445 28117445 PMC8382232 · DOI · Paper page
  10. [12]Kelly CJ Karthikesalingam A Suleyman M Corrado G King D Key challenges for delivering clinical impact with artificial intelligence BMC Med 2019 10 29 17 1 195 10.1186/s12916-019-1426-2 31665002 31665002 PMC6821018 · DOI · Paper page
  11. [13]Mnih V Kavukcuoglu K Silver D Graves A Antonoglou I Wierstra D Playing Atari with deep reinforcement learning arXiv preprint 2013 2021-06-15 https://arxiv.org/abs/1312.5602 · arXiv
  12. [16]Choudhury A Renjilian E Asan O Use of machine learning in geriatric clinical care for chronic diseases: a systematic literature review JAMIA Open 2020 10 3 3 459 471 10.1093/jamiaopen/ooaa034 33215079 33215079 PMC7660963 · DOI
  13. [17]Lau N Hildebrandt M Althoff T Boyle LN Iqbal ST Lee JD Mehta RK Poore JC Human in focus: future research and applications of ubiquitous user monitoring 2019 11 20 Proceedings of the Human Factors and Ergonomics Society Annual Meeting. /11/01;63(1) 2019 Philidephia 168 172 10.1177/1071181319631317 · DOI
  14. [18]Lau N Hildebrandt M Jeon M Ergonomics in AI: designing and interacting with machine learning and AI Ergon Des 2020 6 05 28 3 3 10.1177/1064804620915238 · DOI
  15. [19]Panch T Mattie H Celi LA The "inconvenient truth" about AI in healthcare NPJ Digit Med 2019 8 16 2 1 77 73 10.1038/s41746-019-0155-4 31453372 31453372 PMC6697674 · DOI · Paper page
  16. [20]Rajkomar A Dean J Kohane I Machine learning in medicine N Engl J Med 2019 4 04 380 14 1347 1358 10.1056/nejmra1814259 30943338 · DOI · Paper page
  17. [23]Holden RJ Cornet VP Valdez RS Patient ergonomics: 10-year mapping review of patient-centered human factors Appl Ergon 2020 1 82 102972 10.1016/j.apergo.2019.102972 31654954 31654954 · DOI
  18. [24]Aldape-Pérez M Yáñez-Márquez C Camacho-Nieto O López-Yáñez I Argüelles-Cruz AJ Collaborative learning based on associative models: Application to pattern classification in medical datasets Comput Hum Behav 2015 10 51 771 779 10.1016/j.chb.2014.11.091 · DOI
  19. [25]Azari DP Hu YH Miller BL Le BV Radwin RG Using surgeon hand motions to predict surgical maneuvers Hum Factors 2019 12 61 8 1326 1339 10.1177/0018720819838901 31013463 31013463 · DOI
  20. [26]Balani S De Choudhury M Detecting and characterizing mental health related self-disclosure in social media 2015 Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15 2015 Seoul 1373 1378 10.1145/2702613.2732733 · DOI
  21. [28]Cvetković J Cvetković M Investigation of the depression in breast cancer patients by computational intelligence technique Comput Hum Behav 2017 3 68 228 231 10.1016/j.chb.2016.11.042 · DOI
  22. [29]Ding X Jiang Y Qin X Chen Y Zhang W Qi L Reading Face, Reading Health 2019 Proceedings of the CHI Conference on Human Factors in Computing Systems - CHI '19 2019 Glasgow 1 13 10.1145/3290605.3300435 · DOI
  23. [30]Erebak S Turgut T Caregivers’ attitudes toward potential robot coworkers in elder care Cogn Tech Work 2018 7 24 21 2 327 336 10.1007/s10111-018-0512-0 · DOI
  24. [32]Chang M Cheung W Determinants of the intention to use Internet/WWW at work: a confirmatory study Inf Manag 2001 11 39 1 1 14 10.1016/s0378-7206(01)00075-1 · DOI
  25. [34]Gao J Tian F Fan J Wang D Fan X Zhu Y Implicit detection of motor impairment in Parkinson's disease from everyday smartphone interactions 2018 CHI Conference on Human Factors in Computing Systems 2018 Montreal 1 6 10.1145/3170427.3188502 · DOI
  26. [35]Hawkins JB Brownstein JS Tuli G Runels T Broecker K Nsoesie EO McIver DJ Rozenblum R Wright A Bourgeois FT Greaves F Measuring patient-perceived quality of care in US hospitals using Twitter BMJ Qual Saf 2016 6 25 6 404 413 10.1136/bmjqs-2015-004309 26464518 PMC4878682 26464518 · DOI
  27. [36]Hu B Kim C Ning X Xu X Using a deep learning network to recognise low back pain in static standing Ergonomics 2018 10 61 10 1374 1381 10.1080/00140139.2018.1481230 29792576 29792576 · DOI
  28. [37]Jin H Qu Q Munechika M Sano M Kajihara C Duffy VG Chen H Applying intelligent algorithms to automate the identification of error factors J Patient Saf 2018 5 03 online ahead of print 10.1097/PTS.0000000000000498 29733301 29733301 · DOI
  29. [38]Kandaswamy S Hettinger AZ Ratwani RM What did you order? Developing models to measure the impact of usability on emergency physician accuracy using computerized provider order entry 2019 11 20 Human Factors and Ergonomics Society Annual Meeting 2019 Philidephia 713 717 10.1177/1071181319631461 · DOI
  30. [39]Komogortsev O Holland C The application of eye movement biometrics in the automated detection of mild traumatic brain injury 2014 Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA '14 2014 Toronto 1711 1716 10.1145/2559206.2581150 · DOI
  31. [40]Krause J Perer A Ng K Interacting with predictions 2016 Proceedings of the CHI Conference on Human Factors in Computing Systems 2016 Montreal 5686 5697 10.1145/2858036.2858529 · DOI · Paper page
  32. [41]Ladstätter F Garrosa E Moreno-Jiménez B Ponsoda V Reales Aviles JM Dai J Expanding the occupational health methodology: A concatenated artificial neural network approach to model the burnout process in Chinese nurses Ergonomics 2016 59 2 207 221 10.1080/00140139.2015.1061141 26230967 26230967 · DOI
  33. [42]Ladstätter F Garrosa E Badea C Moreno B Application of artificial neural networks to a study of nursing burnout Ergonomics 2010 9 53 9 1085 1096 10.1080/00140139.2010.502251 20737334 20737334 · DOI
  34. [43]Lee J Cho D Kim J Im E Bak J Itchtector: A wearable-based mobile system for managing itching conditions 2017 Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems 2017 Montreal 893 905 10.1145/3025453.3025569 · DOI
  35. [44]Marella WM Sparnon E Finley E Screening electronic health record-related patient safety reports using machine learning J Patient Saf 2017 3 13 1 31 36 10.1097/PTS.0000000000000104 24721977 24721977 · DOI
  36. [45]Mazilu S Blanke U Hardegger M Tröster G Gazit E Hausdorff J GaitAssist 2014 Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems - CHI '14 2014 Toronto 2531 2540 10.1145/2556288.2557278 · DOI
  37. [46]McKnight SD Semi-supervised classification of patient safety event reports J Patient Saf 2012 6 8 2 60 64 10.1097/PTS.0b013e31824ab987 22543364 22543364 · DOI
  38. [47]Moore CR Farrag A Ashkin E Using natural language processing to extract abnormal results from cancer screening reports J Patient Saf 2017 9 13 3 138 143 10.1097/PTS.0000000000000127 25025472 25025472 PMC4294990 · DOI
  39. [48]Morrison C D'Souza M Huckvale K Dorn JF Burggraaff J Kamm CP Steinheimer SM Kontschieder P Criminisi A Uitdehaag B Dahlke F Kappos L Sellen A Usability and acceptability of ASSESS MS: assessment of motor dysfunction in multiple sclerosis using depth-sensing computer vision JMIR Hum Factors 2015 6 24 2 1 e11 10.2196/humanfactors.4129 27025782 27025782 PMC4797664 · DOI
  40. [49]Muñoz M Cobos A Campos A Low vacuum re-infusion drains after total knee arthroplasty: is there a real benefit? Blood Transfus 2014 1 12 Suppl 1 s173 s175 10.2450/2013.0236-13 24120601 24120601 PMC3934272 · DOI
  41. [50]Nobles A Glenn J Kowsari K Teachman B Barnes L Identification of imminent suicide risk among young adults using text messages 2018 SIGCHI Conference on Human Factors in Computing Systems 2018 San Fransisco 10.1145/3173574.3173987 PMC6442737 30944915 · DOI
  42. [51]Ong M Magrabi F Coiera E Automated categorisation of clinical incident reports using statistical text classification Qual Saf Health Care 2010 12 19 6 e55 10.1136/qshc.2009.036657 20724392 20724392 · DOI
  43. [52]Park A Conway M Chen AT Examining thematic similarity, difference, and membership in three online mental health communities from Reddit: a text mining and visualization approach Comput Human Behav 2018 1 78 98 112 10.1016/j.chb.2017.09.001 29456286 29456286 PMC5810583 · DOI
  44. [53]Patterson ES Hansen CJ Allen TT Yang Q Moffatt-Bruce SD Predicting mortality with applied machine learning: Can we get there? Proc Int Symp Hum Factors Ergon Healthc 2019 9 8 1 115 119 10.1177/2327857919081026 33024791 33024791 PMC7534840 · DOI
  45. [54]Pryor M Ebert D Byrne V Richardson K Jones Q Cole R Collins McLaughlin A Diagnosis behaviors of physicians and non-physicians when supported by an electronic differential diagnosis aid 2019 11 20 Proceedings of the Human Factors and Ergonomics Society Annual Meeting. /11/01;63(1) 2019 Philidephia 68 72 10.1177/1071181319631420 · DOI
  46. [55]Putnam C Cheng J Rusch D Berthiaume A Burke R Supporting therapists in motion-based gaming for brain injury rehabilitation 2013 CHI '13 Extended Abstracts on Human Factors in Computing Systems 2013 Paris 391 10.1145/2468356.2468426 · DOI
  47. [56]Sbernini L Quitadamo L Riillo F Lorenzo N Gaspari A Saggio G Sensory-glove-based open surgery skill evaluation IEEE Trans Hum Mach Syst 2018 4 48 2 213 218 10.1109/thms.2017.2776603 · DOI
  48. [57]Shiner B Neily J Mills PD Watts BV Identification of inpatient falls using automated review of text-based medical records J Patient Saf 2020 9 16 3 e174 e178 10.1097/PTS.0000000000000275 27331601 27331601 · DOI
  49. [58]Sonğur C Top M Regional clustering of medical imaging technologies Comput Hum Behav 2016 8 61 333 343 10.1016/j.chb.2016.03.056 · DOI
  50. [59]Swangnetr M Kaber D Emotional state classification in patient–robot interaction using wavelet analysis and statistics-based feature selection IEEE Trans Hum Mach Syst 2013 1 43 1 63 75 10.1109/tsmca.2012.2210408 · DOI
  51. [60]Wagland R Recio-Saucedo A Simon M Bracher M Hunt K Foster C Downing A Glaser A Corner J Development and testing of a text-mining approach to analyse patients' comments on their experiences of colorectal cancer care BMJ Qual Saf 2016 8 25 8 604 614 10.1136/bmjqs-2015-004063 26512131 26512131 · DOI
  52. [61]Wang SV Rogers JR Jin Y DeiCicchi D Dejene S Connors JM Bates DW Glynn RJ Fischer MA Stepped-wedge randomised trial to evaluate population health intervention designed to increase appropriate anticoagulation in patients with atrial fibrillation BMJ Qual Saf 2019 10 28 10 835 842 10.1136/bmjqs-2019-009367 31243156 PMC7812610 31243156 · DOI
  53. [62]Waqar M Majeed N Dawood H Daud A Aljohani N An adaptive doctor-recommender system Behav Inf Technol 2019 959 973 10.1080/0144929x.2019.1625441 · DOI
  54. [63]Xiao C Wang S Zheng L Zhang X Chaovalitwongse W A patient-specific model for predicting tibia soft tissue insertions from bony outlines using a spatial structure supervised learning framework IEEE Trans Hum Mach Syst 2016 10 46 5 638 646 10.1109/thms.2016.2545924 · DOI
  55. [64]Valik JK Ward L Tanushi H Müllersdorf K Ternhag A Aufwerber E Färnert A Johansson AF Mogensen ML Pickering B Dalianis H Henriksson A Herasevich V Nauclér P Validation of automated sepsis surveillance based on the Sepsis-3 clinical criteria against physician record review in a general hospital population: observational study using electronic health records data BMJ Qual Saf 2020 9 06 29 9 735 745 10.1136/bmjqs-2019-010123 32029574 PMC7467502 32029574 · DOI
  56. [65]Bailey S Hunt C Brisley A Howard S Sykes L Blakeman T Implementation of clinical decision support to manage acute kidney injury in secondary care: an ethnographic study BMJ Qual Saf 2020 5 03 29 5 382 389 10.1136/bmjqs-2019-009932 31796574 PMC7241968 31796574 · DOI
  57. [66]Carayon P Hoonakker P Hundt AS Salwei M Wiegmann D Brown RL Kleinschmidt P Novak C Pulia M Wang Y Wirkus E Patterson B Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study BMJ Qual Saf 2020 4 27 29 4 329 340 10.1136/bmjqs-2019-009857 31776197 PMC7490974 31776197 · DOI
  58. [67]Parekh N Ali K Davies JG Stevenson JM Banya W Nyangoma S Schiff R van der Cammen T Harchowal J Rajkumar C Medication-related harm in older adults following hospital discharge: development and validation of a prediction tool BMJ Qual Saf 2020 2 16 29 2 142 153 10.1136/bmjqs-2019-009587 31527053 PMC7045783 31527053 · DOI
  59. [68]Gilbank P Johnson-Cover K Truong T Designing for physician trust: toward a machine learning decision aid for radiation toxicity risk Ergon Design 2019 12 29 28 3 27 35 10.1177/1064804619896172 · DOI
  60. [71]Acosta J Ward N Achieving rapport with turn-by-turn, user-responsive emotional coloring Speech Commun 2011 11 53 9-10 1137 1148 10.1016/j.specom.2010.11.006 · DOI
  61. [73]Liang Y Fan H Fang Z Miao L Li W Zhang X OralCam: enabling self-examination and awareness of oral health using a smartphone camera 2020 2020 CHI Conference on Human Factors in Computing Systems 2020 Honolulu USA Association for Computing Machinery 10.1145/3313831.3376238 · DOI
  62. [74]Chatterjee S Rahman M Ahmed T Saleheen N Nemati E Nathan V Assessing severity of pulmonary obstruction from respiration phase-based wheeze-sensing using mobile sensors 2020 2020 CHI Conference on Human Factors in Computing Systems 2020 Honolulu USA Association for Computing Machinery 10.1145/3313831.3376444 · DOI
  63. [76]Phelps EA Ling S Carrasco M Emotion facilitates perception and potentiates the perceptual benefits of attention Psychol Sci 2006 4 17 4 292 299 10.1111/j.1467-9280.2006.01701.x 16623685 16623685 PMC1555625 · DOI
  64. [77]Ruotsalainen JH Verbeek JH Mariné A Serra C Preventing occupational stress in healthcare workers Cochrane Database Syst Rev 2015 4 07 4 CD002892 10.1002/14651858.CD002892.pub5 25847433 25847433 PMC6718215 · DOI
  65. [79]McVicar A Workplace stress in nursing: a literature review J Adv Nurs 2003 12 44 6 633 642 10.1046/j.0309-2402.2003.02853.x 14651686 14651686 · DOI
  66. [81]Maslach C Leiter M The truth about burnout: How organizations cause personal stress and what to do about it: 2008 Hoboken, NJ John Wiley & Sons
  67. [82]Anderson JE Ross AJ Macrae C Wiig S Defining adaptive capacity in healthcare: A new framework for researching resilient performance Appl Ergon 2020 9 87 103111 10.1016/j.apergo.2020.103111 32310111 32310111 · DOI
  68. [84]Sujan M Furniss D Grundy K Grundy H Nelson D Elliott M White S Habli I Reynolds N Human factors challenges for the safe use of artificial intelligence in patient care BMJ Health Care Inform 2019 11 26 1 e100081 10.1136/bmjhci-2019-100081 31780459 PMC7252977 31780459 · DOI · Paper page
  69. [85]Felmingham CM Adler NR Ge Z Morton RL Janda M Mar VJ The importance of incorporating human factors in the design and implementation of artificial intelligence for skin cancer diagnosis in the real world Am J Clin Dermatol 2021 3 22 22 2 233 242 10.1007/s40257-020-00574-4 33354741 33354741 · DOI · Paper page
  70. [87]Muehlematter UJ Daniore P Vokinger KN Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015–20): a comparative analysis Lancet Digital Health 2021 3 3 3 e195 e203 10.1016/s2589-7500(20)30292-2 33478929 · DOI · Paper page
  71. [88]Plsek PE Greenhalgh T Complexity science: The challenge of complexity in health care BMJ 2001 9 15 323 7313 625 628 10.1136/bmj.323.7313.625 11557716 11557716 PMC1121189 · DOI · Paper page
  72. [89]Plsek PE Wilson T Complexity, leadership, and management in healthcare organisations BMJ 2001 9 29 323 7315 746 749 10.1136/bmj.323.7315.746 11576986 11576986 PMC1121291 · DOI
  73. [90]Patel VL Zhang J Yoskowitz NA Green R Sayan OR Translational cognition for decision support in critical care environments: a review J Biomed Inform 2008 6 41 3 413 431 10.1016/j.jbi.2008.01.013 18343731 18343731 PMC2459228 · DOI
  74. [92]De Vito Dabbs A Myers BA Mc Curry KR Dunbar-Jacob J Hawkins RP Begey A Dew MA User-centered design and interactive health technologies for patients Comput Inform Nurs 2009 27 3 175 183 10.1097/NCN.0b013e31819f7c7c 19411947 19411947 PMC2818536 · DOI · Paper page
  75. [93]Schnall R Cho H Liu J Health Information Technology Usability Evaluation Scale (Health-ITUES) for usability assessment of mobile health technology: validation study JMIR Mhealth Uhealth 2018 1 05 6 1 e4 10.2196/mhealth.8851 29305343 29305343 PMC5775483 · DOI · Paper page
  76. [95]Kieras D Polson PG An approach to the formal analysis of user complexity Int J Hum Comput Stud 1999 8 51 2 405 434 10.1006/ijhc.1983.0317 · DOI
  77. [96]Schuetz S Venkatesh V The rise of human machines: how cognitive computing systems challenge assumptions of user-system interaction J Assoc Inf Syst 2020 460 482 10.17705/1jais.00608 · DOI
  78. [97]Davis F User acceptance of information technology: system characteristics, user perceptions and behavioral impacts Int J Man Machine Stud 1993 3 38 3 475 487 10.1006/imms.1993.1022 · DOI
  79. [98]Bainbridge L Ironies of automation. Analysis, design and evaluation of man-machine systems 1982 Proceedings of IFAC/IFIP/IFORS/IEA Conference 1982 Baden 151 157 10.1016/b978-0-08-029348-6.50026-9 · DOI · Paper page
  80. [99]Salvendy G Handbook of human factors and ergonomics. 4th edition 2012 Hoboken, NJ John Wiley & Sons
  81. [100]Sarter N Woods D Billings C Salvendy G Automation surprises Handbook of human factors and ergonomics 1997 Hoboken, NJ Wiley 1926 1943
  82. [101]Ruskin K Ruskin A O'Connor M Automation failures and patient safety Curr Opin Anaesthesiol 2020 12 33 6 788 792 10.1097/ACO.0000000000000935 33093302 33093302 · DOI
  83. [102]Alberdi E Povykalo A Strigini L Ayton P Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography Acad Radiol 2004 8 11 8 909 918 10.1016/j.acra.2004.05.012 15354301 15354301 · DOI · Paper page
  84. [103]Tschandl P Rinner C Apalla Z Argenziano G Codella N Halpern A Janda M Lallas A Longo C Malvehy J Paoli J Puig S Rosendahl C Soyer HP Zalaudek I Kittler H Human-computer collaboration for skin cancer recognition Nat Med 2020 8 22 26 8 1229 1234 10.1038/s41591-020-0942-0 32572267 32572267 · DOI · Paper page
  85. [105]Parasuraman R Mouloua M Molloy R Effects of adaptive task allocation on monitoring of automated systems Hum Factors 1996 12 38 4 665 679 10.1518/001872096778827279 11536753 11536753 · DOI
  86. [107]Endsley M Toward a theory of situation awareness in dynamic systems Hum Factors 2016 11 23 37 1 32 64 10.1518/001872095779049543 · DOI · Paper page
  87. [109]Kass S Cole K Stanny C Effects of distraction and experience on situation awareness and simulated driving Transport Res F Traffic Psychol Behav 2007 7 10 4 321 329 10.1016/j.trf.2006.12.002 · DOI
  88. [110]Carlesi KC Padilha KG Toffoletto MC Henriquez-Roldán C Juan MAC Patient safety incidents and nursing workload Rev Lat Am Enfermagem 2017 4 06 25 e2841 10.1590/1518-8345.1280.2841 28403334 28403334 PMC5396482 · DOI
  89. [111]Fagerström L Kinnunen M Saarela J Nursing workload, patient safety incidents and mortality: an observational study from Finland BMJ Open 2018 4 24 8 4 e016367 10.1136/bmjopen-2017-016367 29691240 PMC5922465 29691240 · DOI
  90. [112]Koch S Weir C Haar M Staggers N Agutter J Görges M Westenskow D Intensive care unit nurses' information needs and recommendations for integrated displays to improve nurses' situation awareness J Am Med Inform Assoc 2012 19 4 583 590 10.1136/amiajnl-2011-000678 22437074 22437074 PMC3384123 · DOI
  91. [113]Fairbanks R Caplan S Poor interface design and lack of usability testing facilitate medical error Joint Commiss J Qual Safety 2004 10 30 10 579 584 10.1016/s1549-3741(04)30068-7 15518362 · DOI
  92. [115]van Berkel N Clarkson MJ Xiao G Dursun E Allam M Davidson BR Blandford A Dimensions of ecological validity for usability evaluations in clinical settings J Biomed Inform 2020 10 110 103553 10.1016/j.jbi.2020.103553 32891762 32891762 · DOI
  93. [116]Liu X Faes L Kale A Wagner S Fu D Bruynseels A Mahendiran T Moraes G Shamdas M Kern C Ledsam JR Schmid Mk Balaskas K Topol EJ Bachmann LM Keane PA Denniston AK A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis Lancet Digital Health 2019 10 1 6 e271 e297 10.1016/s2589-7500(19)30123-2 33323251 · DOI · Paper page
  94. [117]van Smeden M Van Calster B Groenwold RHH Machine learning compared with pathologist assessment JAMA 2018 4 24 319 16 1725 1726 10.1001/jama.2018.1466 29710156 29710156 · DOI

Cited by

Sorted by citations

Authors
Asan, O., & Choudhury, A.
Affiliations
Stevens Institute of Technology (US)
Corresponding author
Asan, Onur
ORCID
Venue
JMIR Human Factors
Publisher
JMIR Publications Inc.
Published
2021-06-18
Volume
8
Issue
2
Pages
e28236
ISSN
2292-9495 (ISSN-L)
DOI
10.2196/28236
PMC
PMC8277302
OpenAlex
W3157986414
Cited by
114
References
13
Citations by year
2026: 31 · 2025: 20 · 2024: 26 · 2023: 19 · 2022: 15 · 2021: 3
FWCI
1.28
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Human Factors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
artificial intelligence in healthcare, human factors, user-centered design, clinical workflow integration, usability, mental workload, trust in AI, user perception, ecological validity, socio-technical systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review