CoChat
Sign Up
Cited by 27Hospital Admissions and OutcomesArtificial Intelligence in Healthcare and EducationHealthcare Technology and Patient Monitoring

Potential uses of AI for perioperative nursing handoffs: a qualitative study

Open access
King, Christopher Ryan, Shambe, Ayanna, Abraham, Joanna·2023·JAMIA Open
DOI 10.1093/jamiaopen/ooad015PMC PMC10019806
Open this paper in CoChat — free accountOpen PDF

Abstract

Objective: Situational awareness and anticipatory guidance for nurses receiving a patient after surgery are keys to patient safety. Little work has defined the role of artificial intelligence (AI) to support these functions during nursing handoff communication or patient assessment. We used interviews to better understand how AI could work in this context. Materials and Methods: Eleven nurses participated in semistructured interviews. Mixed inductive-deductive thematic analysis was used to extract major themes and subthemes around roles for AI supporting postoperative nursing. Results: Five themes were generated from the interviews: (1) nurse understanding of patient condition guides care decisions, (2) handoffs are important to nurse situational awareness, but multiple barriers reduce their effectiveness, (3) AI may address barriers to handoff effectiveness, (4) AI may augment nurse care decision making and team communication outside of handoff, and (5) user experience in the electronic health record and information overload are likely barriers to using AI. Important subthemes included that AI-identified problems would be discussed at handoff and team communications, that AI-estimated elevated risks would trigger patient re-evaluation, and that AI-identified important data may be a valuable addition to nursing assessment. Discussion and Conclusion: Most research on postoperative handoff communication relies on structured checklists. Our results suggest that properly designed AI tools might facilitate postoperative handoff communication for nurses by identifying specific elevated risks faced by a patient, triggering discussion on those topics. Limitations include a single center, many participants lacking of applied experience with AI, and limited participation rate.

AI snapshot

Written from the paper’s full text.

TL;DR

In interviews with 11 postoperative nurses at a single academic center, nurses were receptive to AI that identifies elevated patient risks and relevant EHR data to focus handoff communication and post-handoff assessment. They also warned that EHR usability and information overload would be barriers, and preferences for AI presentation varied.

Key findings

  • Five themes were generated: nurse understanding of patient condition guides care decisions; handoffs are important for situational awareness but have barriers; AI may address handoff barriers; AI may augment care decision-making and team communication outside handoff; and EHR user experience and information overload are likely barriers to AI use.
  • Almost all participants agreed that if AI identified a patient at high risk for a complication, that topic would be prioritized at handoff, and receiving nurses would ask follow-up questions about patient state and plan.
  • Participants believed AI-identified elevated risks could prompt closer review of data, trigger patient re-evaluation, support nurse-driven interventions such as fall, delirium, and pneumonia prevention, and help allocate resources such as high-dependency beds.
  • Participants did not agree on a single presentation of AI predictions; they variously endorsed absolute risk estimates, relative risks, simplified high-medium-low flags, and plots, and noted existing alerts already cause alarm fatigue and that complex outputs could cause information overload.

Methods

  • Single-center qualitative study at Barnes-Jewish Hospital, a 1400-bed academic medical center; the Acute and Critical Care Surgery division performs approximately 1600 inpatient surgeries annually, and the PACU is a 30-bed area.
  • Eleven nurses participated in semistructured interviews: 7 PACU nurses and 4 ward nurses; direct observations of handoffs were also conducted to provide context but were not separately reported.
  • Interviews were double-coded using mixed inductive-deductive reflexive thematic analysis; after 10 interviews most topics were addressed by multiple participants, and no new topics appeared in the 11th interview, so recruitment stopped.

Limitations

  • The paper states: single center; ward nurses worked in a small number of units; low number of participants and low recruitment rate; participants had limited experience with AI; academic medical center setting may not reflect other settings; semistructured interviews with participants informed of the study may have led to agreeable responses, though participants seemed free to disagree.
  • Evident from the text: no specific AI product or technical implementation was evaluated, and the observation findings were not separately reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/jamiaopen/ooad015

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

36 of 72 cited sources checked · 16 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Handoff-related information gaps are common for postoperative patients.

    “Nevertheless, handoff-related information gaps are common for postoperative patients.10,25–28”
    Quote in the paperSubstantial claim1 of 5 sources checked
    • cites [10] Wheeler DS, Sheets AM, Ryckman FC. Improving transitions of care between the operating room and intensive care unit. Transl Pediatr 2018; 7 (4): 299– 307. 30460182 PMC6212379 source
    • cites [25] Siddiqui N, Arzola C, Iqbal M, et al Deficits in information transfer between anaesthesiologist and postanaesthesia care unit staff: an analysis of patient handover. Eur J Anaesthesiol 2012; 29 (9): 438– 45. 22569028 source
    • cites [26] Milby A, Böhmer A, Gerbershagen MU, Joppich R, Wappler F. Quality of post-operative patient handover in the post-anaesthesia care unit: a prospective analysis. Acta Anaesthesiol Scand 2014; 58 (2): 192– 7. 24355063 source
    • cites [27] Halladay ML, Thompson JA, Vacchiano CA. Enhancing the quality of the anesthesia to postanesthesia care unit patient transfer through use of an electronic medical record-based handoff tool. J Perianesth Nurs 2019; 34 (3): 622– 32. 30528308 source
    • cites [28] Pucher PH, Johnston MJ, Aggarwal R, Arora S, Darzi A. Effectiveness of interventions to improve patient handover in surgery: a systematic review. Surgery 2015; 158 (1): 85– 95. 25999255 source
  • Premisecitation

    Incomplete or inaccurate handoffs are a source of subsequent medical errors and patient injury, particularly for patients undergoing major surgery.

    “Incomplete or inaccurate handoffs are a source of subsequent medical errors and patient injury,1–3 particularly for patients undergoing major surgery.4–7”
    Quote in the paperSubstantial claim1 of 7 sources checked
    • cites [1] Keebler JR, Lazzara EH, Patzer BS, et al Meta-analyses of the effects of standardized handoff protocols on patient, provider, and organizational outcomes. Hum Factors 2016; 58 (8): 1187– 205. 27821676 source
    • cites [2] Solet DJ, Norvell JM, Rutan GH, Frankel RM. Lost in translation: challenges and opportunities in physician-to-physician communication during patient handoffs. Acad Med 2005; 80 (12): 1094– 9. 16306279 source
    • cites [3] Humphrey KE, Sundberg M, Milliren CE, Graham DA, Landrigan CP. Frequency and nature of communication and handoff failures in medical malpractice claims. J Patient Saf 2022; 18 (2): 130– 7. 35188927 source
    • cites [4] Segall N, Bonifacio AS, Schroeder RA, et al; on behalf of the Durham VA Patient Safety Center of Inquiry. Can we make postoperative patient handovers safer? A systematic review of the literature. Anesth. Analg 2012; 115 (1): 102– 15. 22543067 PMC6152818 source

      The Results and Conclusions (page not available) link poor postoperative handovers with adverse outcomes, but explicitly say that causality cannot be established. The review covers surgical patients generally; it does not establish that the risk is particular to patients undergoing major surgery. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [5] Nagpal K, Arora S, Abboudi M, et al Postoperative handover: problems, pitfalls, and prevention of error. Ann Surg 2010; 252 (1): 171– 6. 20505507 source
    • cites [6] Nagpal K, Arora S, Vats A, et al Failures in communication and information transfer across the surgical care pathway: interview study. BMJ Qual Saf 2012; 21 (10): 843– 9. 22773891 source
    • cites [7] Douglas RN, Stephens LS, Posner KL, et al Communication failures contributing to patient injury in anaesthesia malpractice claims. Br J Anaesth 2021; 127 (3): 470– 8. 34238547 PMC8563338 source
  • Findingcitation

    Similar to others, we found that extraction of directly interpretable patient data and actionable needs was a high priority.

    “Similar to others,73–75 we found that extraction of directly interpretable patient data and actionable needs was a high priority (Subtheme 4.d).”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites [73] Schwartz JM, George M, Rossetti SC, et al Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study. JMIR Hum Factors 2022; 9 (2): e33960. 35550304 PMC9136656 source

      In Results > Inductive Coding (page unavailable), clinicians wanted to see patient-specific data behind predictions, and some wanted a clear recommendation for what action to take. The paper does not describe extracting those data and needs as a high priority; its Discussion > Implications for Research, Practice, and Policy says preferences for recommended actions varied. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [74] Barda AJ, Horvat CM, Hochheiser H. A qualitative research framework for the design of user-centered displays of explanations for machine learning model predictions in healthcare. BMC Med Inform Decis Mak 2020; 20 (1): 257. 33032582 PMC7545557 source
    • cites [75] Helman S, Terry MA, Pellathy T, et al Engaging clinicians early during the development of a graphical user display of an intelligent alerting system at the bedside. Int J Med Inf 2022; 159: 104643. PMC9040820 34973608 source
  • Premisecitation

    ML identification of patients with a high risk of pressure ulcers or falls can trigger CDS for nursing interventions.

    “For example, ML identification of patients with a high risk of pressure ulcers51,52 or falls53 can trigger CDS for nursing interventions.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [51] Ting JJ, Garnett A. E-health decision support technologies in the prevention and management of pressure ulcers: a systematic review. Comput Inform Nurs 2021; 39 (12): 955– 73. 34132227 source
    • cites [52] Hu Y-H, Lee Y-L, Kang M-F, Lee P-J. Constructing inpatient pressure injury prediction models using machine learning techniques. Comput Inform Nurs 2020; 38 (8): 415– 23. 32205474 source
    • cites [53] Lindberg DS, Prosperi M, Bjarnadottir RI, et al Identification of important factors in an inpatient fall risk prediction model to improve the quality of care using EHR and electronic administrative data: a machine-learning approach. Int J Med Inf 2020; 143: 104272. PMC8562928 32980667 source
  • Interpretationcitation

    Hunter and Forbes's work suggests a distinct role for AI prediction from traditional CDS: facilitating problem-based report and assessment during handoffs.

    “Hunter and Forbes's work56,63,64 suggests a distinct role for AI prediction from traditional CDS: facilitating problem-based report and assessment during handoffs.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [56] Forbes A, Surdeanu M, Jansen P, Carrington J. Transmitting narrative: an interactive shift-summarization tool for improving nurse communication. In: proceedings of 3rd IEEE Workshop on Interactive Visual Text Analytics. 2013. http://clulab.cs.arizona.edu/papers/textvis2013.pdf. Accessed July 7, 2021.

      Not checked in the free run.

    • cites [63] Hunter J, Freer Y, Gatt A, Reiter E, Sripada S, Sykes C. Automatic generation of natural language nursing shift summaries in neonatal intensive care: BT-Nurse. Artif Intell Med 2012; 56 (3): 157– 72. 23068882 source

      The Conclusions (p. 15) say BT-Nurse generates shift summaries that include current status and potential problems, supporting its role in handoff reporting. The paper does not present BT-Nurse as an AI prediction system or establish a distinct predictive role from traditional clinical decision support; its Computerised shift handover section (p. 2) describes the summaries as complementing other handover communication. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [64] Chetta A, Carrington JM, Forbes AG. Augmenting EHR interfaces for enhanced nurse communication and decision making. In: proc. 2015 Workshop Visual Analytics in Healthcare. 2015; Association for Computing Machinery: 1– 6; Chicago, IL. source
  • May combine two claims. 19
  • Classified as probably not a substantial claim. 36

  • May combine two claims.

    Premisecitation

    The data surrounding surgical patients are voluminous and diverse while simultaneously incomplete, which strains the ability of receiving nurses to review and assimilate it de novo.

    “The data surrounding surgical patients are voluminous and diverse while simultaneously incomplete, which strains the ability of receiving nurses to review and assimilate it de novo.8–10”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [8] Hughes HK, Serwint JR, O'Toole JK, Spector ND, Ngo TL. I-PASS adherence and implications for future handoff training. J Grad Med Educ 2019; 11 (3): 301– 6. 31210861 PMC6570451 source

      The Background and Conclusion (pages unavailable) address I-PASS use during pediatric resident handoffs, not receiving nurses reviewing surgical-patient data. The Results mention complex patients and time constraints, but do not address the claimed volume, diversity, or incompleteness of surgical-patient data. Searches for surgical patients, receiving nurses, and incomplete handoff information found no supporting passage. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [9] Hughes TM, Dossett LA, Hawley ST, Telem DA. Recognizing heuristics and bias in clinical decision-making. Ann Surg 2020; 271 (5): 813– 4. 31850992 source
    • cites [10] Wheeler DS, Sheets AM, Ryckman FC. Improving transitions of care between the operating room and intensive care unit. Transl Pediatr 2018; 7 (4): 299– 307. 30460182 PMC6212379 source

      The Review Article section says OR-to-ICU teams must communicate complete, accurate patient information while continuing to provide care. The Conclusions section also describes ICU nurses needing clarification of missing information after a hand-off. The paper does not say that surgical-patient data are voluminous and diverse or that reviewing and assimilating those data de novo strains receiving nurses. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    AI using EHR data has become much more general and accurate in the last few years, allowing prediction of perioperative events and learning effective treatment strategies.

    “AI using EHR data has become much more general and accurate in the last few years,35,36 allowing prediction of perioperative events37–41 and learning effective treatment strategies.42”
    Quote in the paperNot a substantial claim7 of 8 sources checked
    • cites [35] Yang X, Chen A, PourNejatian N, et al A large language model for electronic health records. NPJ Digit Med 2022; 5 (1): 194. 36572766 PMC9792464 source
    • cites [36] Singhal K, Azizi S, Tu T et al Large language models encode clinical knowledge. 2022. Preprint at 10.48550/arXiv.2212.13138. source

      The Abstract (page unavailable) describes recent advances that make medical language models usable across diverse tasks and reports improved accuracy on medical question-answering benchmarks. It does not establish that AI using EHR data has become more general or accurate. The later claims about perioperative prediction and treatment strategies are attached to other citations, not this source. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [37] Ke JXC, McIsaac DI, George RB, et al Postoperative mortality risk prediction that incorporates intraoperative vital signs: development and internal validation in a historical cohort. J Can Anesth 2022; 69 (9): 1086– 98. 35996071 source
    • cites [38] Fritz BA, Cui Z, Zhang M, et al Deep-learning model for predicting 30-day postoperative mortality. Br J Anaesth 2019; 123 (5): 688– 95. 31558311 PMC6993109 source
    • cites [39] Mathis MR, Engoren MC, Williams AM, et al; BCIL Collaborators Group. Prediction of postoperative deterioration in cardiac surgery patients using electronic health record and physiologic waveform data. Anesthesiology 2022; 137 (5): 586– 601. 35950802 PMC10227693 source
    • cites [40] Chiew CJ, Liu N, Wong TH, Sim YE, Abdullah HR. Utilizing machine learning methods for preoperative prediction of postsurgical mortality and intensive care unit admission. Ann Surg 2020; 272 (6): 1133– 9. 30973386 PMC7668340 source
    • cites [41] Nakatani H, Nakao M, Uchiyama H, Toyoshiba H, Ochiai C. Predicting inpatient falls using natural language processing of nursing records obtained from Japanese Electronic Medical Records: case-control study. JMIR Med Inform 2020; 8 (4): e16970. 32319959 PMC7203618 source

      The Discussion (Principal Results; page unavailable) says recent machine-learning developments have broadened applications, and the Results report accurate prediction of inpatient falls from nursing records. The paper does not establish that EHR-based AI has become more accurate generally over the last few years. The perioperative-event and treatment-strategy claims carry other citations, so they are not attributed to this source. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [42] Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med 2018; 24 (11): 1716– 20. 30349085 source

  • May combine two claims.

    Premisecitation

    In our work on OR to ICU handoffs, participants endorsed similar desires to integrate AI into summaries of patient data like laboratory results and vital signs and the need to focus on actionability.

    “In our work on OR to ICU handoffs,66 participants endorsed similar desires to integrate AI into summaries of patient data like laboratory results and vital signs and the need to focus on actionability.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [66] Abraham J, King CR, Meng A. Ascertaining design requirements for postoperative care transition interventions. Appl Clin Inform 2021; 12 (1): 107– 15. 33626584 PMC7904383 source

      The Conclusion (page unavailable) proposes an AI-generated patient handoff summary, and “Flexible Elements and Machine Learning-Generated Risk Predictions” reports that most residents and nurses wanted recent vital signs included. The Discussion links patient-specific risks to anticipatory guidance and care planning. But the copy does not establish that participants endorsed AI-generated summaries specifically of laboratory results and vital signs; it also reports mixed views on whether AI-generated risks would affect care. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Home care nurses in a prior study expressed a similar use case for AI to modify the intensity of their services but did not discuss its role in transitions of care.

    “Home care nurses in a prior study expressed a similar use case for AI to modify the intensity of their services but did not discuss its role in transitions of care.71”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [71] Dowding DW, Russell D, Onorato N, Merrill JA. Technology solutions to support care continuity in home care: a focus group study. J Healthc Qual 2018; 40 (4): 236– 46. 28885241 PMC5832509 source

      The available indexed passages contain the title and publication metadata, but no account of what the focus-group nurses said. Searches for AI, service intensity, visit frequency, and transitions of care found no passage addressing the claim, so the attribution cannot be verified from our copy. (LLM, conf=0.70)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    AI is able to interpret nursing documentation to recognize patient types and predict clinical deterioration.

    “AI is able to interpret nursing documentation to recognize patient types and predict clinical deterioration.43–47”
    Quote in the paperSubstantial claim4 of 5 sources checked
    • cites [43] Korach ZT, Cato KD, Collins SA, et al Unsupervised machine learning of topics documented by nurses about hospitalized patients prior to a rapid-response event. Appl Clin Inform 2019; 10 (5): 952– 63. 31853936 PMC6920051 source

      The Outcome Calculation section says unsupervised machine learning extracted clinical concepts from nursing notes for a model of rapid-response events. The Discussion says those topics improved prediction of rapid-response-event hazard beyond vital signs alone. The paper identifies concepts in notes, not patient types, and studies rapid-response events as the outcome rather than clinical deterioration broadly. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [44] Korach ZT, Yang J, Rossetti SC, et al Mining clinical phrases from nursing notes to discover risk factors of patient deterioration. Int J Med Inf 2020; 135: 104053. PMC7103062 31884312 source

      The Discussion reports that phrases mined from nursing notes can be used to predict rapid-response event risk, supporting the deterioration-prediction part of the claim. The paper describes identifying phrases and risk factors, not recognizing patient types. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [45] Rossetti SC, Knaplund C, Albers D, et al Leveraging clinical expertise as a feature - not an outcome - of predictive models: evaluation of an early warning system use case. AMIA Annu Symp Proc 2019; 2019: 323– 32. 32308825 PMC7153052
    • cites [46] Rossetti SC, Knaplund C, Albers D, et al Healthcare process modeling to phenotype clinician behaviors for exploiting the signal gain of clinical expertise (HPM-ExpertSignals): development and evaluation of a conceptual framework. J Am Med Inform Assoc 2021; 28 (6): 1242– 51. 33624765 PMC8200261 source

      The Methods section (page unavailable) says the CONCERN model uses natural language processing of nursing notes and comments and assigns patients low, moderate, or high risk scores. The Results section (page unavailable) reports that those scores predict adverse clinical events. The paper supports interpreting nursing documentation to predict deterioration, but it describes risk categories and phenotypes of clinician behavior, not recognition of patient types. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [47] Cato KD, McGrow K, Rossetti SC. Transforming clinical data into wisdom: artificial intelligence implications for nurse leaders. Nurs Manag (Harrow) 2020; 51 (11): 24– 30. PMC8018525 33086364 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several small studies from other nursing contexts have found similar themes.

    “Several small studies from other nursing contexts have found similar themes.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [71] Dowding DW, Russell D, Onorato N, Merrill JA. Technology solutions to support care continuity in home care: a focus group study. J Healthc Qual 2018; 40 (4): 236– 46. 28885241 PMC5832509 source

      The available copy identifies Dowding et al. as a focus group study in home care, but the passages returned contain only bibliographic information, not findings. Searches for small studies in other nursing contexts and for similar themes found no passage addressing the claim. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
    • cites [56] Forbes A, Surdeanu M, Jansen P, Carrington J. Transmitting narrative: an interactive shift-summarization tool for improving nurse communication. In: proceedings of 3rd IEEE Workshop on Interactive Visual Text Analytics. 2013. http://clulab.cs.arizona.edu/papers/textvis2013.pdf. Accessed July 7, 2021.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [64] Chetta A, Carrington JM, Forbes AG. Augmenting EHR interfaces for enhanced nurse communication and decision making. In: proc. 2015 Workshop Visual Analytics in Healthcare. 2015; Association for Computing Machinery: 1– 6; Chicago, IL. source

  • May combine two claims.

    Premisecitation

    In the authors' previous study at OR to ICU handoff, key findings were difficulty of making EHR information universally accessible, need to focus on AI with direct relevance to patient care, and general acceptance of blending AI risk prediction with current summaries of patient data into a handoff tool.

    “Key findings of that study were the difficulty of making EHR information universally accessible, the need to focus on AI with direct relevance to patient care, and general acceptance of blending AI risk prediction with current summaries of patient data into a handoff tool.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [65] Abraham J, Bartek B, Meng A, et al Integrating machine learning predictions for perioperative risk management: towards an empirical design of a flexible-standardized risk assessment tool. J Biomed Inform 2023; 137: 104270. 36516944 source
    • cites [66] Abraham J, King CR, Meng A. Ascertaining design requirements for postoperative care transition interventions. Appl Clin Inform 2021; 12 (1): 107– 15. 33626584 PMC7904383 source

      The Conclusion (page unavailable) proposes an EHR-integrated handoff summary with AI-generated risks that is accessible to sending and receiving teams, and the Discussion emphasizes information relevant to postoperative care. But the study does not establish a general acceptance of AI risk predictions: the Abstract reports divergent views, and the “Flexible Elements and Machine Learning-Generated Risk Predictions” section says nurses and fellows did not expect the risk information to affect their care. The “Intervention Modality and Access Considerations” section notes occasional lack of computer access, rather than a general difficulty making EHR information universally accessible. (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Nurse users largely accepted a prototype system for shift change in the neonatal intensive care unit which focused on summarizing data in natural language and included expert decision rules as a minor component.

    “Nurse users largely accepted a prototype system for shift change in the neonatal intensive care unit which focused on summarizing data in natural language and included expert decision rules as a minor component.63”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [63] Hunter J, Freer Y, Gatt A, Reiter E, Sripada S, Sykes C. Automatic generation of natural language nursing shift summaries in neonatal intensive care: BT-Nurse. Artif Intell Med 2012; 56 (3): 157– 72. 23068882 source

      The abstract (p. 1) reports that nurses rated most generated summaries understandable, accurate and helpful, supporting a broadly positive reception of the NICU shift-handover prototype. Section 3.2.6 (p. 8) describes medical-knowledge if–then rules used in data interpretation, but the paper does not establish that expert decision rules were a minor component. The evaluation measured nurses’ ratings of partial summaries, not acceptance of a deployable system. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Expanding nursing engagement in design of AI projects is a recognized priority.

    “Expanding nursing engagement in design of AI projects is a recognized priority,61”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [61] Ronquillo CE, Peltonen L-M, Pruinelli L, et al Artificial intelligence in nursing: priorities and opportunities from an international invitational think-tank of the Nursing and Artificial Intelligence Leadership Collaborative. J Adv Nurs 2021; 77 (9): 3707– 17. 34003504 PMC7612744 source

  • May combine two claims.

    Premisecitation

    Very few AI studies have gone beyond initial development phases or shown benefits to stakeholders.

    “Very few AI studies have gone beyond initial development phases or shown benefits to stakeholders,49,50”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [49] Seibert K, Domhoff D, Bruch D, et al Application scenarios for artificial intelligence in nursing care: rapid review. J Med Internet Res 2021; 23 (11): e26522. 34847057 PMC8669587 source

      The Discussion’s Principal Findings state that only a few publications went beyond proof-of-concept studies or laboratory experiments, and few assessed clinical or organizational outcomes. The review is limited to AI in nursing care; it identifies limited evidence about benefits but does not establish the same claim for AI studies generally or specifically quantify benefits shown to stakeholders. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [50] von Gerich H, Moen H, Block LJ, et al Artificial intelligence-based technologies in nursing: a scoping literature review of the evidence. Int J Nurs Stud 2022; 127: 104153. 35092870 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some implementation studies have found that AI-based alerts are relatively more salient to nurses than physicians in this regard.

    “Some implementation studies have found that AI-based alerts are relatively more salient to nurses than physicians in this regard.79”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [79] Ginestra JC, Giannini HM, Schweickert WD, et al Clinician perception of a machine learning-based early warning system designed to predict severe sepsis and septic shock. Crit. Care Med 2019; 47: 1477– 84. 31135500 PMC6791738 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Current methods of AI explainability have been found to have limited usefulness in practice.

    “Physicians and nurses rate explainability in terms of patient data and personal understanding as highly related to trust in AI;73,77 however, current methods of AI explainability have been found to have limited usefulness in practice.78”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [78] Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health 2021; 3 (11): e745– 50. 34711379 source

      The Conclusions say current explainability methods are unlikely to help with patient-level clinical decisions or justify trust in individual AI recommendations. But they also call the methods “incredibly useful” for model troubleshooting and systems audits, so the unqualified claim of limited usefulness in practice is broader than the paper supports. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Physicians and nurses rate explainability in terms of patient data and personal understanding as highly related to trust in AI.

    “Physicians and nurses rate explainability in terms of patient data and personal understanding as highly related to trust in AI;73,77 however, current methods of AI explainability have been found to have limited usefulness in practice.78”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [73] Schwartz JM, George M, Rossetti SC, et al Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study. JMIR Hum Factors 2022; 9 (2): e33960. 35550304 PMC9136656 source

      The Results (Deductive Coding and Inductive Coding; page not available) identify understandability as an important influence on trust and report that clinicians wanted explanations tied to individual patients’ data. The study interviewed clinicians about a predictive decision-support system for in-hospital deterioration; it did not have physicians and nurses rate explainability or establish how highly it relates to trust in AI generally. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [77] Diprose WK, Buist N, Hua N, Thurier Q, Shand G, Robinson R. Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator. J Am Med Inform Assoc 2020; 27 (4): 592– 600. 32106285 PMC7647292 source

  • May combine two claims.

    Findingown data

    Participants acknowledged barriers where the documentation they relied on was incomplete, the handoff-giver did not know the relevant information, or they did not understand what needed to be conveyed.

    “However, they acknowledged barriers where the documentation they relied on was incomplete (Subtheme 2.d), the handoff-giver did not know the relevant information, or they did not understand what needed to be conveyed.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants believed that well-functioning AI risk assessment would lead to allocating resources (such as high-dependency beds) more efficiently.

    “Yes, participants believed that well-functioning AI risk assessment would lead to activating nurse-driven interventions, allocating resources (such as high-dependency beds) more efficiently, and prioritizing monitoring for higher-risk outcomes.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    AI-based pattern matching would make the nurse's holistic view of patient risk more concrete and easier to request midlevel clinicians act on or personally evaluate.

    “Second, the nurse's holistic view of patient risk might be difficult to communicate, and AI-based pattern matching would make this more concrete and easier to request midlevel clinicians act on or personally evaluate.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Automatic identification of EHR data elements which increased the patients' risk could mitigate data omissions, especially if that data was in an unusual location.

    “Finally, automatic identification of EHR data elements which increased the patients’ risk could mitigate data omissions, especially if that data was in an unusual location (Subtheme 3.d).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    AI-identified risks for adverse events would allow the nurse to better target their assessment and monitoring independent of any effect on handoff.

    “Additionally, AI-identified risks for adverse events would allow the nurse to better target their assessment and monitoring independent of any effect on handoff (Subtheme 4.d).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Awareness that a patient was overall high-risk would prompt nurses to closely review all available data and prioritize shared careful patient evaluation.

    “Third, awareness that a patient was overall high-risk would prompt nurses to closely review all available data and prioritize shared careful patient evaluation (Subtheme 3.c).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Almost all participants agreed that if AI identified a patient at high risk for a complication, that topic would be prioritized for discussion at handoff, and those receiving handoff would ask follow-up questions regarding the patient state and the current plan.

    “First, almost all participants agreed that if AI identified a patient at high risk for a complication, that this topic would be prioritized for discussion at handoff, and that those receiving handoff would ask follow-up questions regarding the patient state and the current plan (Subtheme 3.a).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    AI-identified elevated risks could allow participants to target interventions within their scope of practice, such as fall prevention, delirium prevention, and pneumonia prevention.

    “Participants noted that AI-identified elevated risks could allow them to target interventions within their scope of practice, such as fall prevention, delirium prevention, and pneumonia prevention (Subtheme 4.d).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Several participants noted that existing clinical decision support and alerts already generate alarm fatigue, and that additional flags would likely be ignored unless they had high value.

    “Several participants noted that existing clinical decision support and alerts already generate alarm fatigue, and that additional flags would likely be ignored unless they had high value (Subtheme 5.c).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants stressed that their bedside presence allowed rapid detection and hopefully mitigation of complications.

    “Participants stressed that their bedside presence allowed rapid detection and hopefully mitigation of complications.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants believed that AI which identified patients at elevated risk would increase situational awareness.

    “Yes, participants believed that AI which identified patients at elevated risk would lead to focused handoff communication and physician-nurse team communication on those topics, increasing anticipatory guidance and situational awareness.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Several participants stated that although almost all treatment changes required a team discussion, their recommendations were likely to be considered or acted on.

    “Several participants stated that although almost all treatment changes required a team discussion, their recommendations were likely to be considered or acted on.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants believed that well-functioning AI risk assessment would lead to activating nurse-driven interventions.

    “Yes, participants believed that well-functioning AI risk assessment would lead to activating nurse-driven interventions, allocating resources (such as high-dependency beds) more efficiently, and prioritizing monitoring for higher-risk outcomes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants universally agreed that their understanding of the issues facing a patient modified what signs and symptoms they were alert for, what issues they communicated to the PACU or midlevel clinician, and what treatments they recommended.

    “They universally agreed that their understanding of the issues facing a patient modified what signs and symptoms they were alert for (Subtheme 1.b), what issues they communicated to the PACU or midlevel clinician (Subtheme 1.c), and what treatments they recommended.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    It was frequent for participants to describe receiving handoffs focusing on details they found to be irrelevant or unintelligible, and for handoff participants to not value topics on which their counterparty asked questions.

    “It was frequent for participants to describe receiving handoffs focusing on details they found to be irrelevant or unintelligible, and for handoff, participants to not value topics on which their counterparty asked questions (Subtheme 2.c).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants believed that well-functioning AI risk assessment would lead to prioritizing monitoring for higher-risk outcomes.

    “Yes, participants believed that well-functioning AI risk assessment would lead to activating nurse-driven interventions, allocating resources (such as high-dependency beds) more efficiently, and prioritizing monitoring for higher-risk outcomes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants agreed that problem-focused handoffs with anticipatory guidance were extremely useful, but that many topics in handoffs were not relevant or recited data without context.

    “Participants agreed that problem-focused handoffs with anticipatory guidance were extremely useful, but that many topics in handoffs were not relevant or recited data without context (Subtheme 2.b).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    A high calculated risk could alert participants that a known comorbidity was more severe than they expected, which was information frequently absent from documentation.

    “Second, a high calculated risk could alert them that a known comorbidity was more severe than they expected (Subtheme 3.b), which was information frequently absent from documentation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    If a patient had been identified as high risk, resistance to contacting midlevel clinicians to discuss that topic would be lowered.

    “First, if a patient had been identified as high risk, the resistance to contacting the midlevel clinicians to discuss that topic would be lowered (Subtheme 4.b).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Findingown data

    AI identification of alternative key data would be valuable.

    “AI identification of alternative key data would then be valuable.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Existing EHR handoff summaries were too rigid and incomplete to be useful and interfered with the receiving nurse's encoding of information via note taking.

    “Staggers et al31 found that existing EHR handoff summaries were too rigid and incomplete to be useful; additionally, they interfered with the receiving nurse's encoding of information via note taking.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Staggers N, Clark L, Blaz JW, Kapsandoy S. Why patient summaries in electronic health records do not provide the cognitive support necessary for nurses’ handoffs on medical and surgical units: insights from interviews and observations. Health Informatics J 2011; 17 (3): 209– 23. 21937463 source

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The ICU shift-change and OR-ICU handoffs previously studied are quite different from the OR-PACU-ward transition.

    “However, the ICU shift-change and OR-ICU handoffs previously studied are quite different from the OR-PACU-ward transition.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Many participants lacked applied experience with AI.

    “Limitations include a single center, many participants lacking of applied experience with AI, and limited participation rate.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prior work has found that nurses more frequently use a "bottom-up" (data and needs first) approach to patient summarization.

    “Prior work has also found that nurses more frequently use a “bottom-up” (data and needs first) approach to patient summarization,76 which agrees with our finding of specific risk-increasing data and conditions being important for handoff support (Subtheme 3.c).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [76] Kannampallil T, Jones S, Abraham J. ‘This is our liver patient…’: use of narratives during resident and nurse handoff conversations. BMJ Qual Saf 2020; 29 (2): 135– 41. 31270253 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Authors King and Shambe conducted interviews using the same guide.

    “Authors King and Shambe conducted interviews using the same guide (Supplementary Appendix S2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study was approved by Washington University IRB.

    “We conducted interviews under Washington University IRB approval (#201812137 and #202009066) with the consent of the participant.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Two researchers double-coded interviews using a mixed inductive-deductive reflexive thematic analysis approach.

    “Two researchers (King and Shambe) double-coded interviews using a mixed inductive-deductive reflexive thematic analysis approach.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Our findings can also be related to work with dashboards intended to detect change in patient status which lack explicit AI predictions.

    “Our findings can also be related to work with dashboards intended to detect change in patient status which lack explicit AI predictions.54”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54] Dunn Lopez K, Gephart SM, Raszewski R, Sousa V, Shehorn LE, Abraham J. Integrative review of clinical decision support for registered nurses in acute care settings. J Am Med Inform Assoc 2017; 24 (2): 441– 50. 27330074 PMC7651925 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    At each stage, coders compared codes and resolved disagreements.

    “At each stage, coders compared codes and resolved disagreements.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The ability to easily integrate AI into multiple EHR workflows and choose a personalized presentation will be necessary for AI to succeed.

    “The ability to easily integrate AI into multiple EHR workflows and choose a personalized presentation will be necessary for it to succeed.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Forbes and colleagues envisioned a dynamic EHR integrated shift-report summary for nurses including key data, diagnoses, and predicted adverse events.

    “Forbes and colleagues56,64 envisioned a dynamic EHR integrated shift-report summary for nurses including key data, diagnoses, and predicted adverse events.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [56] Forbes A, Surdeanu M, Jansen P, Carrington J. Transmitting narrative: an interactive shift-summarization tool for improving nurse communication. In: proceedings of 3rd IEEE Workshop on Interactive Visual Text Analytics. 2013. http://clulab.cs.arizona.edu/papers/textvis2013.pdf. Accessed July 7, 2021.

      Not checked in the free run.

    • cites [64] Chetta A, Carrington JM, Forbes AG. Augmenting EHR interfaces for enhanced nurse communication and decision making. In: proc. 2015 Workshop Visual Analytics in Healthcare. 2015; Association for Computing Machinery: 1– 6; Chicago, IL. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    After construction of the coding tree, coders checked statements to validate their applicability to the higher-level themes.

    “After the construction of the coding tree, coders checked statements to validate their applicability to the higher-level themes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The coders and a third researcher reviewed and revised themes.

    “The coders and a third researcher (Abraham) reviewed and revised themes.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study did not evaluate a specific AI product or technical implementation.

    “We intend these findings to guide subsequent design and implementation efforts, but we did not evaluate a specific AI product or technical implementation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Interviews were transcribed verbatim.

    “We conducted interviews over the phone or voice application with audio recording, which was transcribed verbatim.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    The number of participants and recruitment rate from those potentially eligible were both low.

    “The number of participants and recruitment rate from those potentially eligible were both low.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Participants seemed to feel free to disagree.

    “They may have endorsed ideas to be agreeable, but participants seemed to feel free to disagree.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The setting was an academic medical center, so the views may not reflect the experiences of those outside this type of setting.

    “The setting was an academic medical center, so the views may not reflect the experiences of those outside this type of setting.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    The participants had limited experience with AI, which limits the reliability of the findings.

    “The participants had limited experience with AI, which limits the reliability of the findings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Participants may have endorsed ideas to be agreeable.

    “They may have endorsed ideas to be agreeable, but participants seemed to feel free to disagree.”
    Quote in the paperNot a substantial claim
53 of 53

  • The paper relies on this without support.

    Participants' self-reported beliefs about hypothetical AI reflect their likely future acceptance and use of AI in practice.

    “This interview study of perioperative nurses at an academic medical center found that participants were receptive to AI as a potential adjunct for postoperative handoff communication.”
1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Risk-increasing data attributed to the wrong subtheme

    Minor

    The discussion attributes identification of risk-increasing data to Subtheme 3.c, but the results assign that finding to Subtheme 3.d. Subtheme 3.c instead concerns nurses reviewing data when a patient is identified as high-risk overall.

1 of 1

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
King, C. R., Shambe, A., & Abraham, J.
Affiliations
Washington University in St. Louis (US)
Corresponding author
King, Christopher Ryan
ORCID
Venue
JAMIA Open
Publisher
Oxford University Press (OUP)
Published
2023-01-06
Volume
6
Issue
1
Pages
ooad015-ooad015
ISSN
2574-2531 (ISSN-L)
DOI
10.1093/jamiaopen/ooad015
PMC
PMC10019806
OpenAlex
W4327661996
Semantic Scholar
d3d87b63caf4fe31655876818b97848623f71cb9
Cited by
27
References
84
Citations by year
2026: 6 · 2025: 15 · 2024: 6
FWCI
7.74
Type
journal article
Language
en
Open access
Licence
Locations
  • JAMIA Open (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by-nd)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by)
  • Digital Commons@Becker (Washington University School of Medicine) (submittedVersion, cc-by)
Funders
National Institutes of Health (KL2 TR002346)
Topics
  • Health Sciences › Medicine › Emergency Medicine › Hospital Admissions and Outcomes
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
Keywords
artificial intelligence, nursing handover, perioperative nursing, postoperative care, situational awareness, anticipatory guidance, patient safety, team communication, electronic health records, information overload, risk identification, nursing decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-27.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-27.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review