Cited by 50Artificial Intelligence in Healthcare and EducationCardiac, Anesthesia and Surgical OutcomesMachine Learning in Healthcare

Integrating machine learning predictions for perioperative risk management: towards an empirical design of a flexible-standardized risk assessment tool

Open access
Abraham, Joanna, Bartek, Brian, Meng, Alicia, Ryan King, Christopher, Xue, Bing, Lu, Chenyang, Avidan, Michael S.·2023·Journal of Biomedical Informatics

Abstract

Background Surgical patients are complex, vulnerable, and prone to postoperative complications that can potentially be mitigated with quality perioperative risk assessment and management. Several institutions have incorporated machine learning (ML) into their patient care to improve awareness and support clinician decision-making along the perioperative spectrum. Recent research suggests that ML risk prediction can support perioperative patient risk monitoring and management across several situations, including the operating room (OR) to intensive care unit (ICU) handoffs. Objectives Our study objectives were threefold: (1) evaluate whether ML-generated postoperative predictions are concordant with clinician-generated risk rankings for acute kidney injury, delirium, pneumonia, deep vein thrombosis, and pulmonary embolism, and establish their associated risk factors; (2) ascertain clinician end-user suggestions to improve adoption of ML-generated risks and their integration into the perioperative workflow; and (3) develop a user-friendly visualization format for a tool to display ML-generated risks and risk factors to support postoperative care planning, for example, within the context of OR-ICU handoffs. Methods Graphical user interfaces for postoperative risk prediction models were assessed for end-user usability through cognitive walkthroughs and interviews with anesthesiologists, surgeons, certified registered nurse anesthetists, registered nurses, and critical care physicians. Thematic analysis relying on an explanation design framework was used to identify feedback and suggestions for improvement. Results 17 clinicians participated in the evaluation. ML estimates of complication risks aligned with clinicians' independent rankings, and related displays were perceived as valuable for decision-making and care planning for postoperative care. During OR-ICU handoffs, the tool could speed up report preparation and remind clinicians to address patient-specific complications, thus providing more tailored care information. Suggestions for improvement centered on electronic tool delivery; methods to build trust in ML models; modifiable risks and risk mitigation strategies; and additional patient information based on individual preferences (e.g., surgical procedure). Conclusions ML estimates of postoperative complication risks can provide anticipatory guidance, potentially increasing the efficiency of care planning. We have offered an ML visualization framework for designing future ML-augmented tools and anticipate the development of tools that recommend specific actions to the user based on ML model output.

Integrity signals

Registry

Crossref · 10.1016/j.jbi.2022.104270

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Abraham, J., Bartek, B., Meng, A., Ryan King, C., Xue, B., Lu, C., & Avidan, M. S.
Affiliations
Washington University in St. Louis (US)
Corresponding author
Abraham, Joanna
Venue
Journal of Biomedical Informatics
Publisher
Elsevier BV
Published
2023-01
Volume
137
Pages
104270
ISSN
1532-0464 (ISSN-L)
OpenAlex
W4311761926
Cited by
50
References
47
Citations by year
2026: 18 · 2025: 15 · 2024: 11 · 2023: 6
FWCI
2.41
Type
journal article
Language
en
Open access
Locations
  • Journal of Biomedical Informatics (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
machine learning, perioperative risk assessment, postoperative complication prediction, clinical decision support, risk visualization, end-user usability, risk factors, trust in AI, risk mitigation, postoperative care planning, clinical workflow integration, explanation design, acute kidney injury, delirium, deep vein thrombosis, pulmonary embolism, pneumonia
MeSH terms
Machine Learning, Critical Care, Patient Care, Humans, Risk Assessment, Surgeons

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review