CoChat

Improved predictive models for acute kidney injury with IDEA: Intraoperative Data Embedded Analytics.

Open access

The DOI matches its Crossref record; 68 of the 70 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

BACKGROUND: Acute kidney injury (AKI) is a common complication after surgery that is associated with increased morbidity and mortality. The majority of existing perioperative AKI risk prediction models are limited in their generalizability and do not fully utilize intraoperative physiological time-series data. Thus, there is a need for intelligent, accurate, and robust systems to leverage new information as it becomes available to predict the risk of developing postoperative AKI. METHODS: A retrospective single-center cohort of 2,911 adults who underwent surgery at the University of Florida Health between 2000 and 2010 was utilized for this study. Machine learning and statistical analysis techniques were used to develop perioperative models to predict the risk of developing AKI during the first three days after surgery, first seven days after surgery, and overall (after surgery during the index hospitalization). The improvement in risk prediction was examined by incorporating intraoperative physiological time-series variables. Our proposed model enriched a preoperative model that produced a probabilistic AKI risk score by integrating intraoperative statistical features through a machine learning stacking approach inside a random forest classifier. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC), accuracy, and Net Reclassification Improvement (NRI). RESULTS: The predictive performance of the proposed model is better than the preoperative data only model. The proposed model had an AUC of 0.86 (accuracy of 0.78) for the seven-day AKI outcome, while the preoperative model had an AUC of 0.84 (accuracy of 0.76). Furthermore, by integrating intraoperative features, the algorithm was able to reclassify 40% of the false negative patients from the preoperative model. The NRI for each outcome was AKI at three days (8%), seven days (7%), and overall (4%). CONCLUSIONS: Postoperative AKI prediction was improved with high sensitivity and specificity through a machine learning approach that dynamically incorporated intraoperative data.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0214904

Cites

68 of 69 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Clinical workflow of the Intraoperative Data Embedded Analytics (IDEA) algorithm for postoperative acute kidney injury prediction. Phase I, all available health care data from the electronic health record and other public datasets is fed into the preoperative model. The preoperative model calculates a risk score for postoperative acute kidney injury (AKI) and shares the risk score with the clinical team before the surgery. Phase II, the preoperative data is combined with intraoperative data and fed into the IDEA algorithm which calculates a risk score for postoperative acute kidney injury and shares the risk score with the clinical team after surgery.
Fig 1 Clinical workflow of the Intraoperative Data Embedded Analytics (IDEA) algorithm for postoperative acute kidney injury prediction. Phase I, all available health care data from the electronic health record and other public datasets is fed into the preoperative model. The preoperative model calculates a risk score for postoperative acute kidney injury (AKI) and shares the risk score with the clinical team before the surgery. Phase II, the preoperative data is combined with intraoperative data and fed into the IDEA algorithm which calculates a risk score for postoperative acute kidney injury and shares the risk score with the clinical team after surgery.CC BY · PubMed Central
Fig 2 Conceptual diagram of the Intraoperative Data Embedded Analytics (IDEA) AKI prediction model. This diagram shows the aggregation of the data transformer, data engineering, and data analytics modules in the preoperative and intraoperative layers. The two layers can be integrated by either (1) stacking the preoperative prediction scores with the cleaned and feature engineered intraoperative data (blue arrow) or (2) obtaining the full perioperative dataset by merging all the clean features from both layers (orange arrow).
Fig 2 Conceptual diagram of the Intraoperative Data Embedded Analytics (IDEA) AKI prediction model. This diagram shows the aggregation of the data transformer, data engineering, and data analytics modules in the preoperative and intraoperative layers. The two layers can be integrated by either (1) stacking the preoperative prediction scores with the cleaned and feature engineered intraoperative data (blue arrow) or (2) obtaining the full perioperative dataset by merging all the clean features from both layers (orange arrow).CC BY · PubMed Central
Fig 3 Intraoperative physiological time series variations stratified by the occurrence of acute kidney injury. Intraoperative physiological time series variations for 100 randomly selected patients during the first 200 minutes of surgery stratified by the occurrence of acute kidney injury (AKI) within the first seven days after surgery. Mean and 95% Confidence Interval (CI) stratified by the occurrence of AKI are shown for (A) intraoperative mean arterial blood pressure, MAP (mmHg), (B) intraoperative heart rate, HR (beats/minute), and (C) intraoperative mean alveolar concentration of anesthetics, MAC.
Fig 3 Intraoperative physiological time series variations stratified by the occurrence of acute kidney injury. Intraoperative physiological time series variations for 100 randomly selected patients during the first 200 minutes of surgery stratified by the occurrence of acute kidney injury (AKI) within the first seven days after surgery. Mean and 95% Confidence Interval (CI) stratified by the occurrence of AKI are shown for (A) intraoperative mean arterial blood pressure, MAP (mmHg), (B) intraoperative heart rate, HR (beats/minute), and (C) intraoperative mean alveolar concentration of anesthetics, MAC.CC BY · PubMed Central
Fig 4 Association between risk of postoperative acute kidney injury and magnitude and duration of intraoperative physiological variations. The risk for the development of postoperative acute kidney injury (AKI) within the first seven days after surgery, as predicted by the postoperative stacked model, was aggregated across the entire cohort. The color represents the risk of developing postoperative AKI, where red is high-risk, and green is low-risk. The y-axis is time (minutes) and the x-axes are (A) diastolic blood pressure (mmHg), (B) systolic blood pressure (mmHg), (C) Mean arterial blood pressure (mmHg), and (D) Heart rate (beats/minute).
Fig 4 Association between risk of postoperative acute kidney injury and magnitude and duration of intraoperative physiological variations. The risk for the development of postoperative acute kidney injury (AKI) within the first seven days after surgery, as predicted by the postoperative stacked model, was aggregated across the entire cohort. The color represents the risk of developing postoperative AKI, where red is high-risk, and green is low-risk. The y-axis is time (minutes) and the x-axes are (A) diastolic blood pressure (mmHg), (B) systolic blood pressure (mmHg), (C) Mean arterial blood pressure (mmHg), and (D) Heart rate (beats/minute).CC BY · PubMed Central
Fig 5 Nonlinear association between risk of postoperative acute kidney injury and intraoperative features. Each blue dot represents a patient that developed postoperative AKI. The y-axis represents the risk probability for postoperative acute kidney injury, which ranges from 0 to 1. The red line represents the median acute kidney injury risk score given the values of x. The x-axes represent (A) maximum intraoperative percentage of methemoglobin in arterial blood gasses, (B) intraoperative variance of bicarbonate (mmol/L)2 in arterial blood gasses, (C) intraoperative mean of lactic acid (mmol/L), (D) minimum intraoperative platelet count (thou/mm3), (E) intraoperative variance of mean corpuscular hemoglobin, MCH (pg/cell)2, (F) intraoperative mean of red cell distribution (%), (G) total of blood products administered during surgery (mL), and (H) duration of surgery (minutes).
Fig 5 Nonlinear association between risk of postoperative acute kidney injury and intraoperative features. Each blue dot represents a patient that developed postoperative AKI. The y-axis represents the risk probability for postoperative acute kidney injury, which ranges from 0 to 1. The red line represents the median acute kidney injury risk score given the values of x. The x-axes represent (A) maximum intraoperative percentage of methemoglobin in arterial blood gasses, (B) intraoperative variance of bicarbonate (mmol/L)2 in arterial blood gasses, (C) intraoperative mean of lactic acid (mmol/L), (D) minimum intraoperative platelet count (thou/mm3), (E) intraoperative variance of mean corpuscular hemoglobin, MCH (pg/cell)2, (F) intraoperative mean of red cell distribution (%), (G) total of blood products administered during surgery (mL), and (H) duration of surgery (minutes).CC BY · PubMed Central
Fig 6 Receiver operating characteristic curves and other performance metrics for prediction of postoperative acute kidney injury. The graph at the top of the figure is the receiver operating characteristic curves for prediction of postoperative acute kidney injury (AKI) within the first seven days after surgery for the preoperative model (pink), the postoperative stacked model (green), and the postoperative full model (blue). The diamonds on each curve represent the cutoffs calculated by maximizing the Youden Index. The y-axis is sensitivity, which ranges from zero to one. The x-axis is one minus specificity, which ranges from zero to one. The table in the bottom of the figure contains the following performance metrics for the preoperative and both postoperative models: (A) accuracy (percentage of patients correctly classified), (B) sensitivity (percentage of patients that developed AKI that were classified as high-risk), (C) positive predictive value (percentage of high-risk patients that developed AKI), (D) negative predictive value (percentage of low-risk patients that did not develop AKI), (E) net reclassification improvement for patients that developed postoperative AKI (F) net reclassification improvement for patients that did not develop postoperative AKI, (H) overall net reclassification improvement.
Fig 6 Receiver operating characteristic curves and other performance metrics for prediction of postoperative acute kidney injury. The graph at the top of the figure is the receiver operating characteristic curves for prediction of postoperative acute kidney injury (AKI) within the first seven days after surgery for the preoperative model (pink), the postoperative stacked model (green), and the postoperative full model (blue). The diamonds on each curve represent the cutoffs calculated by maximizing the Youden Index. The y-axis is sensitivity, which ranges from zero to one. The x-axis is one minus specificity, which ranges from zero to one. The table in the bottom of the figure contains the following performance metrics for the preoperative and both postoperative models: (A) accuracy (percentage of patients correctly classified), (B) sensitivity (percentage of patients that developed AKI that were classified as high-risk), (C) positive predictive value (percentage of high-risk patients that developed AKI), (D) negative predictive value (percentage of low-risk patients that did not develop AKI), (E) net reclassification improvement for patients that developed postoperative AKI (F) net reclassification improvement for patients that did not develop postoperative AKI, (H) overall net reclassification improvement.CC BY · PubMed Central
Fig 7 Reclassification performance of the postoperative stacked and full models for predicting postoperative acute kidney injury. The postoperative risk scores in the first column are from the stacked model, while the postoperative risk scores for the second column are from the full model. The patients in the first row developed postoperative AKI within seven days of surgery, whereas the patients in the second row did not develop postoperative AKI within seven days of surgery. The y-axis on each plot is the postoperative model acute kidney injury risk score, which ranges from zero to one. The x-axis on each plot is the preoperative risk group. The red dots are patients at high-risk for AKI according to the postoperative model, whereas the green dots are patients at low-risk for AKI according to the postoperative model. (A & B) The proposed postoperative stacked and full models effectively reclassified false negative patients from the preoperative model as high AKI risk patients.
Fig 7 Reclassification performance of the postoperative stacked and full models for predicting postoperative acute kidney injury. The postoperative risk scores in the first column are from the stacked model, while the postoperative risk scores for the second column are from the full model. The patients in the first row developed postoperative AKI within seven days of surgery, whereas the patients in the second row did not develop postoperative AKI within seven days of surgery. The y-axis on each plot is the postoperative model acute kidney injury risk score, which ranges from zero to one. The x-axis on each plot is the preoperative risk group. The red dots are patients at high-risk for AKI according to the postoperative model, whereas the green dots are patients at low-risk for AKI according to the postoperative model. (A & B) The proposed postoperative stacked and full models effectively reclassified false negative patients from the preoperative model as high AKI risk patients.CC BY · PubMed Central
Fig 8 Association between intraoperative variables (mean arterial pressure and administered blood products) and postoperative risk group reclassification. The first column represents patients that developed postoperative acute kidney injury (AKI), yet were classified as low-risk by the preoperative model. The second column represents patients that did not develop postoperative AKI, but were classified as high-risk by the preoperative model. The boxes represent 95% confidence intervals for the given variables. (A) Patients correctly classified as having high-risk for AKI by the postoperative stacked model tended to have lower mean arterial pressure. (B) Patients correctly classified as having low-risk for AKI by the postoperative stacked model tended to have higher mean arterial pressure. (C) Patients correctly classified as having high-risk for AKI by the postoperative stacked model tended to have greater volumes of administered intraoperative blood products. (D) Patients correctly classified as having low-risk for AKI by the postoperative stacked model tended to have lesser volumes of administered intraoperative blood products.
Fig 8 Association between intraoperative variables (mean arterial pressure and administered blood products) and postoperative risk group reclassification. The first column represents patients that developed postoperative acute kidney injury (AKI), yet were classified as low-risk by the preoperative model. The second column represents patients that did not develop postoperative AKI, but were classified as high-risk by the preoperative model. The boxes represent 95% confidence intervals for the given variables. (A) Patients correctly classified as having high-risk for AKI by the postoperative stacked model tended to have lower mean arterial pressure. (B) Patients correctly classified as having low-risk for AKI by the postoperative stacked model tended to have higher mean arterial pressure. (C) Patients correctly classified as having high-risk for AKI by the postoperative stacked model tended to have greater volumes of administered intraoperative blood products. (D) Patients correctly classified as having low-risk for AKI by the postoperative stacked model tended to have lesser volumes of administered intraoperative blood products.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 3

  1. C Hobson, 2015, Cost and Mortality Associated With Postoperative Acute Kidney Injury, Annals of Surgery, 261, 1207, 10.1097/SLA.0000000000000732 · DOI · Paper page
  2. 2012, Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group—KDIGO Clinical Practice Guideline for Acute Kidney Injury, Kidney Int, 1
  3. OE Jones E, 2001, SciPy: Open Source Scientific Tools for Python, SciPy: Open Source Scientific Tools for Python

Cited by

See all 98

Metadata

Authors
Adhikari, L., Ozrazgat-Baslanti, T., Ruppert, M., Madushani, R. W. M. A., Paliwal, S., Hashemighouchani, H., Zheng, F., Tao, M., Lopes, J. M., Li, X., Rashidi, P., & Bihorac, A.
Affiliations
University of Florida (US)
Corresponding author
Bihorac, Azra
Venue
PLOS ONE
Published
2019-04-04
Volume
14
Issue
4
Pages
e0214904
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2804227301
Cited by
98
References
70
Citations by year
2026: 11 · 2025: 12 · 2024: 12 · 2023: 16 · 2022: 19 · 2021: 13 · 2020: 9 · 2019: 5
FWCI
8.87
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • National Institute of General Medical Sciences (RO1 GM110240, P50 GM-111152)
  • University of Florida (97071, 127900)
  • Society of Critical Care Medicine (Vision Grant)
  • National Center for Advancing Translational Sciences (UL1 TR000064)
  • I Heermann Anesthesia Foundation
Topics
  • Health Sciences › Medicine › Nephrology › Acute Kidney Injury Research
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
acute kidney injury, stacking ensemble, random forest, perioperative complications, surgical patients
MeSH terms
Machine Learning, Adult, Aged, Algorithms, Female, Humans, Intraoperative Period, Male, Middle Aged, Postoperative Complications, Predictive Value of Tests, Prognosis, Retrospective Studies, Risk Factors, Sensitivity and Specificity, Models, Statistical, Cohort Studies, Area Under Curve, Acute Kidney Injury

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.