The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Use of Machine Learning to Develop and Evaluate Models Using Preoperative and Intraoperative Data to Identify Risks of Postoperative Complications
Open accessAbstract
Importance: Postoperative complications can significantly impact perioperative care management and planning. Objectives: To assess machine learning (ML) models for predicting postoperative complications using independent and combined preoperative and intraoperative data and their clinically meaningful model-agnostic interpretations. Design, Setting, and Participants: This retrospective cohort study assessed 111 888 operations performed on adults at a single academic medical center from June 1, 2012, to August 31, 2016, with a mean duration of follow-up based on the length of postoperative hospital stay less than 7 days. Data analysis was performed from February 1 to September 31, 2020. Main Outcomes and Measures: Outcomes included 5 postoperative complications: acute kidney injury (AKI), delirium, deep vein thrombosis (DVT), pulmonary embolism (PE), and pneumonia. Patient and clinical characteristics available preoperatively, intraoperatively, and a combination of both were used as inputs for 5 candidate ML models: logistic regression, support vector machine, random forest, gradient boosting tree (GBT), and deep neural network (DNN). Model performance was compared using the area under the receiver operating characteristic curve (AUROC). Model interpretations were generated using Shapley Additive Explanations by transforming model features into clinical variables and representing them as patient-specific visualizations. Results: A total of 111 888 patients (mean [SD] age, 54.4 [16.8] years; 56 915 [50.9%] female; 82 533 [73.8%] White) were included in this study. The best-performing model for each complication combined the preoperative and intraoperative data with the following AUROCs: pneumonia (GBT), 0.905 (95% CI, 0.903-0.907); AKI (GBT), 0.848 (95% CI, 0.846-0.851); DVT (GBT), 0.881 (95% CI, 0.878-0.884); PE (DNN), 0.831 (95% CI, 0.824-0.839); and delirium (GBT), 0.762 (95% CI, 0.759-0.765). Performance of models that used only preoperative data or only intraoperative data was marginally lower than that of models that used combined data. When adding variables with missing data as input, AUROCs increased from 0.588 to 0.905 for pneumonia, 0.579 to 0.848 for AKI, 0.574 to 0.881 for DVT, 0.5 to 0.831 for PE, and 0.6 to 0.762 for delirium. The Shapley Additive Explanations analysis generated model-agnostic interpretation that illustrated significant clinical contributors associated with risks of postoperative complications. Conclusions and Relevance: The ML models for predicting postoperative complications with model-agnostic interpretation offer opportunities for integrating risk predictions for clinical decision support. Such real-time clinical decision support can mitigate patient risks and help in anticipatory management for perioperative contingency planning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
MissForest—Non-parametric missing value imputation for mixed-type data
Stekhoven, D. J., & Bühlmann, P. (2012) · Bioinformatics · doi:10.1093/bioinformatics/btr597 · Cited by 6,156
Multiple imputation by chained equations: what is it and how does it work?
Azur, M. J., Stuart, E. A., Frangakis, C., & Leaf, P. J. (2011) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.329 · Cited by 3,542
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967
Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning
Lemaître, G., Nogueira, F., & Aridas, C. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.06570 · Cited by 1,514
Bratzler, D. W., & Houck, P. M. (2005) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2005.01.015 · Cited by 1,163
Postoperative pulmonary complications
Miskovic, A., & Lumb, A. B. (2017) · British Journal of Anaesthesia · doi:10.1093/bja/aex002 · Cited by 1,123
Surgical Risk Factors, Morbidity, and Mortality in Elderly Patients
Turrentine, F. E., Wang, H., Simpson, V. B., & Jones, R. S. (2006) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2006.08.026 · Cited by 1,061
Aas, K., Jullum, M., & Løland, A. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103502 · Cited by 1,023
The incidence and nature of surgical adverse events in Colorado and Utah in 1992
Gawande, A. A., Thomas, E. J., Zinner, M. J., & Brennan, T. A. (1999) · Surgery · doi:10.1067/msy.1999.98664 · Cited by 977
Meersch, M., Schmidt, C., Hoffmeier, A., Van Aken, H., Wempe, C., Gerss, J., & Zarbock, A. (2017) · Intensive Care Medicine · doi:10.1007/s00134-016-4670-3 · Cited by 892
Clinical Interventions in Aging
(2020) · Clinical Interventions in Aging · doi:10.2147/cia · Cited by 882
Surgical risk factors, morbidity, and mortality in elderly patients
Turrentine, F. E., Wang, H., Simpson, V. B., & Jones, R. S. (2008) · Critical Reviews in Oncology/Hematology · doi:10.1016/s1040-8428(08)70016-7 · Cited by 711
(2026) · Journal of Thoracic Disease · doi:10.21037/jtd · Cited by 710
Hamel, M. B., Henderson, W. G., Khuri, S. F., & Daley, J. (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53159.x · Cited by 631
Adverse events in surgical patients in Australia
KABLE, A. K., GIBBERD, R. W., & SPIGELMAN, A. D. (2002) · International Journal for Quality in Health Care · doi:10.1093/intqhc/14.4.269 · Cited by 450
Machine learning for real-time prediction of complications in critical care: a retrospective study
Meyer, A., Zverinski, D., Pfahringer, B., Kempfert, J., Kuehne, T., Sündermann, S. H., Stamm, C., Hofmann, T., Falk, V., & Eickhoff, C. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30300-x · Cited by 350
Complications in surgical patients.
Healey, M. A. (2002) · Archives of Surgery · doi:10.1001/archsurg.137.5.611 · Cited by 343 · cited 2×
“We depicted the following to support practitioners’ model interpretation: (1) the accumulated risk with the each of the top 10 clinical variables, measured by SHAP values and scaled to on a 0- to 1-point scale, with 0 indicating lowest risk score and 1 indicating highest risk score, using a logistic function; (2) comp…”
MethodDelirium in the ICU: an overview
Cavallazzi, R., Saad, M., & Marik, P. E. (2012) · Annals of Intensive Care · doi:10.1186/2110-5820-2-49 · Cited by 275
Postoperative complications and implications on patient-centered outcomes
Tevis, S. E., & Kennedy, G. D. (2013) · Journal of Surgical Research · doi:10.1016/j.jss.2013.01.032 · Cited by 224
Janssen, T. L., Alberts, A. R., Hooft, L., Mattace-Raso, F. U. S., Mosk, C. A., & van der Laan, L. (2019) · Clinical Interventions in Aging · doi:10.2147/cia.s201323 · Cited by 211
Young, P. Y., & Khadaroo, R. G. (2014) · Surgical Clinics of North America · doi:10.1016/j.suc.2014.08.008 · Cited by 185
Brajer, N., Cozzi, B., Gao, M. C., Nichols, M., Revoir, M., Balu, S., Futoma, J., Bae, J., Setji, N., Hernandez, A. F., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.20733 · Cited by 153
Deep-learning model for predicting 30-day postoperative mortality
Fritz, B. A., Cui, Z., Zhang, M., He, Y., Chen, Y., Kronzer, A., Ben Abdallah, A., King, C. R., & Avidan, M. S. (2019) · British Journal of Anaesthesia · doi:10.1016/j.bja.2019.07.025 · Cited by 127
Self-citationAlotaibi, G., Wu, C. M., Senthilselvan, A., & McMurtry, M. S. (2015) · Vascular Medicine · doi:10.1177/1358863x15573839 · Cited by 115
Recent Advances in Preventing and Managing Postoperative Delirium
Vlisides, P. E., & Avidan, M. S. (2019) · F1000Research · doi:10.12688/f1000research.16780.1 · Cited by 112
Self-citationFitzHenry, F., Murff, H. J., Matheny, M. E., Gentry, N., Fielstein, E. M., Brown, S. H., Reeves, R., Aronsky, D., Elkin, P. L., Messina, V. P., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31828d1210 · Cited by 85
Hofer, I. S., Lee, C., Gabel, E., Baldi, P., & Cannesson, M. P. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0248-0 · Cited by 81
Leveraging electronic health records for predictive modeling of post-surgical complications
Weller, G. B., Lovely, J. K., Larson, D. W., Earnshaw, B. A., & Huebner, M. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280217696115 · Cited by 68
Postoperative Delirium in Cardiac Surgery: An Unavoidable Menace?
Hollinger, A., Siegemund, M., Goettel, N., & Steiner, L. A. (2015) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2014.08.021 · Cited by 59
Higgins, T. L., Deshpande, A., Zilberberg, M. D., Lindenauer, P. K., Imrey, P. B., Yu, P.-C., Haessler, S., Richter, S. S., & Rothberg, M. B. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.7750 · Cited by 53
Kaafarani, H. M. A., & Rosen, A. K. (2009) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2009.08.008 · Cited by 47
Warner, J. L., Zhang, P., Liu, J. X., & Alterovitz, G. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.12.008 · Cited by 44
Fritz, B. A., Chen, Y., Murray-Torres, T. M., Gregory, S., Ben Abdallah, A., Kronzer, A., McKinnon, S. L., Budelier, T., Helsten, D. L., Wildes, T. S., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020124 · Cited by 42
Self-citationEvaluating risk-prediction models using data from electronic health records
Wang, L., Shaw, P. A., Mathelier, H. M., Kimmel, S. E., & French, B. C. (2016) · The Annals of Applied Statistics · doi:10.1214/15-aoas891 · Cited by 28
Multidimensional Attitudes of Medical Residents and Geriatrics Fellows Toward Older People
Lee, M., Reuben, D. B., & Ferrell, B. A. (2005) · Journal of the American Geriatrics Society · doi:10.1111/jgs.2005.53.issue-3 · Cited by 21
Cited by
The Integration of Artificial Intelligence into Clinical Practice
Karalis, V. D. (2024) · Applied Biosciences · doi:10.3390/applbiosci3010002 · Cited by 255
Arina, P., Kaczorek, M. R., Hofmaenner, D. A., Pisciotta, W., Refinetti, P., Singer, M., Mazomenos, E. B., & Whittle, J. (2024) · Anesthesiology · doi:10.1097/aln.0000000000004764 · Cited by 81
Abraham, J., Bartek, B., Meng, A., Ryan King, C., Xue, B., Lu, C., & Avidan, M. S. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104270 · Cited by 50
Self-citationJiang, Z., Bo, L., Wang, L., Xie, Y., Cao, J., Yao, Y., Lu, W., Deng, X., Yang, T., & Bian, J. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107772 · Cited by 38
Potential uses of AI for perioperative nursing handoffs: a qualitative study
King, C. R., Shambe, A., & Abraham, J. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad015 · Cited by 27
Self-citation
Metadata
- Authors
- Xue, B., Li, D., Lu, C., King, C. R., Wildes, T., Avidan, M. S., Kannampallil, T., & Abraham, J.
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2021-03-30
- Volume
- 4
- Issue
- 3
- Pages
- e212240
- ISSN
- 2574-3805 (ISSN-L)
- OpenAlex
- W3150419368
- Cited by
- 365
- References
- 38
- FWCI
- 29.18
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.