Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): a retrospective, single-site study
Open accessAbstract
BACKGROUND: Pythia is an automated, clinically curated surgical data pipeline and repository housing all surgical patient electronic health record (EHR) data from a large, quaternary, multisite health institute for data science initiatives. In an effort to better identify high-risk surgical patients from complex data, a machine learning project trained on Pythia was built to predict postoperative complication risk. METHODS AND FINDINGS: A curated data repository of surgical outcomes was created using automated SQL and R code that extracted and processed patient clinical and surgical data across 37 million clinical encounters from the EHRs. A total of 194 clinical features including patient demographics (e.g., age, sex, race), smoking status, medications, comorbidities, procedure information, and proxies for surgical complexity were constructed and aggregated. A cohort of 66,370 patients that had undergone 99,755 invasive procedural encounters between January 1, 2014, and January 31, 2017, was studied further for the purpose of predicting postoperative complications. The average complication and 30-day postoperative mortality rates of this cohort were 16.0% and 0.51%, respectively. Least absolute shrinkage and selection operator (lasso) penalized logistic regression, random forest models, and extreme gradient boosted decision trees were trained on this surgical cohort with cross-validation on 14 specific postoperative outcome groupings. Resulting models had area under the receiver operator characteristic curve (AUC) values ranging between 0.747 and 0.924, calculated on an out-of-sample test set from the last 5 months of data. Lasso penalized regression was identified as a high-performing model, providing clinically interpretable actionable insights. Highest and lowest performing lasso models predicted postoperative shock and genitourinary outcomes with AUCs of 0.924 (95% CI: 0.901, 0.946) and 0.780 (95% CI: 0.752, 0.810), respectively. A calculator requiring input of 9 data fields was created to produce a risk assessment for the 14 groupings of postoperative outcomes. A high-risk threshold (15% risk of any complication) was determined to identify high-risk surgical patients. The model sensitivity was 76%, with a specificity of 76%. Compared to heuristics that identify high-risk patients developed by clinical experts and the ACS NSQIP calculator, this tool performed superiorly, providing an improved approach for clinicians to estimate postoperative risk for patients. Limitations of this study include the missingness of data that were removed for analysis. CONCLUSIONS: Extracting and curating a large, local institution's EHR data for machine learning purposes resulted in models with strong predictive performance. These models can be used in clinical settings as decision support tools for identification of high-risk patients as well as patient evaluation and care management. Further work is necessary to evaluate the impact of the Pythia risk calculator within the clinical workflow on postoperative outcomes and to optimize this data flow for future machine learning efforts.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A growing nationwide effort is seeking to enhance preoperative and perioperative support for high-risk patients and high-cost populations.
“A growing nationwide effort is seeking to enhance preoperative and perioperative support for high-risk patients and high-cost populations (16, 17).”
From Accessing Artificial Intelligence for Clinical Decision-Making (2021) · cites this paper as [16]
ML models have been shown to better predict mortality than conventional logistic regression after liver cancer surgery, aortic aneurysm surgery, and cardiac surgery.
“These ML models have been shown to better predict mortality than conventional logistic regression after liver cancer surgery, aortic aneurysm surgery, and cardiac surgery.”
From Accessing Artificial Intelligence for Clinical Decision-Making (2021) · cites this paper as [16]
Healthcare systems have increasingly sought to use ML to assist in risk stratification, and these ML models may outperform statistical models in calibration and discrimination.
“Healthcare systems have increasingly sought to use ML to assist in risk stratification, and these ML models may outperform statistical models in calibration and discrimination.”
From Accessing Artificial Intelligence for Clinical Decision-Making (2021) · cites this paper as [16]
Multiple ML models that risk-stratify patients with a disease or prepare patients for surgery have been recently developed and validated.
“Multiple ML models that risk-stratify patients with a disease or prepare patients for surgery have been recently developed and validated (16, 20–25).”
From Accessing Artificial Intelligence for Clinical Decision-Making (2021) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927 · see citing works
Classification and Regression by randomForest
Liaw, A. I., & Wiener, M. C. (2007) · Cited by 18,594 · see citing works
Comorbidity measures for use with administrative data.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998) · Medical Care · doi:10.1097/00005650-199801000-00004 · Cited by 9,921 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104 · see citing works
Friedman, J. H. (2002) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(01)00065-2 · Cited by 7,067 · see citing works
Collins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,312 · see citing works
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · see citing works
Scalable and accurate deep learning with electronic health records.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · eScholarship (California Digital Library) · Cited by 2,242 · see citing works
Bilimoria, K. Y., Liu, Y., Paruch, J. L., Zhou, L., Kmiecik, T. E., Ko, C. Y., & Cohen, M. E. (2013) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2013.07.385 · Cited by 1,929 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. (2015) · BMC Medicine · doi:10.1186/s12916-014-0241-z · Cited by 1,927 · see citing works
Big Data In Health Care: Using Analytics To Identify And Manage High-Risk And High-Cost Patients
Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0041 · Cited by 1,235 · see citing works
Dimick, J. B., Chen, S. L., Taheri, P. A., Henderson, W. G., Khuri, S. F., & Campbell, D. A. (2004) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2004.05.276 · Cited by 933 · see citing works
Hersh, W. R., Weiner, M. G., Embi, P. J., Logan, J. R., Payne, P. R. O., Bernstam, E. V., Lehmann, H. P., Hripcsak, G., Hartzog, T. H., Cimino, J. J., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1dbd · Cited by 567 · see citing works
Complications in surgical patients.
Healey, M. A. (2002) · Archives of Surgery · doi:10.1001/archsurg.137.5.611 · Cited by 343 · see citing works
Osborne, N. H., Nicholas, L. H., Ryan, A. M., Thumma, J. R., & Dimick, J. B. (2015) · JAMA · doi:10.1001/jama.2015.25 · Cited by 324 · see citing works
Development and validation of the Surgical Outcome Risk Tool (SORT)
Protopapa, K., Simpson, J. C., Smith, N. C. E., & Moonesinghe, R. (2014) · British journal of surgery · doi:10.1002/bjs.9638 · Cited by 308 · see citing works
McDonald, S. R., Heflin, M. T., Whitson, H. E., Dalton, T. O., Lidsky, M. E., Liu, P., Poer, C. M., Sloane, R., Thacker, J. K., White, H. K., et al. (2018) · JAMA Surgery · doi:10.1001/jamasurg.2017.5513 · Cited by 210 · see citing works
Self-citationEtzioni, D. A., Wasif, N., Dueck, A. C., Cima, R. R., Hohmann, S. F., Naessens, J. M., Mathur, A. K., & Habermann, E. B. (2015) · JAMA · doi:10.1001/jama.2015.90 · Cited by 201 · see citing works
Escaping the EHR Trap — The Future of Health IT
Mandl, K. D., & Kohane, I. S. (2012) · New England Journal of Medicine · doi:10.1056/nejmp1203102 · Cited by 174 · see citing works
Surgical Risk Preoperative Assessment System (SURPAS)
Meguid, R. A., Bronsert, M. R., Juarez‐Colunga, E., Hammermeister, K. E., & Henderson, W. G. (2016) · Annals of Surgery · doi:10.1097/sla.0000000000001678 · Cited by 115 · see citing works
Amrock, L. G., Neuman, M. D., Lin, H., & Deiner, S. G. (2014) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2014.04.018 · Cited by 88 · see citing works
Cologne, K. G., Keller, D. S., Liwanag, L., Devaraj, B., & Senagore, A. J. (2014) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2014.12.007 · Cited by 82 · see citing works
Predicting colorectal surgical complications using heterogeneous clinical data and kernel methods
Soguero-Ruíz, C., Hindberg, K., Mora-Jiménez, I., Rojo‐Álvarez, J. L., Skrøvseth, S. O., Godtliebsen, F., Mortensen, K. E., Revhaug, A., Lindsetmo, R., Augestad, K. M., et al. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.03.008 · Cited by 78 · see citing works
Desebbe, O., Lanz, T., Kain, Z. N., & Cannesson, M. P. (2015) · Anaesthesia Critical Care & Pain Medicine · doi:10.1016/j.accpm.2015.08.001 · Cited by 73 · see citing works
Rivard, C. L., Nahum, R., Slagle, E., Duininck, M., Vogel, R. I., & Teoh, D. G. K. (2016) · Gynecologic Oncology · doi:10.1016/j.ygyno.2016.02.015 · Cited by 72 · see citing works
Prasad, K. G., Nelson, B. G., Deig, C. R., Schneider, A. L., & Moore, M. G. (2016) · Otolaryngology · doi:10.1177/0194599816655976 · Cited by 67 · see citing works
da Graca, B., Filardo, G., & Nicewander, D. A. (2013) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.112.968826 · Cited by 63 · see citing works
Leveraging electronic health records for predictive modeling of post-surgical complications
Weller, G. B., Lovely, J. K., Larson, D. W., Earnshaw, B. A., & Huebner, M. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280217696115 · Cited by 62 · see citing works
Samson, P. P., Robinson, C. G., Bradley, J. D., Lee, A., Broderick, S., Kreisel, D., Krupnick, A. S., Patterson, G. A., Puri, V., Meyers, B. F., et al. (2015) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2015.08.058 · Cited by 57 · see citing works
Using Electronic Health Records for Surgical Quality Improvement in the Era of Big Data
Anderson, J. E., & Chang, D. C. (2014) · JAMA Surgery · doi:10.1001/jamasurg.2014.947 · Cited by 51 · see citing works
Barriers to Achieving Economies of Scale in Analysis of EHR Data: A Cautionary Tale
Sendak, M. P., Balu, S., & Schulman, K. A. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2017-03-cr-0046 · Cited by 45 · see citing works
Self-citationMassoumi, R. L., Trevino, C. M., & Webb, T. P. (2016) · World Journal of Surgery · doi:10.1007/s00268-016-3816-3 · Cited by 37 · see citing works
Risk Prediction With Electronic Health Records
Goldstein, B. A., Návar, A. M., & Pencina, M. J. (2016) · JAMA Cardiology · doi:10.1001/jamacardio.2016.3826 · Cited by 33 · see citing works
Resolving the Productivity Paradox of Health Information Technology
Wachter, R. M., & Howell, M. D. (2018) · JAMA · doi:10.1001/jama.2018.5605 · Cited by 27 · see citing works
Flagged first, then by citations
2 of the 39 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Accessing Artificial Intelligence for Clinical Decision-Making
Giordano, C., Brennan, M., Mohamed, B., Rashidi, P., Modave, F., & Tighe, P. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.645232 · Cited by 326 · see citing works
The potential of artificial intelligence to improve patient safety: a scoping review
Bates, D. W., Levine, D. M., Syrowatka, A., Kuznetsova, M., Craig, K. J. T., Rui, A., Jackson, G. P., & Rhee, K. B. L. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00423-6 · Cited by 329 · see citing works
Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287 · see citing works
Self-citationDevelopment and Validation of a Deep Learning Model for Non–Small Cell Lung Cancer Survival
She, Y., Jin, Z., Wu, J., Deng, J., Zhang, L., Su, H., Jiang, G., Liu, H., Xie, D., Cao, N., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.5842 · Cited by 270 · see citing works
A Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172 · see citing works
Self-citationArtificial intelligence surgery: how do we get to autonomous actions in surgery?
Gumbs, A. A., Frigerio, I., Spolverato, G., Croner, R., Illanes, A., Chouillard, E., & Elyan, E. (2021) · Sensors · doi:10.3390/s21165526 · Cited by 161 · see citing works
Brajer, N., Cozzi, B., Gao, M., Nichols, M., Revoir, M., Balu, S., Futoma, J., Bae, J., Setji, N. P., Hernandez, A. F., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.20733 · Cited by 107 · see citing works
Self-citationLee, T. C., Shah, N. U., Haack, A., & Baxter, S. L. (2020) · Informatics · doi:10.3390/informatics7030025 · Cited by 133 · see citing works
The Secondary Use of Electronic Health Records for Data Mining: Data Characteristics and Challenges
Sarwar, T., Seifollahi, S., Chan, J., Zhang, X., Aksakallı, V., Hudson, I. L., Verspoor, K. M., & Cavedon, L. (2022) · ACM Computing Surveys · doi:10.1145/3490234 · Cited by 129 · see citing works
Better medicine through machine learning: What’s real, and what’s artificial?
Saria, S., Butte, A. J., & Sheikh, A. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002721 · Cited by 128 · see citing works
Artificial intelligence in sleep medicine: background and implications for clinicians
Goldstein, C. A., Berry, R. B., Kent, D. T., Kristo, D. A., Seixas, A. A., Redline, S. S., & Westover, M. B. (2020) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.8388 · Cited by 145 · see citing works
Mahajan, A., Esper, S. A., Oo, T. H., McKibben, J., Garver, M., Artman, J., Klahre, C., Ryan, J. P., Sadhasivam, S., Holder‐Murray, J., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.22285 · Cited by 112 · see citing works
Bonde, A., Varadarajan, K. M., Bonde, N., Troelsen, A., Muratoglu, O. K., Malchau, H., Yang, A. D., Alam, H. B., & Sillesen, M. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00084-4 · Cited by 113 · see citing works
Bishara, A. M., Chiu, C., Whitlock, E. L., Douglas, V. C., Lee, S., Butte, A. J., Leung, J. M., & Donovan, A. L. (2022) · BMC Anesthesiology · doi:10.1186/s12871-021-01543-y · Cited by 112 · see citing works
Machine learning for early detection of sepsis: an internal and temporal validation study
Bedoya, A. D., Futoma, J., Clement, M. E., Corey, K., Brajer, N., Lin, A., Simons, M. G., Gao, M., Nichols, M., Balu, S., et al. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa006 · Cited by 109 · see citing works
Self-citationPredictive Utility of a Machine Learning Algorithm in Estimating Mortality Risk in Cardiac Surgery
Kilic, A., Goyal, A., Miller, J. K., Gjekmarkaj, E., Tam, W. L., Gleason, T. G., Sultan, I. S., & Dubrawksi, A. (2019) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2019.09.049 · Cited by 102 · see citing works
Wong, D. J. N., Harris, S. K., Sahni, A., Bedford, J., Cortes, L., Shawyer, R., Wilson, A. M., Lindsay, H. A., Campbell, D. A., Popham, S., et al. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003253 · Cited by 98 · see citing works
Geriatric Preoperative Optimization: A Review
Zietlow, K. E., Wong, S. P., Heflin, M. T., McDonald, S. R., Sickeler, R. A., Devinney, M. J., Blitz, J. D., Lagoo‐Deenadayalan, S. A., & Berger, M. (2021) · The American Journal of Medicine · doi:10.1016/j.amjmed.2021.07.028 · Cited by 98 · see citing works
Self-citationMachine Learning in Healthcare Data Analysis: A Survey
Dhillon, A. P., & Singh, A. (2019) · Journal of Biology and Today s World · Cited by 86 · see citing works
The Role of Artificial Intelligence in Surgery
Hashimoto, D. A., Ward, T. M., & Meireles, O. R. (2020) · Advances in Surgery · doi:10.1016/j.yasu.2020.05.010 · Cited by 78 · see citing works
Cabitza, F., & Zeitoun, J.-D. (2019) · Annals of Translational Medicine · doi:10.21037/atm.2019.04.07 · Cited by 72 · see citing works
Surgical data science – from concepts toward clinical translation
Maier‐Hein, L., Eisenmann, M., Sarıkaya, D., März, K., Collins, T., Malpani, A., Fallert, J., Feußner, H., Giannarou, S., Mascagni, P., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102306 · Cited by 88 · see citing works
Yuan, H., Yu, K., Xie, F., Liu, M., & Sun, S. (2024) · Medicine Advances · doi:10.1002/med4.75 · Cited by 61 · see citing works
Zanella-Calzada, L. A., Galvan-Tejada, C. E., Chávez-Lamas, N. M., Gracia-Cortés, M. D. C., Magallanes‐Quintanar, R., Celaya-Padilla, J. M., Galván-Tejada, J. I., & Gamboa-Rosales, H. (2019) · Diagnostics · doi:10.3390/diagnostics9010008 · Cited by 60 · see citing works
Thompson, M., Hill, B. L., Rakocz, N., Chiang, J. N., Geschwind, D. H., Sankararaman, S., Hofer, I. S., Cannesson, M. P., Zaitlen, N., & Halperin, E. (2022) · npj Genomic Medicine · doi:10.1038/s41525-022-00320-1 · Cited by 56 · see citing works
Nationwide prediction of type 2 diabetes comorbidities
Dworzynski, P., Aasbrenn, M., Rostgaard, K., Melbye, M., Gerds, T., Hjalgrim, H., & Pers, T. (2019) · Scientific Reports · doi:10.1038/s41598-020-58601-7 · Cited by 51 · see citing works
Maurer, L. R., Bertsimas, D., Bouardi, H. T., Hechi, M. W. E., Moheb, M. N. E., Giannoutsou, K., Zhuo, Y. D., Dunn, J., Velmahos, G. C., & Kaafarani, H. M. A. (2021) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000003158 · Cited by 58 · see citing works
Bompelli, A., Wang, Y., Wan, R., Singh, E., Zhou, Y., Xu, L., Oniani, D., Kshatriya, B. S. A., Balls-Berry, J. (. E., & Zhang, R. (2021) · Health Data Science · doi:10.34133/2021/9759016 · Cited by 58 · see citing works
Artificial intelligence and telemedicine in anesthesia: potential and problems
Bellini, V., Valente, M., Gaddi, A. V., Pelosi, P. P., & Bignami, E. G. (2022) · Minerva Anestesiologica · doi:10.23736/s0375-9393.21.16241-8 · Cited by 58 · see citing works
Nevin, L. M., & Editors, O. B. O. T. P. M. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002708 · Cited by 58 · see citing works
Vernooij, J. E. M., Koning, N. J., Geurts, J. W., Holewijn, S., Preckel, B., Kalkman, C. J., & Vernooij, L. M. (2023) · Anaesthesia · doi:10.1111/anae.15988 · Cited by 59 · see citing works
Song, B., Zhou, M. L., & Zhu, J. (2023) · Medical Science Monitor · doi:10.12659/msm.938835 · Cited by 50 · see citing works
Hyer, J. M., Ejaz, A., Tsilimigras, D. I., Paredes, A. Z., Mehta, R., & Pawlik, T. M. (2019) · JAMA Surgery · doi:10.1001/jamasurg.2019.2979 · Cited by 49 · see citing works
Salazar-Reyna, R., Aleu, F. G., Gutiérrez, E. M. A. G., Díaz-Ramírez, J., Garza‐Reyes, J. A., & Kumar, A., V (2020) · Management Decision · doi:10.1108/md-01-2020-0035 · Cited by 53 · see citing works
Distinctive Oculomotor Behaviors in Alzheimer's Disease and Frontotemporal Dementia
Lage, C., López‐García, S., Bejanin, A., Kazimierczak, M., Aracil‐Bolaños, I., Calvo-Córdoba, A., Pozueta, A., García‐Martínez, M., Fernández-Rodríguez, A., Bravo-González, M., et al. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2020.603790 · Cited by 49 · see citing works
Abraham, A., Le, B. L., Kosti, I., Straub, P., Velez-Edwards, D. R., Davis, L. K., Newton, J. M., Muglia, L. J., Rokas, A., Bejan, C. A., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02522-x · Cited by 47 · see citing works
Exploring Artificial Intelligence in Anesthesia: A Primer on Ethics, and Clinical Applications
Cascella, M., Tracey, M. C., Petrucci, E., & Bignami, E. G. (2023) · Surgeries · doi:10.3390/surgeries4020027 · Cited by 47 · see citing works
A framework for understanding label leakage in machine learning for health care
Davis, S. E., Matheny, M. E., Balu, S., & Sendak, M. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad178 · Cited by 55 · see citing works
Self-citationFenn, A., Davis, C., Buckland, D. M., Kapadia, N. N., Nichols, M., Gao, M. C., Knechtle, W. S., Balu, S., Sendak, M., & Theiling, J. (2021) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2021.02.029 · Cited by 45 · see citing works
Self-citationMøller, J. K., Sørensen, M., & Hardahl, C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248636 · Cited by 44 · see citing works
MacNell, N., Feinstein, L., Wilkerson, J., Salo, P. M., Molsberry, S. A., Fessler, M. B., Thorne, P. S., Motsinger‐Reif, A. A., & Zeldin, D. C. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280387 · Cited by 44 · see citing works
A Comprehensive Analysis and Review of Artificial Intelligence in Anaesthesia
Singhal, M., Gupta, L. K., & Hirani, K. (2023) · Cureus · doi:10.7759/cureus.45038 · Cited by 42 · see citing works
The digital transformation of surgery
Marwaha, J. S., Raza, M. M., & Kvedar, J. C. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00846-3 · Cited by 44 · see citing works
Afshar, M., Dligach, D., Sharma, B., Cai, X., Boyda, J., Birch, S., Valdez, D., Zelisko, S., Joyce, C., Modave, F. P., et al. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz068 · Cited by 34 · see citing works
Rossi, L. A., Melstrom, L. G., Fong, Y., & Sun, V. C.-Y. (2021) · Journal of Surgical Oncology · doi:10.1002/jso.26413 · Cited by 34 · see citing works
The automaton as a surgeon: the future of artificial intelligence in emergency and general surgery
Rimmer, L. J., Howard, C., Picca, L., & Bashir, M. (2020) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-020-01444-8 · Cited by 33 · see citing works
Burns, C. M., Pung, L., Witt, D., Gao, M. C., Sendak, M., Balu, S., Krakower, D., Marcus, J. L., Okeke, N. L., & Clement, M. E. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac775 · Cited by 33 · see citing works
Self-citationWang, W., Sheng, R., Liao, S., Wu, Z., Wang, L., Liu, C., Yang, C., & Jiang, R. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01172-0 · Cited by 31 · see citing works
Henn, J., Buneß, A., Schmid, M. C., Kalff, J. C., & Matthaei, H. (2021) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-021-02348-w · Cited by 30 · see citing works
Wagner, M., Brandenburg, J. M., Bodenstedt, S., Schulze, A., Jenke, A. C., Stern, A., Daum, M. T. J., Mündermann, L., Kolbinger, F. R., Bhasker, N., et al. (2022) · Surgical Endoscopy · doi:10.1007/s00464-022-09611-1 · Cited by 30 · see citing works
Reilly, J. R., Gabbe, B. J., Brown, W. A., Hodgson, C. L., & Myles, P. S. (2020) · ANZ Journal of Surgery · doi:10.1111/ans.16255 · Cited by 29 · see citing works
Innovations in Spine Surgery: A Narrative Review of Current Integrative Technologies
Bcharah, G., Gupta, N. K., Panico, N., Winspear, S., Bagley, A., Turnow, M., D’Amico, R. S., & Ukachukwu, A.-E. K. (2023) · World Neurosurgery · doi:10.1016/j.wneu.2023.12.124 · Cited by 28 · see citing works
Wang, Z., Zhe, S., Zimmerman, J., Morrissey, C. K., Tonna, J. E., Sharma, V., & Metcalf, R. A. (2022) · Scientific Reports · doi:10.1038/s41598-022-05445-y · Cited by 27 · see citing works
Melstrom, L. G., Rodin, A. S., Rossi, L. A., Fu, P. C., Fong, Y., & Sun, V. C.-Y. (2020) · Journal of Surgical Oncology · doi:10.1002/jso.26232 · Cited by 28 · see citing works
Ning, F., Gu, W., Zhao, Z., Du, W.-Y., Sun, M., Cao, S., Zeng, Y., Abe, M., & Zhang, C. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000269 · Cited by 25 · see citing works
Zhuang, Y., Dyas, A. R., Meguid, R. A., Henderson, W. G., Bronsert, M. R., Madsen, H. J., & Colborn, K. L. (2023) · Annals of Surgery · doi:10.1097/sla.0000000000006106 · Cited by 25 · see citing works
Zhang, L., Wang, Y., Niu, M., Wang, C., & Wang, Z. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3077114 · Cited by 25 · see citing works
Hechi, M. W. E., Eddine, S. N., Maurer, L. R., & Kaafarani, H. M. A. (2020) · Surgery · doi:10.1016/j.surg.2020.06.049 · Cited by 24 · see citing works
Artificial Intelligence, Machine Learning, and Surgical Science: Reality Versus Hype
Hechi, M. W. E., Ward, T. M., An, G. C., Maurer, L. R., Moheb, M. N. E., Tsoulfas, G., & Kaafarani, H. M. A. (2021) · Journal of Surgical Research · doi:10.1016/j.jss.2021.01.046 · Cited by 23 · see citing works
ChatGPT provides inconsistent risk-stratification of patients with atraumatic chest pain
Heston, T. F., & Lewis, L. M. (2023) · medRxiv · doi:10.1371/journal.pone.0301854 · Cited by 20 · see citing works
Systematic review of machine learning models for personalised dosing of heparin
Falconer, N., Abdel‐Hafez, A., Scott, I. A., Marxen, S., Canaris, S., & Barras, M. A. (2021) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14852 · Cited by 22 · see citing works
Lifting Hospital Electronic Health Record Data Treasures: Challenges and Opportunities
Maletzky, A., Böck, C., Tschoellitsch, T., Roland, T., Ludwig, H., Thumfart, S., Giretzlehner, M., Hochreiter, S., & Meier, J. M. (2022) · JMIR Medical Informatics · doi:10.2196/38557 · Cited by 23 · see citing works
Nammari, R. H. A., Simsekler, M. C. E., Gabor, A. F., & Qazi, A. J. (2022) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2022.3209879 · Cited by 21 · see citing works
Impact of Intraoperative Data on Risk Prediction for Mortality After Intra-Abdominal Surgery
Yan, X., Goldsmith, J., Mohan, S., Turnbull, Z. A., Freundlich, R. E., Billings, F. T., Kiran, R. P., Li, G., & Kim, M. (2021) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000005694 · Cited by 22 · see citing works
Machine-Learning-Based Prediction Modelling in Primary Care: State-of-the-Art Review
Sherbini, A. E., Virk, H. U. H., Wang, Z., Glicksberg, B. S., & Krittanawong, C. (2023) · AI · doi:10.3390/ai4020024 · Cited by 22 · see citing works
Chen, M., Kong, W., Li, B., Tian, Z., Yin, C., Zhang, M., Pan, H., & Bai, W. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1325179 · Cited by 20 · see citing works
Greenbury, S. F., Ougham, K., Wu, J., Battersby, C., Gale, C., Modi, N., & Angelini, E. D. (2021) · Scientific Reports · doi:10.1038/s41598-021-85878-z · Cited by 20 · see citing works
Artificial intelligence in vascular surgical decision making
Lareyre, F., Yeung, K. K., Guzzi, L., Lorenzo, G. D., Chaudhuri, A., Behrendt, C., Spanos, K. T., & Raffort, J. (2023) · Seminars in Vascular Surgery · doi:10.1053/j.semvascsurg.2023.05.004 · Cited by 19 · see citing works
Drossopoulos, P. N., Sharma, A., Ononogbu-Uche, F. C., Tabarestani, T. Q., Bartlett, A. M., Wang, T. Y., Huie, D., Gottfried, O. N., Blitz, J. D., Erickson, M. M., et al. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13082410 · Cited by 19 · see citing works
Niu, Z., Wu, X., Zhu, Y., Yang, L., Shi, Y., Wang, Y., Qiu, H. X., Gu, W., Wu, Y., Long, X., et al. (2022) · Neuroscience Bulletin · doi:10.1007/s12264-022-00871-4 · Cited by 19 · see citing works
van de Sande, D., van Genderen, M. E., Verhoef, C. K., Huiskens, J., Gommers, D. A. M. P. J., van Unen, E., Schasfoort, R. A., Schepers, J., van Bommel, J., & Grünhagen, D. J. (2022) · Surgery · doi:10.1016/j.surg.2022.03.031 · Cited by 18 · see citing works
Artificial Intelligence in Urology
Abid, R., Hussein, A. A., & Guru, K. A. (2023) · Urologic Clinics of North America · doi:10.1016/j.ucl.2023.06.005 · Cited by 18 · see citing works
New Horizons of Artificial Intelligence in Medicine and Surgery
Luțenco, V., Țocu, G., Guliciuc, M., Moraru, M., Candussi, I.-L., Dănilă, D. M., Luțenco, V., Dimofte, F., Mihailov, O. M., & Mihailov, R. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13092532 · Cited by 18 · see citing works
Integrating Artificial Intelligence into Causal Research in Epidemiology
Matthay, E. C., Neill, D. B., Titus, A. R., Desai, S. M., Troxel, A. B., Cerdá, M., Diaz, I. M., Santacatterina, M., & Thorpe, L. E. (2025) · Current Epidemiology Reports · doi:10.1007/s40471-025-00359-5 · Cited by 18 · see citing works
Wang, J., Lin, J., Wang, M., Meng, Z., Zhou, D., & Li, J. (2021) · Frontiers in Immunology · doi:10.3389/fimmu.2021.774664 · Cited by 17 · see citing works
Zhu, F., Zhong, R., Li, F., Li, C., Din, N., Sweidan, H., Potluri, L. B., Xiong, S., Li, J., Cheng, B., et al. (2023) · Translational Lung Cancer Research · doi:10.21037/tlcr-23-84 · Cited by 17 · see citing works
Samareh, A., Chang, X., Lober, W. B., Evans, H. L., Wang, Z., Qian, X., & Huang, S. (2019) · Surgical Infections · doi:10.1089/sur.2019.150 · Cited by 17 · see citing works
Management of abdominal emergencies in adults using telemedicine and artificial intelligence
Gorincour, G., Monneuse, O., Cheikh, A. B., Avondo, J., Chaillot, P.-F., Journe, C., Youssof, E., Lecomte, J.-C., & Thomson, V. (2021) · Journal of Visceral Surgery · doi:10.1016/j.jviscsurg.2021.01.008 · Cited by 17 · see citing works
van de Sande, D., van Genderen, M. E., Verhoef, C., van Bommel, J., Gommers, D., van Unen, E., Huiskens, J., & Grünhagen, D. J. (2021) · Surgery · doi:10.1016/j.surg.2021.05.005 · Cited by 15 · see citing works
Artificial intelligence and nonoperating room anesthesia
Pardo, E., Cam, E. L., & Verdonk, F. (2024) · Current Opinion in Anaesthesiology · doi:10.1097/aco.0000000000001388 · Cited by 15 · see citing works
Doherty, J. U., Daugherty, S. L., Kort, S., London, M. J., Mehran, R., Merli, G. J., Schoenhagen, P., Soman, P., Starling, R. C., Johnson, D. M., et al. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.07.022 · Cited by 15 · see citing works
Zhang, H., Cao, Y., Jiang, H., Zhou, Q., Yang, Q., & Cheng, L. (2025) · Clinical and Translational Allergy · doi:10.1002/clt2.70020 · Cited by 14 · see citing works
Steiger, E., & Kroll, L. E. (2023) · JMIR AI · doi:10.2196/40755 · Cited by 13 · see citing works
Machine Learning in Spinal Cord Stimulation for Chronic Pain
Hariharan, V., Harland, T., Young, C. M., Sagar, A., Gomez, M. M., & Pilitsis, J. G. (2023) · Operative Neurosurgery · doi:10.1227/ons.0000000000000774 · Cited by 13 · see citing works
Machine Learning and Artificial Intelligence in Surgical Research
Srinivas, S., & Young, A. J. (2023) · Surgical Clinics of North America · doi:10.1016/j.suc.2022.11.002 · Cited by 12 · see citing works
Cognitive Bias and Dissonance in Surgical Practice
Richburg, C. E., Dossett, L. A., & Hughes, T. M. (2023) · Surgical Clinics of North America · doi:10.1016/j.suc.2022.11.003 · Cited by 12 · see citing works
Esper, S. A., Holder‐Murray, J., Meister, K., Lin, H. S., Hamilton, D. K., Groff, Y. J., Zuckerbraun, B. S., & Mahajan, A. (2024) · JMIR Perioperative Medicine · doi:10.2196/52125 · Cited by 12 · see citing works
Testing Machine Learning Models to Predict Postoperative Ileus after Colorectal Surgery
Brydges, G., Chang, G. J., Gan, T. J., Konishi, T., Gottumukkala, V. N. R., & Uppal, A. (2024) · Current Oncology · doi:10.3390/curroncol31060262 · Cited by 13 · see citing works
Stacked Inverse Probability of Censoring Weighted Bagging: A Case Study In the InfCareHIV Register
Ginestet, P. G., Kotalik, A., Vock, D. M., Wolfson, J. A., & Gabriel, E. E. (2020) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/rssc.12448 · Cited by 12 · see citing works
Using the Surgical Risk Preoperative Assessment System to Define the “High Risk” Surgical Patient
Dyas, A. R., Bronsert, M. R., Meguid, R. A., Colborn, K. L., Lambert‐Kerzner, A., Hammermeister, K. E., Rozeboom, P. D., Velopulos, C. G., & Henderson, W. G. (2021) · Journal of Surgical Research · doi:10.1016/j.jss.2021.08.045 · Cited by 12 · see citing works
Assessing Quality of Surgical Real-World Data from an Automated Electronic Health Record Pipeline
Corey, K., Helmkamp, J. K., Simons, M. G., Curtis, L. H., Marsolo, K. A., Balu, S., Gao, M. C., Nichols, M., Watson, J. A., Mureebe, L., et al. (2020) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2019.12.005 · Cited by 12 · see citing works
Self-citationLessons from other disciplines for setting management thresholds for biodiversity conservation
Hilton, M., Walsh, J. C., Liddell, E., & Cook, C. N. (2021) · Conservation Biology · doi:10.1111/cobi.13865 · Cited by 12 · see citing works
Kansal, A., Gao, M. C., Balu, S., Nichols, M., Corey, K., Kashyap, S., & Sendak, M. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104466 · Cited by 11 · see citing works
Self-citationMatsumoto, K., Nohara, Y., Sakaguchi, M., Takayama, Y., Fukushige, S., Soejima, H., Nakashima, N., & Kamouchi, M. (2023) · JMIR Perioperative Medicine · doi:10.2196/50895 · Cited by 12 · see citing works
Velmahos, C. S., Paschalidis, A., & Paranjape, C. N. (2023) · The American Surgeon · doi:10.1177/00031348231167397 · Cited by 10 · see citing works
Amin, K. D., Weissler, E. H., Ratliff, W., Sullivan, A. E., Holder, T. A., Bury, C., Francis, S., Theiling, B. J., Hintze, B. J., Gao, M. C., et al. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.01.036 · Cited by 10 · see citing works
Self-citationAI-delirium guard: Predictive modeling of postoperative delirium in elderly surgical patients
Boppana, S. H., Tyagi, D., Komati, S. S. K., Boppana, S. L., Raj, R., & Mintz, C. D. (2025) · PLoS ONE · doi:10.1371/journal.pone.0322032 · Cited by 10 · see citing works
The Association Between Hearing Loss and Surgical Complications in Older Adults
Huang, R. J., Riska, K. M., Gordee, A., Peskoe, S. B., Francis, H. W., Witsell, D. L., & Smith, S. L. (2021) · Ear and Hearing · doi:10.1097/aud.0000000000001152 · Cited by 10 · see citing works
Osuwa, A. A., & Öztoprak, H. (2021) · International Conference on Engineering and Emerging Technologies (ICEET) · doi:10.1109/iceet53442.2021.9659597 · Cited by 10 · see citing works
Integration of AI in surgical decision support: improving clinical judgment
Balch, J. A., Shickel, B., Bihorac, A., Upchurch, G. R., & Loftus, T. J. (2024) · Global Surgical Education - Journal of the Association for Surgical Education · doi:10.1007/s44186-024-00257-2 · Cited by 9 · see citing works
- Authors
- Corey, K. M., Kashyap, S., Lorenzi, E., Lagoo-Deenadayalan, S. A., Heller, K., Whalen, K., Balu, S., Heflin, M. T., McDonald, S. R., Swaminathan, M., & Sendak, M.
- Affiliations
- Duke University (US)
- Corresponding author
- Corey, Kristin M.
- ORCID
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-11-27
- Volume
- 15
- Issue
- 11
- Pages
- e1002701
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC6258507
- OpenAlex
- W2901997892
- Cited by
- 204
- References
- 39
- Citations by year
- 2026: 14 · 2025: 17 · 2024: 25 · 2023: 34 · 2022: 30 · 2021: 40 · 2020: 28 · 2019: 14 · 2018: 2
- FWCI
- 11.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Duke Institue for Health Innovation (Clinical Research and Innovation Scholarship)
- Duke Forge (Graduate Fellowship Award)
- Health Resources and Services Administration (U1QHP28708)
- Duke Institute for Health Innovation (Internal Pilot Grant)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- postoperative complication prediction, automated data processing pipeline, random forest, extreme gradient boosting, 30-day postoperative mortality, postoperative shock, genitourinary complications, surgical risk calculator, clinical decision support, Pythia, model validation, surgical complexity
- MeSH terms
- Machine Learning, Adolescent, Adult, Aged, Automation, Female, Health Status, Humans, Male, Middle Aged, Postoperative Complications, Retrospective Studies, Risk Factors, Surgical Procedures, Operative, Reproducibility of Results, Comorbidity, Risk Assessment, Young Adult, Data Mining, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
