The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
An automated machine learning-based model predicts postoperative mortality using readily-extractable preoperative electronic health record data
Open accessPublished version of a preprint
Abstract
Background Rapid, preoperative identification of patients with the highest risk for medical complications is necessary to ensure that limited infrastructure and human resources are directed towards those most likely to benefit. Existing risk scores either lack specificity at the patient level or utilise the American Society of Anesthesiologists (ASA) physical status classification, which requires a clinician to review the chart.Methods We report on the use of machine learning algorithms, specifically random forests, to create a fully automated score that predicts postoperative in-hospital mortality based solely on structured data available at the time of surgery. Electronic health record data from 53 097 surgical patients (2.01% mortality rate) who underwent general anaesthesia between April 1, 2013 and December 10, 2018 in a large US academic medical centre were used to extract 58 preoperative features.Results Using a random forest classifier we found that automatically obtained preoperative features (area under the curve [AUC] of 0.932, 95% confidence interval [CI] 0.910-0.951) outperforms Preoperative Score to Predict Postoperative Mortality (POSPOM) scores (AUC of 0.660, 95% CI 0.598-0.722), Charlson comorbidity scores (AUC of 0.742, 95% CI 0.658-0.812), and ASA physical status (AUC of 0.866, 95% CI 0.829-0.897). Including the ASA physical status with the preoperative features achieves an AUC of 0.936 (95% CI 0.917-0.955).Conclusions This automated score outperforms the ASA physical status score, the Charlson comorbidity score, and the POSPOM score for predicting in-hospital mortality. Additionally, we integrate this score with a previously published postoperative score to demonstrate the extent to which patient risk changes during the perioperative period.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another advantage is the promise that the use of ML in medicine will facilitate an understanding of what features drive outcomes (26).”
From Accessing Artificial Intelligence for Clinical Decision-Making (2021) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
Validation of a combined comorbidity index
Charlson, M., Szatrowski, T. P., Peterson, J., & Gold, J. (1994) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)90129-5 · Cited by 6,587
Quan, H., Li, B., Couris, C. M., Fushimi, K., Graham, P. J., Hider, P., Januel, J., & Sundararajan, V. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwq433 · Cited by 5,967
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648
Scalable and accurate deep learning with electronic health records.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · eScholarship (California Digital Library) · Cited by 2,242
Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine LearningPreprint
Lemaître, G., Nogueira, F., & Aridas, C. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.06570 · Cited by 1,923
Bilimoria, K., Liu, Y., Paruch, J., Zhou, L., Kmiecik, T. E., Ko, C. Y., & Cohen, M. E. (2013) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2013.07.385 · Cited by 1,854
Regularization methods for learning incomplete matrices
Mazumder, R., Hastie, T. J., & Tibshirani, R. J. (2009) · PubMed · doi:10.48550/arxiv.0906.2034 · Cited by 1,175
ASA classification and perioperative variables as predictors of postoperative outcome
Wolters, U., Wolf, T., Stützer, H., & Schröder, T. (1996) · British Journal of Anaesthesia · doi:10.1093/bja/77.2.217 · Cited by 1,072
American Society of Anaesthesiologists physical status classification
Daabiss, M. A. (2011) · Indian Journal of Anaesthesia · doi:10.4103/0019-5049.79879 · Cited by 919
Early Surgery versus Conventional Treatment for Infective Endocarditis
Kang, D.-H., Kim, Y.-J., Kim, S.-H., Sun, B. J., Kim, D.-H., Yun, S.-C., Song, J.-M., Choo, S. J., Chung, C.-H., Song, J.-K., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1112843 · Cited by 902
Spectral Regularization Algorithms for Learning Large Incomplete Matrices
MazumderRahul, HastieTrevor, & TibshiraniRobert (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859931 · Cited by 671
Identification and characterisation of the high-risk surgical population in the United Kingdom
Pearse, R. M., Harrison, D. A., James, P., Watson, D. I., Hinds, C., Rhodes, A., Grounds, R. M., & Bennett, E. (2006) · Critical Care · doi:10.1186/cc4928 · Cited by 669
Hackett, N., Oliveira, G. S. D., Jain, U. K., & Kim, J. Y. S. (2015) · International Journal of Surgery · doi:10.1016/j.ijsu.2015.04.079 · Cited by 653
Reliability of the American Society of Anesthesiologists physical status scale in clinical practice
Sankar, A., Johnson, S. R., Beattie, W. S., Tait, G., & Wijeysundera, D. N. (2014) · British Journal of Anaesthesia · doi:10.1093/bja/aeu100 · Cited by 555
Artificial intelligence powers digital medicine
Fogel, A. L., & Kvedar, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-017-0012-2 · Cited by 473
Vacanti, C. J., VanHOUTEN, R. J., & HILL, R. C. (1970) · Anesthesia & Analgesia · doi:10.1213/00000539-197007000-00010 · Cited by 390
The ASA classification and peri-operative risk
Fitz-Henry, J. (2011) · Annals of The Royal College of Surgeons of England · doi:10.1308/rcsann.2011.93.3.185a · Cited by 373
for the CONSORT Group*, Moher, D., Schulz, K. F., & Altman, D. G. (2003) · Clinical Oral Investigations · doi:10.1007/s00784-002-0188-x · Cited by 259
Preoperative Score to Predict Postoperative Mortality (POSPOM)
Manach, Y. L., Collins, P. G. S., Rodseth, R. N., Bihan-Benjamin, C. L., Biccard, B. M., Riou, B., Devereaux, P. J., & Landais, P. (2015) · Anesthesiology · doi:10.1097/aln.0000000000000972 · Cited by 199
Lee, C. K., Hofer, I., Gabel, E., Baldi, P., & Cannesson, M. (2018) · Anesthesiology · doi:10.1097/aln.0000000000002186 · Cited by 185
Self-citationSessler, D. I., Sigl, J. C., Manberg, P. J., Kelley, S. D., Schubert, A., & Chamoun, N. G. (2010) · Anesthesiology · doi:10.1097/aln.0b013e3181f79a8d · Cited by 105
Dalton, J. E., Kurz, A. M., Turan, A., Mascha, E. J., Sessler, D. I., & Saager, L. (2011) · Anesthesiology · doi:10.1097/aln.0b013e318219d5f9 · Cited by 94
A Systematic Approach to Creation of a Perioperative Data Warehouse
Hofer, I. S., Gabel, E., Pfeffer, M. A., Mahbouba, M., & Mahajan, A. (2016) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000001201 · Cited by 75
Self-citationLeeds, I. L., Truta, B., Parian, A. M., Chen, S. Y., Efron, J. E., Gearhart, S. L., Safar, B. A., & Fang, S. H. (2017) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-017-3538-3 · Cited by 56
The Validity of Discharge Billing Codes Reflecting Severe Maternal Morbidity
Sigakis, M. J. G., Leffert, L. R., Mirzakhani, H., Sharawi, N., Rajala, B., Callaghan, W. M., Kuklina, E. V., Creanga, A. A., Mhyre, J. M., & Bateman, B. T. (2016) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000001436 · Cited by 47
Impact of Present-on-admission Indicators on Risk-adjusted Hospital Mortality Measurement
Dalton, J. E., Glance, L. G., Mascha, E. J., Ehrlinger, J., Chamoun, N. G., & Sessler, D. I. (2013) · Anesthesiology · doi:10.1097/aln.0b013e31828e12b3 · Cited by 39
Rubinsteyn, A., Feldman, S., O’Donnell, T., & Beaulieu‐Jones, B. K. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.594235 · Cited by 15
Machine Learning Tool for Automatic ASA Detection
Lazouni, M. E. A., Daho, M. E. H., Settouti, N., Chikh, M. A., & Mahmoudi, S. (2013) · Studies in computational intelligence · doi:10.1007/978-3-319-00560-7_5 · Cited by 7
Data-Driven System for Perioperative Acuity Prediction.
Zhang, L., Fabbri, D., & Wanderer, J. P. (2016) · AMIA · Cited by 2
Nonstoichiometry and stability in water of undoped SrCeO3
Mather, G. C., & Egea, J. R. J. (2003) · DIGITAL.CSIC (Spanish National Research Council (CSIC)) · doi:10.13039/501100000780 · Cited by 1
The Validity of Discharge Billing Codes Reflecting Severe Maternal Morbidity
Sigakis, M. J. G., Leffert, L. R., Mirzakhani, H., Sharawi, N., Rajala, B., Callaghan, W. M., Kuklina, E. V., Creanga, A. A., Mhyre, J. M., & Bateman, B. T. (2017) · Obstetric Anesthesia Digest · doi:10.1097/01.aoa.0000512008.03839.93 · Cited by 1
Scalable and accurate deep learning for electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Liu, P. J., Liu, X., Sun, M., Sundberg, P., Yee, H., et al. (2018) · arXiv · doi:10.48550/arxiv.1801.07860
Flagged first, then by citations
Cited by
Sorted by citations
Accessing Artificial Intelligence for Clinical Decision-Making
Giordano, C., Brennan, M., Mohamed, B., Rashidi, P., Modave, F., & Tighe, P. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.645232 · Cited by 326
“Another advantage is the promise that the use of ML in medicine will facilitate an understanding of what features drive outcomes (26).”
BackgroundMachine Learning and Surgical Outcomes Prediction: A Systematic Review
Elfanagely, O., Toyoda, Y., Othman, S., Mellia, J. A., Basta, M. N., Liu, T., Kording, K. P., Ungar, L., & Fischer, J. P. (2021) · Journal of Surgical Research · doi:10.1016/j.jss.2021.02.045 · Cited by 141
Large Language Model Capabilities in Perioperative Risk Prediction and Prognostication
Chung, P., Fong, C. T., Walters, A. M., Aghaeepour, N., Yetişgen, M., & O’Reilly-Shah, V. N. (2024) · JAMA Surgery · doi:10.1001/jamasurg.2024.1621 · Cited by 140
Klug, M., Barash, Y., Bechler, S., Resheff, Y. S., Tron, T., Ironi, A., Soffer, S., Zimlichman, E., & Klang, E. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05512-7 · Cited by 120
Bishara, A. M., Chiu, C., Whitlock, E. L., Douglas, V. C., Lee, S., Butte, A. J., Leung, J. M., & Donovan, A. L. (2022) · BMC Anesthesiology · doi:10.1186/s12871-021-01543-y · Cited by 118
“…prediction models have been developed to predict delirium in hospitalized patients [17], postoperative delirium in focused patient populations [18], non-delirium-related intraoperative complications [19, 20], and postoperative mortality [21], in addition to applications in many other contexts [22].”
Background
Metadata
- Authors
- Hill, B. L., Brown, R., Gabel, E., Rakocz, N., Lee, C., Cannesson, M., Baldi, P., Olde Loohuis, L., Johnson, R., Jew, B., Maoz, U., Mahajan, A., Sankararaman, S., Hofer, I., & Halperin, E.
- Affiliations
- University of California, Los Angeles (US)
- University of California, Irvine (US)
- California Institute of Technology (US)
- University of Pittsburgh (US)
- Corresponding author
- Hofer, Ira
- ORCID
- Venue
- British Journal of Anaesthesia
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 123
- Issue
- 6
- Pages
- 877-886
- ISSN
- 0007-0912 (ISSN-L)
- PMC
- PMC6883494
- OpenAlex
- W2980769368
- Cited by
- 103
- References
- 39
- Citations by year
- 2026: 14 · 2025: 16 · 2024: 19 · 2023: 13 · 2022: 21 · 2021: 12 · 2020: 7 · 2019: 1
- FWCI
- 7.40
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- British Journal of Anaesthesia (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion, cc-by-nc-nd)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- Chapman University Digital Commons (Chapman University) (submittedVersion)
- Funders
- National Institute of Mental Health (K99MH116115)
- University of California, Los Angeles (DGE- 1650604)
- National Institute of Neurological Disorders and Stroke (T32NS048004)
- National Institutes of Health (III-1705121, R01 NR013912, R01 GM117622, R00GM111744, R35GM125055)
- Alfred P. Sloan Foundation
- National Science Foundation (III-1705121, 1705197)
- Masimo
- Edwards Lifesciences
- Okawa Foundation for Information and Telecommunications
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- postoperative mortality prediction, preoperative risk assessment, electronic health record data, random forest, general anesthesia, surgical patients, ASA physical status, Charlson Comorbidity Index
- MeSH terms
- Machine Learning, Adolescent, Adult, Aged, Aged, 80 and over, California, Female, Health Status, Humans, Male, Middle Aged, Postoperative Complications, Risk Factors, Comorbidity, Hospital Mortality, Risk Assessment, Young Adult, Preoperative Period, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
