Surgical risk is not linear: derivation and validation of a novel, user-friendly, and machine-learning-based Predictive Optimal Trees in Emergency Surgery Risk (POTTER) calculator.
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Introduction: Most risk assessment tools assume that the impact of risk factors is linear and cumulative. Using novel machine-learning techniques, we sought to design an interactive, nonlinear risk calculator for Emergency Surgery (ES). Methods: All ES patients in the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) 2007 to 2013 database were included (derivation cohort). Optimal Classification Trees (OCT) were leveraged to train machine-learning algorithms to predict postoperative mortality, morbidity, and 18 specific complications (eg, sepsis, surgical site infection). Unlike classic heuristics (eg, logistic regression), OCT is adaptive and reboots itself with each variable, thus accounting for nonlinear interactions among variables. An application [Predictive OpTimal Trees in Emergency Surgery Risk (POTTER)] was then designed as the algorithms’ interactive and user-friendly interface. POTTER performance was measured (c-statistic) using the 2014 ACS-NSQIP database (validation cohort) and compared with the American Society of Anesthesiologists (ASA), Emergency Surgery Score (ESS), and ACS-NSQIP calculators’ performance. Results: Based on 382,960 ES patients, comprehensive decision-making algorithms were derived, and POTTER was created where the provider's answer to a question interactively dictates the subsequent question. For any specific patient, the number of questions needed to predict mortality ranged from 4 to 11. The mortality c-statistic was 0.9162, higher than ASA (0.8743), ESS (0.8910), and ACS (0.8975). The morbidity c-statistics was similarly the highest (0.8414). Conclusion: POTTER is a highly accurate and user-friendly ES risk calculator with the potential to continuously improve accuracy with ongoing machine-learning. POTTER might prove useful as a tool for bedside preoperative counseling of ES patients and families.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, efforts to use manual data entry to feed machine learning models for surgical risk prediction on mobile devices have been successful.14”
From Artificial Intelligence and Surgical Decision-making. (2020) · cites this paper as [14]
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Comorbidity measures for use with administrative data.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998) · Medical Care · doi:10.1097/00005650-199801000-00004 · Cited by 9,921
Validation of a combined comorbidity index
Charlson, M., Szatrowski, T. P., Peterson, J., & Gold, J. (1994) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)90129-5 · Cited by 6,587
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
Bilimoria, K. Y., Liu, Y., Paruch, J. L., Zhou, L., Kmiecik, T. E., Ko, C. Y., & Cohen, M. E. (2013) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2013.07.385 · Cited by 1,905
Khuri, S. F., Daley, J., Henderson, W., Hur, K., Demakis, J., Aust, J. B., Chong, V., Fabri, P. J., Gibbs, J. O., Grover, F., et al. (1998) · Annals of Surgery · doi:10.1097/00000658-199810000-00006 · Cited by 1,532
Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations
Chen, J. H., & Asch, S. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1702071 · Cited by 1,171
ASA classification and perioperative variables as predictors of postoperative outcome
Wolters, U., Wolf, T., Stützer, H., & Schröder, T. (1996) · British Journal of Anaesthesia · doi:10.1093/bja/77.2.217 · Cited by 1,073
Hall, B. L., Hamilton, B. H., Richards, K., Bilimoria, K. Y., Cohen, M. E., & Ko, C. Y. (2009) · Annals of Surgery · doi:10.1097/sla.0b013e3181b4148f · Cited by 753
Bertsimas, D., & Dunn, J. (2017) · Machine Learning · doi:10.1007/s10994-017-5633-9 · Cited by 638
Self-citation
Cited by
Artificial Intelligence and Surgical Decision-making.
Loftus, T. J., Tighe, P. J., Filiberto, A. C., Efron, P. A., Brakenridge, S. C., Mohr, A. M., Rashidi, P., Upchurch, G. R., & Bihorac, A. (2020) · JAMA Surgery · doi:10.1001/jamasurg.2019.4917 · Cited by 650
“14 Human Intuition— Human intuition seems to arise from dopaminergic limbic system neurons that modify their connections with one another when a certain pattern or situation leads to a reward or penalty such as pleasure or pain.”
BackgroundThe potential of artificial intelligence to improve patient safety: a scoping review
Bates, D. W., Levine, D., Syrowatka, A., Kuznetsova, M., Craig, K. J. T., Rui, A., Jackson, G. P., & Rhee, K. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00423-6 · Cited by 323
Artificial intelligence in surgery
Varghese, C., Harrison, E. M., O’Grady, G., & Topol, E. J. (2024) · Nature Medicine · doi:10.1038/s41591-024-02970-3 · Cited by 287
Coimbra, R., Edwards, S., Kurihara, H., Bass, G. A., Balogh, Z. J., Tilsed, J., Faccincani, R., Carlucci, M., Martínez Casas, I., Gaarder, C., et al. (2020) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-020-01364-7 · Cited by 211
Dixon, D., Sattar, H., Moros, N., Kesireddy, S. R., Ahsan, H., Lakkimsetti, M., Fatima, M., Doshi, D., Sadhu, K., & Junaid Hassan, M. (2024) · Cureus · doi:10.7759/cureus.59954 · Cited by 203
“The POTTER risk calculator outperformed the ASA, Emergency Surgery Score, and NSQIP in terms of predicting mortality and morbidity [86].”
Method
Metadata
- Authors
- Bertsimas, D., Dunn, J., Velmahos, G. C., & Kaafarani, H. M. A.
- Affiliations
- Massachusetts Institute of Technology (US)
- Harvard University (US)
- Corresponding author
- Bertsimas, Dimitris
- Dunn, Jack
- Venue
- Annals of Surgery
- Published
- 2018-10
- Volume
- 268
- Issue
- 4
- Pages
- 574-583
- ISSN
- 0003-4932 (ISSN-L)
- 1528-1140
- OpenAlex
- W2888528836
- Cited by
- 321
- References
- 36
- Citations by year
- 2026: 42 · 2025: 47 · 2024: 34 · 2023: 51 · 2022: 52 · 2021: 48 · 2020: 34 · 2019: 13
- FWCI
- 17.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Surgery (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Trauma and Emergency Care Studies
- Keywords
- emergency surgery, surgical risk prediction, postoperative mortality, postoperative morbidity, surgical complications, National Surgical Quality Improvement Program, preoperative counseling
- MeSH terms
- Machine Learning, Decision Support Techniques, Emergencies, Humans, Postoperative Complications, Predictive Value of Tests, Surgical Procedures, Operative, User-Computer Interface, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Surgical risk is not linear: derivation and validation of a novel, user-friendly, and machine-learning-based Predictive Optimal Trees in Emergency Surgery Risk (POTTER) calculator. https://app.cochat.ai/papers/10.1097/sla.0000000000002956
- The paper
- Bertsimas, D., Dunn, J., Velmahos, G. C., & Kaafarani, H. M. A. (2018). Surgical risk is not linear: derivation and validation of a novel, user-friendly, and machine-learning-based Predictive Optimal Trees in Emergency Surgery Risk (POTTER) calculator. Annals of Surgery. https://doi.org/10.1097/sla.0000000000002956
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.