Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative Complications and Report on a Mobile Platform
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Importance: Predicting postoperative complications has the potential to inform shared decisions regarding the appropriateness of surgical procedures, targeted risk-reduction strategies, and postoperative resource use. Realizing these advantages requires that accurate real-time predictions be integrated with clinical and digital workflows; artificial intelligence predictive analytic platforms using automated electronic health record (EHR) data inputs offer an intriguing possibility for achieving this, but there is a lack of high-level evidence from prospective studies supporting their use. Objective: To examine whether the MySurgeryRisk artificial intelligence system has stable predictive performance between development and prospective validation phases and whether it is feasible to provide automated outputs directly to surgeons' mobile devices. Design, Setting, and Participants: In this prognostic study, the platform used automated EHR data inputs and machine learning algorithms to predict postoperative complications and provide predictions to surgeons, previously through a web portal and currently through a mobile device application. All patients 18 years or older who were admitted for any type of inpatient surgical procedure (74 417 total procedures involving 58 236 patients) between June 1, 2014, and September 20, 2020, were included. Models were developed using retrospective data from 52 117 inpatient surgical procedures performed between June 1, 2014, and November 27, 2018. Validation was performed using data from 22 300 inpatient surgical procedures collected prospectively from November 28, 2018, to September 20, 2020. Main Outcomes and Measures: Algorithms for generalized additive models and random forest models were developed and validated using real-time EHR data. Model predictive performance was evaluated primarily using area under the receiver operating characteristic curve (AUROC) values. Results: Among 58 236 total adult patients who received 74 417 major inpatient surgical procedures, the mean (SD) age was 57 (17) years; 29 226 patients (50.2%) were male. Results reported in this article focus primarily on the validation cohort. The validation cohort included 22 300 inpatient surgical procedures involving 19 132 patients (mean [SD] age, 58 [17] years; 9672 [50.6%] male). A total of 2765 patients (14.5%) were Black or African American, 14 777 (77.2%) were White, 1235 (6.5%) were of other races (including American Indian or Alaska Native, Asian, Native Hawaiian or Pacific Islander, and multiracial), and 355 (1.9%) were of unknown race because of missing data; 979 patients (5.1%) were Hispanic, 17 663 (92.3%) were non-Hispanic, and 490 (2.6%) were of unknown ethnicity because of missing data. A greater number of input features was associated with stable or improved model performance. For example, the random forest model trained with 135 input features had the highest AUROC values for predicting acute kidney injury (0.82; 95% CI, 0.82-0.83); cardiovascular complications (0.81; 95% CI, 0.81-0.82); neurological complications, including delirium (0.87; 95% CI, 0.87-0.88); prolonged intensive care unit stay (0.89; 95% CI, 0.88-0.89); prolonged mechanical ventilation (0.91; 95% CI, 0.90-0.91); sepsis (0.86; 95% CI, 0.85-0.87); venous thromboembolism (0.82; 95% CI, 0.81-0.83); wound complications (0.78; 95% CI, 0.78-0.79); 30-day mortality (0.84; 95% CI, 0.82-0.86); and 90-day mortality (0.84; 95% CI, 0.82-0.85), with accuracy similar to surgeons' predictions. Compared with the original web portal, the mobile device application allowed efficient fingerprint login access and loaded data approximately 10 times faster. The application output displayed patient information, risk of postoperative complications, top 3 risk factors for each complication, and patterns of complications for individual surgeons compared with their colleagues. Conclusions and Relevance: In this study, automated real-time predictions of postoperative complications with mobile device outputs had good performance in clinical settings with prospective validation, matching surgeons' predictive accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Burnout and Medical Errors Among American Surgeons
Shanafelt, T. D., Balch, C. M., Bechamps, G., Russell, T., Dyrbye, L., Satele, D., Collicott, P., Novotny, P. J., Sloan, J., & Freischlag, J. (2010) · Annals of Surgery · doi:10.1097/sla.0b013e3181bfdab3 · Cited by 2,099
Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731
Dimick, J. B., Chen, S. L., Taheri, P. A., Henderson, W. G., Khuri, S. F., & Campbell, D. A. (2004) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2004.05.276 · Cited by 933
Shahian, D. M., Jacobs, J. P., Badhwar, V., Kurlansky, P. A., Furnary, A. P., Cleveland, J. C., Lobdell, K. W., Vassileva, C., Wyler von Ballmoos, M. C., Thourani, V. H., et al. (2018) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2018.03.002 · Cited by 459
Bihorac, A., Ozrazgat-Baslanti, T., Ebadi, A., Motaei, A., Madkour, M., Pardalos, P. M., Lipori, G., Hogan, W. R., Efron, P. A., Moore, F., et al. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000002706 · Cited by 377
Self-citationComplications in surgical patients.
Healey, M. A. (2002) · Archives of Surgery · doi:10.1001/archsurg.137.5.611 · Cited by 343
Bertsimas, D., Dunn, J., Velmahos, G. C., & Kaafarani, H. M. A. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002956 · Cited by 321
An examination of American College of Surgeons NSQIP surgical risk calculator accuracy.
Cohen, M. E., Liu, Y., Ko, C. Y., & Hall, B. L. (2017) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2016.12.057 · Cited by 176
Shickel, B., Loftus, T. J., Adhikari, L., Ozrazgat-Baslanti, T., Bihorac, A., & Rashidi, P. (2019) · Scientific Reports · doi:10.1038/s41598-019-38491-0 · Cited by 165
Self-citationBrennan, M., Puri, S., Ozrazgat-Baslanti, T., Feng, Z., Ruppert, M., Hashemighouchani, H., Momcilovic, P., Li, X., Wang, D. Z., & Bihorac, A. (2019) · Surgery · doi:10.1016/j.surg.2019.01.002 · Cited by 131
Self-citation
Cited by
Artificial intelligence in surgery
Varghese, C., Harrison, E. M., O’Grady, G., & Topol, E. J. (2024) · Nature Medicine · doi:10.1038/s41591-024-02970-3 · Cited by 287
Sun, J., Feng, T., Wang, B., Li, F., Han, B., Chu, M., Gong, F., Yi, Q., Zhou, X., Chen, S., et al. (2025) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00245-0 · Cited by 9
Electronic health record use and perceptions among urologic surgeons
Tan, H.-J., Chung, A. E., Gotz, D., Deal, A. M., Heiling, H. M., Teal, R., Vu, M. B., Meeks, W. D., Fang, R., Bennett, A. V., et al. (2023) · Applied Clinical Informatics · doi:10.1055/s-0043-1763513 · Cited by 8
Metadata
- Authors
- Ren, Y., Loftus, T. J., Datta, S., Ruppert, M. M., Guan, Z., Miao, S., Shickel, B., Feng, Z., Giordano, C., Upchurch, G. R., Rashidi, P., Ozrazgat-Baslanti, T., & Bihorac, A.
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2022-05-16
- Volume
- 5
- Issue
- 5
- Pages
- e2211973
- ISSN
- 2574-3805 (ISSN-L)
- OpenAlex
- W4280528360
- Cited by
- 136
- References
- 15
- FWCI
- 8.40
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative Complications and Report on a Mobile Platform. https://app.cochat.ai/papers/10.1001/jamanetworkopen.2022.11973
- The paper
- Ren, Y., Loftus, T. J., Datta, S., Ruppert, M. M., Guan, Z., Miao, S., Shickel, B., Feng, Z., Giordano, C., Upchurch, G. R., Rashidi, P., Ozrazgat-Baslanti, T., & Bihorac, A. (2022). Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative Complications and Report on a Mobile Platform. JAMA Network Open. https://doi.org/10.1001/jamanetworkopen.2022.11973
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.