Predicting Unplanned Transfers to the Intensive Care Unit: A Machine Learning Approach Leveraging Diverse Clinical Elements
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Early warning scores aid in the detection of pediatric clinical deteriorations but include limited data inputs, rarely include data trends over time, and have limited validation. OBJECTIVE: Machine learning methods that make use of large numbers of predictor variables are now commonplace. This work examines how different types of predictor variables derived from the electronic health record affect the performance of predicting unplanned transfers to the intensive care unit (ICU) at three large children's hospitals. METHODS: We trained separate models with data from three different institutions from 2011 through 2013 and evaluated models with 2014 data. Cases consisted of patients who transferred from the floor to the ICU and met one or more of 5 different priori defined criteria for suspected unplanned transfers. Controls were patients who were never transferred to the ICU. Predictor variables for the models were derived from vitals, labs, acuity scores, and nursing assessments. Classification models consisted of L1 and L2 regularized logistic regression and neural network models. We evaluated model performance over prediction horizons ranging from 1 to 16 hours. RESULTS: Across the three institutions, the c-statistic values for our best models were 0.892 (95% CI 0.875-0.904), 0.902 (95% CI 0.880-0.923), and 0.899 (95% CI 0.879-0.919) for the task of identifying unplanned ICU transfer 6 hours before its occurrence and achieved 0.871 (95% CI 0.855-0.888), 0.872 (95% CI 0.850-0.895), and 0.850 (95% CI 0.825-0.875) for a prediction horizon of 16 hours. For our first model at 80% sensitivity, this resulted in a specificity of 80.5% (95% CI 77.4-83.7) and a positive predictive value of 5.2% (95% CI 4.5-6.2). CONCLUSIONS: Feature-rich models with many predictor variables allow for patient deterioration to be predicted accurately, even up to 16 hours in advance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · cited 1×
“Finally, rich information is present in various free-text fields [37,38,24] that may provide indicators of clinician concern.”
BackgroundIntelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
Feature selection, L 1 vs. L 2 regularization, and rotational invariance
Ng, A. Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015435 · Cited by 1,572
Churpek, M. M., Yuen, T. C., Winslow, C., Meltzer, D. O., Kattan, M. W., & Edelson, D. P. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001571 · Cited by 638
Horng, S., Sontag, D. A., Halpern, Y., Jernite, Y., Shapiro, N. I., & Nathanson, L. A. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174708 · Cited by 310
Tsoukalas, A., Albertson, T., & Tagkopoulos, I. (2015) · JMIR Medical Informatics · doi:10.2196/medinform.3445 · Cited by 96 · cited 1×
“Most machine learning methods for predicting deterioration have focused on logistic regression models preceded by careful variable selection [12,13].”
MethodLuo, G. (2016) · Health Information Science and Systems · doi:10.1186/s13755-016-0015-4 · Cited by 85 · cited 1×
“In many cases, however, model interpretability can suffer [15,16].”
Background
Cited by
Emergency department triage prediction of clinical outcomes using machine learning models
Raita, Y., Goto, T., Faridi, M. K., Brown, D. F. M., Camargo, C. A., & Hasegawa, K. (2019) · Critical Care · doi:10.1186/s13054-019-2351-7 · Cited by 519
Artificial Intelligence -based technologies in nursing: A scoping literature review of the evidence
von Gerich, H., Moen, H., Block, L. J., Chu, C. H., DeForest, H., Hobensack, M., Michalowski, M., Mitchell, J., Nibber, R., Olalia, M. A., et al. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.104153 · Cited by 360
“…et al., 2016 , Singh et al., 2018 , Soufi et al., 2018 , Su et al., 2019 , Sullivan et al., 2019 , Tateno et al., 2020 , Topaz et al., 2019b , Wellner et al., 2017 , Wojtusiak et al., 2021 , Wong et al., 2018 , Yokota et al., 2017 , Zachariah et al., 2020 Natural language processing (7)…”
InfluentialGoto, T., Camargo, C. A., Faridi, M. K., Freishtat, R. J., & Hasegawa, K. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.6937 · Cited by 285
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269
Machine learning for microalgae detection and utilization
Ning, H., Li, R., & Zhou, T. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2022.947394 · Cited by 83
Early Warning Systems: The Neglected Importance of Timing
Rolnick, J. A. (2019) · Journal of Hospital Medicine · doi:10.12788/jhm.3229 · Cited by 7
Metadata
- Authors
- Wellner, B., Grand, J., Canzone, E., Coarr, M., Brady, P. W., Simmons, J., Kirkendall, E., Dean, N., Kleinman, M., & Sylvester, P.
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2017-11-22
- Volume
- 5
- Issue
- 4
- Pages
- e45
- ISSN
- 2291-9694 (ISSN-L)
- OpenAlex
- W2770222727
- Cited by
- 78
- References
- 8
- FWCI
- 4.26
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.