An attention based deep learning model of clinical events in the intensive care unit
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
This study trained long short-term memory (LSTM) recurrent neural networks (RNNs) incorporating an attention mechanism to predict daily sepsis, myocardial infarction (MI), and vancomycin antibiotic administration over two week patient ICU courses in the MIMIC-III dataset. These models achieved next-day predictive AUC of 0.876 for sepsis, 0.823 for MI, and 0.833 for vancomycin administration. Attention maps built from these models highlighted those times when input variables most influenced predictions and could provide a degree of interpretability to clinicians. These models appeared to attend to variables that were proxies for clinician decision-making, demonstrating a challenge of using flexible deep learning approaches trained with EHR data to build clinical decision support. While continued development and refinement is needed, we believe that such models could one day prove useful in reducing information overload for ICU physicians by providing needed clinical decision support for a variety of clinically important tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250
Vincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“As Lipton has noted, deep learning can facilitate interpretation by “learn[ing] rich representations that can be visualized, verbalized, or used for clustering” [39].”
BackgroundSeymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939
Vincent, J.-L., de Mendonca, A., Cantraine, F., Moreno, R., Takala, J., Suter, P. M., Sprung, C. L., Colardyn, F., & Blecher, S. (1998) · Critical Care Medicine · doi:10.1097/00003246-199811000-00016 · Cited by 3,463
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648
Cited by
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717
“Kaji et al. [80] demonstrated particular occasions when the input features have mostly influenced the predictions of clinical events in ICU patients using the attention mechanism.”
MethodExplainable AI for Healthcare 5.0: Opportunities and Challenges
Saraswat, D., Bhattacharya, P., Verma, A., Prasad, V. K., Tanwar, S., Sharma, G., Bokoro, P. N., & Sharma, R. (2022) · IEEE Access · doi:10.1109/access.2022.3197671 · Cited by 523
Lauritsen, S. M., Kristensen, M., Olsen, M. V., Larsen, M. S., Lauritsen, K. M., Jørgensen, M. J., Lange, J., & Thiesson, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17431-x · Cited by 448
Armoundas, A. A., Narayan, S. M., Arnett, D. K., Spector-Bagdady, K., Bennett, D. A., Celi, L. A., Friedman, P. A., Gollob, M. H., Hall, J. L., Kwitek, A. E., et al. (2024) · Circulation · doi:10.1161/cir.0000000000001201 · Cited by 347
“112 Implantable devices provide monitoring data that can also improve cardiovascular care.”
BackgroundPayrovnaziri, S. N., Chen, Z., Rengifo-Moreno, P., Miller, T., Bian, J., Chen, J. H., Liu, X., & He, Z. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa053 · Cited by 313
Early Prediction of Sepsis in the ICU Using Machine Learning: A Systematic Review
Moor, M., Rieck, B., Horn, M., Jutzeler, C. R., & Borgwardt, K. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.607952 · Cited by 231
Clinical applications of artificial intelligence in sepsis: A narrative review
Schinkel, M., Paranjape, K., Nannan Panday, R. S., Skyttberg, N., & Nanayakkara, P. W. B. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103488 · Cited by 130
Integrating multi-omics data with EHR for precision medicine using advanced artificial intelligence
Tong, L., Shi, W., Isgut, M., Zhong, Y., Lais, P., Gloster, L., Sun, J., Swain, A., Giuste, F., & Wang, M. D. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2023.3324264 · Cited by 130
Sheu, R.-K., & Pardeshi, M. S. (2022) · Sensors · doi:10.3390/s22208068 · Cited by 129
Mohd Faizal, A. S., Thevarajah, T. M., Khor, S. M., & Chang, S.-W. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106190 · Cited by 113
Hong, N., Liu, C., Gao, J., Han, L., Chang, F., Gong, M., & Su, L. (2022) · JMIR Medical Informatics · doi:10.2196/28781 · Cited by 76
Zilker, S., Weinzierl, S., Kraus, M., Zschech, P., & Matzner, M. (2024) · Health Care Management Science · doi:10.1007/s10729-024-09673-8 · Cited by 29
“To date, however, little work exists in this area and current approaches often do not distinguish between sequential and static features [e.g., 39, 40].”
BackgroundD’Hondt, E., Ashby, T. J., Chakroun, I., Koninckx, T., & Wuyts, R. (2022) · Communications Medicine · doi:10.1038/s43856-022-00225-1 · Cited by 29
“Putting objective numerical values on explainability is hard, but clearly this does not prevent explainability from being an important consideration; Kaji et al. 39 show that, especially for challenging and clinically relevant models, explainability can help to highlight problems with the undertaking of building ML mo…”
Enhancing sepsis management through machine learning techniques: A review
Ocampo-Quintero, N., Vidal-Cortés, P., del Río Carbajo, L., Fdez-Riverola, F., Reboiro-Jato, M., & Glez-Peña, D. (2022) · Medicina Intensiva · doi:10.1016/j.medin.2020.04.003 · Cited by 20
Yang, M., Liu, S., Hao, T., Ma, C., Chen, H., Li, Y., Wu, C., Xie, J., Qiu, H., Li, J., et al. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102785 · Cited by 10
Multimorbidity Measurement Strategies for Predicting Hospital Visits
Neves, B., Moreira, J. M., Gonçalves, S., Cerejo, J., Mota, I., Silva, N. A., Leite, F., & Silva, M. J. (2025) · doi:10.21203/rs.3.rs-6515465/v1
Metadata
- Authors
- Kaji, D. A., Zech, J. R., Kim, J. S., Cho, S. K., Dangayach, N. S., Costa, A. B., & Oermann, E. K.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-02-13
- Volume
- 14
- Issue
- 2
- Pages
- e0211057
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2913977338
- Cited by
- 198
- References
- 19
- FWCI
- 11.74
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.