The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data
Abstract
With the improvement of medical data capturing, vast amount of continuous patient monitoring data, e.g., electrocardiogram (ECG), real-time vital signs and medications, become available for clinical decision support at intensive care units (ICUs). However, it becomes increasingly challenging to model such data, due to high density of the monitoring data, heterogeneous data types and the requirement for interpretable models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“All of these studies have shown that multimodal models outperform single-modality models in predictive performance.”
From Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review (2026) · cites this paper as [18]
In the citing paper
“Intermediate fusion methods modeled each modality separately over time before integrating learned representations using concatenation [57,58] or attention [18,59], but most relied on retrospective alignment within relatively coarse time windows.”
From Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review (2026) · cites this paper as [18]
In the citing paper
“To improve performance and provide a degree of interpretability to the model, an attention mechanism is adopted in 13 studies [6,14,27,31,43,57,64,72,76–78,80,88].”
From Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review (2021) · cites this paper as [14]
In the citing paper
“On the contrary, for those outcomes that happen in hospital, such as inpatient mortality [14,57,61,65], length-of-stay [14,83], and intervention prediction [32,55,65], observation windows at hour level are developed.”
From Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review (2021) · cites this paper as [14]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“Extensive attention-based deep models have been developed for images [14, 21, 36, 48], audios and videos [23, 49], machine translations [1, 30, 46].”
BackgroundHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 1×
“For the multiclass prediction task of forecasting length of stay, we evaluate the models in terms of Cohen’s Kappa [10] and Accuracy.”
Method(2008) · Journal of Machine Learning Research · Cited by 33,051
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 3×
“Intervention-based guidance Gt[1] For predicting output yt”
BackgroundBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
(2017) · Cited by 11,625
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 1×
“Extensive attention-based deep models have been developed for images [14, 21, 36, 48], audios and videos [23, 49], machine translations [1, 30, 46].”
BackgroundShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 1×
“Extensive attention-based deep models have been developed for images [14, 21, 36, 48], audios and videos [23, 49], machine translations [1, 30, 46].”
Background
Research works printed but not linked: 16
- Choi Edward, 2016, Elizabeth Searles, Catherine Coffey, and Jimeng Sun . 2016 b. Multi-layer Representation Learning for Medical Concepts. arXiv preprint arXiv:1602.05568 · arXiv
- Erhan Dumitru, 2009, Visualizing higher-layer features of a deep network, University of Montreal, 1341, 1
- Jagannatha Abhyuday N, 2016, Association for Computational Linguistics. North American Chapter. Meeting, 2016
- Jagannatha Abhyuday N, 2016, Conference on Empirical Methods in Natural Language Processing, 2016
- Johnson Alistair EW, 2016, Leo Anthony Celi, and Roger G Mark
- Kim Suyoun, 2015, Recurrent models for auditory attention in multi-microphone distance speech recognition. arXiv preprint arXiv:1511.06407 · arXiv
- Kim You Jin, 2017, Jin Joo Park, Borim Ryu, and Jung-Woo Ha .
- Kingma Diederik, 2014, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 · arXiv
- Le Quoc V, 2013, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 8595--8598
- Lipton Zachary C, 2015, Learning to diagnose with LSTM recurrent neural networks. arXiv preprint arXiv:1511.03677 · arXiv
- Mnih Volodymyr, 2014, et almbox
- Nguyen Phuoc, 2017, Deep Learning to Attend to Risk in ICU. arXiv preprint arXiv:1707.05010 · arXiv
- Shickel Benjamin, 2017, Azra Bihorac, and Parisa Rashidi .
- Sun Yan, 2005, Kap Luk Chan, and Shankar Muthu Krishnan
- Suresh Harini, 2017, Clinical Intervention Prediction and Understanding with Deep Neural Networks Machine Learning for Healthcare Conference. 322--337
- Williams B, 2012, et almbox
Cited by
Sorted by citations
Multitask learning and benchmarking with clinical time series data
Harutyunyan, H., Khachatrian, H., Kale, D. C., Ver Steeg, G., & Galstyan, A. (2019) · Scientific Data · doi:10.1038/s41597-019-0103-9 · Cited by 958
“nchmark codebase, several teams used our dataset generation pipeline (fully or partially). In particular, the pipeline was used for in-hospital mortality prediction [38–43], decompensation prediction [44], length-of-stay prediction [42, 44, 45], phenotyping [38, 39, 46] and readmission prediction [47]. Additionally, a…”
MethodRemote patient monitoring using artificial intelligence: Current state, applications, and challenges
Shaik, T., Tao, X., Higgins, N., Li, L., Gururajan, R., Zhou, X., & Acharya, U. R. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1485 · Cited by 620
Hong, S., Zhou, Y., Shang, J., Xiao, C., & Sun, J. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103801 · Cited by 510
“sification (e.g.,patients with paroxysmal atrial fibrillation or Healthy [55]), multi-class classification (e.g., detection of acute cognitive stress detection [31], decompensation of patients detection [131]) and multi-task classification (e.g., detection of prevalent hypertension, sleep apnea, and diabetes [89]). 2) Ann…”
BackgroundSelf-citationDeep learning in ECG diagnosis: A review
Liu, X., Wang, H., Li, Z., & Qin, L. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107187 · Cited by 473
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
“This area remains relatively unexplored in clinical ML explainability literature with a few exceptions (Xu et al., 2018; Choi et al., 2016; Yang et al., 2018) focused primarily on attention based and sensitivity analysis mechanisms in deep learning (Vaswani et al.”
BackgroundSelf-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387317 · Cited by 295
“T IME series data abound in many real-world scenarios [1], [2], including human activity recognition [3], industrial fault diagnosis [4], smart building management [5], and healthcare [6].”
BackgroundSi, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., & Roberts, K. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103671 · Cited by 235
Panigutti, C., Perotti, A., & Pedreschi, D. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372855 · Cited by 170
“Many interpretability approaches related to sequential data modeling focus on adding an attention mechanism [2, 44] to a sequential model and use the attention weights as a form of explanation [3, 11, 33, 47], however recent works have highlighted how this kind of explanation might lack consistency [6, 22, 41] and tha…”
BackgroundExploring Interpretable LSTM Neural Networks over Multi-Variable DataPreprint
Guo, T., Lin, T., & Antulov-Fantulin, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12034 · Cited by 154
M3Care: Learning with Missing Modalities in Multimodal Healthcare Data
Zhang, C., Chu, X., Ma, L., Zhu, Y., Wang, Y., Wang, J., & Zhao, J. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539388 · Cited by 102
Analyzing the role of model uncertainty for electronic health records
Dusenberry, M. W., Tran, D., Choi, E., Kemp, J., Nixon, J., Jerfel, G., Heller, K., & Dai, A. M. (2020) · ACM Conference on Health, Inference, and Learning (CHIL) · doi:10.1145/3368555.3384457 · Cited by 107
Benchmarking machine learning models on multi-centre eICU critical care dataset
Sheikhalishahi, S., Balaraman, V., & Osmani, V. (2020) · PLOS ONE · doi:10.1371/journal.pone.0235424 · Cited by 89
“Differently, Xu et al [35] proposed a deep learning model to predict the decompensation event.”
BackgroundHOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Units
Hong, S., Xu, Y., Khare, A., Priambada, S., Maher, K., Aljiffry, A., Sun, J., & Tumanov, A. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403212 · Cited by 89
“ectrocardiogram (ECG) and electroencephalogram (EEG) are being observed and collected especially in intensive care units (ICUs), which provides invaluable input for training deep learning (DL) models [21, 29, 37]. In fact deep learning models have achieved state-of-the-art performance in many healthcare and medical ap…”
BackgroundInfluentialSelf-citationA Scalable and Transferable Federated Learning System for Classifying Healthcare Sensor Data
Sun, L., & Wu, J. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3171402 · Cited by 86
“Transformer [11] and recurrent attention models [12] are also used in intensive patient monitoring.”
MethodDeep learning for heterogeneous medical data analysis
Yue, L., Tian, D., Chen, W., Han, X., & Yin, M. (2020) · World Wide Web · doi:10.1007/s11280-019-00764-z · Cited by 85
“Xu et al. [71] proposed recurrent attentive and intensive model called RAIM for jointly analyzing continuous monitoring data, which generated a guidance matrix from lab measurements and interventions, used it to guide the multi-channel attention mechanism on processed multimodal input streams, and…”
MethodUsing Clinical Notes with Time Series Data for ICU Management
Khadanga, S., Aggarwal, K., Joty, S., & Srivastava, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1678 · Cited by 91
“Xu et al. (2018) accommodated supplemental information like diagnosis, medications, and lab events to improve model performance.”
BackgroundExplainable Multivariate Time Series Classification
Hsieh, T.-Y., Wang, S., Sun, Y., & Honavar, V. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441815 · Cited by 65
“RAIM [59] introduced a multi-channel attention mechanism guided by discrete clinical events to jointly analyze continuous monitoring data and discrete clinical events.”
BackgroundCheng, W., Ma, T., Wang, X., & Wang, G. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.890563 · Cited by 65
EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation
Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102069 · Cited by 59
Predicting the Survival of Cancer Patients With Multimodal Graph Neural Network
Gao, J., Lyu, T., Xiong, F., Wang, J., Ke, W., & Li, Z. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3083566 · Cited by 55
“…representations from multiple aspects, multimodal representation has been successfully applied to many tasks, such as action or expression recognition [25], [26], [27], image or video classification [28], [29], medical diagnosis [30], [31], [32], disease diagnosis [33] and clinical prediction [34].”
BackgroundO'Reilly, D., McGrath, J., & Martin-Loeches, I. (2024) · Journal of Intensive Medicine · doi:10.1016/j.jointm.2023.10.001 · Cited by 55
“[42] This is a type of model that is easily interpretable for clinicians.”
BackgroundPredicting patient decompensation from continuous physiologic monitoring in the emergency department
Sundrani, S. C., Chen, J., Jin, B. T., Abad, Z. S. H., Rajpurkar, P., & Kim, D. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00803-0 · Cited by 55
MNN: Multimodal Attentional Neural Networks for Diagnosis Prediction
Qiao, Z., Wu, X., Ge, S., & Fan, W. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/823 · Cited by 52
Zhang, C., Gao, X., Ma, L., Wang, Y., Wang, J., & Tang, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16152 · Cited by 55
“However, EMR data are usually sparse (Xu et al. 2018), and while dealing with a patient record with low quality, such methods will learn inadequate representations and lead to inferior performance.”
BackgroundTan, Q., Ye, M., Ma, A. J., Yang, B., Yip, T. C., Wong, G. L., & Yuen, P. C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3025813 · Cited by 52
Xiang, Y., Ji, H., Zhou, Y., Li, F., Du, J., Rasmy, L., Wu, S., Zheng, W. J., Xu, H., Zhi, D., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/16981 · Cited by 54
“Typical prediction tasks include the prediction of diabetes mellitus [23], Parkinson disease [29,33], chronic heart failure [26], sepsis [34], mortality, and readmission [25].”
BackgroundMethodMollura, M., Lehman, L.-W. H., Mark, R. G., & Barbieri, R. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0252 · Cited by 50
“[28] where the authors propose an attention-based deep learning algorithm which predicts patients’ length of stay and 24-h mortality risk combining high-resolution data (ECG sampled at 125Hz), low-resolution data (e.g. mean arterial blood pressure sampled at 2Hz) and uneven and sparse information…”
BackgroundWarpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series
Zhang, J., Zheng, S., Cao, W., Bian, J., & Li, J. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599543 · Cited by 55
“Early methods [24, 51, 52, 56] used either hourly aggregation or forward imputation to obtain uniformly spaced intervals, overlooking meaningful temporal patterns in irregular sampling.”
MethodEnhancing healthcare decision support through explainable AI models for risk prediction
Niu, S., Yin, Q., Ma, J., Song, Y., Xu, R. Y. D., Bai, L., Pan, W., & Yang, X. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114228 · Cited by 54
Kaieski, N., da Costa, C. A., Righi, R. D. R., Lora, P. S., & Eskofier, B. M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106612 · Cited by 48
INSMA: An integrated system for multimodal data acquisition and analysis in the intensive care unit
Sun, Y., Guo, F., Kaffashi, F., Jacono, F. J., DeGeorgia, M. A., & Loparo, K. A. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103434 · Cited by 44
A Drug Recommendation Model Based on Message Propagation and DDI Gating Mechanism
Ren, Y., Shi, Y., Zhang, K., Wang, X., Chen, Z., & Li, H. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3153342 · Cited by 42
“In addition, in health care research, many neural network structures (such as Dipole [14], RETAIN [10], HiTANet [15], and RAIM [16]) adopt attention mechanisms.”
BackgroundA Review of Deep Learning Methods for Irregularly Sampled Medical Time Series Data
Sun, C., Hong, S., Song, M., & Li, H. (2020) · PubMed · doi:10.48550/arxiv.2010.12493 · Cited by 38
“They [60, 61, 62] discretize the time axis into non-overlapping intervals with hand-designed intervals.”
Using neural attention networks to detect adverse medical events from electronic health records
Chu, J., Dong, W., He, K., Duan, H., & Huang, Z. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.10.002 · Cited by 35
Tang, X., Liu, Y., Shah, N., Shi, X., Mitra, P., & Wang, S. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403276 · Cited by 36
“The second group leverages attention methods to generate explanations on-the-fly, and gained tremendous popularity due to their efficiency [6, 9, 25, 31, 37].”
BackgroundExplaining Time Series Predictions with Dynamic MasksPreprint
Crabbé, J., & van der Schaar, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05303 · Cited by 34
“A popular example, building on the success attention mechanisms (Vaswani et al., 2017), is the usage of attention layers to produce importance scores (Choi et al., 2016; Song et al., 2017; Xu et al., 2018; Kwon et al., 2018).”
BackgroundPredicting Optimal Hypertension Treatment Pathways Using Recurrent Neural Networks
Ye, X., Zeng, Q. T., Facelli, J. C., Brixner, D. I., Conway, M., & Bray, B. E. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104122 · Cited by 34
What went wrong and when? Instance-wise Feature Importance for Time-series ModelsPreprint
Tonekaboni, S., Joshi, S. D., Campbell, K. R., Duvenaud, D., & Goldenberg, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02821 · Cited by 44
“Attention models are suitable for sequential data and boost performance by learning to reliably learn from long range dependencies (Song et al., 2018; Kaji et al., 2019; Xu et al., 2018; Choi et al., 2016).”
BackgroundWu, J., He, K., Mao, R., Shang, X., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103283 · Cited by 29
Chou, Y.-H., Hong, S., Zhou, Y., Shang, J., Song, M., & Li, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.045 · Cited by 26
Recommendations to promote fairness and inclusion in biomedical AI research and clinical use
Griffin, A. C., Wang, K., Leung, T. I., & Facelli, J. C. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104693 · Cited by 24
A deep multimodal model for bug localization
Zhu, Z., Li, Y., Wang, Y., Wang, Y., & Tong, H. (2021) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-021-00755-7 · Cited by 23
“To process and correlate information from multiple modalities, multimodal machine learning has received a great deal of attention (Poria et al. 2016; Xu et al. 2018).”
MethodUA-CRNN: Uncertainty-Aware Convolutional Recurrent Neural Network for Mortality Risk Prediction
Tan, Q., Ma, A. J., Ye, M., Yang, B., Deng, H., Wong, V. W., Tse, Y., Yip, T. C., Wong, G. L., Ching, J. Y., et al. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3357884 · Cited by 23
Hong, S., Zhou, Y., Shang, J., Xiao, C., & Sun, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01550 · Cited by 23
“sification (e.g.,patients with paroxysmal atrial fibrillation or Healthy [55]), multi-class classification (e.g., detection of acute cognitive stress detection [31], decompensation of patients detection [131]) and multi-task classification (e.g., detection of prevalent hypertension, sleep apnea, and diabetes [89]). 2) Ann…”
BackgroundSelf-citationLiu, X., Liu, T., Zhang, Z., Kuo, P., Xu, H., Yang, Z., Lan, K., Li, P., Ouyang, Z., Ng, Y. L., et al. (2021) · JMIR Medical Informatics · doi:10.2196/18803 · Cited by 23
“Recently, several researchers have tried to combine the analysis of 2 kinds of materials to represent comprehensive information and improve the performance of the models: Xu et al [56] proposed a model to predict physiological decompensation and length of ICU stay by analyzing ECG and medical records data, and Nemati…”
BackgroundNiu, S., Yin, Q., Song, Y., Guo, Y., & Yang, X. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00056 · Cited by 23
“For example, [33] proposed RAIM by using the self-attention and guided-attention mechanisms, which used clinical events as the prior knowledge to regulate the continuous health status data.”
InfluentialKhan, S., & Kumar, V. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106262 · Cited by 21
CardioLearn: A Cloud Deep Learning Service for Cardiac Disease Detection from Electrocardiogram
Hong, S., Fu, Z., Zhou, R., Jie, Y., Li, Y., Wang, K., & Cheng, G. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366424.3383529 · Cited by 21
Medication adherence prediction through temporal modelling in cardiovascular disease management
Hsu, W. W. Y., Warren, J., & Riddle, P. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-02052-9 · Cited by 24
“These studies include [56] where LSTM is used to model multivariate pediatric intensive care time series to predict diagnoses and [57] where LSTM is used to jointly analyse episodic clinical events and continuous monitoring data in the ICU settings to predict deterioration of patient conditions and their length of sta…”
BackgroundNiu, S., Ma, J., Yin, Q., Wang, Z., Bai, L., & Yang, X. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10513-x · Cited by 23
“In addition, the majority of current CDSS models (Xu et al., 2018; Choi et al., 2016; Ma et al., 2017) mainly process structured sequential EHRs, neglecting the abundant clinical information embedded in unstructured medical notes.”
InfluentialIntegrating Static and Time-Series Data in Deep Recurrent Models for Oncology Early Warning Systems
Li, D., Lyons, P. G., Klaus, J., Gage, B. F., Kollef, M. H., & Lu, C. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482441 · Cited by 19
“Existing models for integrating heterogeneous inputs are to concatenate the learned representations from each input [6, 9, 12, 13, 39] or to feed time-series inputs together with the learned representation of static inputs into RNN at each time step [20, 34].”
BackgroundLearning Saliency Maps to Explain Deep Time Series Classifiers
Parvatharaju, P. S., Doddaiah, R., Hartvigsen, T., & Rundensteiner, E. A. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482446 · Cited by 25
“As machine learning models have become more complex, the need for robust explanations of their predictions has become essential to garner the trust required for experts to react [6, 8, 9, 17, 37, 39, 40, 42, 44, 45, 47].”
BackgroundAlSaad, R., Malluhi, Q. M., Janahi, I. A., & Boughorbel, S. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0951-4 · Cited by 20
“Attention-mechanism-based learning was used in [14– 20] for explaining what part of historical information weighs more in predicting future clinical events.”
MethodPrediction on critically ill patients: The role of “big data”
Bulgarelli, L., Deliberato, R. O., & Johnson, A. E. W. (2020) · Journal of Critical Care · doi:10.1016/j.jcrc.2020.07.017 · Cited by 19
Anomaly detection for sensor data of semiconductor manufacturing equipment using a GAN
Hashimoto, M., Ide, Y., & Aritsugi, M. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.08.090 · Cited by 18
Rare Disease Prediction by Generating Quality-Assured Electronic Health Records
Ma, F., Wang, Y., Gao, J., Xiao, H., & Zhou, J. L. (2020) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611976236.58 · Cited by 17
Artificial Intelligence and Big Data Science in Neurocritical Care
Mainali, S., & Park, S. (2022) · Critical Care Clinics · doi:10.1016/j.ccc.2022.07.008 · Cited by 18
BaT: Beat-aligned Transformer for Electrocardiogram Classification
Li, X., Li, C., Wei, Y., Sun, Y., Wei, J., Li, X., & Qian, B. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00043 · Cited by 15
Predicting Risk of Mortality in Pediatric ICU Based on Ensemble Step-Wise Feature Selection
Hong, S., Hou, X., Jing, J., Ge, W., & Zhang, L. (2021) · Health Data Science · doi:10.34133/2021/9365125 · Cited by 18
“For general ICU patients, some methods mixed the whole population without age stratification [4, 6, 8], others removed patients who are younger than an age threshold [5, 7, 13].”
MethodZheng, K., Cai, S., Chua, H., Herschel, M., Zhang, M., & Ooi, B. C. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3554821.3554835 · Cited by 14
“Further, with a guided multi-channel attention mechanism, RAIM provides interpretability for continuous monitoring data while leaving out the modality of time-series clinical data.”
BackgroundMethodInfluentialPyHealth: A Python Library for Health Predictive ModelsPreprint
Zhao, Y., Qiao, Z., Xiao, C., Glass, L. M., & Sun, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04209 · Cited by 24
Self-citationEBBE-Text: Explaining Neural Networks by Exploring Text Classification Decision Boundaries
Delaforge, A., Azé, J., Bringay, S., Mollévi, C., Sallaberry, A., & Servajean, M. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3184247 · Cited by 13
Temporal-Spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit
Nie, W., Yu, Y., Zhang, C., Song, D., Zhao, L., & Bai, Y. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3309956 · Cited by 13
“The calculation in the j-th Fusion-Encoder is as follows: The core is a multi-head cross-attention mechanism in which Q is derived from self-attentive output of the j-th time series X ij , K from the previous Fusion-Encoder’s output, and V from a mapping that combines Q and K.”
BackgroundMethodInfluentialShickel, B., & Rashidi, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12524 · Cited by 20
“Similar to RETAIN 71 and RAIM 73 , Qin et al. 74 implemented dual attention mechanisms along with LSTMs for forecasting real-valued financial time series that focused on both the timestep and the feature dimension of input multivariate sequences.”
MethodResultDCMN: Double Core Memory Network for Patient Outcome Prediction with Multimodal Data
Feng, Y., Xu, Z., Gan, L., Chen, N., Yu, B., Chen, T., & Wang, F. (2019) · 2019 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2019.00030 · Cited by 15
“values have been explored in many research to overcome the problems of multi-resolution and sampling irregularities of multimodal data, [3], [7].”
BackgroundMethodAdvances in Mining Heterogeneous Healthcare Data
Ma, F., Ye, M., Luo, J., Xiao, C., & Sun, J. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3470789 · Cited by 11
Self-citationKhope, S. R., & Elias, S. (2023) · Healthcare · doi:10.3390/healthcare11050710 · Cited by 15
Stage-Aware Hierarchical Attentive Relational Network for Diagnosis Prediction
Wang, L., Liu, Q., Zhang, M., Hu, Y., Wu, S., & Wang, L. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3310478 · Cited by 10
“To validate the effectiveness of the proposed HAR framework, without loss of generality, we choose four baseline models: LSTM, RETAIN [30], Dipole, [26], RAIM [59], StageNet [32], HiTANet [60].”
MethodResearch on Multimodal Fusion of Temporal Electronic Medical Records
Ma, M., Wang, M., Gao, B., Li, Y., Huang, J., & Chen, H. (2024) · Bioengineering · doi:10.3390/bioengineering11010094 · Cited by 11
“In our future work, we plan to explore the applicability of the model in scenarios involving representations of other modalities of medical data, such as integrating clinical imaging or adapting the model for English-language datasets.”
MethodZhou, Y., Hong, S., Shang, J., Wu, M., Wang, Q., Li, H., & Xie, J. (2020) · Sensors · doi:10.3390/s20247307 · Cited by 10
“To explain the results of Deep Neural Network (DNN) models, most methods use the attention mechanism to highlight segments of an input record which are strongly associated with the model prediction and consider them as an explanation of prediction results [37,38], while others measure the role of each feature in the p…”
MethodGeneralized and transferable patient language representation for phenotyping with limited data
Si, Y., Bernstam, E. V., & Roberts, K. E. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103726 · Cited by 10
Length of stay prediction for hospital management using domain adaptation
Momo, L. N. W., Moorosi, N., Nsoesie, E. O., Rademakers, F. E., & Moor, B. D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108088 · Cited by 10
Liu, T., Liang, L., Che, C., Liu, Y., & Jin, B. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104826 · Cited by 11
DynEHR: Dynamic adaptation of models with data heterogeneity in electronic health records
Zhang, L., Chen, X., Chen, T., Wang, Z., & Mortazavi, B. J. (2021) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi50953.2021.9508558 · Cited by 10
“This heterogeneity is a challenge in machine learning and deep learning applications and is rarely addressed in existing work [7], [13], [15], [17].”
BackgroundDMRNet: Effective Network for Accurate Discharge Medication Recommendation
Shi, J., Wang, Y., Zhang, C., Luo, Z., Chai, C., & Zhang, M. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00262 · Cited by 8
“Specifically, in hospitals, Electronic Health Records (EHRs) [13]–[18], which contain different sources of structured data for the patients [19]–[22], have been widely adopted.”
BackgroundAID: Active Distillation Machine to Leverage Pre-Trained Black-Box Models in Private Data Settings
Hoang, T. N., Hong, S., Xiao, C., Low, B. K. H., & Sun, J. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449944 · Cited by 9
“First, attention-based methods often exploit the attention weights [5, 6, 27] to distil a local model for each subject (e.”
MethodSelf-citationShukla, S. N., & Marlin, B. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11059 · Cited by 8
“Xu et al. (2018) developed clinical predictive models by integrating continuous monitoring data with discrete clinical event sequences.”
MethodCURENet: combining unified representations for efficient chronic disease prediction
Dao, C.-T., Phan, T. T., Ding, J.-E., Wu, C., Restrepo, D. S., Luo, D., Zhao, F., Liao, C.-C., Peng, W.-C., Wang, C., et al. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00396-w · Cited by 11
“By aligning LLM-derived semantic features with transformer-based temporal modeling, CURENet addresses a key limitation of prior single-modality or shallow fusion methods, which often fail to capture cross-modal dependencies [6, 7, 9, 44].”
Forecasting Treatment Outcomes Over Time Using Alternating Deep Sequential Models
Wu, F. C., Zhao, G., Zhou, Y., Qian, X., Elias, B.-K., & Lehman, L.-W. H. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3331298 · Cited by 7
Niu, S., Ma, J., Yin, Q., Bai, L., Chen, L., & Yang, X. (2024) · Annals of Operations Research · doi:10.1007/s10479-023-05817-1 · Cited by 11
How to Leverage Multimodal EHR Data for Better Medical Predictions?Preprint
Yang, B., & Wu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15763 · Cited by 7
“Xu et al. (2018) propose a recurrent attentive model to fuse continuous patient monitoring data, such as electrocardiogram (ECG), and discrete clinical events for predicting length of ICU stay.”
BackgroundELDA: Learning Explicit Dual-Interactions for Healthcare Analytics
Cai, Q., Zheng, K., Ooi, B. C., Wang, W., & Yao, C. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00034 · Cited by 6
Hong, S., Wang, C., & Fu, Z. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105847 · Cited by 6
Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models
Zhong, Y., Wang, X., Wang, J., Zhang, X., Wang, Y., Huai, M., Xiao, C., & Ma, F. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671836 · Cited by 5
Label-dependent and event-guided interpretable disease risk prediction using EHRs
Niu, S., Song, Y., Yin, Q., Guo, Y., & Yang, X. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669675 · Cited by 5
“[13] treated clinical events as interventions and also adopted the attention mechanism to generate the weighted embedding of electrocardiogram (ECG) for patients’ mortality prediction.”
BackgroundMethodZou, C., Zhang, Y., & Yuan, Z. (2023) · Computer Methods in Biomechanics & Biomedical Engineering · doi:10.1080/10255842.2023.2262663 · Cited by 5
“Most clinical prediction methods convert irregular time-series data into regular sequences (Xu et al. 2018; Tan et al. 2019).”
Attention-based Multimodal Fusion with Adversarial Network for In-Hospital Mortality Prediction
An, Y., Qiu, R., Lin, G., & Chen, X. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821815 · Cited by 6
Time-Guided High-Order Attention Model of Longitudinal Heterogeneous Healthcare Data
Huang, Y., Yang, X., & Xu, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29908-8_5 · Cited by 5
“ep Learning on Heterogeneous EHRs data. [12] designed a heterogeneous LSTM structure to explore multiple inter-correlations of dierent medical sequences with dierent lengths and record frequencies. [25] proposed an e- cient multi-channel attention model of multimodal EHRs time series. However, these models only focus…”
BackgroundMethodInfluentialHsieh, T.-Y., Wang, S., Sun, Y., & Honavar, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.11631 · Cited by 8
“RAIM [61] introduced a multi-channel attention mechanism guided by discrete clinical events to jointly analyze continuous monitoring data and discrete clinical events.”
Li, B., Du, B., & Ye, J. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3744251 · Cited by 4
A Review of Deep Learning Methods for Irregularly Sampled Medical Time Series Data
Sun, C., Song, M., Cai, D., Zhang, B., Li, H., & Hong, S. (2026) · Health Data Science · doi:10.34133/hds.0456 · Cited by 79
“They [60, 61, 62] discretize the time axis into non-overlapping intervals with hand-designed intervals.”
Xie, J., Wang, Z., Yu, Z., & Guo, B. (2022) · Sensors · doi:10.3390/s22166104 · Cited by 4
“Next, 1-D convolution is applied to the obtained segments to extract features P = conv(s), where s stands for g k or l (cl) k , P∈R U×J , U is the number of filters, and J is the length of the segment after convolution (a hyperparameter of CNN [29,34]).”
BackgroundMethodResultInfluentialA Disease Progression Prediction Model Based on EHR data
Zhao, F., Yu, X., Zhang, J., Li, X., & Li, R. (2022) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys57074.2022.00247 · Cited by 4
“Currently, EHR data play a major role in mortality prediction [2,3], disease diagnoses [4], length-of-stay forecasting [2,4,5] and admission prediction [6,7].”
BackgroundZhang, Y., Wittrup, E. M., Hodgman, M. W., Mathis, M. R., & Najarian, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3551312 · Cited by 4
Explainable AI for the Operating Theater
Rudzicz, F., & Joshi, S. D. (2020) · Springer eBooks · doi:10.1007/978-3-030-49100-0_25 · Cited by 3
Machine learning for early prediction of circulatory failure in the intensive care unitPreprint
Hyland, S. L., Faltys, M., Hüser, M., Lyu, X., Gumbsch, T., Esteban, C., Bock, C., Horn, M., Moor, M., Rieck, B., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07990 · Cited by 3
Explainable Health Risk Predictor with Transformer-based Medicare Claim EncoderPreprint
Lahlou, C., Crayton, A., Trier, C., & Willett, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09428 · Cited by 7
“A number of studies have leveraged dimensional reduction techniques, such as t-distributed stochastic neighbor embeddings (t-SNE), to construct clusters that may lend itself to an interpretation of learned representations [13, 25, 31, 41, 43, 50, 54].”
BackgroundMultimodal Artificial Intelligence for Precision Critical Care: A Scoping Review
Yang, M., Shi, N., Chen, H., Po, E. Y. P., Ng, P. Y., Lai, P. C. K., Wu, Y.-C., & Fong, D. Y. T. (2026) · Health Data Science · doi:10.34133/hds.0356 · Cited by 3
“…specific clinical tasks using a single data modality, such as using structured EHR data for sepsis prediction [ 9 ], patient subphenotyping [ 10 , 11 ], or medication dosing optimization [ 12 , 13 ], and utilizing medical imaging, such as MRI for neurological outcome assessment [ 14 , 15 ].”
GRU-TV: Time- and Velocity-aware Gated Recurrent Unit for patient representation
Liu, N., Gou, S., Gao, R., Su, B., Liu, W., Park, C., Xing, S., Yuan, J. Y., & Fenster, A. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104855 · Cited by 2
Time Series Classification Based on Supervised Contrastive Learning and Homoscedastic Uncertainty
Zhang, T., Li, K., & Wang, S. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3607901 · Cited by 2
“T IME series data exist in many real-world scenarios, including wind forecasting [1], stock forecasting [2], human activity recognition (HAR) [3], industrial fault diagnosis [4], [5], and healthcare [6].”
Metadata
- Authors
- Xu, Y., Biswal, S., Deshpande, S. R., Maher, K. O., & Sun, J.
- Affiliations
- Georgia Institute of Technology (US)
- Emory University (US)
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 2565-2573
- OpenAlex
- W2884555939
- Cited by
- 144
- References
- 43
- Citations by year
- 2026: 13 · 2025: 18 · 2024: 15 · 2023: 11 · 2022: 15 · 2021: 26 · 2020: 31 · 2019: 14 · 2018: 1
- FWCI
- 7.16
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- Funders
- National Institutes of Health (1R01MD011682-01, R56HL138415)
- National Science Foundation (IIS-#1418511, CCF-#1533768)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
- Keywords
- clinical decision support, intensive care unit, continuous patient monitoring, electrocardiogram, vital signs, drug information, interpretable models, heterogeneous data types
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
