Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks

Ma, Fenglong; Chitta, Radha; Zhou, Jing; You, Quanzeng; Sun, Tong; Gao, Jing

doi:10.1145/3097983.3098088

Cited by 576Machine Learning in HealthcareMedical Coding and Health InformationArtificial Intelligence in Healthcare

Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks

Open access

Ma, Fenglong, Chitta, Radha, Zhou, Jing, You, Quanzeng, Sun, Tong, Gao, Jing·2017·Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

DOI 10.1145/3097983.3098088

Open this paper in CoChat — free account Open PDF

Abstract

Predicting the future health information of patients from the historical Electronic Health Records (EHR) is a core research task in the development of personalized healthcare. Patient EHR data consist of sequences of visits over time, where each visit contains multiple medical codes, including diagnosis, medication, and procedure codes. The most important challenges for this task are to model the temporality and high dimensionality of sequential EHR data and to interpret the prediction results. Existing work solves this problem by employing recurrent neural networks (RNNs) to model EHR data and utilizing simple attention mechanism to interpret the results. However, RNN-based approaches suffer from the problem that the performance of RNNs drops when the length of sequences is large, and the relationships between subsequent visits are ignored by current RNN-based approaches. To address these issues, we propose Dipole, an end-to-end, simple and robust model for predicting patients' future health information. Dipole employs bidirectional recurrent neural networks to remember all the information of both the past visits and the future visits, and it introduces three attention mechanisms to measure the relationships of different visits for the prediction. With the attention mechanisms, Dipole can interpret the prediction results effectively. Dipole also allows us to interpret the learned medical code representations which are confirmed positively by medical experts. Experimental results on two real world EHR datasets show that the proposed Dipole can significantly improve the prediction accuracy compared with the state-of-the-art diagnosis prediction approaches and provide clinically meaningful interpretation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1145/3097983.3098088

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

RETAIN and Dipole outperform the clinical time series partial model on mortality and readmission prediction but fail on the long length of stay task.
“RETAIN [10] and Dipole [16] outperform our clinical time series partial model on mortality and readmission prediction but failed on the long length of stay task.”
From Multimodal risk prediction with physiological signals, medical images and clinical notes (2024) · cites this paper as [16]
An attention mechanism was adopted in 13 of the 49 included studies to improve performance and provide a degree of interpretability.
“To improve performance and provide a degree of interpretability to the model, an attention mechanism is adopted in 13 studies [6,14,27,31,43,57,64,72,76–78,80,88].”
From Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review (2021) · cites this paper as [76]
RETAIN was commonly used as a baseline in subsequent advanced models (GRAM, Dipole, Health-ATM).
“RETAIN was also commonly used as baselines in the subsequent series of advanced models (i.e., GRAM [27], Dipole [76], Health-ATM [72]).”
From Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review (2021) · cites this paper as [76]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · see citing works
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · see citing works
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · see citing works
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · see citing works
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,859 · see citing works
Self-citation
Mining electronic health records: towards better research applications and clinical care
Jensen, P. B., Jensen, L. J., & Brunak, S. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3208 · Cited by 1,680 · see citing works
On the Properties of Neural Machine Translation: Encoder-Decoder ApproachesPreprint
Cho, K., van Merriënboer, B., Bahdanau, D., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.1259 · Cited by 1,112 · see citing works
Theano: A CPU and GPU Math Compiler in Python
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J. P., Warde-Farley, D., & Bengio, Y. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-003 · Cited by 1,063 · see citing works
GRAM: Graph-based Attention Model for Healthcare Representation Learning
Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098126 · Cited by 637 · see citing works
A multi-task learning formulation for predicting disease progression
Zhou, J., Yuan, L., Liu, J., & Ye, J. (2011) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2020408.2020549 · Cited by 258 · see citing works
Self-citation
Deep Computational Phenotyping
Che, Z., Kale, D. C., Li, W., Bahadori, M. T., & Liu, Y. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783365 · Cited by 247 · see citing works
Recurrent Neural Networks for Multivariate Time Series with Missing ValuesPreprint
Che, Z., Purushotham, S., Cho, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01865 · Cited by 230 · see citing works
Temporal Phenotyping from Longitudinal Electronic Health Records
Liu, C., Wang, F., Hu, J., & Xiong, H. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783352 · Cited by 150 · see citing works
From micro to macro
Zhou, J., Wang, F., Hu, J., & Ye, J. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623711 · Cited by 115 · see citing works
Self-citation
Constructing Disease Network and Temporal Progression Model via Context-Sensitive Hawkes Process
Choi, E., Du, N., Chen, R. E., Song, L., & Sun, J. (2015) · Proceedings · doi:10.1109/icdm.2015.144 · Cited by 100 · see citing works

Flagged first, then by citations

Research works printed but not linked: 17

[1][1] Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. 2015. Multiple Object Recognition with Visual Attention. In Proceedings of the 3rd International Conference on Learning Representations (ICLR'15) . -
[2][2] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR'15) . -
[7][7] Yu Cheng, Fei Wang, Ping Zhang, and Jianying Hu. 2016. Risk Prediction with Electronic Health Records: A Deep Learning Approach. In Proceedings of the 2016 SIAM International Conference on Data Mining (SDM'16) . 432-440. -
[9][9] Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. 2016. 10. Multi-layer Representation Learning for Medical Concepts. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'16) . 1495-1504. -
[11][11] Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. 2016. RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism. In Advances in Neural Information Processing Systems (NIPS'16) . 3504-3512. -
[13][13] Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. 2015. Attention-based Models for Speech Recognition. In Advances in Neural Information Processing Systems (NIPS'15) . 577-585. -
[14][14] Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching Machines to Read and Comprehend. In Advances in Neural Information Processing Systems (NIPS'15) . 1693-1701. -
[17][17] Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. 2016. Professor Forcing: A New Algorithm for Training Recurrent Networks. In Advances In Neural Information Processing Systems (NIPS'16) . 4601-4609. -
[18][18] Zachary C Lipton, David C Kale, and Randall Wetzel. 2016. Modeling Missing Data in Clinical Time Series with RNNs. In Proceedings of Machine Learning for Healthcare (MLHC'16) . -
[21][21] Fenglong Ma, Chuishi Meng, Houping Xiao, Qi Li, Jing Gao, Lu Su, and Aidong Zhang. 2017. Unsupervised Discovery of Drug Side-Effects from Heterogeneous Data Sources. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'17) . ACM. -
[22][22] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and Their Compositionality. In Advances in Neural Information Processing Systems (NIPS'13) . 3111-3119. -
[23][23] Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, and Svetha Venkatesh. 2016. Deepr: A Convolutional Net for Medical Records. IEEE Journal of Biomedical and Health Informatics (2016). -
[25][25] Qiuling Suo, Fenglong Ma, Giovanni Canino, Jing Gao, Aidong Zhang, Pierangelo Veltri, and Agostino Gnasso. 2017. A Multi-task Framework for Monitoring Health Conditions via Attention-based Recurrent Neural Networks. In Proceedings of the AMIA 2017 Annual Symposium (AMIA'17) . -
[26][26] Xiang Wang, David Sontag, and Fei Wang. 2014. Unsupervised Learning of Disease Progression Models. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'14) . ACM, 85-94. -
[27][27] Houping Xiao, Jing Gao, Long Vu, and Deepak S. Turaga. 2017. Learning Temporal State of Diabetes Patients via Combining Behavioral and Demographic Data. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'17) . ACM. -
[28][28] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S Zemel, and Yoshua Bengio. 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. In Proceedings of the 32nd International Conference on Machine Learning (ICML'15) . CoRR, 20482057. -
[31][31] Jiayu Zhou, Jimeng Sun, Yashu Liu, Jianying Hu, and Jieping Ye. 2013. Patient Risk Prediction Model via Top-k Stability Selection. In Proceedings of the 13th SIAM International Conference on Data Mining (SDM'13) . SIAM, 55-63. -

Sorted by citations

Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · see citing works
The importance of interpretability and visualization in machine learning for applications in medicine and health care
Vellido, A. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04051-w · Cited by 635 · see citing works
RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
Kwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294 · see citing works
Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review
Si, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., & Roberts, K. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103671 · Cited by 235 · see citing works
KAME: Knowledge-based Attention Model for Diagnosis Prediction in Healthcare
Ma, F., You, Q., Xiao, H., Chitta, R., Zhou, J., & Gao, J. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271701 · Cited by 213 · see citing works
Self-citation
Readmission prediction using deep learning on electronic health records
Ashfaq, A., Sant’Anna, A., Lingman, M., & Nowaczyk, S. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103256 · Cited by 194 · see citing works
Interpretable Representation Learning for Healthcare via Capturing Disease Progression through Time
Bai, T., Zhang, S., Egleston, B. L., & Vucetic, S. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219904 · Cited by 180 · see citing works
RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data
Xu, Y., Biswal, S., Deshpande, S. R., Maher, K. O., & Sun, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220051 · Cited by 144 · see citing works
Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99 · see citing works
Disease Prediction via Graph Neural Networks
Sun, Z., Yin, H., Chen, H., Chen, T., Cui, L., & Yang, F. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3004143 · Cited by 98 · see citing works
EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation
Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102069 · Cited by 59 · see citing works
Multimodal risk prediction with physiological signals, medical images and clinical notes
Wang, Y., Yin, C., & Zhang, P. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26772 · Cited by 54 · see citing works
“It can be shown from the result that RETAIN [10] and Dipole [16] outperform our clinical time series partial model on mortality and readmission prediction but failed on the long length of stay task.”
BackgroundMethod
HeteroMed: Heterogeneous Information Network for Medical Diagnosis
Hosseini, A., Chen, T., Wu, W., Sun, Y., & Sarrafzadeh, M. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271805 · Cited by 47 · see citing works
Modeling asynchronous event sequences with RNNs
Wu, S., Liu, S., Sohn, S., Moon, S., Wi, C.-I., Juhn, Y., & Liu, H. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.05.016 · Cited by 45 · see citing works
Harmonized representation learning on dynamic EHR graphs
Lee, D., Jiang, X., & Yu, H. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103426 · Cited by 34 · see citing works
Hospital Admission Location Prediction via Deep Interpretable Networks for the Year-Round Improvement of Emergency Patient Care
El-Bouri, R., Eyre, D. W., Watkinson, P., Zhu, T., & Clifton, D. A. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2990309 · Cited by 30 · see citing works

Authors: Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T., & Gao, J.
Affiliations: University at Buffalo, State University of New York (US)
Conduent Labs US, Rochester, NY, USA
University of Rochester (US)
United Technologies Research Center & Xerox, East Hartford, CT, USA
Venue: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Publisher: ACM
Published: 2017-08-13
Pages: 1903-1911
DOI: 10.1145/3097983.3098088
OpenAlex: W2690721124
Cited by: 576
References: 16
Citations by year: 2026: 25 · 2025: 54 · 2024: 66 · 2023: 78 · 2022: 71 · 2021: 99 · 2020: 77 · 2019: 62 · 2018: 38 · 2017: 6
FWCI: 22.83
Type: conference paper
Language: en
Open access: green
arxiv.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: National Science Foundation (IIS-1514204)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords: electronic health records, diagnostic prediction, bidirectional recurrent neural network, attention mechanism, temporal modeling, interpretability, personalized healthcare

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review