Multi-layer Representation Learning for Medical Concepts
Open accessAbstract
Proper representations of medical concepts such as diagnosis, medication, procedure codes and visits from Electronic Health Records (EHR) has broad applications in healthcare analytics. Patient EHR data consists of a sequence of visits over time, where each visit includes multiple medical concepts, e.g., diagnosis, procedure, and medication codes. This hierarchical structure provides two types of relational information, namely sequential order of visits and co-occurrence of the codes within a visit. In this work, we propose Med2Vec, which not only learns the representations for both medical codes and visits from large EHR datasets with over million visits, but also allows us to interpret the learned representations confirmed positively by clinical experts. In the experiments, Med2Vec shows significant improvement in prediction accuracy in clinical applications compared to baselines such as Skip-gram, GloVe, and stacked autoencoder, while providing clinically meaningful interpretation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The lack of an industry standard for health information exchange was the main reason behind the slow adoption of the EHR system in healthcare organizations even if their adoption was highly beneficial.
“It was the main reason behind the slow adoption of the EHR system in healthcare organizations even if their adoption was highly beneficial for them [25].”
From Secondary Use of Electronic Health Record: Opportunities and Challenges (2020) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
(2012) · Cited by 9,062 · see citing works
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 7,048 · see citing works
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · see citing works
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · see citing works
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · see citing works
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 5,037 · see citing works
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · see citing works
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051 · see citing works
(2010) · Cited by 1,906 · see citing works
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695 · see citing works
Theano: A CPU and GPU Math Compiler in Python
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J. P., Warde-Farley, D., & Bengio, Y. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-003 · Cited by 1,063 · see citing works
A Scalable Hierarchical Distributed Language Model
Mnih, A., & Hinton, G. E. (2008) · Cited by 843 · see citing works
Doctor AI: Predicting Clinical Events via Recurrent Neural Networks
Choi, E., Bahadori, M. T., Schuetz, A., Stewart, W. F., & Sun, J. (2015) · Europe PMC (PubMed Central) · doi:10.48550/arxiv.1511.05942 · Cited by 793 · see citing works
Self-citationLearning Low-Dimensional Representations of Medical Concepts.
Choi, Y., Chiu, C. Y.-I., & Sontag, D. A. (2016) · PubMed Central · Cited by 256 · see citing works
Ghassemi, M., Naumann, T., Doshi‐Velez, F., Brimmer, N., Joshi, R., Rumshisky, A., & Szolovits, P. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623742 · Cited by 198 · see citing works
Improved musical onset detection with Convolutional Neural Networks
Schlüter, J., & Böck, S. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854953 · Cited by 195 · see citing works
Supervised patient similarity measure of heterogeneous patient records
Sun, J., Wang, F., Hu, J., & Edabollahi, S. (2012) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2408736.2408740 · Cited by 149 · see citing works
Self-citationMedical Semantic Similarity with a Neural Language Model
Vine, L. D., Zuccon, G., Koopman, B., Sitbon, L., & Bruza, P. (2014) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2661829.2661974 · Cited by 143 · see citing works
Epileptic seizures in cerebral arterial occlusive disease.
Cocito, L., FAVALE, E., & Reni, L. (1982) · Stroke · doi:10.1161/01.str.13.2.189 · Cited by 132 · see citing works
Choi, E., Schuetz, A., Stewart, W. F., & Sun, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03686 · Cited by 110 · see citing works
Self-citationDistilling Knowledge from Deep Networks with Applications to Healthcare DomainPreprint
Che, Z., Purushotham, S., Khemani, R. G., & Liu, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.03542 · Cited by 109 · see citing works
Exploring the Application of Deep Learning Techniques on Medical Text Corpora
Miñarro-Giménez, J. A., Oscar, M.-A., & Samwald, M. (2014) · Studies in health technology and informatics · doi:10.3233/978-1-61499-432-9-584 · Cited by 86 · see citing works
Sun, J., McNaughton, C. D., Zhang, P., Perer, A., Gkoulalas-Divanis, A., Denny, J. C., Kirby, J., Lasko, T. A., Saip, A., & Malin, B. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002033 · Cited by 73 · see citing works
Self-citationWang, Y., Ng, K., Byrd, R. J., et al. (2015) · doi:10.1109/embc.2015.7318907 · Cited by 69 · see citing works
Lung volumes and mechanics of breathing in tetraplegics
Forner, J. (1980) · Spinal Cord · doi:10.1038/sc.1980.45 · Cited by 57 · see citing works
A Multiplicative Model for Learning Distributed Text-Based Attribute RepresentationsPreprint
Kiros, R., Zemel, R. S., & Salakhutdinov, R. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2710 · Cited by 37 · see citing works
Cloud-based Predictive Modeling System and its Application to Asthma Readmission Prediction.
Chen, R. F., Su, H., Khalilia, M. A., et al. (2017) · PubMed · Cited by 18 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin. A neural probabilistic language model. JMLR, 2003.
- M. Ghassemi, T. Naumann, F. Doshi-Velez, N. Brim- mer, R. Joshi, A. Rumshisky, and P. Szolovits. Unfold- ing physiological state: Mortality modelling in intensive care units. In KDD, 2014.
- R. Kiros, Y. Zhu, R. Salakhutdinov, R. S. Zemel, A. Torralba, R. Urtasun, and S. Fidler. Skip-thought vectors. In NIPS, 2015.
- Q. Le and T. Mikolov. Distributed representations of sentences and documents. In ICML, 2014.
- Z. C. Lipton, D. C. Kale, C. Elkan, and R. Wetzell. Learning to diagnose with lstm recurrent neural net- works. In ICLR, 2016.
- T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, 2013.
Cited by
Sorted by citations
Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2767063 · Cited by 1,688 · see citing works
GRAM: Graph-based Attention Model for Healthcare Representation Learning
Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098126 · Cited by 637 · see citing works
Self-citationKwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294 · see citing works
Self-citationAyala Solares, J. R., Diletta Raimondi, F. E., Zhu, Y., Rahimian, F., Canoy, D., Tran, J., Pinho Gomes, A. C., Payberah, A. H., Zottoli, M., Nazarzadeh, M., et al. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103337 · Cited by 247 · see citing works
Si, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., & Roberts, K. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103671 · Cited by 235 · see citing works
KAME: Knowledge-based Attention Model for Diagnosis Prediction in Healthcare
Ma, F., You, Q., Xiao, H., Chitta, R., Zhou, J., & Gao, J. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271701 · Cited by 213 · see citing works
Readmission prediction using deep learning on electronic health records
Ashfaq, A., Sant’Anna, A., Lingman, M., & Nowaczyk, S. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103256 · Cited by 194 · see citing works
Min, X., Yu, B., & Wang, F. (2019) · Scientific Reports · doi:10.1038/s41598-019-39071-y · Cited by 170 · see citing works
Emerging technologies and analytics for a new era of value-centered marketing in healthcare
Agarwal, R., Dugas, M., Gao, G. (., & Kannan, P. K. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00692-4 · Cited by 175 · see citing works
Secondary Use of Electronic Health Record: Opportunities and Challenges
Shah, S. M., & Khan, R. A. (2020) · IEEE Access · doi:10.1109/access.2020.3011099 · Cited by 158 · see citing works
Che, Z., Cheng, Y., Zhai, S., Sun, Z., & Liu, Y. (2017) · 2017 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2017.93 · Cited by 162 · see citing works
Language models are an effective representation learning technique for electronic health record data
Steinberg, E., Jung, K., Fries, J. A., Corbin, C. K., Pfohl, S. R., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103637 · Cited by 132 · see citing works
MedGCN: Medication recommendation and lab test imputation via graph convolutional networks
Mao, C., Yao, L., & Luo, Y. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104000 · Cited by 70 · see citing works
Raket, L. L., Jaskolowski, J., Kinon, B. J., Brasen, J. C., Jönsson, L., Wehnert, A., & Fusar-Poli, P. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30024-8 · Cited by 60 · see citing works
Multimodal risk prediction with physiological signals, medical images and clinical notes
Wang, Y., Yin, C., & Zhang, P. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26772 · Cited by 54 · see citing works
HeteroMed: Heterogeneous Information Network for Medical Diagnosis
Hosseini, A., Chen, T., Wu, W., Sun, Y., & Sarrafzadeh, M. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271805 · Cited by 47 · see citing works
Zhao, C., Jiang, J., Guan, Y., Guo, X., & He, B. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.03.005 · Cited by 41 · see citing works
Performing an Informatics Consult: Methods and Challenges
Schuler, A., Callahan, A., Jung, K., & Shah, N. H. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.12.023 · Cited by 37 · see citing works
Harmonized representation learning on dynamic EHR graphs
Lee, D., Jiang, X., & Yu, H. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103426 · Cited by 34 · see citing works
Zhang, Y., Qian, S., Fang, Q., & Xu, C. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351033 · Cited by 33 · see citing works
- Authors
- Choi, E., Bahadori, M. T., Searles, E., Coffey, C., Thompson, M., Bost, J., Tejedor-Sojo, J., & Sun, J.
- Affiliations
- Georgia Institute of Technology (US)
- Children's Healthcare of Atlanta (US)
- Children's Healthcare of Atlanta, Atlanta, GA, USA
- Venue
- Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
- Publisher
- ACM
- Published
- 2016-08-13
- Pages
- 1495-1504
- OpenAlex
- W2284851926
- Cited by
- 469
- References
- 34
- Citations by year
- 2026: 13 · 2025: 25 · 2024: 31 · 2023: 53 · 2022: 43 · 2021: 63 · 2020: 85 · 2019: 80 · 2018: 48 · 2017: 22 · 2016: 6
- FWCI
- 28.02
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Samsung Scholarship
- Centers for Disease Control and Prevention
- National Science Foundation (1418511)
- Children's Healthcare of Atlanta
- UCB
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- electronic health records, clinical codes, diagnosis codes, medication codes, procedure codes, clinical interpretability, medical data analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
