Explainable Prediction of Medical Codes from Clinical Text
Open accessAbstract
James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, Jacob Eisenstein. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A deep learning-based NLP model leveraging CAML has been applied to predict DRGs based on clinical notes and yielded promising outcomes.
“More recently, a deep learning-based NLP model leveraging adjusted Convolutional Attention for Multi-Label Classification (CAML) has been applied to predict DRGs based on clinical notes and yielded promising outcomes12,13.”
From DRG-LLaMA : tuning LLaMA model to predict diagnosis-related group for hospitalized patients (2024) · cites this paper as [12]
The CAML architecture was shown to be effective for automatic ICD coding.
“In this study, we adopted an architecture shown to be effective for automatic ICD coding named CAML12 that pools CNN features using the attention mechanism, modifying it for the single-label DRG prediction task.”
From Early prediction of diagnostic-related groups and estimation of hospital cost by processing clinical notes (2021) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Language Modeling with Gated Convolutional NetworksPreprint
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1612.08083 · Cited by 3,097 · see citing works
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · see citing works
Improving palliative care with deep learning
Avati, A., Jung, K., Harman, S. M., Downing, N. L., Ng, A. Y., & Shah, N. H. (2017) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2017.8217669 · Cited by 445 · see citing works
Attentive Pooling NetworksPreprint
dos Santos, C. N., Tan, M., Xiang, B., & Zhou, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03609 · Cited by 347 · see citing works
Towards Automated ICD Coding Using Deep LearningPreprint
Shi, H., Xie, P., Hu, Z., Zhang, M., & Xing, E. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04075 · Cited by 120 · see citing works
Condensed Memory Networks for Clinical Diagnostic Inferencing
Prakash, A., Zhao, S., Hasan, S. A., Datla, V. V., Lee, K., Qadir, A., Liu, J., & Farri, O. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10964 · Cited by 39 · see citing works
Grounded Recurrent Neural NetworksPreprint
Vani, A., Jernite, Y., & Sontag, D. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08557 · Cited by 19 · see citing works
Flagged first, then by citations
Research works printed but not linked: 25
- Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning. pages 2091-2100.
- Tal Baumel, Jumana Nassour-Kassis, Michael Elhadad, and Noémie Elhadad. 2018. Multi-label classifica- tion of patient notes a case study on ICD code assign- ment. In AAAI Workshop on Health Intelligence.
- Elena Birman-Deych, Amy D. Waterman, Yan Yan, David S. Nilasena, Martha J. Radford, and Brian F Gage. 2005. Accuracy of ICD-9-CM codes for iden- tifying cardiovascular and stroke risk factors. Medi- cal care 43(5):480-485.
- Edward Choi, Mohammad Taha Bahadori, Andy Schuetz, Walter F Stewart, and Jimeng Sun. 2016. Doctor AI: Predicting clinical events via recurrent neural networks. In Machine Learning for Health- care Conference. pages 301-318.
- Luciano R.S. de Lima, Alberto H.F. Laender, and Berthier A. Ribeiro-Neto. 1998. A hierarchical ap- proach to the automatic categorization of medical documents. In Proceedings of the seventh inter- national conference on Information and knowledge management. ACM, pages 132-139.
- Joshua C. Denny, Marylyn D. Ritchie, Melissa A. Bas- ford, Jill M. Pulley, Lisa Bastarache, Kristin Brown- Gentry, Deede Wang, Dan R. Masys, Dan M. Ro- den, and Dana C. Crawford. 2010. PheWAS: demon- strating the feasibility of a phenome-wide scan to discover gene-disease associations. Bioinformatics 26(9):1205-1210.
- Ramakanth Kavuluru, Anthony Rios, and Yuan Lu. 2015. An empirical evaluation of supervised learn- ing approaches in assigning diagnosis codes to elec- tronic medical records. Artificial intelligence in medicine 65(2):155-166.
- Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Lan- guage Processing. pages 1746-1751.
- Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016. Visualizing and understanding neural models in NLP. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies. Association for Computational Linguistics, San Diego, California, pages 681-691. http:// www.aclweb.org/anthology/N16-1082.
- Andrew McCallum. 1999. Multi-label text classifica- tion with a mixture model trained by EM. In AAAI workshop on Text Learning. pages 1-7.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems. pages 3111-3119.
- Aurélie Névéol, Robert N Anderson, K Bretonnel Co- hen, Cyril Grouin, Thomas Lavergne, Grégoire Rey, Aude Robert, Claire Rondet, and Pierre Zweigen- baum. 2017. CLEF ehealth 2017 multilingual in- formation extraction task overview: ICD10 coding of death certificates in english and french. In CLEF 2017 Evaluation Labs and Workshop: Online Work- ing Notes, CEUR-WS. page 17. Adler Perotte, Rimma Pivovarov, Karthik Natarajan, Nicole Weiskopf, Frank Wood, and Noémie El- hadad. 2013. Diagnosis code assignment: models and evaluation metrics. Journal of the American Medical Informatics Association 21(2):231-237.
- Rajesh Ranganath, Adler J. Perotte, Noémie Elhadad, and David M. Blei. 2015. The survival filter: Joint survival analysis with a latent time series. In UAI. pages 742-751.
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explain- ing the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, pages 1135-1144.
- Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. 2016. Reasoning about entailment with neural attention. In International Conference on Learning Representa- tions.
- Elyne Scheurwegs, Boris Cule, Kim Luyckx, Léon Luyten, and Walter Daelemans. 2017. Selecting rel- evant features from the electronic health record for clinical code prediction. Journal of Biomedical In- formatics 74:92-103.
- Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, Cur- ran Associates, Inc., pages 2951-2959.
- Michael Subotin and Anthony R. Davis. 2014. A sys- tem for predicting ICD-10-PCS codes from elec- tronic health records. In Proceedings of the 2014 Workshop on Biomedical Natural Language Pro- cessing.
- Michael Subotin and Anthony R. Davis. 2016. A method for modeling co-occurrence propensity of clinical codes with application to ICD-10-PCS auto- coding. Journal of the American Medical Informat- ics Association 23(5):866-871.
- Kevin Swersky, Jasper Snoek, and Ryan P Adams. 2013. Multi-task bayesian optimization. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Infor- mation Processing Systems 26, Curran Associates, Inc., pages 2004-2012.
- Sen Wang, Xiaojun Chang, Xue Li, Guodong Long, Lina Yao, and Quan Z Sheng. 2016. Diagnosis code assignment using sparsity-based disease correlation embedding. IEEE Transactions on Knowledge and Data Engineering 28(12):3191-3202.
- Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hi- erarchical attention networks for document classifi- cation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies. Association for Computational Linguistics, San Diego, California, pages 1480-1489.
- Wenpeng Yin and Hinrich Schütze. 2017. Attentive convolution. arXiv preprint arXiv:1710.00519 . · arXiv
- Wenpeng Yin, Hinrich Schütze, Bing Xiang, and Bowen Zhou. 2016. ABCNN: Attention-based con- volutional neural network for modeling sentence pairs. Transactions of the Association for Compu- tational Linguistics 4:259-272.
- Danchen Zhang, Daqing He, Sanqiang Zhao, and Lei Li. 2017. Enhancing automatic ICD-9-CM code as- signment for medical texts with pubmed. BioNLP 2017 pages 263-271.
Cited by
Sorted by citations
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966 · see citing works
Self-citationXiao, C., Choi, E., & Sun, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy068 · Cited by 792 · see citing works
Self-citationPadChest: A large chest x-ray image dataset with multi-label annotated reports
Bustos, A., Pertusa, A., Salinas, J.-M., & de la Iglesia-Vayá, M. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101797 · Cited by 691 · see citing works
Neural Legal Judgment Prediction in English
Chalkidis, I., Androutsopoulos, I., & Aletras, N. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1424 · Cited by 352 · see citing works
Limitations of transformers on clinical text classification
Gao, S., Alawad, M., Young, M. T., Gounley, J., Schaefferkoetter, N., Yoon, H. J., Wu, X.-C., Durbin, E. B., Doherty, J., Stroup, A., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3062322 · Cited by 157 · see citing works
DRG-LLaMA : tuning LLaMA model to predict diagnosis-related group for hospitalized patients
Wang, H., Gao, C., Dantona, C., Hull, B., & Sun, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00989-3 · Cited by 99 · see citing works
“As baseline models for benchmarking, We selected CAML (Mullenbach et al. 2018; Liu et al. 2021) and ClinicalBERT (Alsentzer et al. 2019).”
Self-citationOrder-free Medicine Combination Prediction with Graph Convolutional Reinforcement Learning
Wang, S., Ren, P., Chen, Z., Ren, Z., Ma, J., & de Rijke, M. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357965 · Cited by 76 · see citing works
EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation
Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102069 · Cited by 59 · see citing works
Bardak, B., & Tan, M. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102112 · Cited by 48 · see citing works
Liu, J., Capurro, D., Nguyen, A., & Verspoor, K. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00474-9 · Cited by 50 · see citing works
Kaur, R., Ginige, J. A., & Obst, O. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118997 · Cited by 48 · see citing works
Obeid, J. S., Dahne, J., Christensen, S., Howard, S., Crawford, T., Frey, L. J., Stecker, T., & Bunnell, B. E. (2020) · JMIR Medical Informatics · doi:10.2196/17784 · Cited by 44 · see citing works
Ding, J.-E., Thao, P. N. M., Peng, W.-C., Wang, J.-Z., Chug, C.-C., Hsieh, M.-C., Tseng, Y.-C., Chen, L., Luo, D., Wu, C., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-71020-2 · Cited by 36 · see citing works
Abdaoui, H., Barki, C., Dergaa, I., Tlili, K., Ceylan, H. İ., Bragazzi, N. L., de Giorgio, A., Ben Salah, R., & Rahmouni, H. B. (2025) · Bioengineering · doi:10.3390/bioengineering13010030 · Cited by 6 · see citing works
- Authors
- Mullenbach, J., Wiegreffe, S., Duke, J., Sun, J., & Eisenstein, J.
- Affiliations
- Georgia Institute of Technology (US)
- ORCID
- Venue
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
- Published
- 2018
- Pages
- 1101-1111
- OpenAlex
- W2787132559
- Cited by
- 597
- References
- 10
- Citations by year
- 2026: 46 · 2025: 57 · 2024: 74 · 2023: 92 · 2022: 87 · 2021: 105 · 2020: 70 · 2019: 51 · 2018: 15
- FWCI
- 24.59
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1418511, 1533768)
- Georgia Institute of Technology
- Children's Healthcare of Atlanta
- National Institutes of Health (1r56hl138415-01)
- Defense Threat Reduction Agency (HDTRA1)
- National Institute on Minority Health and Health Disparities (R01MD011682)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- clinical text, interpretable prediction, clinical natural language processing, medical coding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
