Deep learning in clinical natural language processing: a methodical review
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
Objective: This article methodically reviews the literature on deep learning (DL) for natural language processing (NLP) in the clinical domain, providing quantitative analysis to answer 3 research questions concerning methods, scope, and context of current research. Materials and Methods: We searched MEDLINE, EMBASE, Scopus, the Association for Computing Machinery Digital Library, and the Association for Computational Linguistics Anthology for articles using DL-based approaches to NLP problems in electronic health records. After screening 1,737 articles, we collected data on 25 variables across 212 papers. Results: DL in clinical NLP publications more than doubled each year, through 2018. Recurrent neural networks (60.8%) and word2vec embeddings (74.1%) were the most popular methods; the information extraction tasks of text classification, named entity recognition, and relation extraction were dominant (89.2%). However, there was a “long tail” of other methods and specific tasks. Most contributions were methodological variants or applications, but 20.8% were new methods of some kind. The earliest adopters were in the NLP community, but the medical informatics community was the most prolific. Discussion: Our analysis shows growing acceptance of deep learning as a baseline for NLP research, and of DL-based NLP in the medical community. A number of common associations were substantiated (eg, the preference of recurrent neural networks for sequence-labeling named entity recognition), while others were surprisingly nuanced (eg, the scarcity of French language clinical NLP with deep learning). Conclusion: Deep learning has not yet fully penetrated clinical NLP and is growing rapidly. This review highlighted both the popular and unique trends in this active field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
A guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Recurrent Convolutional Neural Networks for Text Classification
Lai, S., Xu, L., Liu, K., & Zhao, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9513 · Cited by 2,280
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“A renewed emphasis on scientific rigor has permeated the academic establishment, 108,109 with replicability and reproducibility studies (eg, from ACM’s definitions 110,111 ) showing how much of the literature has fallen short of its scientific goals.”
BackgroundShickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2767063 · Cited by 1,688
Cited by
Using machine learning approaches for multi-omics data analysis: A review
Reel, P. S., Reel, S., Pearson, E., Trucco, E., & Jefferson, E. (2021) · Biotechnology Advances · doi:10.1016/j.biotechadv.2021.107739 · Cited by 903
Hossain, E., Rana, R., Higgins, N., Soar, J., Barua, P. D., Pisani, A. R., & Turner, K. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106649 · Cited by 407
Large language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358
“Clinical text differs significantly from text typically utilized in general NLP, both in syntax and vocabulary (Wu et al., 2020).”
MethodNeural Natural Language Processing for unstructured data in electronic health records: A review
Li, I., Pan, J., Goldwasser, J., Verma, N., Wong, W. P., Nuzumlalı, M. Y., Rosand, B., Li, Y., Zhang, M., Chang, D., et al. (2022) · Computer Science Review · doi:10.1016/j.cosrev.2022.100511 · Cited by 254
Limitations of transformers on clinical text classification
Gao, S., Alawad, M., Young, M. T., Gounley, J., Schaefferkoetter, N., Yoon, H. J., Wu, X.-C., Durbin, E. B., Doherty, J., Stroup, A., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3062322 · Cited by 157
Yan, M. Y., Gustad, L. T., & Nytrø, Ø. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab236 · Cited by 119
A survey on clinical natural language processing in the United Kingdom from 2007 to 2022
Wu, H., Wang, M., Wu, J., Francis, F., Chang, Y.-H., Shavick, A., Dong, H., Poon, M. T. C., Fitzpatrick, N., Levine, A. P., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00730-6 · Cited by 101
Fraile Navarro, D., Ijaz, K., Rezazadegan, D., Rahimi-Ardabili, H., Dras, M., Coiera, E., & Berkovsky, S. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105122 · Cited by 82
Lee, R. Y., Kross, E. K., Torrence, J., Li, K. S., Sibley, J., Cohen, T., Lober, W. B., Engelberg, R. A., & Curtis, J. R. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.1204 · Cited by 74
Joseph, J. W., Leventhal, E. L., Grossestreuer, A. V., Wong, M. L., Joseph, L. J., Nathanson, L. A., Donnino, M. W., Elhadad, N., & Sanchez, L. D. (2020) · doi:10.1101/2020.05.02.20089052 · Cited by 57
Wang, Z., Li, Y., Zhai, J., Yang, S., Sun, B., & Liang, P. (2024) · Talanta · doi:10.1016/j.talanta.2024.126138 · Cited by 55
Bardak, B., & Tan, M. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102112 · Cited by 48
Mitchell, J. R., Szepietowski, P., Howard, R., Reisman, P., Jones, J. D., Lewis, P., Fridley, B. L., & Rollison, D. E. (2022) · Journal of Medical Internet Research · doi:10.2196/27210 · Cited by 45
“More recently, there has been growing interest in more highly automated deep learning approaches for clinical NLP [6,22].”
BackgroundMethodClay, B., Bergman, H. I., Salim, S., Pergola, G., Shalhoub, J., & Davies, A. H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109808 · Cited by 41
GatorTron: A Large Language Model for Clinical Natural Language Processing
Yang, X., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Flores, M. G., Zhang, Y., Magoc, T., et al. (2022) · doi:10.1101/2022.02.27.22271257 · Cited by 33
Fraile Navarro, D., Kocaballi, A. B., Dras, M., & Berkovsky, S. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3569893 · Cited by 28
“The potential uses of NLP in medicine have been explored in previous reviews [138] as well as in the form of computational challenges addressing specific tasks [125].”
BackgroundYang, X., Zhang, H., He, X., Bian, J., & Wu, Y. (2020) · JMIR Medical Informatics · doi:10.2196/22982 · Cited by 29
Hospital-wide natural language processing summarising the health data of 1 million patients
Bean, D. M., Kraljevic, Z., Shek, A., Teo, J., & Dobson, R. J. B. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000218 · Cited by 22
Uyeda, A. M., Curtis, J. R., Engelberg, R. A., Brumback, L. C., Guo, Y., Sibley, J., Lober, W. B., Cohen, T., Torrence, J., Heywood, J., et al. (2022) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2022.02.006 · Cited by 19
Deep learning approach to detection of colonoscopic information from unstructured reports
Seong, D., Choi, Y. H., Shin, S.-Y., & Yi, B.-K. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02121-7 · Cited by 16
“Among deep learning models, recurrent neural network (RNN) has been widely employed in clinical NLP studies [18].”
MethodEthical considerations in the early detection of Alzheimer's disease using speech and AI
Petti, U., Nyrup, R., Skopek, J. M., & Korhonen, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594063 · Cited by 16
Intra-Template Entity Compatibility based Slot-Filling for Clinical Trial Information Extraction.
Witte, C., & Cimiano, P. (2022) · Proceedings of the 21st Workshop on Biomedical Language Processing · doi:10.18653/v1/2022.bionlp-1.18 · Cited by 5
“This problem could be alleviated if publications were machine readable, or could be turned into a structured, machine readable form by information extraction methods (Liu et al., 2016; Wu et al., 2020).”
BackgroundAutomatic literature screening using the PAJO deep-learning model for clinical practice guidelines
Lin, Y., Li, J., Xiao, H., Zheng, L., Xiao, Y., Song, H., Fan, J., Xiao, D., Ai, D., Fu, T., et al. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02328-8 · Cited by 4
“With the development of deep-learning techniques, more complex and advanced methods are now available improving the performance of text mining [8].”
Background
Metadata
- Authors
- Wu, S., Roberts, K., Datta, S., Du, J., Ji, Z., Si, Y., Soni, S., Wang, Q., Wei, Q., Xiang, Y., Zhao, B., & Xu, H.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-12-03
- Volume
- 27
- Issue
- 3
- Pages
- 457-470
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2993873509
- Cited by
- 491
- References
- 61
- FWCI
- 21.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.