Contextualized medication information extraction using Transformer-based deep learning architectures
Open accessAbstract
Objective To develop a natural language processing (NLP) system to extract medications and contextual information that help understand drug changes. This project is part of the 2022 n2c2 challenge.Materials and methods We developed NLP systems for medication mention extraction, event classification (indicating medication changes discussed or not), and context classification to classify medication changes context into 5 orthogonal dimensions related to drug changes. We explored 6 state-of-the-art pretrained transformer models for the three subtasks, including GatorTron, a large language model pretrained using > 90 billion words of text (including > 80 billion words from > 290 million clinical notes identified at the University of Florida Health). We evaluated our NLP systems using annotated data and evaluation scripts provided by the 2022 n2c2 organizers.Results Our GatorTron models achieved the best F1-scores of 0.9828 for medication extraction (ranked 3rd), 0.9379 for event classification (ranked 2nd), and the best micro-average accuracy of 0.9126 for context classification. GatorTron outperformed existing transformer models pretrained using smaller general English text and clinical text corpora, indicating the advantage of large language models.Conclusion This study demonstrated the advantage of using large transformer models for contextual medication information extraction from clinical narratives.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Specialized LLMs have shown strong abilities on these and more complex extraction tasks.
“More recently, specialized LLMs have shown strong abilities on these and more complex extraction tasks.3”
From A Scalable Method for Validated Data Extraction from Electronic Health Records with Large Language Models (2025) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,953 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519 · see citing works
Alsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., & McDermott, M. (2019) · Proceedings of the 2nd Clinical Natural Language Processing Workshop · doi:10.18653/v1/w19-1909 · Cited by 1,736 · see citing works
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000203 · Cited by 1,258 · see citing works
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847 · see citing works
Self-citationMedEx: a medication information extraction system for clinical narratives
Xu, H., Stenner, S. P., Doan, S., Johnson, K. B., Waitman, L. R., & Denny, J. C. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3378 · Cited by 532 · see citing works
Evaluating temporal relations in clinical text: 2012 i2b2 Challenge
Sun, W., Rumshisky, A., & Uzuner, Ö. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001628 · Cited by 477 · see citing works
Extracting medication information from clinical text
Uzuner, Ö., Solti, I., & Cadag, E. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003947 · Cited by 466 · see citing works
Bidirectional RNN for Medical Event Detection in Electronic Health Records
Jagannatha, A. N., & Yu, H. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1056 · Cited by 314 · see citing works
2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records
Henry, S., Buchan, K., Filannino, M., Stubbs, A., & Uzuner, Ö. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz166 · Cited by 307 · see citing works
de Bruijn, B., Cherry, C. A., Kiritchenko, S., Martin, J., & Zhu, X. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000150 · Cited by 235 · see citing works
Audio Albert: A Lite Bert for Self-Supervised Learning of Audio Representation
Chi, P.-H., Chung, P.-H., Wu, T.-H., Hsieh, C.-C., Chen, Y.-H., Li, S.-W. D., & Lee, H.-Y. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383575 · Cited by 176 · see citing works
Clinical concept extraction using transformers
Yang, X., Bian, J., Hogan, W. R., & Wu, Y. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa189 · Cited by 155 · see citing works
Self-citationNamed Entity Recognition in Chinese Clinical Text Using Deep Neural Network
Wu, Y., Min, J., Jianbo, L., & Xu, H. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-564-7-624 · Cited by 148 · see citing works
Self-citationWei, Q., Ji, Z., Li, Z., Du, J., Wang, J., Xu, J., Xiang, Y., Tiryaki, F., Wu, S., Zhang, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz063 · Cited by 129 · see citing works
A hybrid system for temporal information extraction from clinical text
Tang, B., Wu, Y., Jiang, M., Chen, Y., Denny, J. C., & Xu, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001635 · Cited by 117 · see citing works
Self-citationMedical Information Extraction in the Age of Deep Learning
Hahn, U., & Oleynik, M. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1702001 · Cited by 87 · see citing works
Yang, X., Bian, J., Fang, R., Bjarnadottir, R. I., Hogan, W. R., & Wu, Y. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz144 · Cited by 74 · see citing works
Self-citationYang, X., Bian, J., Gong, Y., Hogan, W. R., & Wu, Y. (2019) · Drug Safety · doi:10.1007/s40264-018-0761-0 · Cited by 66 · see citing works
Self-citationExtracting and standardizing medication information in clinical text - the MedEx-UIMA system.
Jiang, M., Wu, Y., Shah, A., Priyanka, P., Denny, J. C., & Xu, H. (2014) · PubMed Central · Cited by 41 · see citing works
Self-citationKinlay, M., Zheng, W. Y., Burke, R. A., Juraskova, I., Moles, R. J., & Baysari, M. T. (2020) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2020.12.004 · Cited by 18 · see citing works
Toward Understanding Clinical Context of Medication Change Events in Clinical Narratives
Mahajan, D., Liang, J. J., & Tsou, C.-H. (2020) · PubMed Central · doi:10.48550/arxiv.2011.08835 · Cited by 17 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
Sohail, S. S., Farhat, F., Himeur, Y., Nadeem, M., Madsen, D. Ø., Singh, Y., Atalla, S., & Mansoor, W. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101675 · Cited by 192 · see citing works
“ChatGPT utilizes a transformer architecture consisting of encoder-decoder layers that collaborate to process and generate natural language text (Chen et al., 2023).”
Opportunities and risks of large language models in psychiatry
Obradovich, N., Khalsa, S., Khan, W., Suh, J., Perlis, R. H., Ajilore, O., & Paulus, M. P. (2024) · NPP—Digital Psychiatry and Neuroscience · doi:10.1038/s44277-024-00010-z · Cited by 107 · see citing works
Shen, Y., Yu, J., Zhou, J., & Hu, G. (2024) · Journal of Medical Internet Research · doi:10.2196/59024 · Cited by 91 · see citing works
Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis
Yu, H., Fan, L., Li, L., Zhou, J., Ma, Z., Xian, L., Hua, W., He, S., Jin, M., Zhang, Y., et al. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00171-8 · Cited by 69 · see citing works
Large language models: a primer and gastroenterology applications
Shahab, O., Kurdi, B. E., Shaukat, A., Nadkarni, G., & Soroush, A. (2024) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848241227031 · Cited by 43 · see citing works
A clinical named entity recognition model using pretrained word embedding and deep neural networks
Dash, A., Darshana, S., Yadav, D. K., & Gupta, V. (2024) · Decision Analytics Journal · doi:10.1016/j.dajour.2024.100426 · Cited by 38 · see citing works
Cheng, P., Yang, X., Chen, A., Yu, Z., Smith, K. E., Costa, A. B., Flores, M. G., Bian, J., & Wu, Y. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae078 · Cited by 30 · see citing works
Self-citationLarge language models to process, analyze, and synthesize biomedical texts: a scoping review
Doneva, S. E., Qin, S., Sick, B., Ellendorff, T., Goldman, J.-P., Schneider, G., & Ineichen, B. V. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00197-2 · Cited by 43 · see citing works
Bai, E., Luo, X., Zhang, Z., Adelgais, K., Ali, H., Finkelstein, J., & Kutzin, J. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02197-w · Cited by 24 · see citing works
Siepmann, R. M., Baldini, G., Schmidt, C. S., Truhn, D., Müller‐Franzes, G. A., Dada, A., Kleesiek, J., Nensa, F., & Hosch, R. (2025) · Healthcare Analytics · doi:10.1016/j.health.2024.100378 · Cited by 16 · see citing works
From Transformers to ChatGPT: An Analysis of Large Language Models Research
Cotfas, L., Sandu, A., Delcea, C., Diaconu, P., Frăsineanu, C., & Stănescu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3600739 · Cited by 16 · see citing works
Luo, X., Tahabi, F. M., Marc, T., Haunert, L., & Storey, S. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae145 · Cited by 14 · see citing works
Stuhlmiller, T. J., Rabe, A., Rapp, J., Awawda, A., Kouser, H., Lui, K., Salamon, H., Chuyka, D., Mahoney, W., Furgason, J. M., et al. (2025) · medRxiv · doi:10.1101/2025.02.25.25322898 · Cited by 13 · see citing works
“3 Here, we present a highly scalable solution using general purpose LLMs for accurate clinical text classification and structured data extraction that can be implemented and adapted quickly, achieves targeted performance, enables efficient and effective data retrieval, and integrates with standard…”
Bibliometric analysis of natural language processing using CiteSpace and VOSviewer
Chen, X., Tian, W., & Fang, H. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100123 · Cited by 20 · see citing works
What can artificial intelligence do for EUS?
Mahajan, S., Siyu, S., & Bhutani, M. S. (2025) · Endoscopic Ultrasound · doi:10.1097/eus.0000000000000102 · Cited by 11 · see citing works
Generative Artificial Intelligence-Based Medical Entity Data Extractor Using Large Language ModelsPreprint
Ghali, M.-K., Farrag, A., Sakai, H., baz, H. E., Jin, Y., & Lam, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5063216 · Cited by 11 · see citing works
AI-driven report-generation tools in mental healthcare: A review of commercial tools
Bouguettaya, A., Victoria Team, Stuart, E. M., & Aboujaoude, E. N. (2025) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2025.02.018 · Cited by 10 · see citing works
Clinical natural language processing for secondary uses
Gao, Y., Mahajan, D., Uzuner, Ö., & Yetisgen, M. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104596 · Cited by 8 · see citing works
Natural Language Processing of Electronic Health Records for Predicting Alzheimer's Disease
Joshi, H. (2024) · Advances in bioinformatics and biomedical engineering book series · doi:10.4018/979-8-3693-6442-0.ch006 · Cited by 8 · see citing works
Salem, M., Mohamed, A., & Shaalan, K. F. (2025) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-81308-5_42 · Cited by 8 · see citing works
Large Language Models to process, analyze, and synthesize biomedical texts – a scoping reviewPreprint
Doneva, S. E., Qin, S., Sick, B., Ellendorff, T., Goldman, J.-P., Schneider, G., & Ineichen, B. V. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.04.19.588095 · Cited by 5 · see citing works
GPT-4o in radiology: In-context learning based automatic generation of radiology impressions
Mahyoub, M. A., Wang, Y., & Khasawneh, M. T. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100145 · Cited by 4 · see citing works
Zhang, J., Zhou, J., Zhou, L., & Ba, Z. (2025) · JMIR Medical Informatics · doi:10.2196/70096 · Cited by 4 · see citing works
A Joint Classification Method for COVID-19 Lesions Based on Deep Learning and Radiomics
Ma, G., Wang, K., Zeng, T., Sun, B., & Yang, L. (2024) · Tomography · doi:10.3390/tomography10090109 · Cited by 4 · see citing works
FANS: A framework for automatic assessment of nutritional status based on free-text clinical notes
Hu, J., Wang, X., Yao, K., (49484), X. Z., Zhao, W., Fu, J., Lou, P., Chen, W., & Fang, A. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106168 · Cited by 4 · see citing works
How LLMs can advance safety case intake—points to consider and insights from a proof of concept
Roemming, H.-J., Hauben, M., Wannhoff, W., Schaffer, C., Tihaa, I., Heitmann, M., & Mengling, V. (2025) · Therapeutic Advances in Drug Safety · doi:10.1177/20420986251386222 · Cited by 3 · see citing works
AI-Based Automation for Medication Reconciliation: Scoping Review
Bortesi, J. P. T., Becerra, M. P., Ranisau, J., Wen, B., Nadesan, P., Devereaux, P. J., McGillion, M. H., & Petch, J. (2026) · Journal of Medical Internet Research · doi:10.2196/86760 · Cited by 2 · see citing works
Wen, Z., Huang, J., Yu, K., Li, Y., Wang, Z., Liao, X., Li, B., Tao, Z., & He, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-30050-0 · Cited by 2 · see citing works
Koustas, S. G., Kirschbaum, J., & Möslein, K. M. (2024) · Procedia CIRP · doi:10.1016/j.procir.2024.07.068 · Cited by 2 · see citing works
Cheng, P., Dong, X., Lyu, M., Paredes, D., Zhang, Y., & Wu, Y. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105034 · Cited by 1 · see citing works
Self-citationHall, L., Donohue, J. M., Larochelle, M. R., Rossom, R. C., Wang, X., Afshar, M., Gellad, W. F., Groenwold, R. H. H., Han, B., Jones, C. M., et al. (2026) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2026.210065 · Cited by 1 · see citing works
Iftikhar, H. (2025) · medRxiv · doi:10.1101/2025.05.10.25327319 · Cited by 1 · see citing works
Iftikhar, H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5253214 · Cited by 1 · see citing works
Cloud-Driven NLP Architectures for Clinical Note Summarization and Knowledge Extraction
Kaushik, K., Piridi, S., Somanathan, D. S., Modh, N. N., Gandhi, G. C., & Soni, M. (2025) · IEEE Uttar Pradesh Section Women in Engineering International Conference on Electrical, Electronics and Computer Engineering (UPWIECON) · doi:10.1109/upwiecon67212.2025.11390432
Enhancing Outpatient Care Using an LLM-Based Model for Clinical Ward Disposition
Nancy Jane, Y., Kingslin Issac, R., R, V. K., Vidhya Bharathi, S., & B, R. D. (2026) · International Conference on Computing, Communication, Control and Cyber-Physical Systems (I5CPS) · doi:10.1109/i5cps67958.2026.11452531
Artificial Intelligence in Clinical Pharmacy: From Decision Support to Prescription Execution
Jiang, W., Wang, Y., Shi, W., Li, C., Li, M., Tang, Y., Xiaolin, Y. U. E., & Zhao, Z. (2026) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1155/jcpt/4675298
Katyal, A., & Bhatnagar, A. (2026) · Research Square · doi:10.21203/rs.3.rs-11152842/v1
Stuhlmiller, T. J., Rabe, A., Rapp, J., Awawda, A., Kouser, H., Lui, K., Salamon, H., Chuyka, D., Mahoney, W., Furgason, J. M., et al. (2026) · AI in Precision Oncology · doi:10.1177/2993091x261492372
Granular by design: a fine-grained NER approach to context-aware medication extraction
Berzi, A., Berényi, E. L., Arató, V., Kallos-Balogh, P., Aranyi, S. C., Képes, Z., & Emri, M. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1927272
- Authors
- Chen, A., Yu, Z., Yang, X., Guo, Y., Bian, J., & Wu, Y.
- Affiliations
- University of Florida (US)
- Corresponding author
- Wu, Yonghui
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2023-06
- Volume
- 142
- Pages
- 104370
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC10980542
- OpenAlex
- W4366824871
- Cited by
- 40
- References
- 25
- Citations by year
- 2026: 8 · 2025: 17 · 2024: 14 · 2023: 1
- FWCI
- 3.77
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 366 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- PCORI (ME-2018C3-14754)
- National Institute on Aging (R21AG068717, 1R56AG069880)
- National Institute on Drug Abuse (1R01DA050676)
- National Institute of Mental Health (1R01MH121907)
- National Cancer Institute (R21CA253394-01A1, 1R01CA246418, R21CA245858-01A1S1)
- Nvidia
- Centers for Disease Control and Prevention (1U18DP006512)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- clinical natural language processing, large language models, transformer models, clinical text, clinical notes
- MeSH terms
- Deep Learning, Natural Language Processing, Information Storage and Retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
