RxBERT: Enhancing drug labeling text mining and analysis with AI language modeling
Open accessAbstract
The US drug labeling document contains essential information on drug efficacy and safety, making it a crucial regulatory resource for Food and Drug Administration (FDA) drug reviewers. Due to its extensive volume and the presence of free-text, conventional text mining analysis have encountered challenges in processing these data. Recent advances in artificial intelligence (AI) for natural language processing (NLP) have provided an unprecedented opportunity to identify key information from drug labeling, thereby enhancing safety reviews and support for regulatory decisions. We developed RxBERT, a Bidirectional Encoder Representations from Transformers (BERT) model pretrained on FDA human prescription drug labeling documents for an enhanced application of drug labeling documents in both research and drug review. RxBERT was derived from BioBERT with further training on human prescription drug labeling documents. RxBERT was demonstrated in several tasks using regulatory datasets, including those involved in the National Institutes of Technology Text Analysis Challenge Dataset (NIST TAC dataset), the FDA Adverse Drug Event Evaluation Dataset (ADE Eval dataset), and the classification of texts from submission packages into labeling sections (US Drug Labeling dataset). For all these tasks, RxBERT reached 86.5 F1-scores in both TAC and ADE Eval classification, respectively, and prediction accuracy of 87% for the US Drug Labeling dataset. Overall, RxBERT was shown to be as competitive or have better performance compared to other NLP approaches such as BERT, BioBERT, etc. In summary, we developed RxBERT, a transformer-based model specific for drug labeling that outperformed the original BERT model. RxBERT has the potential to be used to assist research scientists and FDA reviewers to better process and utilize drug labeling information toward the advancement of drug effectiveness and safety for public health. This proof-of-concept study also demonstrated a potential pathway to customized large language models (LLMs) tailored to the sensitive regulatory documents for internal application.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The semantic-based performance (recall 0.939, F1 0.911) was higher than previously reported approaches in the TAC 2017 challenge and the NER predictive model based on RxBERT.
“which were higher than previously reported approaches in the TAC 2017 challenge [7] as well as when using the NER predictive model based on RxBERT [31].”
From Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel (2025) · cites this paper as [31]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1166120 · Cited by 708 · see citing works
FDA-approved drug labeling for the study of drug-induced liver injury
Chen, M., Vijay, V., Shi, Q., Liu, Z., Fang, H., & Tong, W. (2011) · Drug Discovery Today · doi:10.1016/j.drudis.2011.05.007 · Cited by 406 · see citing works
Self-citationDetecting formal thought disorder by deep contextualized word representations
Sarzyńska‐Wawer, J., Wawer, A., Pawlak, A., Szymanowska, J., Stefaniak, I., Jarkiewicz, M., & Okruszek, Ł. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.114135 · Cited by 336 · see citing works
COVID-Twitter-BERT: A natural language processing model to analyse COVID-19 content on Twitter
Müller, M., Salathé, M., & Kummervold, P. E. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1023281 · Cited by 301 · see citing works
Progress in Neural NLP: Modeling, Learning, and Reasoning
Zhou, M., Duan, N., Liu, S., & Shum, H. (2020) · Engineering · doi:10.1016/j.eng.2019.12.014 · Cited by 295 · see citing works
Paradigm Shift in Natural Language Processing
Sun, T., Liu, X.-Y., Qiu, X., & Huang, X. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1331-6 · Cited by 94 · see citing works
Study of serious adverse drug reactions using FDA-approved drug labeling and MedDRA
Wu, L., Ingle, T., Liu, Z., Zhao-Wong, A., Harris, S., Thakkar, S., Zhou, G., Yang, J., Xu, J., Mehta, D., et al. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2628-5 · Cited by 80 · see citing works
Self-citationFang, H., Harris, S. C., Liu, Z., Zhou, G., Zhang, G., Xu, J., Rosario, L., Howard, P. C., & Tong, W. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.06.006 · Cited by 56 · see citing works
Self-citationThakkar, S., Slikker, W., Yiannas, F., Silva, P., Blais, B. W., Chng, K. R., Liu, Z., Adholeya, A., Pappalardo, F., Soares, M. D. L. C., et al. (2023) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2023.105388 · Cited by 50 · see citing works
Self-citationPharmBERT: a domain-specific BERT model for drug labels
ValizadehAslani, T., Shi, Y., Ren, P., Wang, J., Zhang, Y., Hu, M., Zhao, L., & Liang, H. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad226 · Cited by 50 · see citing works
Wu, Y., Liu, Z., Wu, L., Chen, M., & Tong, W. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.729834 · Cited by 26 · see citing works
Self-citationBayer, S., Clark, C., Dang, O., Aberdeen, J. S., Brajovic, S., Swank, K. A., Hirschman, L., & Ball, R. (2020) · Drug Safety · doi:10.1007/s40264-020-00996-3 · Cited by 22 · see citing works
Self-citationFDALabel for drug repurposing studies and beyond
Fang, H., Harris, S., Liu, Z., Thakkar, S., Yang, J., Ingle, T., Xu, J., Lesko, L., Rosario, L., & Tong, W. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-00751-0 · Cited by 19 · see citing works
Self-citationSonntagbauer, M., Haar, M., & Kluge, S. (2023) · Medizinische Klinik - Intensivmedizin und Notfallmedizin · doi:10.1007/s00063-023-01019-6 · Cited by 18 · see citing works
Gray, M., Xu, J., Tong, W., & Wu, L. (2023) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.3c00028 · Cited by 11 · see citing works
Self-citationWu, L., Chen, S., Guo, L., Shpyleva, S., Harris, K., Fahmi, T., Flanigan, T., Tong, W., Xu, J., & Ren, Z. X. (2022) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2022.105287 · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
2 of the 19 references couldn’t be matched to a known paper.
Research works printed but not linked: 12
- Roberts K, Demner-Fushman D, Tonning JM. Overview of the TAC 2017 adverse reaction extraction from drug labels track. https://tac. nist.gov/publications/2017/additional.papers/TAC2017.ADR_over- view.proceedings.pdf
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Polosukhin ŁI. Attention is all you need. Adv Neur Inform Process Syst 2017;30:3762
- Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. https://arxiv. org/abs/1810.04805
- Liu P, Yuan W, Fu J, Jiang Z, Hayashi H, Neubig G. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural lan- guage processing. ACM Comput Surv 2023;55:1-35
- Huang K, Altosaar J, Ranganath R. ClinicalBERT: modeling clini- cal notes and predicting hospital readmission. https://arxiv.org/ abs/1904.05342 · arXiv
- Chalkidis I, Fergadiotis M, Malakasiotis P, Aletras N, Androutsopou- los I. LEGAL-BERT: the Muppets straight out of law school. https:// arxiv.org/abs/2010.02559 · arXiv
- Beltagy I, Lo K, Cohan A. SciBERT: a pretrained language model for scientific text. https://arxiv.org/abs/1903.10676 · arXiv
- Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. https://arxiv.org/abs/1301.3781 · arXiv
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A. Language models are few-shot learners. Adv Neur Inform Process Syst 2020;33:1877-901
- Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S. PaLM: scaling language modeling with pathways. https://arxiv.org/abs/2204.02311 · arXiv
- Zhang S, Roller S, Goyal N, Artetxe M, Chen M, Chen S, Dewa n C, Diab M, Li X, Lin XV. OPT: open pre-trained transformer language models. https://arxiv.org/abs/2205.01068 · arXiv
- Sonntagbauer M, Haar M, Kluge S. Artificial intelligence: how will ChatGPT and other AI applications change our everyday medical prac- tice? Med Klin Intensivmed Notfmed 2023;118:366-71
Cited by
Sorted by citations
The applications and advances of artificial intelligence in drug regulation: A global perspective
Lixia, F., Jia, G., Liu, Z., Pang, X., & Cui, Y. (2024) · Acta Pharmaceutica Sinica B · doi:10.1016/j.apsb.2024.11.006 · Cited by 50 · see citing works
Text summarization with ChatGPT for drug labeling documents
Ying, L., Liu, Z., Fang, H., Kusko, R. L., Wu, L., Harris, S., & Tong, W. (2024) · Drug Discovery Today · doi:10.1016/j.drudis.2024.104018 · Cited by 18 · see citing works
Self-citationWu, L., Fang, H., Qu, Y., Xu, J., & Tong, W. (2025) · Drug Safety · doi:10.1007/s40264-025-01520-1 · Cited by 20 · see citing works
“As a result, the average semantic-based recall and F1-score for all 476 samples were 0.939 and 0.911, respectively, which were higher than previously reported approaches in the TAC 2017 challenge [7] as well as when using the NER predictive model based on RxBERT [31].”
InfluentialSelf-citationWu, L., Xu, J., Thakkar, S., Gray, M., Qu, Y., Li, D., & Tong, W. (2024) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2024.105613 · Cited by 16 · see citing works
Self-citationNeyarapally, G. A., Wu, L., Xu, J., Zhou, E. H., Dang, O., Lee, J., Mehta, D., Vaughn, R. D., Pinnow, E. E., & Fang, H. (2024) · Drug Safety · doi:10.1007/s40264-024-01468-8 · Cited by 9 · see citing works
Self-citationBenaïche, A., Billaut-Laden, I., Randriamihaja, H., & Bertocchio, J. (2025) · Journal of Medical Internet Research · doi:10.2196/65651 · Cited by 8 · see citing works
Leveraging Large Language Models in Extracting Drug Safety Information from Prescription Drug Labels
Gisladottir, U., Zietz, M., Kivelson, S., Tanaka, Y., Sirdeshmukh, G., Brown, K. L., & Tatonetti, N. P. (2025) · Drug Safety · doi:10.1007/s40264-025-01594-x · Cited by 6 · see citing works
Artificial Intelligence for Drug Safety Across the Lifecycle and Decision Type: A Scoping Review
Kim, T. H., Park, S., & Kim, M. (2026) · Pharmaceuticals · doi:10.3390/ph19020334 · Cited by 6 · see citing works
Integrating artificial intelligence with bioinformatics promotes public health
Hong, H., & Slikker, W. (2023) · Experimental Biology and Medicine · doi:10.1177/15353702231223575 · Cited by 4 · see citing works
Large Language Models for Pharmaceutical Knowledge Management: A Critical Review
Torres, A., & Fernandez, M. (2025) · Pharmacophore · doi:10.51847/xphozuyfsy · Cited by 2 · see citing works
Martins, B., Pereira, L. H. N., de Azevedo, R., & Costa, P. (2025) · Pharmacophore · doi:10.51847/vwhtdaetbq · Cited by 2 · see citing works
Artificial Intelligence in Clinical Trials: From Protocol Design to Pharmacovigilance
Schulman, A., Fernández‐Torras, A., Gadiya, Y., Saeed, K., Mestres, J., & Tanoli, Z. (2026) · Springer eBooks · doi:10.1007/978-3-031-98022-0_24 · Cited by 1 · see citing works
Carrera, I., Criollo, J., & Dutra, I. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-98287-3_31
Sarwad, A., Housgoudar, S., -, S. B., Patil, V., Akki, A. J., & Parvatikar, P. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-34059-8.00006-3
Retrieval-Augmented Language Models for Checking Safety Consistency in Pharmaceutical Product Labels
Dupont, C., & Martin, J. (2025) · Pharmacophore · doi:10.51847/fzeuaxeluk
Kim, T. Y., Oh, W.-S., & Jeong, D. (2026) · Diagnostics · doi:10.3390/diagnostics16152435 · Cited by 1 · see citing works
- Authors
- Wu, L., Gray, M., Dang, O., Xu, J., Fang, H., & Tong, W.
- Affiliations
- National Center for Toxicological Research (US)
- United States Food and Drug Administration (US)
- Corresponding author
- Wu, Leihong
- Venue
- Experimental Biology and Medicine
- Publisher
- Frontiers Media SA
- Published
- 2023-11
- Volume
- 248
- Issue
- 21
- Pages
- 1937-1943
- ISSN
- 0071-3384 (ISSN-L)
- 1535-3702
- 1535-3699
- PMC
- PMC10798181
- OpenAlex
- W4390546587
- Cited by
- 16
- References
- 19
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 4 · 2023: 1
- FWCI
- 2.84
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Experimental Biology and Medicine (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Oak Ridge Institute for Science and Education
- Topics
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- drug labeling, FDA drug labels, BERT, BioBERT, adverse drug event detection, drug safety review, large language models, FDA drug review
- MeSH terms
- Artificial Intelligence, Drug Labeling, Humans, United States, Prescription Drugs, Data Mining, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
