The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index; 7 of the 18 sources its claims cite were checked for support.
Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel
Open accessAbstract
BACKGROUND: Drug adverse events (AEs) represent a significant public health concern. US Food and Drug Administration (FDA) drug labeling documents are an essential resource for studying drug safety such as assessing a drug's likelihood to cause certain organ toxicities; however, the manual extraction of AEs is labor-intensive, requires specialized expertise, and is challenging to maintain, due to frequent updates of the labeling documents. OBJECTIVE: To automate the extraction of AE data from FDA drug labeling documents, we developed a workflow based on AskFDALabel, a large language model (LLM)-powered framework, and its demonstration in drug safety studies. METHODS: This framework incorporates a retrieval-augmented generation (RAG) component based on FDALabel to enhance standard LLM inference. Key steps include (1) selection of a task-specific template, (2) FDALabel database querying, and (3) content preparation for LLM processing. We evaluated the performance of the framework in three benchmark experiments, including drug-induced liver injury (DILI) classification, drug-induced cardiotoxicity (DICT) classification, and AE term recognition. RESULTS: AskFDALabel achieved F1-scores of 0.978 for DILI, 0.931 for DICT, and 0.911 for AE annotation, outperforming other traditional methods. It also provided cited labeling content and detailed explanations, facilitating manual verification. CONCLUSION: AskFDALabel exhibited high consistency with human AE annotation, particularly in classifying and profiling DILI and DICT. Thus, it can significantly enhance the efficiency and accuracy of AE annotation, with promising potential for advanced AE surveillance and drug safety research.
Overview
CoChat Summary
TL;DR
AskFDALabel is an LLM-powered, retrieval-augmented framework for extracting and classifying drug adverse events from FDA labeling documents. It achieved high agreement with expert annotation across drug-induced liver injury, drug-induced cardiotoxicity, and adverse-event profiling benchmarks.
Key findings
- AskFDALabel achieved F1-scores of 0.978 for DILI classification, 0.931 for DICT classification, and 0.911 for AE annotation, outperforming BERT-, keyword-, XGBoost-, previous-LLM-, and GPT-3.5-based comparators reported in the paper.
- For DILI, 153/154 DILI-positive drugs (99.4%) and 44/57 DILI-negative drugs (77.2%) were labeled consistently with manual curation; after verifying mismatches, the updated F1-score was 0.978.
- For DICT, 780/829 DICT-positive drugs (94.1%) and 271/338 DICT-negative drugs (80.2%) were correctly classified; the overall F1-score was 0.931 versus 0.728 for a previous LLM and 0.756 for GPT-3.5.
- For AE profiling on TAC 2017 section-level data, exact-match recall/F1 were 0.825/0.827 and semantic-match recall/F1 were 0.939/0.911; on up-to-date whole-label retrieval, 188 drugs were processed with semantic recall 0.895 and F1 0.859 at 1024 output tokens, rising to F1 0.870 at 4096 tokens. LLM-based TOI recognition identified 2037/2141 drug trade names (95.14%).
Methods
- The AskFDALabel framework combines FDALabel-based retrieval-augmented generation with term-of-interest recognition, task-specific templates, and a locally hosted Llama 3.1-70B core; key steps are template selection, FDALabel database querying, and content preparation for LLM inference.
- Three benchmark experiments were run: DILI classification on 211 available drugs (154 positive, 57 negative), DICT classification on 1167 drugs (829 positive, 338 negative), and AE profiling on 200 TAC 2017 drugs with over 26,000 manually annotated adverse reaction terms (~13,000 unique), using 476 labeling sections and whole-drug-document analysis.
- AE profiling was evaluated with exact and LLM-based semantic matching; performance was compared with prior methods including BERT, keyword-based classification, XGBoost, previous LLM, GPT-3.5, TAC 2017 state-of-the-art, and RxBERT.
Limitations
- Stated in the paper: fixed structured SQL queries retrieve labeling documents by drug name, limiting adaptability for specific inputs such as routes of administration or dosage forms; future LLM-generated SQL would require reliability testing. Fine-tuning with domain-specific data may improve performance but must balance computational cost and overfitting risk. The study used only public drug labeling documents, though the framework is described as adaptable to secure, non-public data.
- Evident from the text: the only DILI false-negative, clotrimazole, may reflect use of the latest labeling for a topically administered product; the authors state further investigation is needed into revising the template to focus on oral drugs. AE profiling on up-to-date labels showed decreased performance versus using TAC original contents, attributed to labeling retrieval inaccuracy and labeling updates since human annotation; 12 drugs could not be processed because their labeling was no longer available. Output token limits also affected a few labeling documents with many AE terms.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
4 Valid · 0 Not valid · 7 Not checked · 69 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“during the past two decades, the research team at the NCTR has produced several drug-induced toxicity benchmark datasets by manually reading and analyzing drug labeling documents, such as those for drug-induced liver injury (DILI) [9–11], drug-induced cardiotoxicity (DICT) [12], drug-induced renal injury list (DIRIL) [13], and others”
§ Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel > Introduction
What the sources say
- Not checked — no full text of the sourcecites [9] Chen M, Vijay V, Shi Q, Liu Z, Fang H, Tong W. FDA-approved drug labeling for the study of drug-induced liver injury. Drug Discov Today. 2011;16(15–16):697–703. source
- Not checked — no full text of the sourcecites [10] Chen M, Suzuki A, Thakkar S, Yu K, Hu C, Tong W. DILIrank: the largest reference drug list ranked by the risk for developing drug-induced liver injury in humans. Drug Discov Today. 2016;21(4):648–53. source
- Partly supportedcites [11] Thakkar S, Li T, Liu Z, Wu L, Roberts R, Tong W. Drug-induced liver injury severity and toxicity (DILIst): binary classification of 1279 drugs by human hepatotoxicity. Drug Discov Today. 2020;25(1):201–8. sourceExperimental — accuracy not yet measured
The Introduction (p.1) does confirm the DILI part of the claim: 'we developed such an approach to classify DILI potential of drugs using FDA-approved drug labeling documents [29]. We first reported a benchmark dataset of 287 drugs [29] and then expanded the list to generate a larger DILI drug list (DILIrank)', and the Abstract/Discussion present DILIst as the resulting benchmark list. Two attributed parts are not supported by our copy: it never names the NCTR or its research team (the only institutional statement is the FDA disclaimer in 'Additional information'), and DILIst is not itself a labeling-derived list — the Abstract and Introduction state it was generated by augmenting DILIrank with four external literature datasets (LiverTox, Greene, Suzuki, Zhu), i.e. beyond manual reading of drug labeling documents. DICT and DIRIL are not addressed anywhere in our copy. (LLM, conf=0.70)
- Supportedcites [12] Qu Y, Li T, Liu Z, Li D, Tong W. DICTrank: the largest reference list of 1318 human drugs ranked by risk of drug-induced cardiotoxicity using FDA labeling. Drug Discov Today. 2023;15(1):103770. source
The part of the sentence this citation is offered for is the DICT dataset produced by manual reading of drug labeling. Our copy states in the Discussion that the authors 'utilized a systematic process that involved keyword searches and manual curation of FDA labeling documents to create this dataset' (DICTrank, 1318 drugs in four ranks), and Figure 1 describes expert review by pharmacologists of each drug's label (DICTrank Development). The author affiliation is the National Center for Toxicological Research, FDA, Jefferson, AR (title page), so the NCTR attribution holds. The DILI and DIRIL examples are credited to other citations ([9-11], [13]) and are not part of what this source carries; our copy only mentions prior DILI labeling work in passing (Introduction, refs 16-18). (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [13] Connor S, Li T, Qu Y, Roberts RA, Tong W. Generation of a drug-induced renal injury list to facilitate the development of new approach methodologies for nephrotoxicity. Drug Discov Today. 2024;29(4):103938. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“some studies applied standardized terminologies like MedDRA for AE extraction, but this often led to missing some critical AEs, as the FDA does not mandate the use of MedDRA terminology in labeling [6, 19].”
§ Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel > Introduction
What the sources say
- Partly supportedcites [6] Wu L, Ingle T, Liu Z, Zhao-Wong A, Harris S, Thakkar S, et al. Study of serious adverse drug reactions using FDA-approved drug labeling and MedDRA. BMC Bioinform. 2019;20:129–39. sourceExperimental — accuracy not yet measured
The Discussion says FDA-approved drug labeling need not use MedDRA terminology and that differently worded ADRs could be missed by a MedDRA-based query. It does not establish that this often happens or that the missed events are critical. (LLM, conf=0.86)
- Partly supportedcites [19] Wu Y, Liu Z, Wu L, Chen M, Tong W. BERT-based natural language processing of drug labeling documents: a case study for classifying drug-induced liver injury risk. Front Artif Intell. 2021;4: 729834. sourceExperimental — accuracy not yet measured
The Discussion says MedDRA terms have many variants in drug labeling, and the Results report four false negatives from a keyword-based DILI classifier. This supports the concern that term-based extraction can miss adverse reactions. The paper does not say the FDA does not mandate MedDRA terminology or establish that this is why those cases were missed. Page numbers were not available in our copy. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“which were higher than previously reported approaches in the TAC 2017 challenge [7] as well as when using the NER predictive model based on RxBERT [31].”
§ Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel > Result > Drug AE Profiling
What the sources say
- Not checked — no full text of the sourcecites [7] Roberts K, Demner-Fushman D, Tonning JM. Overview of the TAC 2017 adverse reaction extraction from drug labels track. TAC; 2017. source
- Supportedcites [31] Wu L, Gray M, Dang O, Xu J, Fang H, Tong W. RxBERT: enhancing drug labeling text mining and analysis with AI language modeling. Exp Biol Med. 2023;248(21):1937–43. source
The citation to this paper identifies the RxBERT-based NER model used for comparison. In “NER tasks of the TAC 2017 dataset” and Table 2, our copy reports RxBERT recall of 86.3% and F1 of 86.5%, both below the citing document’s 93.9% recall and 91.1% F1. The cited paper does not itself report the semantic-based model’s scores. (LLM, conf=0.91)
plainn = 476- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“This result outperformed previous Bidirectional Encoder Representations from Transformers (BERT)- and keyword-based DILI classification results, even without the needs of training a predictive model [19, 30].”
§ Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel > Result > DILI Classification
What the sources say
- Supportedcites [19] Wu Y, Liu Z, Wu L, Chen M, Tong W. BERT-based natural language processing of drug labeling documents: a case study for classifying drug-induced liver injury risk. Front Artif Intell. 2021;4: 729834. source
The Results section (page unavailable) reports BERT-based and keyword-based DILI classification results, so this paper supports its citation as a source of the earlier results being compared. The statement that AskFDALabel outperformed them without training a predictive model describes the citing authors’ own result, not a finding attributed to this paper. (LLM, conf=0.87)
- Supportedcites [30] Chen M, Wu Y, Wingerd B, Liu Z, Xu J, Thakkar S, et al. Automatic text classification of drug-induced liver injury using document-term matrix and XGBoost. Front Artif Intell. 2024;7:1401810. source
The Introduction and Discussion identify this paper as an earlier DILI-classification study using text terms and a trained XGBoost model. In the citing sentence, the claim that AskFDALabel outperformed earlier results without training a model describes the citing authors’ own result; this source is cited as a prior DILI-classification result, not as evidence that it used no predictive model. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AskFDALabel achieved F1-scores of 0.978 for DILI, 0.931 for DICT, and 0.911 for AE annotation, outperforming other traditional methods.”
§ Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
44 more claims
Create a free account to see all 80 claims and to verify the 2 references still open.
- Probably not a substantial claim. 18
- The quote only partly states this claim. 3
- It may state two separate claims as one. 15
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“recent NLP work on FDA labels targets toxicity or adverse-event extraction [12, 13] or general summarisation [14], not extrapolation.”
From PED-X-Bench: A Benchmark of Adult-to-Pediatric Extrapolation Decisions in FDA Drug Labels (2025) · cites this paper as [13]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“to date, no public dataset has systematically annotated or categorized extrapolation decisions in FDA drug labels.”
From PED-X-Bench: A Benchmark of Adult-to-Pediatric Extrapolation Decisions in FDA Drug Labels (2025) · cites this paper as [13]
Experimental — accuracy not yet measured
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“Word embedding approaches such as Global Vectors for Word Representation (GloVe), Sentence–Bidirectional Encoder Representations from Transformers (SBERT), and Doc2Vec are applied in this transformation process [25–27].”
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Incidence of Adverse Drug Reactions in Hospitalized Patients: A Meta-analysis of Prospective Studies
Lazarou, J., Pomeranz, B. H., & Corey, P. N. (1998) · JAMA · doi:10.1001/jama.279.15.1200 · Cited by 4,807 · cited 1×
“Drug adverse events (AEs) are a major cause of death in the United States (US) [1].”
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817 · cited 1×
“The model was loaded in fp16 format using vLLM [29].”
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023) · arXiv · doi:10.48550/arxiv.2309.06180 · Cited by 1,802
The SIDER database of drugs and side effects
Kuhn, M., Letunic, I., Jensen, L. J., & Bork, P. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1075 · Cited by 1,451
FDA-approved drug labeling for the study of drug-induced liver injury
Chen, M., Vijay, V., Shi, Q., Liu, Z., Fang, H., & Tong, W. (2011) · Drug Discovery Today · doi:10.1016/j.drudis.2011.05.007 · Cited by 406 · cited 4×
“We used 287 DILI annotated drugs, retrieved from the original DILI class publication [9].”
InfluentialSelf-citationChen, M., Suzuki, A., Thakkar, S., Yu, K., Hu, C., & Tong, W. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.02.015 · Cited by 398
Self-citationThakkar, S., Li, T., Liu, Z., Wu, L., Roberts, R., & Tong, W. (2020) · Drug Discovery Today · doi:10.1016/j.drudis.2019.09.022 · Cited by 140 · cited 1×
“…toxicity benchmark datasets by manually reading and analyzing drug labeling documents, such as those for drug-induced liver injury (DILI) [9–11], drug-induced cardiotoxicity (DICT) [12], drug-induced renal injury list (DIRIL) [13], and others, which offer large collections of manually…”
Self-citation
Research works printed but not linked: 3
- Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M-A, Lacroix T, et al. Llama: open and efficient foundation language models. arXiv:230213971; 2023.
- Almazrouei E, Alobeidli H, Alshamsi A, Cappelli A, Cojocaru R, Debbah M, et al. The Falcon series of open language models. arXiv:231116867. 2023.
- (NIST) USNIoSaT. Text Analysis Conference (TAC) 2017. 2017 [cited 2024; https://tac.nist.gov/2017/
Cited by
Fawzy, M. P., Nafie, M. S., Nabih, N. W., Diab, M. K., Babker, A. M. A., Mohamed, H., Hassan, H. A. F. M., & Fahmy, S. A. (2026) · Artificial Intelligence in the Life Sciences · doi:10.1016/j.ailsci.2026.100168 · Cited by 17
Prompt-to-Pill: Multi-Agent Drug Discovery and Clinical Simulation Pipeline
Vichentijevikj, I., Mishev, K., & Simjanoska, M. (2025) · Bioinformatics Advances · doi:10.1093/bioadv/vbaf323 · Cited by 10
“Because Prompt-to-Pill connects these components sequentially, the effective AD of the full system corresponds to the intersection of all model-specific ADs.”
Kathuria, D., Aggarwal, S., Negi, A., Barthwal, R., Joshi, A., & Singh, N. (2025) · Food Chemistry X · doi:10.1016/j.fochx.2025.103360 · Cited by 8
Application and Prospects of Large Language Models in Small-Molecule Drug Discovery
Ma, J., Liu, J., Xu, D., Song, X., & Zhang, Z. (2025) · Analytical Chemistry · doi:10.1021/acs.analchem.5c04083 · Cited by 10
Waikar, S., Bhat, A. G., & Ramanathan, M. K. (2026) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.70201 · Cited by 8
Metadata
- Authors
- Wu, L., Fang, H., Qu, Y., Xu, J., & Tong, W.
- Affiliations
- National Center for Toxicological Research (US)
- Corresponding author
- Wu, Leihong
- Venue
- Drug Safety
- Published
- 2025-02-20
- Volume
- 48
- Issue
- 6
- Pages
- 655-665
- ISSN
- 0114-5916 (ISSN-L)
- 1179-1942
- PMC
- PMC12098182
- OpenAlex
- W4407793327
- Semantic Scholar
- d6db744894b076eb0c8c78765985e7c125b88cab
- Cited by
- 20
- References
- 23
- Citations by year
- 2026: 12 · 2025: 8
- FWCI
- 10.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Drug Safety (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- drug adverse events, FDA drug labeling, large language models, retrieval-augmented generation, drug-induced liver injury, drug-induced cardiotoxicity, adverse event monitoring
- MeSH terms
- Large Language Models, Drug Labeling, Humans, United States, United States Food and Drug Administration, Databases, Factual, Adverse Drug Reaction Reporting Systems, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel. https://app.cochat.ai/papers/10.1007/s40264-025-01520-1
- The paper
- Wu, L., Fang, H., Qu, Y., Xu, J., & Tong, W. (2025). Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel. Drug Safety. https://doi.org/10.1007/s40264-025-01520-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.