Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index; 7 of the 18 sources its claims cite were checked for support.

Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel

Open access

Abstract

BACKGROUND: Drug adverse events (AEs) represent a significant public health concern. US Food and Drug Administration (FDA) drug labeling documents are an essential resource for studying drug safety such as assessing a drug's likelihood to cause certain organ toxicities; however, the manual extraction of AEs is labor-intensive, requires specialized expertise, and is challenging to maintain, due to frequent updates of the labeling documents. OBJECTIVE: To automate the extraction of AE data from FDA drug labeling documents, we developed a workflow based on AskFDALabel, a large language model (LLM)-powered framework, and its demonstration in drug safety studies. METHODS: This framework incorporates a retrieval-augmented generation (RAG) component based on FDALabel to enhance standard LLM inference. Key steps include (1) selection of a task-specific template, (2) FDALabel database querying, and (3) content preparation for LLM processing. We evaluated the performance of the framework in three benchmark experiments, including drug-induced liver injury (DILI) classification, drug-induced cardiotoxicity (DICT) classification, and AE term recognition. RESULTS: AskFDALabel achieved F1-scores of 0.978 for DILI, 0.931 for DICT, and 0.911 for AE annotation, outperforming other traditional methods. It also provided cited labeling content and detailed explanations, facilitating manual verification. CONCLUSION: AskFDALabel exhibited high consistency with human AE annotation, particularly in classifying and profiling DILI and DICT. Thus, it can significantly enhance the efficiency and accuracy of AE annotation, with promising potential for advanced AE surveillance and drug safety research.

Overview

CoChat Summary

TL;DR

AskFDALabel is an LLM-powered, retrieval-augmented framework for extracting and classifying drug adverse events from FDA labeling documents. It achieved high agreement with expert annotation across drug-induced liver injury, drug-induced cardiotoxicity, and adverse-event profiling benchmarks.

Key findings

  • AskFDALabel achieved F1-scores of 0.978 for DILI classification, 0.931 for DICT classification, and 0.911 for AE annotation, outperforming BERT-, keyword-, XGBoost-, previous-LLM-, and GPT-3.5-based comparators reported in the paper.
  • For DILI, 153/154 DILI-positive drugs (99.4%) and 44/57 DILI-negative drugs (77.2%) were labeled consistently with manual curation; after verifying mismatches, the updated F1-score was 0.978.
  • For DICT, 780/829 DICT-positive drugs (94.1%) and 271/338 DICT-negative drugs (80.2%) were correctly classified; the overall F1-score was 0.931 versus 0.728 for a previous LLM and 0.756 for GPT-3.5.
  • For AE profiling on TAC 2017 section-level data, exact-match recall/F1 were 0.825/0.827 and semantic-match recall/F1 were 0.939/0.911; on up-to-date whole-label retrieval, 188 drugs were processed with semantic recall 0.895 and F1 0.859 at 1024 output tokens, rising to F1 0.870 at 4096 tokens. LLM-based TOI recognition identified 2037/2141 drug trade names (95.14%).

Methods

  • The AskFDALabel framework combines FDALabel-based retrieval-augmented generation with term-of-interest recognition, task-specific templates, and a locally hosted Llama 3.1-70B core; key steps are template selection, FDALabel database querying, and content preparation for LLM inference.
  • Three benchmark experiments were run: DILI classification on 211 available drugs (154 positive, 57 negative), DICT classification on 1167 drugs (829 positive, 338 negative), and AE profiling on 200 TAC 2017 drugs with over 26,000 manually annotated adverse reaction terms (~13,000 unique), using 476 labeling sections and whole-drug-document analysis.
  • AE profiling was evaluated with exact and LLM-based semantic matching; performance was compared with prior methods including BERT, keyword-based classification, XGBoost, previous LLM, GPT-3.5, TAC 2017 state-of-the-art, and RxBERT.

Limitations

  • Stated in the paper: fixed structured SQL queries retrieve labeling documents by drug name, limiting adaptability for specific inputs such as routes of administration or dosage forms; future LLM-generated SQL would require reliability testing. Fine-tuning with domain-specific data may improve performance but must balance computational cost and overfitting risk. The study used only public drug labeling documents, though the framework is described as adaptable to secure, non-public data.
  • Evident from the text: the only DILI false-negative, clotrimazole, may reflect use of the latest labeling for a topically administered product; the authors state further investigation is needed into revising the template to focus on oral drugs. AE profiling on up-to-date labels showed decreased performance versus using TAC original contents, attributed to labeling retrieval inaccuracy and labeling updates since human annotation; 12 drugs could not be processed because their labeling was no longer available. Output token limits also affected a few labeling documents with many AE terms.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s40264-025-01520-1

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 An example of drug AE profiling contents for raxibacumab generated by AskFDALabel. We matched the generated AE profile with human annotation (ground truth) in two ways: (1) hash tags (##) indicated human-annotated AEs were matched exactly in the AE profiling; (2) dollar tags ($$) indicated human-annotated AEs were matched semantically based on LLM determination. AE adverse event, FDA Food and Drug Administration, LLM large language model
Fig. 1 An example of drug AE profiling contents for raxibacumab generated by AskFDALabel. We matched the generated AE profile with human annotation (ground truth) in two ways: (1) hash tags (##) indicated human-annotated AEs were matched exactly in the AE profiling; (2) dollar tags ($$) indicated human-annotated AEs were matched semantically based on LLM determination. AE adverse event, FDA Food and Drug Administration, LLM large language modelCC BY · PubMed Central
Fig. 2 The AskFDALabel framework for drug AE analysis. This framework introduces a structured, end-to-end pipeline that processes user queries through a series of specialized steps, rather than directly using an LLM. Upon receiving a user query, the framework determines the appropriate template to apply based on the query type. For example, in a DILI classification task (first template in figure), the framework identifies and extracts the drug name, retrieves the most recent labeling document from the FDALabel database, and extracts AE sections, such as “Boxed Warnings,” “Warnings and Precautions,” and “Adverse Reactions.” The LLM then identifies DILI-related keywords from these sections. A post-processing step evaluates the extracted keywords to assess whether the drug poses a DILI risk. The pipeline’s output is then provided to the user. AE adverse event, DILI drug-induced liver injury, FDA Food and Drug Administration, LLM large language model, TOI term-of-interest
Fig. 2 The AskFDALabel framework for drug AE analysis. This framework introduces a structured, end-to-end pipeline that processes user queries through a series of specialized steps, rather than directly using an LLM. Upon receiving a user query, the framework determines the appropriate template to apply based on the query type. For example, in a DILI classification task (first template in figure), the framework identifies and extracts the drug name, retrieves the most recent labeling document from the FDALabel database, and extracts AE sections, such as “Boxed Warnings,” “Warnings and Precautions,” and “Adverse Reactions.” The LLM then identifies DILI-related keywords from these sections. A post-processing step evaluates the extracted keywords to assess whether the drug poses a DILI risk. The pipeline’s output is then provided to the user. AE adverse event, DILI drug-induced liver injury, FDA Food and Drug Administration, LLM large language model, TOI term-of-interestCC BY · PubMed Central

Verification

Claims the paper makes

4 Valid · 0 Not valid · 7 Not checked · 69 Own results and interpretations

44 more claims

Create a free account to see all 80 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 18
  • The quote only partly states this claim. 3
  • It may state two separate claims as one. 15

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 23 references

Research works printed but not linked: 3

  1. Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M-A, Lacroix T, et al. Llama: open and efficient foundation language models. arXiv:230213971; 2023.
  2. Almazrouei E, Alobeidli H, Alshamsi A, Cappelli A, Cojocaru R, Debbah M, et al. The Falcon series of open language models. arXiv:231116867. 2023.
  3. (NIST) USNIoSaT. Text Analysis Conference (TAC) 2017. 2017 [cited 2024; https://tac.nist.gov/2017/

Cited by

See all 20

Metadata

Authors
Wu, L., Fang, H., Qu, Y., Xu, J., & Tong, W.
Affiliations
National Center for Toxicological Research (US)
Corresponding author
Wu, Leihong
Venue
Drug Safety
Published
2025-02-20
Volume
48
Issue
6
Pages
655-665
ISSN
  • 0114-5916 (ISSN-L)
  • 1179-1942
OpenAlex
W4407793327
Cited by
20
References
23
Citations by year
2026: 12 · 2025: 8
FWCI
10.16
Type
journal article
Language
en
Open access
Licence
Locations
  • Drug Safety (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Topics
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
drug adverse events, FDA drug labeling, large language models, retrieval-augmented generation, drug-induced liver injury, drug-induced cardiotoxicity, adverse event monitoring
MeSH terms
Large Language Models, Drug Labeling, Humans, United States, United States Food and Drug Administration, Databases, Factual, Adverse Drug Reaction Reporting Systems, Drug-Related Side Effects and Adverse Reactions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel. https://app.cochat.ai/papers/10.1007/s40264-025-01520-1
The paper
Wu, L., Fang, H., Qu, Y., Xu, J., & Tong, W. (2025). Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel. Drug Safety. https://doi.org/10.1007/s40264-025-01520-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.