Machine learning-based identification and rule-based normalization of adverse drug reactions in drug labels
Open accessThe DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Background: Use of medication can cause adverse drug reactions (ADRs), unwanted or unexpected events, which are a major safety concern. Drug labels, or prescribing information or package inserts, describe ADRs. Therefore, systematically identifying ADR information from drug labels is critical in multiple aspects; however, this task is challenging due to the nature of the natural language of drug labels. Results: In this paper, we present a machine learning- and rule-based system for the identification of ADR entity mentions in the text of drug labels and their normalization through the Medical Dictionary for Regulatory Activities (MedDRA) dictionary. The machine learning approach is based on a recently proposed deep learning architecture, which integrates bi-directional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network (CNN), and Conditional Random Fields (CRF) for entity recognition. The rule-based approach, used for normalizing the identified ADR mentions to MedDRA terms, is based on an extension of our in-house text-mining system, SciMiner. We evaluated our system on the Text Analysis Conference (TAC) Adverse Drug Reaction 2017 challenge test data set, consisting of 200 manually curated US FDA drug labels. Our ML-based system achieved 77.0% F1 score on the task of ADR mention recognition and 82.6% micro-averaged F1 score on the task of ADR normalization, while rule-based system achieved 67.4 and 77.6% F1 scores, respectively. Conclusion: Our study demonstrates that a system composed of a deep learning architecture for entity recognition and a rule-based model for entity normalization is a promising approach for ADR extraction from drug labels.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“These research efforts present promising results in extracting true ADE mentions from drug label text, indicating the suitability of using NLP methods for this task.”
From OnSIDES database: Extracting adverse drug events from drug labels using natural language processing models (2025) · cites this paper as [21]
The cited source says what the claim attributes to it.
In the citing paper
“A number of NLP methods have been proposed to extract ADE terms from drug label text utilizing this dataset20–21.”
From OnSIDES database: Extracting adverse drug events from drug labels using natural language processing models (2025) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“LSTM is from the family of Recurrent Neural Networks (RNNs), which are designed to learn patterns within sequences [24].”
BackgroundMethod(2014) · Cited by 33,419
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Even though these components are theoretically capable of learning long distance dependencies, it is hard to train them with gradient descent due to the problems of gradient vanishing or explosion [25].”
BackgroundNeural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401 · cited 1×
“Two recent studies [20, 27] used two LSTMs which run on opposite directions on the input sequences.”
MethodOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 1×
“As a solution to the gradient explosion problem, our model used gradient normalization [26] with the value of 1, since it has been shown to be effective in the NER task [21].”
MethodEnd-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567 · cited 4×
“This representation has been shown to be powerful in encoding morphological information [20], which we expect to be useful in the biochemical domain as well.”
BackgroundMethodInfluentialIncorporating Nesterov Momentum into Adam
Dozat, T. (2016) · Cited by 1,595
The Medical Dictionary for Regulatory Activities (MedDRA)
Brown, E. G., Wood, L., & Wood, S. (1999) · Drug Safety · doi:10.2165/00002018-199920020-00002 · Cited by 1,390 · cited 2×
“For ADR, Medical Dictionary for Regulatory Activities (Med-DRA) terms [9], SNOMED CT [10] as well as a few bio-medical ontologies developed by us such as Ontology of Adverse Events (OAE) [11] and Ontology of Drug Neur-opathy Adverse Events (ODNAE) [12] can be used.”
BackgroundMethodDistributional Semantics Resources for Biomedical Text Processing
Pyysalo, S., Ginter, F., Moen, H., Salakoski, T., & Ananiadou, S. (2013) · Research Explorer (The University of Manchester) · Cited by 548
Research works printed but not linked: 4
- Gurulingappa H, Fluck J, Hofmann-Apitius M, Toldo L. Identification of adverse drug event assertive sentences in medical case reports. In: First international workshop on knowledge discovery and health care management (KD-HCM), European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD); 2011. p. 16–27.
- Bird S, Loper E, Klein E. Natural language processing with Python: analyzing text with the natural language toolkit; 2009.
- Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems; 2013. p. 3111–9.
- Gal Y, Ghahramani Z. A theoretically grounded application of dropout in recurrent neural networks. In: Advances in neural information processing systems; 2016. p. 1019–27
Cited by
Extracting adverse drug events from clinical Notes: A systematic review of approaches used
Modi, S., Kasmiran, K. A., Sharef, N. M., & Sharum, M. Y. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104603 · Cited by 54
Drug Safety Data Curation and Modeling in ChEMBL: Boxed Warnings and Withdrawn Drugs
Hunter, F. M. I., Bento, A. P., Bosc, N., Gaulton, A., Hersey, A., & Leach, A. R. (2021) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.0c00296 · Cited by 28
“16 Trends in boxed warnings in medicinal product labels have been evaluated over time to see whether safety concerns could be predicted (e.g., refs 17 and 18).”
BackgroundHuffman, A., Ong, E., Hur, J., D’Mello, A., Tettelin, H., & He, Y. O. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac190 · Cited by 24
Self-citationTanaka, Y., Chen, H. Y., Belloni, P., Gisladottir, U., Kefeli, J., Patterson, J., Srinivasan, A., Zietz, M., Sirdeshmukh, G., Berkowitz, J., et al. (2025) · Med · doi:10.1016/j.medj.2025.100642 · Cited by 17
Li, X., Sun, S., Tang, T., Lu, J., Zhang, L., Yin, J., Qian, G., & Wu, Y. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02322-0 · Cited by 18
“There are three types of NER methods that can be used to extract BI-RADS variables from EMRs: rule-based methods [22, 23], machine learning-based methods [24–26], and deep learning-based methods [27–29].”
Method
Metadata
- Authors
- Tiftikci, M., Özgür, A., He, Y., & Hur, J.
- Affiliations
- Boğaziçi University (TR)
- University of Michigan (US)
- University of North Dakota (US)
- Corresponding author
- Hur, Junguk
- Venue
- BMC Bioinformatics
- Published
- 2019-12
- Volume
- 20
- Issue
- S21
- Pages
- 707-707
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC6927101
- OpenAlex
- W2995450470
- Cited by
- 23
- References
- 32
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 4 · 2023: 3 · 2022: 5 · 2021: 3 · 2020: 1 · 2019: 1
- FWCI
- 0.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor, 22 d)
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute of General Medical Sciences (P20GM113123)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- adverse drug reactions, drug labels, prescribing information, entity recognition, text mining, BiLSTM, conditional random fields, FDA drug labels
- MeSH terms
- Machine Learning, Deep Learning, Drug Labeling, United States, United States Food and Drug Administration, Neural Networks, Computer, Data Mining, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Machine learning-based identification and rule-based normalization of adverse drug reactions in drug labels. https://app.cochat.ai/papers/10.1186/s12859-019-3195-5
- The paper
- Tiftikci, M., Özgür, A., He, Y., & Hur, J. (2019). Machine learning-based identification and rule-based normalization of adverse drug reactions in drug labels. BMC Bioinformatics. https://doi.org/10.1186/s12859-019-3195-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.