Joint extraction of entity and relation based on fine-tuning BERT for long biomedical literatures
Open accessAbstract
Motivation: Joint extraction of entity and relation is an important research direction in Information Extraction. The number of scientific and technological biomedical literature is rapidly increasing, so automatically extracting entities and their relations from these literatures are key tasks to promote the progress of biomedical research. Results: The joint extraction of entity and relation model achieves both intra-sentence extraction and cross-sentence extraction, alleviating the problem of long-distance information dependence in long literature. Joint extraction of entity and relation model incorporates a variety of advanced deep learning techniques in this paper: (i) a fine-tuning BERT text classification pre-training model, (ii) Graph Convolutional Network learning method, (iii) Robust Learning Against Textual Label Noise with Self-Mixup Training, (iv) Local regularization Conditional Random Fields. The model implements the following functions: identifying entities from complex biomedical literature effectively, extracting triples within and across sentences, reducing the effect of noisy data during training, and improving the robustness and accuracy of the model. The experiment results prove that the model performs well on the self-built BM_GBD dataset and public datasets, enabling precise large language model enhanced knowledge graph construction for biomedical tasks. Availability and implementation: The model and partial code are available on GitHub at https://github.com/zhaix922/Joint-extraction-of-entity-and-relation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Unlike knowledge graph approaches that prioritize coverage across thousands of abstracts and can tolerate some extraction errors, parameter calibration requires higher per-target stringency because each value directly influences model predictions.
“Unlike knowledge graph approaches that prioritize coverage across thousands of abstracts [14, 32] and can tolerate some extraction errors, parameter calibration requires higher per‐target stringency because each value directly influences model predictions.”
From Structured Schemas for Provenance‐Rich, <scp>LLM</scp> ‐Assisted <scp>QSP</scp> Model Calibration (2026) · cites this paper as [32]
Split into 2 atoms — 0 confirmed, 1 partially supported, 1 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredUnlike knowledge graph approaches that prioritize coverage across thousands of abstracts14,32 and can tolerate some extraction errors, parameter calibration requires higher per-target stringency because each value directly influences model predictions.
“Unlike knowledge graph approaches that prioritize coverage across thousands of abstracts14,32 and can tolerate some extraction errors, parameter calibration requires higher per-target stringency”
From Structured Schemas for Provenance-Rich, LLM-Assisted QSP Model Calibration (2026) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,882 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · see citing works
Named Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J., & Nichols, E. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00104 · Cited by 1,925 · see citing works
A Frustratingly Easy Approach for Entity and Relation Extraction
Zhong, Z., & Chen, D. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.5 · Cited by 555 · see citing works
A Frustratingly Easy Approach for Entity and Relation Extraction
Zhong, Z., & Chen, D. (2020) · arXiv · doi:10.48550/arxiv.2010.12812 · Cited by 531 · see citing works
A comprehensive review of conditional random fields: variants, hybrids and applications
Yu, B., & Fan, Z. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09793-6 · Cited by 36 · see citing works
A Neural Joint Model for Extracting Bacteria and Their Locations
Li, F., Zhang, M., Fu, G., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-57529-2_2 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- Devlin J, Chang M-W, Lee K et al Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p. 4171– 86. Minneapolis, Minnesota: Association for Computational Linguistics, 2019.
- Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:.01991, 2015.
- Katiyar A, Cardie C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, p.917– 28. Vancouver, Canada: Association for Computational Linguistics, 2017.
- Li Q, Ji H. Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, p. 402–12. Baltimore, Maryland: Association for Computational Linguistics, 2014.
- Londt T, Gao X, Xue B et al Evolving character-level convolutional neural networks for text classification. arXiv preprint arXiv:.02223, 2020.
- Miwa M, Bansal M. End-to-end relation extraction using lstms on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, p. 1105–16. Berlin, Germany: Association for Computational Linguistics, 2016.
- Ning J, Yang Z, Sun Y et al Od-rte: a one-stage object detection framework for relational triple extraction. In: Paper Presented at the Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, p. 11120–35. Toronto, Canada: Association for Computational Linguistics, 2023.
- Qiao D, Dai C, Ding Y et al Selfmix: robust learning against textual label noise with self-mixup training. In: Proceedings of the 29th International Conference on Computational Linguistics, p. 960–70. Gyeongju, Republic of Korea: International Committee on Computational Linguistics, 2022.
- Rei M. Semi-supervised multitask learning for sequence labeling. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, p. 2121–30. Vancouver, Canada: Association for Computational Linguistics, 2017.
- Santos CNd, Guimaraes V. Boosting named entity recognition with neural character embeddings. In: Proceedings of the Fifth Named Entity Workshop, p. 25–33. Beijing, China: Association for Computational Linguistics, 2015.
- Shang Y-M, Huang H, Mao X. Onerel: joint entity and relation extraction with one module in one step. In: Proceedings of the AAAI Conference on Artificial Intelligence, p. 11285–93. Vancouver, Canada: Association for the Advancement of Artificial Intelligence, 2022.
- Vaswani A, Shazeer N, Parmar N et al Attention is all you need. Adv Neural Inf Process Syst 2017; 30: 5998–6008.
- Wang Y, Yu B, Zhang Y et al TPLinker: single-stage joint extraction of entities and relations through token pair linking. In: Proceedings of the 28th International Conference on Computational Linguistics, p. 1572–82. Barcelona, Spain: International Committee on Computational Linguistics, 2020.
- Wen Y, Wang Z, Sun J. Mindmap: knowledge graph prompting sparks graph of thoughts in large language models. arXiv preprint arXiv:.09729, 2023, preprint: not peer reviewed.
- Yang J, Jin H, Tang R et al Harnessing the power of llms in practice: a survey on ChatGPT and beyond. arXiv preprint arXiv:.13712, 2023, preprint: not peer reviewed.
- Yao L, Mao C, Luo Y. Graph convolutional networks for text classification. In: Paper Presented at the Proceedings of the AAAI Conference on Artificial Intelligence, p. 7370–7. Honolulu, Hawaii: Association for the Advancement of Artificial Intelligence, 2019.
- Ye J, Gui T, Luo Y et al One2Set: generating diverse keyphrases as a set. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, p. 4598–608. Association for Computational Linguistics, Online, 2021.
- Yu W, Lu N, Qi X et al PICK: processing key information extraction from documents using improved graph learning-convolutional networks. In: 2020 25th International Conference on Pattern Recognition (ICPR), p. 4363–70. Milan, Italy: IEEE, 2021.
- Zhang T, Tian X, Sun X et al Overview on knowledge graph embedding technology research. J Softw 2021; 34: 277– 311.
- Zheng S, Wang F, Bao H et al Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, p. 1227–36. Vancouver, Canada: Association for Computational Linguistics, 2017.
- Zhou W, Huang K, Ma T et al Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI Conference on Artificial Intelligence, p. 14612–20. Association for the Advancement of Artificial Intelligence, Virtual Conference, 2021.
Cited by
Sorted by citations
Structured Schemas for Provenance-Rich, LLM-Assisted QSP Model Calibration
Eliason, J., & Popel, A. S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.05.709623
Structured Schemas for Provenance‐Rich, <scp>LLM</scp> ‐Assisted <scp>QSP</scp> Model Calibration
Eliason, J., & Popel, A. S. (2026) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.70306
- Authors
- Gao, T., Zhai, X., Yang, C., Lv, L., & Wang, H.
- Affiliations
- Northeast Normal University (CN)
- Corresponding author
- Wang, Han
- Venue
- Bioinformatics Advances
- Publisher
- Oxford University Press (OUP)
- Published
- 2024
- Volume
- 4
- Issue
- 1
- Pages
- vbae194-vbae194
- ISSN
- 2635-0041 (ISSN-L)
- PMC
- PMC11665630
- OpenAlex
- W4405228207
- Cited by
- 2
- References
- 8
- Citations by year
- 2026: 2
- FWCI
- 0.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Bioinformatics Advances (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (62372099, 22273010)
- Jilin Scientific and Technological Development Program (20230401092YY, 20230201090GX)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- joint entity and relation extraction, biomedical literature, BERT fine-tuning, graph convolutional networks, label noise robustness, conditional random fields, knowledge graph construction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
