The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Revisiting Relation Extraction in the era of Large Language Models
Open accessAbstract
Relation extraction (RE) is the core NLP task of inferring semantic relationships between entities from text.Standard supervised RE techniques entail training modules to tag tokens comprising entity spans and then predict the relationship between them.Recent work has instead treated the problem as a sequence-tosequence task, linearizing relations between entities as target strings to be generated conditioned on the input.Here we push the limits of this approach, using larger language models (GPT-3 and Flan-T5 large) than considered in prior work and evaluating their performance on standard RE tasks under varying levels of supervision.We address issues inherent to evaluating generative approaches to RE by doing human evaluations, in lieu of relying on exact matching.Under this refined evaluation, we find that: (1) Few-shot prompting with GPT-3 achieves near SOTA performance, i.e., roughly equivalent to existing fully supervised models; (2) Flan-T5 is not as capable in the fewshot setting, but supervising and fine-tuning it with Chain-of-Thought (CoT) style explanations (generated via GPT-3) yields SOTA results.We release this model as a new baseline for RE tasks 1 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This dependency is a shared limitation of agentic memory systems that rely on LLM-based structural inference, as they are susceptible to extraction errors and hallucinations (Pan et al., 2024; Xi et al., 2025; Wadhwa et al., 2023).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Wadhwa et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Support vector machine learning for interdependent and structured output spaces
Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015341 · Cited by 1,493
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Modeling Relations and Their Mentions without Labeled Text
Riedel, S., Yao, L., & McCallum, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15939-8_10 · Cited by 1,333
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610
Research works printed but not linked: 2
- Brown Tom, Mann Benjamin, Ryder Nick, Subbiah Melanie, Jared D Kaplan Prafulla Dhariwal, Neelakantan Arvind, Shyam Pranav, Sastry Girish, Askell Amanda, Agarwal Sandhini, Herbert-Voss Ariel, Krueger Gretchen, Henighan Tom, Child Rewon, Ramesh Aditya, Ziegler Daniel, Wu Jeffrey, Winter Clemens, Hesse Chris, Chen Mark, Sigler Eric, Litwin Mateusz, Gray Scott, Chess Benjamin, Clark Jack, Berner Christopher, Sam McCandlish Alec Radford, Sutskever Ilya, and Amodei Dario. 2020a. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877– 1901. Curran Associates, Inc.
- Paolini Giovanni, Athiwaratkun Ben, Krone Jason, Ma Jie, Achille Alessandro, ANUBHAI RISHITA, Santos Cicero Nogueira dos, Xiang Bing, and Soatto Stefano. 2021. Structured prediction as translation between augmented natural languages. In International Conference on Learning Representations.
Cited by
Sorted by citations
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335
Miao, J., Thongprayoon, C., Suppadungsuk, S., Valencia, O. A. G., & Cheungpasitporn, W. (2024) · Medicina · doi:10.3390/medicina60030445 · Cited by 198
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 193
Dennstädt, F., Zink, J., Putora, P. M., Hastings, J., & Čihorić, N. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02575-4 · Cited by 191
A survey of large language models for cyber threat detection
Chen, Y., Cui, M., Wang, D., Cao, Y., Yang, P., Jiang, B., Lü, Z., & Liu, B. (2024) · Computers & Security · doi:10.1016/j.cose.2024.104016 · Cited by 172
Ma, Y., Cao, Y., Hong, Y. S., & Sun, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.710 · Cited by 193
Data extraction methods for systematic review (semi)automation: a living systematic review
Schmidt, L., Olorisade, B. K., McGuinness, L. A., Thomas, J., & Higgins, J. P. T. (2021) · F1000Research · doi:10.12688/f1000research.51117.1 · Cited by 106
Chain of Thought Utilization in Large Language Models and Application in Nephrology
Miao, J., Thongprayoon, C., Suppadungsuk, S., Krisanapan, P., Radhakrishnan, Y., & Cheungpasitporn, W. (2024) · Medicina · doi:10.3390/medicina60010148 · Cited by 114
A review of AI edge devices and lightweight CNN and LLM deployment
Sun, K., Wang, X., Miao, X., & Zhao, Q. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128791 · Cited by 84
Actionable Cyber Threat Intelligence Using Knowledge Graphs and Large Language Models
Fieblinger, R., Alam, T., & Rastogi, N. (2024) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw61312.2024.00018 · Cited by 87
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 71
Fan, Z., & Chen, C. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103646 · Cited by 50
Data extraction methods for systematic review (semi)automation: Update of a living systematic review
Schmidt, L., Finnerty, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2023) · F1000Research · doi:10.12688/f1000research.51117.2 · Cited by 54
Enhancing supply chain visibility with knowledge graphs and large language models
AlMahri, S., Xu, L., & Brintrup, A. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2575841 · Cited by 74
Jain, R., Sojitra, D., Acharya, A., Saha, S., Jatowt, A., & Dandapat, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.418 · Cited by 39
LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction
Zhou, H., Li, M., Xiao, Y., Yang, H., & Zhang, R. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae147 · Cited by 47
An LLM-assisted ETL pipeline to build a high-quality knowledge graph of the Italian legislation
Colombo, A., Bernasconi, A., & Ceri, S. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104082 · Cited by 39
Li, J., Jia, Z., & Zheng, Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.334 · Cited by 40
Šuster, S., Baldwin, T., & Verspoor, K. M. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1749 · Cited by 31
Data extraction methods for systematic review (semi)automation: Update of a living systematic review
Schmidt, L., Finnerty Mutlu, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2025) · F1000Research · doi:10.12688/f1000research.51117.3 · Cited by 27
Retrieval-Augmented Code Generation for Universal Information Extraction
Guo, Y., Li, Z., Jin, X., Liu, Y., Zeng, Y., Liu, W., Li, X., Pan, Y., Bai, L., Guo, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_3 · Cited by 65
Barron, R., Grantcharov, V., Wanna, S., Eren, M. E., Bhattarai, M., Solovyev, N., Tompkins, G., Nicholas, C., Rasmussen, K. Ø., Matuszek, C., et al. (2024) · International Conference on Machine Learning and Applications · doi:10.1109/icmla61862.2024.00258 · Cited by 31
KLLMs4Rec: Knowledge graph-enhanced LLMs sentiment extraction for personalized recommendations
Cui, Y., Wang, K., Yu, H., Guo, X., & Cao, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127430 · Cited by 29
Testing prompt engineering methods for knowledge extraction from text
Polat, F., Tiddi, I., & Groth, P. (2024) · Semantic Web · doi:10.3233/sw-243719 · Cited by 26
Retrieval And Structuring Augmented Generation with Large Language Models
Jiang, P., Ouyang, S., Jiao, Y., Zhong, M., Tian, R., & Han, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736557 · Cited by 28
Information Extraction in Low-Resource Scenarios: Survey and Perspective
Deng, S., Ma, Y., Zhang, N., Cao, Y., & Hooi, B. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00013 · Cited by 25
Are Large Language Models Reliable Argument Quality Annotators?
Mirzakhmedova, N., Gohsen, M., Chang, C. H., & Stein, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63536-6_8 · Cited by 22
Core Concept Identification in Educational Resources via Knowledge Graphs and Large Language Models
Reales, D., Manrique, R., & Grévisse, C. (2024) · SN Computer Science · doi:10.1007/s42979-024-03341-y · Cited by 17
A survey on cutting-edge relation extraction techniques based on language models
Diaz-Garcia, J. A., & Amador, J. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11280-0 · Cited by 32
LLM-IE: a python package for biomedical generative information extraction with large language models
Hsu, E., & Roberts, K. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf012 · Cited by 16
InstructIE: A Bilingual Instruction-based Information Extraction Dataset
Gui, H., Qiao, S., Zhang, J., Ye, H., Sun, M., Liang, L., Pan, J. Z., Chen, H., & Zhang, N. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77847-6_4 · Cited by 30
LTACL: long-tail awareness contrastive learning for distantly supervised relation extraction
Yan, T., Zhang, X., & Luo, Z. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01226-w · Cited by 14
Dietrich, J., & Hollstein, A. (2024) · Drug Safety · doi:10.1007/s40264-024-01499-1 · Cited by 22
Wu, H., Zhang, Y., Han, Z., Hou, Y., Wang, L., Liu, S., Gong, Q., & Ge, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113057 · Cited by 20
GEOSatDB: global civil earth observation satellite semantic database
Lin, M., Jin, M., Li, J., & Bai, Y. (2024) · Big Earth Data · doi:10.1080/20964471.2024.2331992 · Cited by 13
A review on the reliability of knowledge graph: from a knowledge representation learning perspective
Yang, Y., Chen, J., & Xiang, Y. (2024) · World Wide Web · doi:10.1007/s11280-024-01316-w · Cited by 13
Text-to-structure interpretation of user requests in BIM interaction
Wei, Y., Li, X., & Petzold, F. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106119 · Cited by 14
PIVOINE: Instruction Tuning for Open-world Entity Profiling
Lu, K., Pan, X., Song, K., Zhang, H., Yu, D., & Chen, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1009 · Cited by 12
Enhancing Relation Extraction from Biomedical Texts by Large Language Models
Asada, M., & Fukuda, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60615-1_1 · Cited by 13
From document-centric to data-centric public service provision
Konstantinidis, I., Kapantai, E., Michailidis, A., Deligiannis, A. P., Berberidis, C., Magnisalis, I. D., & Peristeras, V. (2024) · Digital Government Research and Practice · doi:10.1145/3676279 · Cited by 17
A closer look at ChatGPT's role in concept map generation for education
Schicchi, D., Limongelli, C., Monteleone, V., & Taibi, D. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2025.2497110 · Cited by 11
Zhou, B., & Li, K. (2025) · Geosciences · doi:10.3390/geosciences15100382 · Cited by 13
Automatically learning linguistic structures for entity relation extraction
Yang, W., Chen, Y., Xu, J., Qin, Y., & Chen, P. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103904 · Cited by 13
LLM Instruction-Example Adaptive Prompting (LEAP) Framework for Clinical Relation ExtractionPreprint
Zhou, H., Li, M., Xiao, Y., Yang, H. Q., & Zhang, R. (2023) · medRxiv · doi:10.1101/2023.12.15.23300059 · Cited by 11
Li, Y., Viswaroopan, D., He, W., Li, J., Zuo, X., Xu, H., & Tao, C. (2025) · Research Square · doi:10.21203/rs.3.rs-6201919/v1 · Cited by 11
RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction
Meng, S., Hu, X., Liu, A., Li, S., Ma, F., Yang, Y., & Wen, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.316 · Cited by 16
Kwon, M., Bang, J., Hwang, S., Jang, J., & Lee, W. (2025) · Electronics · doi:10.3390/electronics14040659 · Cited by 10
Few-shot relational triple extraction with hierarchical prototype optimization
Gao, C., Zhang, X., Jin, Z. G., Shang, W., Ma, Y., Li, L., Ding, Z., & Liang, Y. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110779 · Cited by 9
Towards Generating High-Quality Knowledge Graphs by Leveraging Large Language Models
Ezzabady, M. K., Ieng, F., Khorashadizadeh, H., Benamara, F., Groppe, S., & Sahri, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70239-6_31 · Cited by 10
Large language models can extract metadata for annotation of human neuroimaging publications
Turner, M. D., Appaji, A., Rakib, N. A., Golnari, P., Rajasekar, A., Anitha Rathnam, K., V, Sahoo, S. S., Wang, Y., Wang, L., & Turner, J. A. (2025) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2025.1609077 · Cited by 10
LLM With Relation Classifier for Document-Level Relation Extraction
Li, X., Chen, K., Long, Y., & Zhang, M. (2026) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2026.3668580 · Cited by 34
Enhancing the coverage of SemRep using a relation classification approach
Ming, S., Zhang, R., & Halil Kilicoglu, B. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104658 · Cited by 7
TCohPrompt: task-coherent prompt-oriented fine-tuning for relation extraction
Long, J., Yin, Z., Liu, C., & Huang, W. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01563-4 · Cited by 8
Liu, W., Xiao, Y., Cheng, S., Zeng, D., Zhou, L., Kong, W., Zhang, M., & Chen, W. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126099 · Cited by 9
DIRel: Joint relational triple extraction through dual implicit relation
Zhang, L. F., & Zheng, N. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129374 · Cited by 8
A Study of Biomedical Relation Extraction Using GPT Models.
Zhang, J., Wibert, M., Zhou, H., Peng, X., Chen, Q., Keloth, V. K., Hu, Y., Zhang, R., Xu, H., & Raja, K. (2024) · PubMed Central · Cited by 7
Han, P., Liang, G., & Wang, Y. (2025) · Electronics · doi:10.3390/electronics14030593 · Cited by 7
Efficient unified information extraction model based on large language models
Zhang, X., Cai, S., Shen, X.-R., Yang, H. Q., Hu, W., & Zhang, Y. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113302 · Cited by 7
Bhasuran, B., Manoharan, S., Oviya, I. R., Murugesan, G., Prabahar, A., & Raja, K. (2024) · Biomedicines · doi:10.3390/biomedicines12071535 · Cited by 8
Semantically Guided Representation Learning For Action Anticipation
Diko, A., Avola, D., Prenkaj, B., Fontana, F., & Cinque, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73390-1_26 · Cited by 6
Ruixiang, L., Qiu-Jun, D., Xianhui, L., Chenglin, Z., & Weidong, Z. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05969-y · Cited by 6
Detecting Linguistic Indicators for Stereotype Assessment with Large Language Models
Görge, R., Mock, M., & Allende‐Cid, H. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732181 · Cited by 10
Inkeraction: An Interaction Modality Powered by Ink Recognition and Synthesis
Shi, L., Campbell, R., Chi, P., Cirimele, M., Cleron, M., Climer, K., Fleming, C. Q., Ganti, A., Gervais, P., Gonnet, P., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642498 · Cited by 9
Sarol, M. J., Hong, G., Guerra, E., & Halil Kilicoglu, B. (2024) · Database · doi:10.1093/database/baae079 · Cited by 6
Abisheka, P., Deisy, C., & Sharmila, P. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102257 · Cited by 6
Deep mining the textual gold in relation extraction
Sharma, T., & Emmert‐Streib, F. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11042-4 · Cited by 8
A discrete convolutional network for entity relation extraction
Yang, W., Qin, Y., Wang, K., Hu, Y., Huang, R., & Chen, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107117 · Cited by 7
A Knowledge Extraction Framework on Cyber Threat Reports with Enhanced Security Profiles
Cai, Y., Qiu, J., Zhang, F., Li, Q., & Chen, L. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729880 · Cited by 6
Task-Oriented Dynamic Knowledge Distillation for Continuous Few-Shot Relation Extraction
Yang, H., Zhang, X., Gao, C., Shang, W., Du, K., Cai, W., Wang, D., & Li, T. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113914 · Cited by 5
AGLHunter: Automated Threat Hunting Using In-Context Learning-Enhanced LLM
Cui, M., Jiang, Z., Yao, Y., Ma, C., He, Q., Yang, P., & Feng, H. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033364 · Cited by 6
Automated Mining of Structured Knowledge from Text in the Era of Large Language Models
Zhang, Y., Zhong, M., Ouyang, S., Jiao, Y., Zhou, S., Ding, L., & Han, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671469 · Cited by 8
Document-level relation extraction with Double graph guidance for long-tailed distributions
Liu, B., & Qi, G. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110237 · Cited by 5
Integrating Large Language Models and Knowledge Graphs for Next-level AGI
Luo, L., Yang, C., Kharlamov, E., & Pan, S. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3715866 · Cited by 8
Large Language Models for Few-Shot Automatic Term Extraction
Banerjee, S., Chakravarthi, B. R., & McCrae, J. P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70239-6_10 · Cited by 5
Çetinkaya, A. (2025) · Information · doi:10.3390/info16110960 · Cited by 4
Swarup, A., Bhandarkar, A., Dizon-Paradis, O. P., Wilson, R., & Woodard, D. L. (2024) · IEEE Access · doi:10.1109/access.2024.3494737 · Cited by 5
Semantics-enabled biomedical literature analytics
Halil Kilicoglu, B., Ensan, F., McInnes, B. T., & Wang, L. L. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104588 · Cited by 4
EE-LCE: An Event Extraction Framework Based on LLM-Generated CoT Explanation
Yu, Y., Wang, Y., Ma, Y., Li, J., Lu, K., Huang, Z., & Chua, T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5492-2_3 · Cited by 5
Nair, P. C., Gupta, D., & Devi, B. I. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.502 · Cited by 4
Roll, D. S., Kurt, Z., Li, Y.-L., & Woo, W.-L. (2025) · Italian National Conference on Sensors · doi:10.3390/s25113352 · Cited by 5
A survey of Few-Shot Relation Extraction combining meta-learning with prompt learning
Hang, T., Wu, W., Feng, J., Djigal, H., & Huang, J. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130534 · Cited by 5
Nachtegael, C., Stefani, J. D., & Lenaerts, T. (2023) · PLoS ONE · doi:10.1371/journal.pone.0292356 · Cited by 4
Reliable Data Generation and Selection for Low-Resource Relation Extraction
Yu, J., Wang, X., & Chen, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29915 · Cited by 4
Pan, D., Sun, Y., Xu, B., Li, J., Yang, Z., Luo, L., Lin, H., & Wang, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3578781 · Cited by 3
Towards scalable and cross-lingual specialist language models for oncology
Rohanian, M., Mehra, T., Miglino, N., Nooralahzadeh, F., Krauthammer, M. O., & WICKI, A. N. (2025) · Scientific Reports · doi:10.1038/s41598-025-19282-2 · Cited by 3
XLORE 3: A Large-Scale Multilingual Knowledge Graph from Heterogeneous Wiki Knowledge Resources
Zeng, K., Jin, H., Lv, X., Zhu, F., Hou, L., Zhang, Y., Pang, F., Qi, Y., Liu, D., Li, J., et al. (2024) · ACM Transactions on Information Systems · doi:10.1145/3660521 · Cited by 3
A homogeneous multimodality sentence representation for relation extraction
Wang, K., Chen, Y., Yang, W., & Qin, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102968 · Cited by 3
Plant-Disease Relation Model through BERT-BiLSTM-CRF Approach
Riyanto, S., Sitanggang, I. S., Djatna, T., & Atikah, T. D. (2024) · Indonesian Journal of Electrical Engineering and Informatics (IJEEI) · doi:10.52549/ijeei.v12i1.5154 · Cited by 4
Effective Sentence-Level Relation Extraction Model Using Entity-Centric Dependency Tree
Park, S., & Kim, H. (2022) · PeerJ Computer Science · doi:10.2139/ssrn.4272160 · Cited by 4
Zhao, D., Han, D., Yuan, Y., Ning, B., Li, M., He, Z., & Song, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681012 · Cited by 3
How Does Memorization Impact LLMs' Social Reasoning? An Assessment using Seen and Unseen Queries
Amirizaniani, M., Sivachenko, M., Lavergne, A., Shah, C. S., & Mashhadi, A. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703576 · Cited by 4
MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents
Jiang, D., Li, Y., Li, G., & Li, B. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1709 · Cited by 2
Privacy Requirements Acquisition and Analysis
Hosseini, M. B., & Breaux, T. D. (2025) · Springer eBooks · doi:10.1007/978-3-031-73143-3_9 · Cited by 2
Enhancing Zero-Shot Relation Extraction through Staged Interaction with Large Language Models
Zhang, Y., Duan, P., Yang, Y., & Xiong, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887575 · Cited by 2
Knowledge Enhancement with LLMs for Few-Shot Medical Relation Extraction
Zhang, K., Li, Y., Wu, P., Li, Y., Zhang, C., & Zan, H. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-3752-2_10 · Cited by 2
RelCheck: Improving Relation Extraction with Ontology-Guided and LLM-Based Validation
Ourekouch, M., Koulali, M.-A., & Erradi, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-94575-5_24 · Cited by 2
Multilingual Relation Extraction: A Survey
Ali, M., Speck, R., Zahera, H. M., Saleem, M., Moussallem, D., & Ngomo, A.-C. N. (2025) · IEEE Access · doi:10.1109/access.2025.3604258 · Cited by 3
Can an LLM reproduce an expert knowledge base in neuroeducation?
Bukina, T. V., Khramova, M. V., Smirnov, N. M., Kurkin, S., & Hramov, A. E. (2026) · The European Physical Journal Special Topics · doi:10.1140/epjs/s11734-026-02534-1 · Cited by 2
PotatoG-DKB: a potato gene-disease knowledge base mined from biological literature
Xie, C., Gao, J., Chen, J., & Zhao, X. (2024) · PeerJ · doi:10.7717/peerj.18202 · Cited by 2
Metadata
- Authors
- Wadhwa, S., Amir, S., & Wallace, B.
- Affiliations
- Northeastern University (US)
- ORCID
- Wallace, Byron
- Venue
- Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2023
- Volume
- 2023
- Pages
- 15566-15589
- ISSN
- 0736-587X (ISSN-L)
- PMC
- PMC10482322
- OpenAlex
- W4385571451
- Cited by
- 207
- References
- 32
- Citations by year
- 2026: 53 · 2025: 74 · 2024: 65 · 2023: 14
- FWCI
- 36.50
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1750978)
- National Institutes of Health (5r01lm012086-08)
- U.S. National Library of Medicine (R01-LM012086)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- relation extraction, large language models, GPT-3, Flan-T5, few-shot prompting, chain-of-thought prompting, human evaluation, sequence-to-sequence generation, entity relationships, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
