Language Models as Knowledge Bases?
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tracing back to the work [53], large language models have shown their impressive power in storing factual information, such as entities and common sense, and then commonsense knowledge can be reliably transferred to downtown tasks.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [53]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While pure generative LLMs are often capable of answering basic factual questions (Petroni et al., 2019), simply by reciting information memorized from their training sets and stored in the model parameters”
From Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models (2024) · cites this paper as Petroni et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is well-accepted, even among skeptical researchers, that the parameters of modern LLMs encode a remarkable amount of world knowledge [2, 12, 23, 28, 31, 39, 62, 79], which can be leveraged by an LLM in flexible and robust ways [81, 83, 93].”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Pre-training of models on a large corpus is known to result in the model memorizing knowledge in its parameters [169, 201, 221].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [201]
The cited source says what the claim attributes to it.
In the citing paper
“The aforementioned example shows that AI long-term memory can be implicitly stored in the parameters of AI models [248, 249].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [248]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“BERT (base) (Devlin et al., 2018a) Transformer 110M Wikipedia (en) & BookCorpus 3.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“SQuAD (Rajpurkar et al., 2016) is a popular question answering dataset.”
Background(2013) · Cited by 5,726
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 3×
“…2017), recurrent neural networks (Zaremba et al., 2014; Merity et al., 2016; Melis et al., 2017) or self-attention mechanisms (Radford et al., 2018; Dai et al., 2019; Radford et al., 2019). fairseq-fconv: Instead of commonly used recurrent neural networks, Dauphin et al. (2017) use multiple layers…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- [1]1. Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Pushmeet Kohli, and Edward Grefenstette. 2019. Learning to understand goal specifications by 7 https://github.com/allenai/allennlp 8 https://github.com/pytorch/fairseq 9 https://github.com/huggingface/ pytorch-transformers modelling reward. In International Conference on Learning Representations (ICLR).
- [12]12. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Eval- uation (LREC-2018).
- [24]24. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237.
Cited by
Items 401–500 of 1,061
Impact of Co-occurrence on Factual Knowledge of Large Language Models
Kang, C., & Choi, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.518 · Cited by 78
AutoKG: Efficient Automated Knowledge Graph Generation for Language Models
Chen, B., & Bertozzi, A. L. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386454 · Cited by 78
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering
Sachan, D. S., Reddy, S., Hamilton, W. L., Dyer, C., & Yogatama, D. (2021) · neural information processing systems · doi:10.48550/arxiv.2106.05346 · Cited by 129
A Causal View of Entity Bias in (Large) Language Models
Wang, F., Mo, W., Wang, Y., Zhou, W., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1013 · Cited by 78
Li, J., Yang, Y., Bai, Y., Zhou, X., Li, Y., Sun, H., Liu, Y., Si, X., Ye, Y., Wu, Y., et al. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.599 · Cited by 77
Self-citationDo Large Language Models Know about Facts?Preprint
Hu, X., Chen, J., Li, X., Guo, Y., Wen, L., Yu, P. S., & Guo, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05177 · Cited by 77
Mass-Editing Memory in a Transformer
Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., & Bau, D. (2022) · arXiv · doi:10.48550/arxiv.2210.07229 · Cited by 76
Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering
Banerjee, P., & Baral, C. R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.11 · Cited by 75
Knowledge-Augmented Language Model Verification
Baek, J., Jeong, S., Kang, M., Park, J. C., & Hwang, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.107 · Cited by 78
One for All: A Unified Generative Framework for Image Emotion Classification
Shi, G., Deng, S., Wang, B., Feng, C., Zhuang, Y., & Wang, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3341840 · Cited by 76
Exploring Universal Intrinsic Task Subspace for Few-Shot Learning via Prompt Tuning
Qin, Y., Wang, X., Su, Y., Lin, Y., Ding, N., Yi, J., Chen, W., Liu, Z., Li, J., Hou, L., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3430545 · Cited by 96
Joint Language Semantic and Structure Embedding for Knowledge Graph CompletionPreprint
Shen, J., Wang, C., Gong, L., & Song, D. X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.08721 · Cited by 76
Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
Bosselut, A., Bras, R. J. L., & Choi, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i6.16625 · Cited by 75
An Evaluation on Large Language Model Outputs: Discourse and Memorization
de Wynter, A., Wang, X., Sokolov, A., Gu, Q., & Chen, S.-Q. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100024 · Cited by 51
Hierarchical Prompt Learning for Multi-Task Learning
Liu, Y., Lu, Y., Liu, H., An, Y., Xu, Z., Yao, Z., Zhang, B., Xiong, Z., & Gui, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01048 · Cited by 78
Large Language Models and Knowledge Graphs: Opportunities and Challenges
Pan, J. Z., Razniewski, S., Kalo, J.-C., Singhania, S., Chen, J., Dietze, S., Jabeen, H., Omeliyanenko, J., Zhang, W. C., Lissandrini, M., et al. (2023) · UvA-DARE (University of Amsterdam) · doi:10.4230/tgdk.1.1.2 · Cited by 74
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
Kassner, N., Dufter, P., & Schütze, H. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.284 · Cited by 74
TopiOCQA: Open-domain Conversational Question Answering with Topic Switching
Adlakha, V., Dhuliawala, S., Suleman, K., de Vries, H., & Reddy, S. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00471 · Cited by 74
Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations
Yu, L., Cao, M., Cheung, J. C., & Dong, Y. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.466 · Cited by 75
Iteratively Prompt Pre-trained Language Models for Chain of Thought
Wang, B., Deng, X., & Sun, H. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.174 · Cited by 73
Rao, H., & Miao, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02118 · Cited by 74
Zhu, X., Qi, B., Zhang, K., Long, X., Lin, Z., & Zhou, B. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.142 · Cited by 73
Inside-Out: Hidden Factual Knowledge in LLMsPreprint
Gekhman, Z., David, E. B., Orgad, H., Ofek, E. O., Belinkov, Y., Szpektor, I., Herzig, J., & Reichart, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.15299 · Cited by 75
Citation: A Key to Building Responsible and Accountable Large Language Models
Huang, J., & Chang, K. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.31 · Cited by 72
Meta-learning via Language Model In-context Tuning
Chen, Y., Zhong, R., Zha, S., Karypis, G., & He, H. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.53 · Cited by 72
Zhang, H., Liu, X., Pan, H., Ke, H., Ou, J., Fang, T., & Song, Y. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103740 · Cited by 72
A survey on semantic processing techniques
Mao, R., He, K., Zhang, X., Chen, G., Ni, J., Yang, Z., & Cambria, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101988 · Cited by 72
Li, B., Haider, S., & Callison-Burch, C. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.213 · Cited by 73
Knowledge Enhanced Pretrained Language Models: A Compreshensive SurveyPreprint
Wei, X., Wang, S., Zhang, D., Bhatia, P., & Arnold, A. O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08455 · Cited by 72
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.146 · Cited by 71
Towards Few-shot Fact-Checking via Perplexity
Lee, N., Bang, Y., Madotto, A., & Fung, P. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.158 · Cited by 71
Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model
Dai, H., Song, Y., & Wang, H. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.141 · Cited by 71
Synthetic Disinformation Attacks on Automated Fact Verification Systems
Du, Y., Bosselut, A., & Manning, C. D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21302 · Cited by 71
Fun Arabic Teaching With Media Song For Early Child Education School
Yelfi Dewi, S., Umar, A. R., Khan, A. A., & Aziz, A. (2022) · Journal International of Lingua and Technology · doi:10.55849/jiltech.v1i2.80 · Cited by 71
Prompting as Probing: Using Language Models for Knowledge Base Construction
Alivanistos, D., Santamaría, S. B., Cochez, M., Kalo, J.-C., van Krieken, E., & Thanapalasingam, T. (2022) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2208.11057 · Cited by 74
Heterogeneous Region Embedding with Prompt Learning
Zhou, S., He, D., Chen, L., Shang, S., & Han, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25625 · Cited by 72
Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification
Zhai, Y., YaWen, Z., Huang, Z., Qin, Z., Jin, X., & Cao, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28524 · Cited by 73
Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks
Xi, Z., Du, T., Li, C., Pang, R., Ji, S., Chen, J., Ma, F., & Wang, T. (2023) · neural information processing systems · doi:10.52202/075280-1420 · Cited by 72
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Abdou, M., Kulmizev, A., Hershcovich, D., Frank, S., Pavlick, E., & Søgaard, A. (2021) · arXiv · doi:10.48550/arxiv.2109.06129 · Cited by 70
“Beyond this, it has been suggested that text-only training data is enough for LMs to also acquire factual and relational information about the world (Davison et al., 2019; Petroni et al., 2019).”
BackgroundFew-Shot Text Generation with Pattern-Exploiting Training
Schick, T., & Schütze, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.11926 · Cited by 70
Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation
Li, Y., Peng, B., Shen, Y., Mao, Y., Lidén, L., Yu, Z., & Gao, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.15 · Cited by 70
ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection
Bai, Y., Han, Z., Cao, B., Jiang, X., Hu, Q., & Zhang, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01655 · Cited by 72
Contrastive Learning Reduces Hallucination in Conversations
Sun, W., Shi, Z., Gao, S., Ren, P., De Rijke, M., & Ren, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26596 · Cited by 69
Probing Toxic Content in Large Pre-Trained Language Models
Ousidhoum, N., Zhao, X., Fang, T., Song, Y., & Yeung, D.-Y. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.329 · Cited by 69
Commonsense Reasoning for Natural Language Processing
Sap, M., Shwartz, V., Bosselut, A., Choi, Y., & Roth, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts · doi:10.18653/v1/2020.acl-tutorials.7 · Cited by 69
Jiang, Y., Zhang, L., & Wang, W. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.220 · Cited by 69
Fagbohun, O., Harrison, R. M., & Dereventsov, A. (2023) · Journal of Artificial Intelligence Machine Learning and Data Science · doi:10.51219/jaimld/oluwole-fagbohun/15 · Cited by 70
Software Testing of Generative AI Systems: Challenges and Opportunities
Aleti, A. (2023) · IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00009 · Cited by 72
Jansen, P. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.395 · Cited by 69
English Machine Reading Comprehension Datasets: A Survey
Dzendzik, D., Foster, J., & Vogel, C. M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.693 · Cited by 69
A Survey of Knowledge-Intensive NLP with Pre-Trained Language ModelsPreprint
Yin, D., Li, D., Cheng, H., Liu, X., Chang, K.-W., Wei, F., & Gao, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.08772 · Cited by 70
Towards Ontology Construction with Language ModelsPreprint
Funk, M., Hosemann, S., Jung, J. C., & Lutz, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.09898 · Cited by 71
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language ModelsPreprint
Hong, G., Gema, A. P., Saxena, R., Du, X., Nie, P., Zhao, Y., Perez-Beltrachini, L., Ryabinin, M., He, X., Fourrier, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.05904 · Cited by 70
Deduplicating Training Data Mitigates Privacy Risks in Language Models
Kandpal, N., Wallace, E., & Raffel, C. (2022) · arXiv · doi:10.48550/arxiv.2202.06539 · Cited by 68
AI vs. Human -- Differentiation Analysis of Scientific Content Generation
Ma, Y., Liu, J., Fan, Y., Cheng, Q., Huang, Y., Lu, W., & Liu, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.10416 · Cited by 68
It’s not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT
Gonen, H., Ravfogel, S., Elazar, Y., & Goldberg, Y. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.5 · Cited by 69
Relational World Knowledge Representation in Contextual Language Models: A Review
Safavi, T., & Koutra, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.81 · Cited by 69
From Questions to Insightful Answers: Building an Informed Chatbot for University Resources
Neupane, S., Hossain, E., Keith, J., Tripathi, H., Ghiasi, F., Golilarz, N. A., Amirlatifi, A., Mittal, S., & Rahimi, S. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10892994 · Cited by 69
Empower Entity Set Expansion via Language Model Probing
Zhang, Y., Shen, J., Shang, J., & Han, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.725 · Cited by 68
Abedi, M., Alshybani, I., Shahadat, M., & Murillo, M. S. (2023) · Qeios · doi:10.32388/md04b0 · Cited by 69
Optimizing Prompts for Text-to-Image Generation
Hao, Y., Chi, Z., Dong, L., & Wei, F. (2023) · neural information processing systems · doi:10.52202/075280-2923 · Cited by 67
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models
Peng, H. M., Wang, X., Hu, S., Jin, H., Hou, L., Li, J., Liu, Z., & Liu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.335 · Cited by 68
Martins, A., Londral, A. R. M., Nunes, I. L., & Lapão, L. V. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105385 · Cited by 67
Towards Uncertainty-Aware Language Agent
Han, J., Buntine, W., & Shareghi, E. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.398 · Cited by 67
Zero-Shot Information Extraction as a Unified Text-to-Triple Translation
Wang, C., Liu, X., Chen, Z., Hong, H., Tang, J., & Song, D. X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.94 · Cited by 67
Measuring Systematic Generalization in Neural Proof Generation with TransformersPreprint
Gontier, N., Sinha, K., Reddy, S., & Pal, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14786 · Cited by 69
Plansformer: Generating Symbolic Plans using TransformersPreprint
Pallagani, V., Muppasani, B. C., Murugesan, K., Rossi, F., Horesh, L., Srivastava, B., Fabiano, F., & Loreggia, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08681 · Cited by 67
Structured information extraction from complex scientific text with fine-tuned large language models
Dunn, A., Dagdelen, J., Walker, N., Lee, S., Rosen, A. S., Ceder, G., Persson, K., & Jain, A. (2022) · arXiv · doi:10.48550/arxiv.2212.05238 · Cited by 67
“" Use cases for seq2seq are broad[30] and include machine translation [31], answering general factual knowledge questions[32, 28], performing simple arithmetic [28], translating between languages [31, 33], summarizing text [34, 25], and chatbot applications [12, 35].”
BackgroundDo large language models have a legal duty to tell the truth?
Wachter, S., Mittelstadt, B., & Russell, C. (2024) · Royal Society Open Science · doi:10.1098/rsos.240197 · Cited by 66
Can Language Models be Biomedical Knowledge Bases?
Sung, M., Lee, J., Yi, S., Jeon, M., Kim, S., & Kang, J. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.388 · Cited by 66
Semantic Structure in Deep Learning
Pavlick, E. (2022) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-031120-122924 · Cited by 66
Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing
Wang, L., Qin, B., Hui, B., Li, B., Yang, M., Wang, B., Li, B., Sun, J., Huang, F., Si, L., et al. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539305 · Cited by 66
Zhang, Q., Naradowsky, J., & Miyao, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.417 · Cited by 67
An Emulator for Fine-Tuning Large Language Models using Small Language Models
Mitchell, E., Rafailov, R., Sharma, A., Finn, C., & Manning, C. D. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.12962 · Cited by 67
KnowNet:Guided Health Information Seeking from LLMs via Knowledge Graph Integration
Yan, Y., Hou, Y., Xiao, Y., Zhang, R., & Wang, Q. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456364 · Cited by 66
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Ortu, F., Jin, Z., Doimo, D., Sachan, M., Cazzaniga, A., & Schölkopf, B. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.458 · Cited by 66
Identifying Semantic Induction Heads to Understand In-Context Learning
Ren, J., Guo, Q., Yan, H. R., Liu, D., Zhang, Q., Qiu, X., & Lin, D. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.412 · Cited by 66
Detection and Measurement of Syntactic Templates in Generated Text
Shaib, C., Elazar, Y., Li, J. J., & Wallace, B. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.368 · Cited by 67
Neural Data Augmentation via Example ExtrapolationPreprint
Lee, K., Guu, K., He, L., Dozat, T., & Chung, H. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01335 · Cited by 67
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge
Chen, J., Shi, W., Fu, Z., Cheng, S., Li, L., & Xiao, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.550 · Cited by 65
Singha, M., Jha, A., Solanki, B. S., Bose, S., & Banerjee, B. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00196 · Cited by 66
DeepOnto: A Python package for ontology engineering with deep learning
He, Y., Chen, J., Dong, H., Horrocks, I. R., Allocca, C., Kim, T., & Sapkota, B. (2024) · Semantic Web · doi:10.3233/sw-243568 · Cited by 65
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Abdou, M., Kulmizev, A., Hershcovich, D., Frank, S., Pavlick, E., & Søgaard, A. (2021) · Proceedings of the 25th Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.9 · Cited by 64
PopBlends: Strategies for Conceptual Blending with Large Language Models
Wang, S., Petridis, S., Kwon, T., Ma, X., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580948 · Cited by 64
“It generally does not have “awareness of restrictions” [53].”
BackgroundPEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models
Yao, Y., Chen, Q., Zhang, A., Ji, W., Liu, Z., Chua, T., & Sun, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.763 · Cited by 65
Building trustworthy NeuroSymbolic AI Systems: Consistency, reliability, explainability, and safety
Gaur, M., & Sheth, A. (2024) · AI Magazine · doi:10.1002/aaai.12149 · Cited by 65
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Kandpal, N., Lester, B., Raffel, C., Majstorovic, S., Biderman, S., Abbasi, B., Soldaini, L., Shippole, E., Cooper, A. F., Skowron, A., et al. (2025) · neural information processing systems · doi:10.52202/085713-1918 · Cited by 67
Cross-lingual Visual Pre-training for Multimodal Machine Translation
Çağlayan, O., Kuyu, M., Amac, M. S., Madhyastha, P., Erdem, E., Erdem, A., & Specia, L. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.112 · Cited by 64
A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and ModelsPreprint
Wang, H., Wang, L., Du, Y., Liang, C., Zhou, J., Wang, Y., & Wong, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.16789 · Cited by 64
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning
Shen, T., Mao, Y., He, P., Long, G., Trischler, A., & Chen, W. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.722 · Cited by 63
Instruction Induction: From Few Examples to Natural Language Task Descriptions
Honovich, O., Shaham, U., Bowman, S. R., & Levy, O. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.108 · Cited by 63
Schramowski, P., Tauchmann, C., & Kersting, K. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533192 · Cited by 63
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Kassner, N., & Schütze, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.307 · Cited by 63
Pre-trained Language Models Can be Fully Zero-Shot Learners
Zhao, X., Ouyang, S., Yu, Z., Wu, M., & Li, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.869 · Cited by 64
Exploring low-resource medical image classification with weakly supervised prompt learning
Zheng, F., Cao, J., Yu, W., Chen, Z., Xiao, N., & Lu, Y. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110250 · Cited by 63
Large language models as oracles for instantiating ontologies with domain-specific knowledge
Ciatto, G., Agiollo, A., Magnini, M., & Omicini, A. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112940 · Cited by 65
Introduction to Transformers: an NLP PerspectivePreprint
Xiao, T., & Zhu, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.17633 · Cited by 63
A Review on Language Models as Knowledge Bases
AlKhamissi, B., Li, M., Celikyilmaz, A., Diab, M., & Ghazvininejad, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06031 · Cited by 62
“For example, populating KBs involves extracting huge amounts of relational data from unstructured text by using complex NLP pipelines such as entity linking and coreference resolution (Petroni et al., 2019).”
BackgroundMethodInfluentialInductive Learning on Commonsense Knowledge Graph Completion
Wang, B., Wang, G., Huang, J., You, J., Leskovec, J., & Kuo, C. ‐. J. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534355 · Cited by 62
Metadata
- Authors
- Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2463-2473
- OpenAlex
- W2970476646
- Cited by
- 1,834
- References
- 42
- Citations by year
- 2026: 129 · 2025: 230 · 2024: 300 · 2023: 469 · 2022: 334 · 2021: 246 · 2020: 121 · 2019: 5
- FWCI
- 67.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- language models, knowledge base, knowledge representation, knowledge extraction, question answering, commonsense knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.