Language Models as Knowledge Bases?
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tracing back to the work [53], large language models have shown their impressive power in storing factual information, such as entities and common sense, and then commonsense knowledge can be reliably transferred to downtown tasks.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [53]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While pure generative LLMs are often capable of answering basic factual questions (Petroni et al., 2019), simply by reciting information memorized from their training sets and stored in the model parameters”
From Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models (2024) · cites this paper as Petroni et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is well-accepted, even among skeptical researchers, that the parameters of modern LLMs encode a remarkable amount of world knowledge [2, 12, 23, 28, 31, 39, 62, 79], which can be leveraged by an LLM in flexible and robust ways [81, 83, 93].”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Pre-training of models on a large corpus is known to result in the model memorizing knowledge in its parameters [169, 201, 221].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [201]
The cited source says what the claim attributes to it.
In the citing paper
“The aforementioned example shows that AI long-term memory can be implicitly stored in the parameters of AI models [248, 249].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [248]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“BERT (base) (Devlin et al., 2018a) Transformer 110M Wikipedia (en) & BookCorpus 3.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“SQuAD (Rajpurkar et al., 2016) is a popular question answering dataset.”
Background(2013) · Cited by 5,726
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 3×
“…2017), recurrent neural networks (Zaremba et al., 2014; Merity et al., 2016; Melis et al., 2017) or self-attention mechanisms (Radford et al., 2018; Dai et al., 2019; Radford et al., 2019). fairseq-fconv: Instead of commonly used recurrent neural networks, Dauphin et al. (2017) use multiple layers…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- [1]1. Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Pushmeet Kohli, and Edward Grefenstette. 2019. Learning to understand goal specifications by 7 https://github.com/allenai/allennlp 8 https://github.com/pytorch/fairseq 9 https://github.com/huggingface/ pytorch-transformers modelling reward. In International Conference on Learning Representations (ICLR).
- [12]12. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Eval- uation (LREC-2018).
- [24]24. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237.
Cited by
Items 301–400 of 1,061
Salamandra Technical ReportPreprint
González-Agirre, A., Pàmies, M., Palao, J. L., Baucells, I., Dalt, S. D., Tamayo, D. C. L., Saiz, J. J., Espuña, F., Prats, J., Aula-Blasco, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.08489 · Cited by 102
Yu, M., Zhang, Y., Huang, J., Wang, X., Zhang, Y., Ji, H., & Han, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.810 · Cited by 99
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Ye, J., Gao, J., Wu, Z., Feng, J., Yu, C., & Kong, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.269 · Cited by 101
Prompt-Learning for Short Text Classification
Zhu, Y. Q., Wang, Y., Qiang, J., & Wu, X. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3332787 · Cited by 99
Linearity of Relation Decoding in Transformer Language Models
Hernandez, E., Sharma, A. S., Haklay, T., Meng, K., Wattenberg, M., Andreas, J., Belinkov, Y., & Bau, D. (2023) · arXiv · doi:10.48550/arxiv.2308.09124 · Cited by 98
Pre-training Methods in Information Retrieval
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000100 · Cited by 98
Memorisation versus Generalisation in Pre-trained Language Models
Tänzer, M., Ruder, S., & Rei, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.521 · Cited by 98
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT
Ravichander, A., Hovy, E. H., Suleman, K., Trischler, A., & Cheung, J. C. K. (2020) · Joint Conference on Lexical and Computational Semantics · Cited by 97
Retrieval-augmented Generation across Heterogeneous Knowledge
Yu, W. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) · doi:10.18653/v1/2022.naacl-srw.7 · Cited by 98
Towards Open-Vocabulary Scene Graph Generation with Prompt-Based Finetuning
He, T., Gao, L., Song, J., & Li, Y.-F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19815-1_4 · Cited by 99
Briakou, E., Cherry, C., & Foster, G. F. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.524 · Cited by 97
Evaluating GPT-3 Generated Explanations for Hateful Content Moderation
Wang, H., Hee, M. S., Awal, M. R., Choo, K. T. W., & Lee, R. K.-W. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/694 · Cited by 97
Making Language Models Better Tool Learners with Execution Feedback
Qiao, S., Gui, H., Lv, C., Jia, Q., Chen, H., & Zhang, N. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.195 · Cited by 97
On Early Detection of Hallucinations in Factual Question Answering
Snyder, B., Moisescu, M., & Zafar, M. B. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671796 · Cited by 98
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
Dong, G., Zhu, Y., Zhang, C., Wang, Z., Wen, J.-R., & Dou, Z. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714717 · Cited by 98
Huang, Y., Chen, Y., Zhou, Y., & McKeown, K. R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.10670 · Cited by 97
Long Dialogue Emotion Detection Based on Commonsense Knowledge Graph Guidance
Nie, W., Bao, Y., Zhao, Y., & Liu, A. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3267295 · Cited by 96
Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation
Yang, K., Liu, D., Lei, W., Yang, B., Xue, M., Chen, B., & Xie, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.25 · Cited by 96
HONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95
Mitchell, E., Lin, C., Bosselut, A., Finn, C., & Manning, C. D. (2021) · arXiv · doi:10.48550/arxiv.2110.11309 · Cited by 95
Generate rather than Retrieve: Large Language Models are Strong Context Generators
Yu, W., Iter, D., Wang, S., Xu, Y., Ju, M., Sanyal, S., Zhu, C., Zeng, M., & Jiang, M. (2022) · arXiv · doi:10.48550/arxiv.2209.10063 · Cited by 95
Generating Benchmarks for Factuality Evaluation of Language Models
Muhlgay, D., Ram, O., Magar, I., Levine, Y., Ratner, N., Belinkov, Y., Abend, O., Leyton‐Brown, K., Shashua, A., & Shoham, Y. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.4 · Cited by 99
Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech
Chung, Y.-L., Tekiroğlu, S. S., & Guerini, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.79 · Cited by 95
CREAK: A Dataset for Commonsense Reasoning over Entity KnowledgePreprint
Onoe, Y., Zhang, M. J. Q., Choi, E., & Durrett, G. (2021) · NeurIPS Datasets and Benchmarks · doi:10.48550/arxiv.2109.01653 · Cited by 112
Language Models are Few-shot Multilingual Learners
Winata, G. I., Madotto, A., Lin, Z., Liu, R., Yosinski, J., & Fung, P. (2021) · Proceedings of the 1st Workshop on Multilingual Representation Learning · doi:10.18653/v1/2021.mrl-1.1 · Cited by 93
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?
Lehman, E., Jain, S., Pichotta, K., Goldberg, Y., & Wallace, B. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.73 · Cited by 93
Inductive Relation Prediction by BERT
Zha, H., Chen, Z., & Yan, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20537 · Cited by 93
Prompting Contrastive Explanations for Commonsense Reasoning Tasks
Paranjape, B., Michael, J., Ghazvininejad, M., Hajishirzi, H., & Zettlemoyer, L. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.366 · Cited by 93
Understanding Transformer Memorization Recall Through Idioms
Haviv, A., Cohen, I., Gidron, J., Schuster, R., Goldberg, Y., & Geva, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.19 · Cited by 94
Distribution-Aware Prompt Tuning for Vision-Language Models
Cho, E., Kim, J., & Kim, H. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02011 · Cited by 94
Instruction-tuned Language Models are Better Knowledge Learners
Jiang, Z., Sun, Z., Shi, W., Rodríguez, P., Zhou, C., Neubig, G., Lin, X., Yih, W.-T., & Iyer, S. V. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.296 · Cited by 94
Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph CompletionPreprint
Chen, C., Wang, Y., Li, B., & Lam, K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.07299 · Cited by 93
Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-training
Hardalov, M., Arora, A., Nakov, P., & Augenstein, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21318 · Cited by 92
Question Answering Over Temporal Knowledge Graphs
Saxena, A., Chakrabarti, S., & Talukdar, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.520 · Cited by 91
Zero-Shot Recommendation as Language Modeling
Sileo, D., Vossen, W., & Raymaekers, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99739-7_26 · Cited by 91
NORMSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly
Fung, Y. R., Chakrabarty, T., Guo, H., Rambow, O., Muresan, S., & Ji, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.941 · Cited by 91
Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity
Lauscher, A., Vulić, I., Ponti, E. M., Korhonen, A., & Glavaš, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.118 · Cited by 110
Resolving Knowledge Conflicts in Large Language ModelsPreprint
Wang, Y., Feng, S., Wang, H., Shi, W., Balachandran, V., He, T., & Tsvetkov, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.00935 · Cited by 91
Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning
Razeghi, Y., Logan IV, R. L., Gardner, M., & Singh, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.59 · Cited by 89
“Prompting Prompting has been widely applied to study the factual (Petroni et al., 2019), common-sense (Davison et al., 2019; Weir et al., 2020; Lin et al., 2020b), mathematical (Saxton et al., 2019), and other NLP task-related (Radford et al., 2019; Shin et al., 2020) Training Artifacts Challenge…”
MethodLLM-Select: Feature Selection with Large Language Models
Jeong, D. P., Lipton, Z. C., & Ravikumar, P. (2025) · Transactions on machine learning research · doi:10.48550/arxiv.2407.02694 · Cited by 89
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models
Jiang, Z., Anastasopoulos, A., Araki, J., Ding, H., & Neubig, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.479 · Cited by 88
A Survey of Confidence Estimation and Calibration in Large Language Models . -
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.366 · Cited by 88
“Large language models (LLMs) have demonstrated a wide range of capabilities, such as world knowledge storage, sophisticated language-based reasoning, and in-context learning (Petroni et al., 2019; Wei et al., 2022; Brown et al., 2020a).”
BackgroundData Contamination: From Memorization to Exploitation
Magar, I., & Schwartz, R. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.18 · Cited by 88
BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief
Kassner, N., Tafjord, O., Schütze, H., & Clark, P. E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.697 · Cited by 88
Contextual semantic embeddings for ontology subsumption prediction
Chen, J., He, Y., Geng, Y., Jiménez-Ruiz, E., Dong, H., & Horrocks, I. R. (2023) · World Wide Web · doi:10.1007/s11280-023-01169-9 · Cited by 88
Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
Zhao, Y., Devoto, A., Hong, G., Du, X., Gema, A. P., Wang, H., He, X., Wong, K., & Minervini, P. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.264 · Cited by 90
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model BeliefsPreprint
Hase, P., Diab, M., Çelikyılmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., & Iyer, S. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13654 · Cited by 88
Yang, R., Zhu, J., Man, J., Li, F., & Zhou, Y. J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112155 · Cited by 87
Knowledge Graphs for Multi-modal Learning: Survey and Perspective
Chen, Z., Zhang, Y., Fang, Y., Geng, Y., Guo, L., Chen, J., Liu, X., Pan, J. Z., Zhang, N., Chen, H., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103124 · Cited by 87
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Gekhman, Z., Yona, G., Aharoni, R., Eyal, M., Feder, A., Reichart, R., & Herzig, J. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.444 · Cited by 86
Learning from models beyond fine-tuning
Zheng, H., Shen, L., Tang, A., Luo, Y., Hu, H., Du, B., Wen, Y., & Tao, D. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00961-0 · Cited by 86
When Do You Need Billions of Words of Pretraining Data?
Zhang, Y., Warstadt, A., Li, X., & Bowman, S. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.90 · Cited by 85
“Fourth, we probe the models’ world knowledge and commonsense knowledge using unsupervised language model knowledge probing with the LAMA suite (Petroni et al., 2019).”
MethodLM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · arXiv · doi:10.48550/arxiv.2305.13281 · Cited by 85
BERTese: Learning to Speak to BERT
Haviv, A., Berant, J., & Globerson, A. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.316 · Cited by 84
Implicit Representations of Meaning in Neural Language Models
Li, B. Z., Nye, M., & Andreas, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.143 · Cited by 84
“LM pretraining has also been found to be useful for tasks like factoid question answering (Petroni et al., 2019; Roberts et al., 2020).”
MethodInformation Retrieval meets Large Language Models: A strategic report from Chinese IR community
Ai, Q., Bai, T., Cao, Z., Chang, Y., Chen, J., Chen, Z., Cheng, Z., Dong, S., Dou, Z., Feng, F., et al. (2023) · AI Open · doi:10.1016/j.aiopen.2023.08.001 · Cited by 84
Explain and Predict, and then Predict Again
Zhang, Z., Rudra, K., & Anand, A. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441758 · Cited by 84
Frisoni, G., Moro, G., & Carbonaro, A. (2021) · IEEE Access · doi:10.1109/access.2021.3130956 · Cited by 84
An Overview of Language Models: Recent Developments and Outlook
Wei, C., Wang, Y.-C., Wang, B., & Kuo, C. ‐. J. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000010 · Cited by 86
Making Pre-trained Language Models Great on Tabular Prediction
Yan, J., Zheng, B., Xu, H., Zhu, Y., Chen, D., Sun, J., Wu, J., & Chen, J. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2403.01841 · Cited by 84
Liu, Z., Kong, C., Liu, Y., & Sun, M. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.866 · Cited by 84
Exposing Attention Glitches with Flip-Flop Language Modeling
Liu, B., Ash, J. T., Goel, S., Krishnamurthy, A., & Zhang, C. (2023) · neural information processing systems · doi:10.52202/075280-1112 · Cited by 84
A Simple Long-Tailed Recognition Baseline via Vision-Language ModelPreprint
Ma, T., Geng, S., Wang, M., Shao, J., Lu, J., Li, H., Gao, P., & Qiao, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.14745 · Cited by 84
Dang, H., Goller, S., Lehmann, F., & Buschek, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580969 · Cited by 83
“Another approach involves constructing prompt templates to increase the accuracy for probing knowlege [33], for translation tasks [5], or for text classification tasks [39].”
Understanding In-Context Learning via Supportive Pretraining Data
Han, X., Simig, D., Mihaylov, T., Tsvetkov, Y., Celikyilmaz, A., & Wang, T. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.708 · Cited by 18
ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing
Zeng, Z., Zhang, H., Lu, R., Wang, D., Chen, B., & Wang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02247 · Cited by 84
A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection
Yang, S., Sun, R., & Wan, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.256 · Cited by 84
Cross-modal adapter for vision–language retrieval
Jiang, H., Zhang, J., Huang, R., Ge, C., Ni, Z., Song, S., & Huang, G. M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111144 · Cited by 84
Sun, Y., Zheng, Y., Hao, C., & Qiu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03564 · Cited by 84
Centaur: a foundation model of human cognitionPreprint
Binz, M., Akata, E., Bethge, M., Brändle, F., Callaway, F., Coda-Forno, J., Dayan, P., Demircan, C., Eckstein, M. K., Éltető, N., et al. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/d6jeb · Cited by 85
Does ChatGPT have semantic understanding? A problem with the statistics-of-occurrence strategy
Titus, L. M. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101174 · Cited by 82
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems
Madotto, A., Cahyawijaya, S., Winata, G. I., Xu, Y., Liu, Z., Lin, Z., & Fung, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.215 · Cited by 82
Aligning Generative Language Models with Human Values
Liu, R., Zhang, G., Feng, X., & Vosoughi, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.18 · Cited by 82
Prompting Language Models for Linguistic Structure
Blevins, T., Gonen, H., & Zettlemoyer, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.367 · Cited by 82
FinanceMATH: Knowledge-Intensive Math Reasoning in Finance Domains
Zhao, Y., Liu, H., Long, Y., Zhang, R., Zhao, C., & Cohan, A. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.693 · Cited by 84
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 82
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Dziri, N., Madotto, A., Zaïane, O., & Bose, A. J. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.168 · Cited by 81
“Moreover, empirical evidence for hallucination in LMs runs contrary to known studies that these large models are capable of recalling factual knowledge, e.g. entities and relations in a KG, (Roberts et al., 2020; Petroni et al., 2019).”
ResultKnowledge Graph Generation From Text
Melnyk, I., Dognin, P., & Das, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.116 · Cited by 81
Overview of knowledge reasoning for knowledge graph
Liu, X., Mao, T., Shi, Y., & Ren, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127571 · Cited by 81
Fundamental Capabilities and Applications of Large Language Models: A Survey
Li, J., Gao, Y., Yang, Y., Bai, Y., Zhou, X., Li, Y., Sun, H., Liu, Y., Si, X., Ye, Y., et al. (2025) · ACM Computing Surveys · doi:10.1145/3735632 · Cited by 82
Self-citationZheng, G., & Brintrup, A. M. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2543964 · Cited by 82
Training Data Leakage Analysis in Language ModelsPreprint
Inan, H. A., Ramadan, O., Wutschitz, L., Jones, D., Rühle, V., Withers, J., & Sim, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05405 · Cited by 87
Continual Prompt Tuning for Dialog State Tracking
Zhu, Q., Li, B., Mi, F., Zhu, X., & Huang, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.80 · Cited by 80
Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models
Hase, P., Diab, M. T., Celikyilmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., & Iyer, S. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.199 · Cited by 73
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 80
Lakatos, R., Pollner, P., Hajdú, A., & Joó, T. (2025) · Machine Learning and Knowledge Extraction · doi:10.3390/make7010015 · Cited by 80
Knowledge Editing for Large Language Models: A Survey
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., & Li, J. (2025) · ACM Computing Surveys · doi:10.1145/3698590 · Cited by 79
“…& T-REx-1000 [28]: First used in CALINET [24], the authors adopt the classic • CounterFact [70]: CounterFact is established to distinguish superficial alterations in the word • ParaRel [27]: ParaRel is an expert-curated dataset that comprises diverse prompt templates for • NQ-SituatedQA [56]: NQ…”
Improving Biomedical Pretrained Language Models with Knowledge
Yuan, Z., Liu, Y., Tan, C., Huang, S., & Huang, F. (2021) · Proceedings of the 20th Workshop on Biomedical Language Processing · doi:10.18653/v1/2021.bionlp-1.20 · Cited by 79
Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming
Misra, K., Ettinger, A., & Rayz, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.415 · Cited by 81
Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction
Ma, X., Li, J., & Zhang, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.153 · Cited by 79
Amortizing intractable inference in large language models
Hu, J. E., Jain, M., Elmoznino, E., Kaddar, Y., Lajoie, G., Bengio, Y., & Whitammer, E. S. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.04363 · Cited by 81
Rezayi, S., Liu, Z., Wu, Z., Dhakal, C., Ge, B., Dai, H., Mai, G., Liu, N., Zhen, C., Liu, T., et al. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3442542 · Cited by 80
SECURE: Benchmarking Large Language Models for Cybersecurity
Bhusal, D., Alam, T., Nguyen, L., Mahara, A., Lightcap, Z., Frazier, R., Fieblinger, R., Torales, G. L., Blakely, B., & Rastogi, N. (2024) · Annual Computer Security Applications Conference · doi:10.1109/acsac63791.2024.00019 · Cited by 79
A Comprehensive Guide to Explainable AI: From Classical Models to LLMsPreprint
Song, X., HSIEH, W.-C., Bi, Z., Jiang, C., Liu, J., Peng, B., Zhang, S., Pan, X., Xu, J., Wang, J., et al. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/wbk36 · Cited by 79
Heinzerling, B., & Inui, K. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.153 · Cited by 78
GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models
Prasad, A., Hase, P., Zhou, X., & Bansal, M. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.277 · Cited by 78
Efficient Large Scale Language Modeling with Mixtures of Experts
Artetxe, M., Bhosale, S., Goyal, N., Mihaylov, T., Ott, M., Shleifer, S., Lin, X. V., Du, J., Iyer, S., Pasunuru, R., et al. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.804 · Cited by 78
Commonsense Knowledge Base Completion with Structural and Semantic Context
Malaviya, C., Bhagavatula, C., Bosselut, A., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5684 · Cited by 78
Wang, Y., Cheng, J., Chen, Y., Shao, S., Zhu, L., Wu, Z., Liu, T., & Zhu, H. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3306105 · Cited by 78
Liu, K., Casper, S., Hadfield-Menell, D., & Andreas, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.291 · Cited by 80
Metadata
- Authors
- Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2463-2473
- OpenAlex
- W2970476646
- Cited by
- 1,834
- References
- 42
- Citations by year
- 2026: 129 · 2025: 230 · 2024: 300 · 2023: 469 · 2022: 334 · 2021: 246 · 2020: 121 · 2019: 5
- FWCI
- 67.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- language models, knowledge base, knowledge representation, knowledge extraction, question answering, commonsense knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.