Language Models as Knowledge Bases?
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tracing back to the work [53], large language models have shown their impressive power in storing factual information, such as entities and common sense, and then commonsense knowledge can be reliably transferred to downtown tasks.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [53]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While pure generative LLMs are often capable of answering basic factual questions (Petroni et al., 2019), simply by reciting information memorized from their training sets and stored in the model parameters”
From Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models (2024) · cites this paper as Petroni et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is well-accepted, even among skeptical researchers, that the parameters of modern LLMs encode a remarkable amount of world knowledge [2, 12, 23, 28, 31, 39, 62, 79], which can be leveraged by an LLM in flexible and robust ways [81, 83, 93].”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Pre-training of models on a large corpus is known to result in the model memorizing knowledge in its parameters [169, 201, 221].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [201]
The cited source says what the claim attributes to it.
In the citing paper
“The aforementioned example shows that AI long-term memory can be implicitly stored in the parameters of AI models [248, 249].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [248]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“BERT (base) (Devlin et al., 2018a) Transformer 110M Wikipedia (en) & BookCorpus 3.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“SQuAD (Rajpurkar et al., 2016) is a popular question answering dataset.”
Background(2013) · Cited by 5,726
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 3×
“…2017), recurrent neural networks (Zaremba et al., 2014; Merity et al., 2016; Melis et al., 2017) or self-attention mechanisms (Radford et al., 2018; Dai et al., 2019; Radford et al., 2019). fairseq-fconv: Instead of commonly used recurrent neural networks, Dauphin et al. (2017) use multiple layers…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- [1]1. Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Pushmeet Kohli, and Edward Grefenstette. 2019. Learning to understand goal specifications by 7 https://github.com/allenai/allennlp 8 https://github.com/pytorch/fairseq 9 https://github.com/huggingface/ pytorch-transformers modelling reward. In International Conference on Learning Representations (ICLR).
- [12]12. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Eval- uation (LREC-2018).
- [24]24. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237.
Cited by
Items 801–900 of 1,061
Large language models possess some ecological knowledge, but how much?
Dorm, F., Millard, J. W., Purves, D. W., Harfoot, M., & Aodha, O. M. (2026) · Ecological Informatics · doi:10.1016/j.ecoinf.2026.103699 · Cited by 35
A Few More Examples May Be Worth Billions of Parameters
Kirstain, Y., Lewis, P., Riedel, S., & Levy, O. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.72 · Cited by 35
Self-citationPromptGen: Automatically Generate Prompts using Generative Models
Zhang, Y., Fei, H., Li, D., & Li, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.3 · Cited by 35
Prompt-Based Prototypical Framework for Continual Relation Extraction
Zhang, H., Liang, B., Yang, M., Wang, H., & Xu, R. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3199655 · Cited by 35
Language Anisotropic Cross-Lingual Model Editing
Yang, X., Hou, Y., Che, W., & Zhang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.343 · Cited by 35
Does Transliteration Help Multilingual Language Modeling?
Moosa, I. M., Akhter, M. E., & Habib, A. B. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.50 · Cited by 35
Natural Language Decompositions of Implicit Content Enable Better Text Representations
Hoyle, A., Sarkar, R., Goel, P., & Resnik, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.815 · Cited by 35
Discovering Knowledge-Critical Subnetworks in Pretrained Language Models
Bayazit, D., Foroutan, N., Chen, Z., Weiss, G., & Bosselut, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.376 · Cited by 36
DocTER: Evaluating document-based knowledge editing
Wu, S., Wang, A., Peng, M., Lin, Y., Li, W., Sun, M., & Su, J. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104299 · Cited by 35
Language models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K., & Cohn, T. (2023) · arXiv · doi:10.48550/arxiv.2306.08189 · Cited by 34
“Finding 1: Larger LMs are more insensitive to negation MKR-NQ (Jang et al., 2022b) Masked Knowledge Retrieval – Negated Query (MKR-NQ) is a negated version of the LAMA dataset (Petroni et al., 2019), which contains lexicalized statements of triples in ConceptNet (Speer et al., 2017).”
BackgroundRecitation-Augmented Language Models
Sun, Z., Wang, X., Tay, Y., Yang, Y., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2210.01296 · Cited by 34
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks
Wu, Y., Zhao, Y., Hu, B., Minervini, P., Stenetorp, P., & Riedel, S. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.346 · Cited by 34
Self-citationLi, H., Appleby, G., Brumar, C. D., Chang, R., & Suh, A. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3326904 · Cited by 34
Zhao, B., Jin, W., Zhang, Y., Huang, S., & Yang, G. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110928 · Cited by 34
Large Language Model and Text Generation
Wu, Y. (2024) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-55865-8_10 · Cited by 34
Self-citationGong, Y., Shang, L., & Wang, D. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3404236 · Cited by 34
Drinkall, F., Rahimikia, E., Pierrehumbert, J. B., & Zohren, S. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.208 · Cited by 34
Data Contamination Report from the 2024 CONDA Shared Task
Sainz, O., García-Ferrero, I., Jacovi, A., Campos, J. A., Elazar, Y., Agirre, E., Goldberg, Y., Chen, W., Chim, J., Choshen, L., et al. (2024) · doi:10.18653/v1/2024.conda-1.4 · Cited by 34
A survey on neural relation extraction
Liu, K. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1673-6 · Cited by 34
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
Zhang, T., & Hashimoto, T. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.404 · Cited by 34
On Commonsense Cues in BERT for Solving Commonsense Tasks
Cui, L., Cheng, S., Wu, Y., & Zhang, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.61 · Cited by 46
Self-citationCUP: Curriculum Learning based Prompt Tuning for Implicit Event Argument Extraction
Lin, J., Chen, Q., Zhou, J., Jin, J., & He, L. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/589 · Cited by 34
Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?
Santra, P. P., & Majhi, D. (2023) · Journal of Information and Knowledge · doi:10.17821/srels/2023/v60i3/171028 · Cited by 35
Exploring Mode Connectivity for Pre-trained Language Models
Qin, Y., Qian, C., Yi, J., Chen, W., Lin, Y., Xu, H., Liu, Z., Sun, M., & Zhou, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.451 · Cited by 35
Emptying the Ocean with a Spoon: Should We Edit Models?
Pinter, Y., & Elhadad, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1012 · Cited by 34
Turning a CLIP Model Into a Scene Text Spotter
Yu, W., Liu, Y., Zhu, X., Cao, H., Sun, X., & Bai, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3379828 · Cited by 34
Why Does New Knowledge Create Messy Ripple Effects in LLMs?
Qin, J., Zhang, Z., Han, C., Yu, P., Li, M., & Ji, H. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.700 · Cited by 34
Contextualized Representations Using Textual Encyclopedic KnowledgePreprint
Joshi, M., Lee, K., Luan, Y., & Toutanova, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12006 · Cited by 34
Pre-training Methods in Information RetrievalPreprint
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13853 · Cited by 34
SpeechGen: Unlocking the Generative Power of Speech Language Models with PromptsPreprint
Wu, H., Chang, K.-W., Wu, Y.-K., & Lee, H.-Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02207 · Cited by 34
Self-citationTowards Safety and Helpfulness Balanced Responses via Controllable Large Language ModelsPreprint
Tuan, Y.-L., Chen, X., Smith, E. M., Martin, L., Batra, S., Çelikyılmaz, A., Wang, W. Y., & Bikel, D. M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.01295 · Cited by 34
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
Yue, X., Jimenez Gutierrez, B., & Sun, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.410 · Cited by 33
Torbarina, L., Ferkovic, T., Roguski, Ł., Mihelčić, V., Šarlija, B., & Kraljević, Ž. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100076 · Cited by 33
Relational Prompt-Based Pre-Trained Language Models for Social Event Detection
Li, P., Yu, X., Peng, H., Xian, Y., Wang, L., Sun, L., Zhang, J., & Yu, P. S. (2024) · ACM Transactions on Information Systems · doi:10.1145/3695869 · Cited by 33
AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approach
Amirizaniani, M., Martin, E., Roosta, T. G., Chadha, A. R., & Shah, C. S. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679222 · Cited by 33
JM3D & JM3D-LLM: Elevating 3D Representation With Joint Multi-Modal Cues
Ji, J., Wang, H., Wu, C., Ma, Y., Sun, X., & Ji, R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3523675 · Cited by 33
NeOn-GPT: A Large Language Model-Powered Pipeline for Ontology Learning
Fathallah, N., Das, A., Giorgis, S. D., Poltronieri, A., Haase, P., & Kovriguina, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78952-6_4 · Cited by 33
A Reality Check on Context Utilisation for Retrieval-Augmented Generation
Hagström, L., Marjanović, S. V., Yu, H., Arora, A., Lioma, C., Maistro, M., Atanasova, P., & Augenstein, I. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.968 · Cited by 33
PictoBERT: Transformers for next pictogram prediction
Pereira, J., Macêdo, D., Zanchettin, C., Oliveira, A. L. I., & Fidalgo, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117231 · Cited by 33
Kwon, S., Yao, Z., Jordan, H. S., Levy, D., Corner, B., & Yu, H. Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.805 · Cited by 35
Dong, Y., Wieting, J., & Verga, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.76 · Cited by 33
Aspect category sentiment analysis based on prompt-based learning with attention mechanism
Ping, Z., Sang, G., Liu, Z., & Zhang, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126994 · Cited by 34
Adaptive Prompt Routing for Arbitrary Text Style Transfer with Pre-trained Language Models
Liu, Q., Qin, J., Ye, W., Mou, H. L., He, Y., & Wang, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29832 · Cited by 33
What does the Knowledge Neuron Thesis Have to do with Knowledge?
Niu, J., Liu, A., Zhu, Z., & Penn, G. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2405.02421 · Cited by 33
Knowledge Verification to Nip Hallucination in the Bud
Wan, F., Huang, X., Cui, L., Quan, X.-J., Bi, W., & Shi, S.-M. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.152 · Cited by 18
An information theoretic view on selecting linguistic probes
Zhu, Z., & Rudzicz, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.744 · Cited by 33
Lecture Notes on Neural Information RetrievalPreprint
Tonellotto, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.13443 · Cited by 33
MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language ModelsPreprint
Wang, S., Lin, H., Luo, Z., Ye, Z., Chen, G., & Ma, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.11288 · Cited by 34
LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and RegularizationPreprint
Zhang, E., Goto, R., Sagan, N., Mutter, J., Phillips, N., Alizadeh, A., Lee, K., Blanchet, J., Pilanci, M., & Tibshirani, R. (2025) · arXiv.org · doi:10.48550/arxiv.2502.10648 · Cited by 25
Hase, P., Bansal, M., Kim, B., & Ghandeharioun, A. (2023) · neural information processing systems · doi:10.52202/075280-0774 · Cited by 32
Estimating Knowledge in Large Language Models Without Generating a Single Token
Gottesman, D., & Geva, M. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.232 · Cited by 32
Prompt Based Tri-Channel Graph Convolution Neural Network for Aspect Sentiment Triplet Extraction
Peng, K., Jiang, L., Peng, H. M., Liu, R., Yu, Z., Ren, J., Hao, Z., & Yu, P. S. (2024) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611978032.17 · Cited by 32
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
Wang, Y., Chen, Y., Wen, W., Sheng, Y., Li, L., & Zeng, D. D. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.420 · Cited by 32
Pre-trained Data Augmentation for Text Classification
Abonizio, H., & Barbon, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61377-8_38 · Cited by 32
Information to Wisdom: Commonsense Knowledge Extraction and Compilation
Razniewski, S., Tandon, N., & Varde, A. S. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441664 · Cited by 32
Do Language Models Know the Way to Rome?
Liétard, B., Abdou, M., & Søgaard, A. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.40 · Cited by 32
Clues Before Answers: Generation-Enhanced Multiple-Choice QA
Huang, Z., Wu, A., Zhou, J., Gu, Y., Zhao, Y., & Cheng, G. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.239 · Cited by 32
Fusing external knowledge resources for natural language understanding techniques: A survey
Wang, Y., Wang, W., Chen, Q., Huang, K., Nguyen, A., De, S., & Hussain, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.025 · Cited by 32
Lu, J., Li, J., Wallace, B., He, Y., & Pergola, G. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.80 · Cited by 32
Bouyamourn, A. Z. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.192 · Cited by 33
Liu, Y., An, C., & Qiu, X. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-3131-8 · Cited by 32
TaskLAMA: Probing the Complex Task Understanding of Language Models
Yuan, Q., Kazemi, M., Xu, X., Noble, I., Imbrasaitė, V., & Ramachandran, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29918 · Cited by 32
Prompt-based Zero-shot Relation Extraction with Semantic Knowledge Augmentation
Gong, J., & Eldardiry, H. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2yxn5oemh6ks · Cited by 32
Distinguishing Ignorance from Error in LLM HallucinationsPreprint
Simhi, A., Herzig, J., Szpektor, I., & Belinkov, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.22071 · Cited by 32
Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning
Tang, R., Kong, D., Huang, L., & Xue, H. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.284 · Cited by 31
Chen, H.-T., Zhang, M. J. Q., & Choi, E. (2022) · arXiv · doi:10.48550/arxiv.2210.13701 · Cited by 31
“Recent studies (Petroni et al., 2019) have shown that large language models are able to retain vast amounts of factual knowledge seen during pretraining, and closed-book QA systems (Roberts et al., 2020) build upon this foundation by memorizing facts from QA finetuning.”
BackgroundChen, H.-T., Zhang, M., & Choi, E. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.146 · Cited by 31
Entity Cloze By Date: What LMs Know About Unseen Entities
Onoe, Y., Zhang, M., Choi, E., & Durrett, G. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.52 · Cited by 31
Schramowski, P., Turan, C., Jentzsch, S., Rothkopf, C., & Kersting, K. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00036 · Cited by 31
A Large Language Model Agent Based Legal Assistant for Governance Applications
Mamalis, M. E., Kalampokis, E., Fitsilis, F., Theodorakopoulos, G., & Tarabanis, K. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70274-7_18 · Cited by 31
Natural language based analysis of SQuAD: An analytical approach for BERT
Güven, Z. A., & Ünalır, M. O. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116592 · Cited by 31
Soni, N., Matero, M., Balasubramanian, N., & Schwartz, H. A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.52 · Cited by 31
Plug-and-Play Adaptation for Continuously-updated QA
Lee, K., Han, W., Hwang, S.-W., Lee, H., Park, J., & Lee, S.-W. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.37 · Cited by 31
Chen, Q., Li, F.-L., Xu, G., Yan, M., Zhang, J., & Zhang, Y. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/567 · Cited by 31
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
Ma, Z., Pan, J., & Chai, J. Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.31 · Cited by 31
LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
Wei, C., Meng, W., Zhang, Z., Chen, M., Zhao, M., Fang, W., Wang, L., Zhang, Z., & Chen, W. (2024) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.23238 · Cited by 31
Video Question Answering With Semantic Disentanglement and Reasoning
Liu, J., Wang, G., Xie, J., Zhou, F., & Xu, H. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3317447 · Cited by 31
Neural Methods for Data-to-text Generation
Sharma, M., Gogineni, A. K., & Ramakrishnan, N. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3660639 · Cited by 31
Automated Prompting for Non-Overlapping Cross-Domain Sequential Recommendation
Guo, L., Wang, C., Wang, X., Zhu, L., & Yin, H. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3589721 · Cited by 31
Fringe News Networks: Dynamics of US News Viewership following the 2020 Presidential Election
KhudaBukhsh, A. R., Sarkar, R., Kamlet, M. S., & Mitchell, T. M. (2022) · ACM Web Science Conference (WebSci) · doi:10.1145/3501247.3531577 · Cited by 31
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
Brahman, F., Shwartz, V., Rudinger, R., & Choi, Y. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i14.17492 · Cited by 31
Do language models learn typicality judgments from text?Preprint
Misra, K., Ettinger, A., & Rayz, J. T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02987 · Cited by 31
OpenICL: An Open-Source Framework for In-context LearningPreprint
Wu, Z., Wang, Y., Ye, J., Feng, J., Xu, J., Qiao, Y., & Wu, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.02913 · Cited by 31
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization
Ladhak, F., Durmus, E., Suzgun, M., Zhang, T., Jurafsky, D., McKeown, K., & Hashimoto, T. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.234 · Cited by 30
“Prior work has shown that in addition to learning linguistic knowledge such as syntax, grammar, and structure, pre-trained LLMs can also capture and store relational knowledge from their pre-training corpus (Petroni et al., 2019).”
BackgroundHow Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis
Li, S., Li, X., Shang, L., Dong, Z., Sun, C., Liu, B., Ji, Z., Jiang, X., & Liu, Q. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.136 · Cited by 30
Open Domain Question Answering with A Unified Knowledge Interface
Ma, K., Cheng, H., Liu, X., Nyberg, E., & Gao, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.113 · Cited by 30
Answering Complex Queries in Knowledge Graphs with Bidirectional Sequence Encoders
Kotnis, B., Lawrence, C., & Niepert, M. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i6.16630 · Cited by 30
Joint reasoning with knowledge subgraphs for Multiple Choice Question Answering
Zhang, Q., Chen, S., Fang, M., & Chen, X. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103297 · Cited by 30
Sood, S. C., & Kim, A. (2023) · International Journal of Innovation and Economic Development · doi:10.18775/ijied.1849-7551-7020.2015.92.2001 · Cited by 30
Exemplar-Free Lifelong Person Re-identification via Prompt-Guided Adaptive Knowledge Consolidation
Li, Q., Xu, K., Peng, Y., & Zhou, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02110-x · Cited by 30
Language-specific representation of emotion-concept knowledge causally supports emotion inference
Li, M., Su, Y.-S., Huang, H.-Y., Cheng, J., Hu, X., Zhang, X., Wang, H., Qin, Y., Wang, X., Lindquist, K. A., et al. (2024) · iScience · doi:10.1016/j.isci.2024.111401 · Cited by 30
Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts
Chen, L., Dejl, A., & Toni, F. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34529 · Cited by 30
PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge
He, Y., Wang, Z., Zhang, Y., Huang, R., & Caverlee, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.611 · Cited by 30
A little goes a long way: Improving toxic language classification despite data scarcity
Juuti, M., Gröndahl, T., Flanagan, A., & Asokan, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.269 · Cited by 29
Pre-trained Language Models as Prior Knowledge for Playing Text-based Games
Singh, I., Singh, G., & Modi, A. (2022) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/gbmr8139 · Cited by 30
Gender bias, social bias, and representation in Bollywood and Hollywood
Khadilkar, K., KhudaBukhsh, A. R., & Mitchell, T. M. (2021) · Patterns · doi:10.1016/j.patter.2021.100409 · Cited by 31
Layout-Aware Dreamer for Embodied Visual Referring Expression Grounding
Li, M., Wang, Z., Tuytelaars, T., & Moens, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25223 · Cited by 30
Retrieval Contrastive Learning for Aspect-Level Sentiment Classification
Jian, Z., Li, J., Wu, Q., & Yao, J. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103539 · Cited by 30
StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models
Jeoung, S., Ge, Y., & Diesner, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.752 · Cited by 30
Metadata
- Authors
- Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2463-2473
- OpenAlex
- W2970476646
- Cited by
- 1,834
- References
- 42
- Citations by year
- 2026: 129 · 2025: 230 · 2024: 300 · 2023: 469 · 2022: 334 · 2021: 246 · 2020: 121 · 2019: 5
- FWCI
- 67.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- language models, knowledge base, knowledge representation, knowledge extraction, question answering, commonsense knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.