Language Models as Knowledge Bases?
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tracing back to the work [53], large language models have shown their impressive power in storing factual information, such as entities and common sense, and then commonsense knowledge can be reliably transferred to downtown tasks.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [53]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While pure generative LLMs are often capable of answering basic factual questions (Petroni et al., 2019), simply by reciting information memorized from their training sets and stored in the model parameters”
From Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models (2024) · cites this paper as Petroni et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is well-accepted, even among skeptical researchers, that the parameters of modern LLMs encode a remarkable amount of world knowledge [2, 12, 23, 28, 31, 39, 62, 79], which can be leveraged by an LLM in flexible and robust ways [81, 83, 93].”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Pre-training of models on a large corpus is known to result in the model memorizing knowledge in its parameters [169, 201, 221].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [201]
The cited source says what the claim attributes to it.
In the citing paper
“The aforementioned example shows that AI long-term memory can be implicitly stored in the parameters of AI models [248, 249].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [248]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“BERT (base) (Devlin et al., 2018a) Transformer 110M Wikipedia (en) & BookCorpus 3.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“SQuAD (Rajpurkar et al., 2016) is a popular question answering dataset.”
Background(2013) · Cited by 5,726
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 3×
“…2017), recurrent neural networks (Zaremba et al., 2014; Merity et al., 2016; Melis et al., 2017) or self-attention mechanisms (Radford et al., 2018; Dai et al., 2019; Radford et al., 2019). fairseq-fconv: Instead of commonly used recurrent neural networks, Dauphin et al. (2017) use multiple layers…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- [1]1. Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Pushmeet Kohli, and Edward Grefenstette. 2019. Learning to understand goal specifications by 7 https://github.com/allenai/allennlp 8 https://github.com/pytorch/fairseq 9 https://github.com/huggingface/ pytorch-transformers modelling reward. In International Conference on Learning Representations (ICLR).
- [12]12. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Eval- uation (LREC-2018).
- [24]24. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237.
Cited by
Items 501–600 of 1,061
Factual Consistency of Multilingual Pretrained Language Models
Fierro, C., & Søgaard, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.240 · Cited by 62
SemEval-2022 Task 4: Patronizing and Condescending Language Detection
Perez-Almendros, C., Espinosa-Anke, L., & Schockaert, S. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.38 · Cited by 62
NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models
Mei, K., Li, Z., Wang, Z., Zhang, Y., & Ma, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.867 · Cited by 62
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge
Wang, H., Prasad, A., Stengel-Eskin, E., & Bansal, M. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.581 · Cited by 64
mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
Ri, R., Yamada, I., & Tsuruoka, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.505 · Cited by 62
An In-depth Survey of Large Language Model-based Artificial Intelligence AgentsPreprint
Zhao, P., Jin, Z., & Cheng, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.14365 · Cited by 64
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond BenchmarksPreprint
Cao, Y., Hong, S., Li, X., Ying, J., Ma, Y., Liang, H., Liu, Y., Yao, Z., Wang, X., Huang, D., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.18838 · Cited by 62
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
Wang, C., Liu, X., Yue, Y., Tang, X., Zhang, T., Jiayang, C., Yao, Y., Gao, W., Hu, X., Qi, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.07521 · Cited by 61
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning
Tamborrino, A., Pellicanò, N., Pannier, B., Voitot, P., & Naudin, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.357 · Cited by 61
AdaPrompt: Adaptive Model Training for Prompt-based NLP
Chen, Y., Liu, Y., Li, D., Wang, S., Zhu, C., Zeng, M., & Zhang, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.448 · Cited by 61
Temporal Blind Spots in Large Language Models
Wallat, J., Jatowt, A., & Anand, A. (2024) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3616855.3635818 · Cited by 61
OpenPrompt: An Open-source Framework for Prompt-learning
Ding, N., Hu, S., Zhao, W., Chen, Y., Liu, Z., Zheng, H.-T., & Sun, M. (2021) · arXiv · doi:10.48550/arxiv.2111.01998 · Cited by 60
“And the evaluation tasks include WebNLG (Gardent et al., 2017) for conditional generation, GLUE (Wang et al., 2018) and Super-GLUE (Wang et al., 2019) for natural language understanding; SemEval (Hendrickx et al., 2010) for relation extraction; Few-NERD (Ding et al., 2021b) for fine-grained entity typing; MNLI (William…”
BackgroundLM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60
What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
Penha, G., & Hauff, C. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3412249 · Cited by 60
Ye, Z.-X., Chen, Q., Wang, W., & Ling, Z.-H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06725 · Cited by 60
DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense Knowledge
Fang, T., Zhang, H., Wang, W., Song, Y., & He, B. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450117 · Cited by 60
Wang, C., Huang, Z., Wan, Y., Wei, J., Zhao, J., & Wang, P. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.08.002 · Cited by 60
Keleg, A., & Magdy, W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.389 · Cited by 60
Berti, A., Kourani, H., Häfke, H., Li, C.-Y., & Schuster, D. (2024) · Lecture notes in business information processing · doi:10.1007/978-3-031-61007-3_2 · Cited by 60
Context versus Prior Knowledge in Language Models
Du, K., Snæbjarnarson, V., Stoehr, N., White, J., Schein, A., & Cotterell, R. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.714 · Cited by 62
Synergistic Interplay between Search and Large Language Models for Information Retrieval
Feng, J., Tao, C., Geng, X., Shen, T., Xu, C., Long, G., Zhao, D., & Jiang, D. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.517 · Cited by 60
Zeng, Q., Bai, Y., Tan, Z., Feng, S., Liang, Z., Zhang, Z., & Jiang, M. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679608 · Cited by 60
Discovering Latent Concepts Learned in BERTPreprint
Dalvi, F., Khan, A. R., Alam, F., Durrani, N., Xu, J., & Sajjad, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.07237 · Cited by 60
Syntax-Enhanced Pre-trained Model
Xu, Z., Guo, D., Tang, D., Su, Q., Shou, L., Gong, M., Zhong, W., Quan, X., Jiang, D., & Duan, N. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.420 · Cited by 59
Knowledge Enhanced Reflection Generation for Counseling Dialogues
Shen, S., Pérez‐Rosas, V., WELCH, C. A., Poria, S., & Mihalcea, R. F. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.221 · Cited by 59
zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
Ding, Z., Cai, H., Wu, J., Ma, Y., Liao, R., Xiong, B., & Tresp, V. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.104 · Cited by 59
PromptCARE: Prompt Copyright Protection by Watermark Injection and Verification
Yao, H., Lou, J., Qin, Z., & Ren, K. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00209 · Cited by 61
Set the Clock: Temporal Alignment of Pretrained Language Models
Zhao, B., Brumbaugh, Z., Wang, Y., Hajishirzi, H., & Smith, N. A. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.892 · Cited by 59
Yin, X., Zhang, X., Ruan, J., & Wan, X. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.124 · Cited by 59
Testing prompt engineering methods for knowledge extraction from text
Polat, F., Tiddi, I., & Groth, P. (2025) · Semantic Web · doi:10.3233/sw-243719 · Cited by 59
Memorization in Deep Learning: A Survey
Wei, J., Zhang, Y., Zhang, L. Y., Ding, M., Chen, C., Ong, K., Zhang, J., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3769076 · Cited by 60
Transformer networks of human conceptual knowledge.
Bhatia, S., & Richie, R. (2024) · Psychological Review · doi:10.1037/rev0000319 · Cited by 59
Semantic Consistency for Assuring Reliability of Large Language ModelsPreprint
Raj, H., Gupta, V., Rosati, D., & Majumdar, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.09138 · Cited by 59
Retrieval Augmented Generation for Domain-specific Question AnsweringPreprint
Sharma, S., Yoon, D. S., Dernoncourt, F., Sultania, D., Bagga, K., Zhang, M., Bui, T., & Kotte, V. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.14760 · Cited by 60
Learning from Task Descriptions
Weller, O., Lourie, N., Gardner, M., & Peters, M. E. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.105 · Cited by 58
An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models
Xie, Z., & Lukasiewicz, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.876 · Cited by 58
Prompt Tuning in Code Intelligence: An Experimental Evaluation
Wang, C., Yang, Y., Gao, C., Peng, Y., Zhang, H., & Lyu, M. R.-T. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3313881 · Cited by 60
PromptST: Prompt-Enhanced Spatio-Temporal Multi-Attribute Prediction
Zhang, Z., Zhao, X., Liu, Q., Zhang, C., Ma, Q., Wang, W., Zhao, H., Wang, Y., & Liu, Z. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615016 · Cited by 62
Sun, X., Lu, Y., & Chen, J. (2023) · Journal of Advanced Computing Systems · doi:10.69987/jacs.2023.30802 · Cited by 58
Few-Shot Learning with Siamese Networks and Label Tuning
Müller, T., Pérez-Torró, G., & Franco-Salvador, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.584 · Cited by 58
Probing Pre-Trained Language Models for Cross-Cultural Differences in Values
Arora, A., Kaffee, L.-A., & Augenstein, I. (2023) · Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.12 · Cited by 57
Mayer, C. W. F., Ludwig, S., & Brandt, S. (2023) · Journal of Research on Technology in Education · doi:10.1080/15391523.2022.2142872 · Cited by 57
Language Models as Hierarchy Encoders
He, Y., Yuan, Z., Chen, J., & Horrocks, I. R. (2024) · neural information processing systems · doi:10.52202/079017-0469 · Cited by 57
BERTnesia: Investigating the capture and forgetting of knowledge in BERT
Wallat, J., Singh, J., & Anand, A. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.17 · Cited by 56
Undesirable Memorization in Large Language Models: A SurveyPreprint
Satvaty, A., Verberne, S., & Türkmen, F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.02650 · Cited by 57
Adaptive Semiparametric Language Models
Yogatama, D., de Masson d’Autume, C., & Kong, L. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00371 · Cited by 56
Prompt Tuning for Discriminative Pre-trained Language Models
Yao, Y., Dong, B., Zhang, A., Zhang, Z., Xie, R., Liu, Z., Lin, L., Sun, M., & Wang, J. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.273 · Cited by 56
FolkScope: Intention Knowledge Graph Construction for E-commerce Commonsense Discovery
Yu, C., Wang, W., Liu, X., Bai, J., Song, Y., Li, Z., Gao, Y., Cao, T., & Yin, B. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.76 · Cited by 56
Speaking Multiple Languages Affects the Moral Bias of Language Models
Hämmerl, K., Deiseroth, B., Schramowski, P., Libovický, J., Rothkopf, C. A., Fraser, A., & Kersting, K. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.134 · Cited by 56
CoTDet: Affordance Knowledge Prompting for Task Driven Object Detection
Tang, J., Zheng, G., Yu, J., & Yang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00285 · Cited by 57
AI safety landscape for large language models: taxonomy, state-of-the-art, and future directions
Chen, C., Gong, X., Liu, Z., Jiang, W. M. W. L. H., Goh, S. Q., & Lam, K. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11590-x · Cited by 56
Holmes ⌕ A Benchmark to Assess the Linguistic Competence of Language Models
Waldis, A., Perlitz, Y., Choshen, L., Hou, Y., & Gurevych, I. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00718 · Cited by 56
Best Practices for Large Language Models in Radiology
Blüthgen, C., Veen, D. V., Zakka, C., Link, K. E., Fanous, A. H., Daneshjou, R., Frauenfelder, T., Langlotz, C. P., Gatidis, S., & Chaudhari, A. S. (2025) · Radiology · doi:10.1148/radiol.240528 · Cited by 56
Asking without Telling: Exploring Latent Ontologies in Contextual Representations
Michael, J., Botha, J. A., & Tenney, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.552 · Cited by 58
MLMLM: Link Prediction with Mean Likelihood Masked Language Model
Clouâtre, L., Trempe, P., Zouaq, A., & Chandar, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.378 · Cited by 56
Coherence boosting: When your pretrained language model is not paying enough attention
Malkin, N., Wang, Z., & Jojić, N. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.565 · Cited by 57
An exploratory study on code attention in BERT
Sharma, R., Chen, F., Fard, F. H., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527921 · Cited by 57
PLeak: Prompt Leaking Attacks against Large Language Model Applications
Hui, B., Yuan, H., Gong, N., Burlina, P., & Cao, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670370 · Cited by 55
Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA?
Wang, C., Liu, P., & Zhang, Y. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.251 · Cited by 55
“Petroni et al. (2019) first solves that discriminative PLMs such as BERT (Devlin et al., 2019) can be used for Cloze-style QA using a mask language modeling task without external resources, such as “Dante was born in [MASK].””
BackgroundMethodInfluential(2023) · doi:10.18653/v1/2023.emnlp-main.971 · Cited by 55
Text-Visual Prompting for Efficient 2D Temporal Video Grounding
Zhang, Y., Chen, X., Jia, J., Liu, S., & Ding, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01421 · Cited by 55
Editing Language Model-Based Knowledge Graph Embeddings
Cheng, S., Zhang, N., Tian, B., Chen, X., Liu, Q., & Chen, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29737 · Cited by 55
Yuan, X., Zhao, Y., Wang, Y., Liu, S., Jun, Z., & Liu, K. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.234 · Cited by 56
DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning
Bai, S., Zhang, J., Guo, S., Li, S. C., Guo, J., Hou, J., Han, T., & Lu, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02576 · Cited by 55
Sentence Analogies: Linguistic Regularities in Sentence Embeddings
Zhu, X., & de Melo, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.300 · Cited by 57
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
Gao, D., Wang, R., Shan, S., & Chen, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3210780 · Cited by 55
HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language ModelsPreprint
Zhang, Y., Li, Q., Du, T., Zhang, X., Zhao, X., Feng, Z. Y., & Yin, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.22832 · Cited by 56
Knowledge of cultural moral norms in large language models
Ramezani, A., & Xu, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.26 · Cited by 54
Visual Classification via Description from Large Language Models
Menon, S., & Vondrick, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.07183 · Cited by 54
Li, Y., Du, M., Wang, X., & Wang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.797 · Cited by 54
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
Li, J., Gao, M., Wei, L., Tang, S., Zhang, W., Li, M., Ji, W., Tian, Q., Chua, T., & Zhuang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00241 · Cited by 54
Quo Vadis ChatGPT? From Large Language Models to Large Knowledge Models
Venkatasubramanian, V., & Chakraborty, A. (2024) · Computers and Chemical Engineering · doi:10.1016/j.compchemeng.2024.108895 · Cited by 42
CommunityLM: Probing Partisan Worldviews from Language ModelsPreprint
Jiang, H., Beeferman, D., Roy, B., & Roy, D. K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.07065 · Cited by 54
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Geva, M., Bastings, J., Filippova, K., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.751 · Cited by 53
Impact of Pretraining Term Frequencies on Few-Shot Reasoning
Razeghi, Y., Logan, R. L., Gardner, M., & Singh, S. (2022) · arXiv · doi:10.48550/arxiv.2202.07206 · Cited by 53
“Prompting Prompting has been widely applied to study the factual (Petroni et al., 2019), commonsense (Davison et al., 2019; Weir et al., 2020; Lin et al., 2020), mathematical (Saxton et al., 2019), and other NLP task-related (Rad-ford et al., 2019; Shin et al., 2020) knowledge LMs acquire during…”
BackgroundDissecting Recall of Factual Associations in Auto-Regressive Language Models
Geva, M., Bastings, J., Filippova, K., & Globerson, A. (2023) · arXiv · doi:10.48550/arxiv.2304.14767 · Cited by 53
“Transformer-based language models (LMs) capture vast amounts of factual knowledge (Roberts et al., 2020; Jiang et al., 2020), which they encode in their parameters and recall during inference (Petroni et al., 2019; Cohen et al., 2023).”
BackgroundEvent Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely
Kauf, C., Ivanova, A. A., Rambelli, G., Chersoni, E., She, J. S., Chowdhury, Z., Fedorenko, E., & Lenci, A. (2023) · Cognitive Science · doi:10.1111/cogs.13386 · Cited by 53
Challenges in Generalization in Open Domain Question Answering
Liu, L., Lewis, P., Riedel, S., & Stenetorp, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.155 · Cited by 53
Self-citationBennetot, A., Franchi, G., Ser, J. D., Chatila, R., & Díaz-Rodríguez, N. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109947 · Cited by 54
Assessing Factual Reliability of Large Language Model Knowledge
Wang, W., Haddow, B., Birch, A., & Peng, W. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.46 · Cited by 54
DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models
Chen, K., Chen, Q., Zhou, J., He, Y., & He, L. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.529 · Cited by 53
A Comprehensive Overview and Analysis of Large Language Models: Trends and Challenges
Mohammed, A., & Kora, R. (2025) · IEEE Access · doi:10.1109/access.2025.3573955 · Cited by 53
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms
Zhou, P., Khanna, R., Lee, S., Lin, B. Y., Ho, D., Pujara, J., & Ren, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.598 · Cited by 53
Zero-shot Generalization in Dialog State Tracking through Generative Question Answering
Li, S., Cao, J. L., Sridhar, M., Zhu, H., Li, S.-W. D., Hamza, W., & McAuley, J. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.91 · Cited by 53
LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection
Chen, Z., Huang, Y., Chen, J., Geng, Y., Yin, F., Pan, J. Z., Zhang, N., & Zhang, W. (2022) · International Joint Conference on Knowledge Graphs (IJCKG) · doi:10.1145/3579051.3579053 · Cited by 53
Mention Memory: incorporating textual knowledge into Transformers through entity mention attentionPreprint
de Jong, M., Zemlyanskiy, Y., FitzGerald, N., Sha, F., & Cohen, W. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06176 · Cited by 53
Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding
Wang, J., Huang, W., Qiu, M., Shi, Q., Wang, H., Li, X., & Gao, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.207 · Cited by 53
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models
Youssef, P., Koraş, O. A., Li, M., Schlötterer, J., & Seifert, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1043 · Cited by 53
ALCUNA: Large Language Models Meet New Knowledge
Yin, X., Huang, B., & Wan, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.87 · Cited by 53
Do LLMs Really Adapt to Domains? An Ontology Learning Perspective
Mai, H. T., Chu, C. X., & Paulheim, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77844-5_7 · Cited by 52
Knowledge Mechanisms in Large Language Models: A Survey and Perspective
Wang, M., Yao, Y., Xu, Z., Qiao, S., Deng, S., Wang, P., Chen, X., Gu, J.-C., Jiang, Y., Xie, P., et al. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.416 · Cited by 52
Boosting Static Resource Leak Detection via LLM-based Resource-Oriented Intention Inference
Wang, C., Liu, J., Peng, X., Liu, Y., & Lou, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00131 · Cited by 53
PROST: Physical Reasoning about Objects through Space and Time
Aroca-Ouellette, S., Paik, C., Roncone, A., & Kann, K. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.404 · Cited by 52
Measuring and Modifying Factual Knowledge in Large Language Models
Pezeshkpour, P. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00122 · Cited by 52
SymbolicAI: A framework for logic-based approaches combining generative models and solversPreprint
Dinu, M.-C., Leoveanu-Condrei, C., Holzleitner, M., Zellinger, W., & Hochreiter, S. (2024) · arXiv (Cornell University) · doi:10.15203/99106-150-2-24 · Cited by 52
Reordering Examples Helps during Priming-based Few-Shot Learning
Kumar, S., & Talukdar, P. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.395 · Cited by 51
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
Baek, J., Aji, A., & Saffari, A. (2023) · Proceedings of the First Workshop on Matching From Unstructured and Structured Data (MATCHING 2023) · doi:10.18653/v1/2023.matching-1.7 · Cited by 51
“…2022) are able to perform zero-shot learning, for example, generating the answer for the input textual prompt, based on the knowledge stored in pre-trained parameters (Petroni et al., 2019; Roberts et al., 2020; Sung et al., 2021), without additional parameter updates as well as labeled datasets.”
InfluentialMarasović, A., Bhagavatula, C., Park, J. S., Le Bras, R., Smith, N. A., & Choi, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.253 · Cited by 51
Automated Storytelling via Causal, Commonsense Plot Ordering
Ammanabrolu, P., Cheung, W., Broniec, W., & Riedl, M. O. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16733 · Cited by 51
Chen, W., Huang, H., Zhang, Z., Wang, T., Lin, Y., Chang, L., & Wan, H. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3545958 · Cited by 51
Metadata
- Authors
- Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2463-2473
- OpenAlex
- W2970476646
- Cited by
- 1,834
- References
- 42
- Citations by year
- 2026: 129 · 2025: 230 · 2024: 300 · 2023: 469 · 2022: 334 · 2021: 246 · 2020: 121 · 2019: 5
- FWCI
- 67.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- language models, knowledge base, knowledge representation, knowledge extraction, question answering, commonsense knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.