Language Models as Knowledge Bases?
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Tracing back to the work [53], large language models have shown their impressive power in storing factual information, such as entities and common sense, and then commonsense knowledge can be reliably transferred to downtown tasks.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [53]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While pure generative LLMs are often capable of answering basic factual questions (Petroni et al., 2019), simply by reciting information memorized from their training sets and stored in the model parameters”
From Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models (2024) · cites this paper as Petroni et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is well-accepted, even among skeptical researchers, that the parameters of modern LLMs encode a remarkable amount of world knowledge [2, 12, 23, 28, 31, 39, 62, 79], which can be leveraged by an LLM in flexible and robust ways [81, 83, 93].”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Pre-training of models on a large corpus is known to result in the model memorizing knowledge in its parameters [169, 201, 221].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [201]
The cited source says what the claim attributes to it.
In the citing paper
“The aforementioned example shows that AI long-term memory can be implicitly stored in the parameters of AI models [248, 249].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [248]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“BERT (base) (Devlin et al., 2018a) Transformer 110M Wikipedia (en) & BookCorpus 3.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“SQuAD (Rajpurkar et al., 2016) is a popular question answering dataset.”
Background(2013) · Cited by 5,726
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 3×
“…2017), recurrent neural networks (Zaremba et al., 2014; Merity et al., 2016; Melis et al., 2017) or self-attention mechanisms (Radford et al., 2018; Dai et al., 2019; Radford et al., 2019). fairseq-fconv: Instead of commonly used recurrent neural networks, Dauphin et al. (2017) use multiple layers…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- [1]1. Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Pushmeet Kohli, and Edward Grefenstette. 2019. Learning to understand goal specifications by 7 https://github.com/allenai/allennlp 8 https://github.com/pytorch/fairseq 9 https://github.com/huggingface/ pytorch-transformers modelling reward. In International Conference on Learning Representations (ICLR).
- [12]12. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Eval- uation (LREC-2018).
- [24]24. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237.
Cited by
Items 701–800 of 1,061
Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA
Li, Z., Bhavan, J., Tang, P., & Ghadar, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01292 · Cited by 41
Understanding Factual Recall in Transformers via Associative Memories
Nichani, E., Lee, J. D., & Bietti, A. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2412.06538 · Cited by 42
Large Language Models for Ontology Engineering: A Systematic Literature Review
Li, J., Garijo, D., & Poveda‐Villalón, M. (2026) · Semantic Web · doi:10.1177/22104968261465514 · Cited by 46
Retrieval Enhanced Model for Commonsense Generation
Wang, H., Liu, Y., Zhu, C., Shou, L., Gong, M., Xu, Y., & Zeng, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.269 · Cited by 41
Refined Commonsense Knowledge from Large-Scale Web Contents
Nguyen, T.-P., Razniewski, S., Romero, J., & Weikum, G. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3206505 · Cited by 42
Zhu, H., Peng, H. M., Lyu, Z., Hou, L., Li, J., & Xiao, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119369 · Cited by 41
Acquiring and modeling abstract commonsense knowledge via conceptualization
He, M., Fang, T., Wang, W., & Song, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104149 · Cited by 41
Configurable Foundation Models: Building LLMs from a Modular PerspectivePreprint
Xiao, C., Zhang, Z., Song, C., Jiang, D., Yao, F., Han, X., Wang, X., Wang, S., Huang, Y., Lin, G.-Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.02877 · Cited by 41
Knowledge Mechanisms in Large Language Models: A Survey and PerspectivePreprint
Wang, M., Yao, Y., Xu, Z., Qiao, S., Deng, S., Wang, P., Chen, X., Gu, J.-C., Jiang, Y., Xie, P., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.15017 · Cited by 41
Unveiling the Implicit Toxicity in Large Language Models
Wen, J., Ke, P., Sun, H., Zhang, Z., Li, C., Bai, J., & Huang, M. (2023) · arXiv · doi:10.48550/arxiv.2311.17391 · Cited by 40
“…of detecting implicit toxicity, we conduct preliminary experiments to explore whether large language models, which have seen extensive linguistic features and extralinguistic knowledge during pre-training (Petroni et al., 2019), can generate implicit toxic responses that evade toxicity classifiers.”
BackgroundDon’t Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
Feng, S., Shi, W., Wang, Y., Ding, W., Balachandran, V., & Tsvetkov, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.786 · Cited by 40
Why Does ChatGPT Fall Short in Providing Truthful Answers?
Zheng, S., Huang, J., & Chang, K. C.-C. (2023) · arXiv · doi:10.48550/arxiv.2304.10513 · Cited by 40
Knowledge-Aware Language Model Pretraining
Rosset, C., Xiong, C., Phan, M., Song, X., Bennett, P. N., & Tiwary, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00655 · Cited by 40
Hase, P., & Bansal, M. (2022) · Proceedings of the First Workshop on Learning with Natural Language Supervision · doi:10.18653/v1/2022.lnls-1.4 · Cited by 40
A prompt model with combined semantic refinement for aspect sentiment analysis
Zhu, X., Kuang, Z., & Zhang, L. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103462 · Cited by 40
Tem-adapter: Adapting Image-Text Pretraining for Video Question Answer
Chen, G., Liu, X., Wang, G., Zhang, K., Torr, P. H. S., Zhang, X.-P. S., & Tang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01282 · Cited by 41
TravelRAG: A Tourist Attraction Retrieval Framework Based on Multi-Layer Knowledge Graph
Song, S., Yang, C., Li, X., Shang, H., Li, Z., & Chang, Y. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13110414 · Cited by 40
ALGAN: Time Series Anomaly Detection with Adjusted-LSTM GAN
Bashar, M. A., & Nayak, R. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00810-2 · Cited by 40
Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset
Fang, T., Wang, W., Choi, S.-H., Hao, S., Zhang, H., Song, Y., & He, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.705 · Cited by 40
Distilling Relation Embeddings from Pretrained Language Models
Ushio, A., Camacho-Collados, J., & Schockaert, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.712 · Cited by 41
Analyzing BERT’s Knowledge of Hypernymy via Prompting
Hanna, M., & Mareček, D. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.20 · Cited by 41
OpenICL: An Open-Source Framework for In-context Learning
Wu, Z., Wang, Y., Ye, J., Wu, Z., Feng, J., Xu, J., & Qiao, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.47 · Cited by 40
Emergent Modularity in Pre-trained Transformers
Zhang, Z., Zeng, Z., Lin, Y., Xiao, C., Wang, X., Han, X., Liu, Z., Xie, R., Sun, M., & Zhou, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.250 · Cited by 41
Chakraborty, T., Gatta, V. L., Moscato, V., & Sperlí, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126680 · Cited by 40
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text ClassificationPreprint
Schick, T., Schmid, H., & Schütze, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13641 · Cited by 40
Ontology-grounded Automatic Knowledge Graph Construction by LLM under Wikidata schemaPreprint
Feng, X., Wu, X., & Meng, H. M. L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.20942 · Cited by 40
Memobert: Pre-Training Model with Prompt-Based Learning for Multimodal Emotion Recognition
Zhao, J., Li, R., Jin, Q., Wang, X., & Li, H. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746910 · Cited by 39
Calibrating Factual Knowledge in Pretrained Language Models
Dong, Q., Dai, D., Song, Y., Xu, J., Sui, Z., & Li, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.438 · Cited by 39
Knowledge Neurons in Pretrained Transformers
Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2104.08696 · Cited by 39
Evidence-based Factual Error Correction
Thorne, J., & Vlachos, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.256 · Cited by 39
Let's Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain
Huang, Q., Wan, Z.-Y., Xing, Z.-C., Wang, C.-J., Chen, J.-S., Xu, X.-W., & Lu, Q.-H. (2023) · International Conference on Automated Software Engineering · doi:10.1109/ase56229.2023.00075 · Cited by 33
Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selection
Ramírez, G., Birch, A., & Titov, I. (2024) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.2405.02134 · Cited by 39
LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models
Wang, R., & Li, P. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.149 · Cited by 39
Learning Contextualized Knowledge Structures for Commonsense Reasoning
Yan, J., Raman, M., Chan, A., Zhang, T., Rossi, R. A., Zhao, H., Kim, S., Lipka, N., & Ren, X. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.354 · Cited by 39
Meng, Z., Liu, F., Shareghi, E., Su, Y., Collins, C., & Collier, N. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.329 · Cited by 39
Low-resource Taxonomy Enrichment with Pretrained Language Models
Takeoka, K., Akimoto, K., & Oyamada, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.217 · Cited by 39
You Only Need One Model for Open-domain Question Answering
Lee, H., Kedia, A., Lee, J., Paranjape, A., Manning, C. D., & Woo, K.-G. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.198 · Cited by 39
Detecting Label Errors by Using Pre-Trained Language Models
Chong, D., Hong, J., & Manning, C. D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.618 · Cited by 39
Language Model Analysis for Ontology Subsumption Inference
He, Y., Chen, J., Jiménez-Ruiz, E., Dong, H., & Horrocks, I. R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.213 · Cited by 39
Medical text classification based on the discriminative pre-training model and prompt-tuning
Wang, Y., Wang, Y., Peng, Z., Zhang, F., Zhou, L., & Yang, F. (2023) · Digital Health · doi:10.1177/20552076231193213 · Cited by 39
Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data SourcesPreprint
Lupidi, A., Gemmell, C., Cancedda, N., Dwivedi-Yu, J., Weston, J., Foerster, J., Răileanu, R., & Lomelí, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.08239 · Cited by 39
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
Roller, S., Boureau, Y.-L., Weston, J., Bordes, A., Dinan, E., Fan, A., Gunning, D., Ju, D., Li, M., Poff, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.12442 · Cited by 38
“…training, methods like BERT (Devlin et al., 2019) do train over Wikipedia and hence can be shown to condense knowledge bases into their weights (Petroni et al., 2019) we contend that this is not the same as reading sentences and learning a compressed, indexable knowledge base (a memory) that…”
BackgroundMethodHow Decoding Strategies Affect the Verifiability of Generated Text
Massarelli, L., Petroni, F., Piktus, A., Ott, M., Rocktäschel, T., Plachouras, V., Silvestri, F., & Riedel, S. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.22 · Cited by 38
Self-citationWeakly supervised veracity classification with LLM-predicted credibility signals
Leite, J. A., Razuvayevskaya, O., Bontcheva, K., & Scarton, C. (2025) · EPJ Data Science · doi:10.1140/epjds/s13688-025-00534-0 · Cited by 38
A Survey on Unifying Large Language Models and Knowledge Graphs for Biomedicine and Healthcare
Xu, R., Jiang, P., Luo, L., Xiao, C., Cross, A., Pan, S., Sun, J., & Yang, C. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736556 · Cited by 38
How Do Multilingual Language Models Remember Facts?
Fierro, C., Foroutan, N., Elliott, D., & Søgaard, A. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.827 · Cited by 38
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations
Abdou, M., Ravishankar, V., Barrett, M., Belinkov, Y., Elliott, D., & Søgaard, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.679 · Cited by 38
Improving Skip-Gram Embeddings Using BERT
Wang, Y., Cui, L., & Zhang, Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3065201 · Cited by 38
Robust Retrieval Augmented Generation for Zero-shot Slot Filling
Glass, M., Rossiello, G., Chowdhury, M. F. M., & Gliozzo, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.148 · Cited by 39
Perspective: Large Language Models in Applied Mechanics
Brodnik, N. R., Carton, S., Muir, C., Ghosh, S., Downey, D., Echlin, M. P., Pollock, T. M., & Daly, S. H. (2023) · Journal of Applied Mechanics · doi:10.1115/1.4062773 · Cited by 38
Selective-LAMA: Selective Prediction for Confidence-Aware Evaluation of Language Models
Yoshikawa, H., & Okazaki, N. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.150 · Cited by 38
Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection
Dai, Y. Q., Lang, H., Zeng, K., Huang, F., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.351 · Cited by 38
IAG: Induction-Augmented Generation Framework for Answering Reasoning Questions
Zhang, Z., Zhang, X., Ren, Y., Shi, S., Han, M., Wu, Y., Lai, R., & Cao, Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1 · Cited by 38
Self-citationHoltzman, A., West, P., & Zettlemoyer, L. (2025) · Journal of Social Computing · doi:10.23919/jsc.2025.0009 · Cited by 38
ConRAG: Contradiction-Aware Retrieval-Augmented Generation under Multi-Source Conflicting Evidence
Sun, X., Chen, J., Zhou, B., & Kuo, M.-J. (2024) · Journal of Advanced Computing Systems · doi:10.69987/jacs.2024.40705 · Cited by 38
Interview: Large-scale Modeling of Media Dialog with Discourse Patterns and Knowledge Grounding
Majumder, B. P., Li, S. X., Ni, J., & McAuley, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.653 · Cited by 38
Protein Representation Learning via Knowledge Enhanced Primary Structure ModelingPreprint
Zhou, H.-Y., Fu, Y., Zhang, Z., Bian, C., & Yu, Y. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.26.525795 · Cited by 38
Align on the Fly: Adapting Chatbot Behavior to Established NormsPreprint
Xu, C., Chern, S., Chern, E., Zhang, G., Wang, Z., Liu, R., Li, J., Fu, J., & Liu, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.15907 · Cited by 38
PIQA: Reasoning about Physical Commonsense in Natural Language
Bisk, Y., Zellers, R., Le bras, R., Gao, J., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6239 · Cited by 37
RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding
Ji, Z., Liu, Z., Lee, N., Yu, T., Wilie, B., Zeng, M., & Fung, P. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.275 · Cited by 37
The MultiBERTs: BERT Reproductions for Robustness Analysis
Sellam, T., Yadlowsky, S., Wei, J., Saphra, N., D'Amour, A., Linzen, T., Bastings, J., Turc, I., Eisenstein, J., Das, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.16163 · Cited by 37
“(Petroni et al., 2019) • Does another model such as RoBERTa (Liu et al., 2019) outperform BERT on tasks like GLUE and SQuAD?”
BackgroundMethodLeveraging Large Language Models for Multiple Choice Question Answering
Robinson, J. A., Rytting, C. M., & Wingate, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.12353 · Cited by 37
Do Neural Language Models Overcome Reporting Bias?
Shwartz, V., & Choi, Y. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.605 · Cited by 37
A comprehensive taxonomy of prompt engineering techniques for large language models
Liu, Y.-Y., Zheng, Z., Zhang, F., Feng, J.-C., Fu, Y.-Y., Zhai, J.-D., He, B.-S., Zhang, X., & Du, X.-Y. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-50058-z · Cited by 37
Panchendrarajan, R., & Zubiaga, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124097 · Cited by 38
Defining Knowledge: Bridging Epistemology and Large Language Models
Fierro, C., Dhar, R., Stamatiou, F., Garneau, N., & Søgaard, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.900 · Cited by 37
Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theory
Li, H., Fan, W., Chen, Y., Jiayang, C., Chu, T., Zhou, X., Hu, P., & Song, Y. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.86 · Cited by 37
Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent
Huang, Z., Yuan, X., Ju, Y., Cai, H., Zhao, J., & Liu, K. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.2184 · Cited by 37
The mosaic memory of large language models
Shilov, I., Meeus, M., & de Montjoye, Y.-A. (2026) · Nature Communications · doi:10.1038/s41467-026-68603-0 · Cited by 38
Gómez-Pérez, J. M., & Ortega, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.441 · Cited by 37
Probing Contextual Language Models for Common Ground with Visual Representations
Ilharco, G., Zellers, R., Farhadi, A., & Hajishirzi, H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.422 · Cited by 40
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Gao, T., Han, X., Bai, Y., Qiu, K., Xie, Z., Lin, Y., Liu, Z., Li, P., Sun, M., & Zhou, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.112 · Cited by 37
Drop Redundant, Shrink Irrelevant: Selective Knowledge Injection for Language Pretraining
Zhang, N., Deng, S., Cheng, X., Chen, X., Zhang, Y., Zhang, W., & Chen, H. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/552 · Cited by 37
Zero-Shot Commonsense Question Answering with Cloze Translation and Consistency Optimization
Dou, Z.-Y., & Peng, N. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21301 · Cited by 37
Improving Pretrained Language Model Fine-Tuning With Noise Stability Regularization
Hua, H., Li, X., Dou, D., Xu, C., & Luo, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3330926 · Cited by 38
Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models
Feng, S., Shi, W., Bai, Y., Balachandran, V., He, T., & Tsvetkov, Y. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2305.09955 · Cited by 38
KILM: Knowledge Injection into Encoder-Decoder Language Models
Xu, Y., Namazifar, M., Hazarika, D., Padmakumar, A., Liu, Y., & Hakkani-Tür, D. Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.275 · Cited by 38
Xu, S., Li, J., & Xiong, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.226 · Cited by 37
Evaluating the External and Parametric Knowledge Fusion of Large Language ModelsPreprint
Zhang, H., Zhang, Y., Li, X., Shi, W., Xu, H., Liu, H., Wang, Y., Shang, L., Liu, Q., Liu, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.19010 · Cited by 37
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social BiasesPreprint
Prabhumoye, S., Kocielnik, R., Shoeybi, M., Anandkumar, A., & Catanzaro, B. (2021) · CaltechAUTHORS (California Institute of Technology) · doi:10.48550/arxiv.2112.07868 · Cited by 37
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T., & Evans, O. (2023) · arXiv · doi:10.48550/arxiv.2309.12288 · Cited by 36
“Knowledge editing in LLMs Previous literature has studied LLMs as knowledge bases (Petroni et al., 2019).”
BackgroundCoPL: Contextual Prompt Learning for Vision-Language Understanding
Goswami, K., Karanam, S., Udhayanan, P., Joseph, K. J., & Srinivasan, B. V. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29766 · Cited by 38
Bonfigli, A., Bacco, L., Merone, M., & Dell’Orletta⋄, F. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.103003 · Cited by 36
Semantic programming by example with pre-trained models
Verbruggen, G., Le, V., & Gulwani, S. (2021) · Proceedings of the ACM on Programming Languages · doi:10.1145/3485477 · Cited by 36
Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification
Wang, H., Xu, C., & McAuley, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.401 · Cited by 36
Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors
Wang, X., Li, G., Kuo, Y., Kocabas, M., Aksan, E., & Hilliges, O. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00047 · Cited by 36
Giordano, V., Puccetti, G., Chiarello, F., Pavanello, T., & Fantoni, G. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120499 · Cited by 36
Evaluating Language Models for Knowledge Base Completion
Veseli, B., Singhania, S., Razniewski, S., & Weikum, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33455-9_14 · Cited by 36
Downstream Datasets Make Surprisingly Good Pretraining Corpora
Krishna, K., Garg, S., Bigham, J. P., & Lipton, Z. C. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.682 · Cited by 36
Can Language Models Understand Physical Concepts?
Li, L., Xu, J., Dong, Q., Zheng, C., Sun, X., Kong, L., & Liu, Q. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.726 · Cited by 36
Wang, L., Joty, S., Gao, W. G., Zeng, X., & Wong, K. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3397321 · Cited by 36
Informing Unsupervised Pretraining with External Linguistic Knowledge
Lauscher, A., Vulić, I., Ponti, E. M., Korhonen, A., & Glavaš, G. (2019) · MADOC (University of Mannheim) · Cited by 36
Probing Neural Language Models for Human Tacit AssumptionsPreprint
Weir, N., Poliak, A., & Durme, B. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04877 · Cited by 38
Are Emergent Abilities in Large Language Models just In-Context Learning?
Lu, S., Bigoulaeva, I., Sachdeva, R., Tayyar Madabushi, H., & Gurevych, I. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.279 · Cited by 35
Probing Across Time: What Does RoBERTa Know and When?
Liu, Z., Wang, Y., Kasai, J., Hajishirzi, H., & Smith, N. A. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.71 · Cited by 35
Suzgun, M., Melas-Kyriazi, L., & Jurafsky, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.141 · Cited by 35
Are Pretrained Language Models Symbolic Reasoners over Knowledge?
Kassner, N., Krojer, B., & Schütze, H. (2020) · Proceedings of the 24th Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.45 · Cited by 35
Bui, T., Tran, O., Nguyen, P., Ho, B., Nguyen, L., Bui, T., & Quan, T. (2024) · Proceedings of the 1st ACM Workshop on AI-Powered Q&A Systems for Multimedia · doi:10.1145/3643479.3662055 · Cited by 35
KnowLog: Knowledge Enhanced Pre-trained Language Model for Log Understanding
Ma, L., Yang, W., Xu, B., Jiang, S., Fei, B., Liang, J., Zhou, M., & Xiao, Y. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3623304 · Cited by 35
Cultural Commonsense Knowledge for Intercultural Dialogues
Nguyen, T.-P., Razniewski, S., & Weikum, G. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679768 · Cited by 36
Metadata
- Authors
- Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2463-2473
- OpenAlex
- W2970476646
- Cited by
- 1,834
- References
- 42
- Citations by year
- 2026: 129 · 2025: 230 · 2024: 300 · 2023: 469 · 2022: 334 · 2021: 246 · 2020: 121 · 2019: 5
- FWCI
- 67.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- language models, knowledge base, knowledge representation, knowledge extraction, question answering, commonsense knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.