How Can We Know What Language Models Know?
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
Recent work has presented intriguing results examining the knowledge contained in language models (LMs) by having the LM fill in the blanks of prompts such as “ Obama is a __ by profession”. These prompts are usually manually created, and quite possibly sub-optimal; another prompt such as “ Obama worked as a __ ” may result in more accurately predicting the correct profession. Because of this, given an inappropriate prompt, we might fail to retrieve facts that the LM does know, and thus any given prompt only provides a lower bound estimate of the knowledge contained in an LM. In this paper, we attempt to more accurately estimate the knowledge contained in LMs by automatically discovering better prompts to use in this querying process. Specifically, we propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. Extensive experiments on the LAMA benchmark for extracting relational knowledge from LMs demonstrate that our methods can improve accuracy from 31.1% to 39.6%, providing a tighter lower bound on what LMs know. We have released the code and the resulting LM Prompt And Query Archive (LPAQA) at https://github.com/jzbjyb/LPAQA .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The ability of Large Language Models to retrieve factual knowledge as explicit knowledge bases [53-56, 56-61] has been stirred discussed, which presents an opportunity to construct more comprehensive knowledge graphs within recommender systems.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [56]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“even experienced prompt designers may fail to manually discover optimal prompts [52].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [52]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Beyond this, we found models trained with Toolformer to often be sensitive to the exact wording of their input when deciding whether or not to call an API; this is perhaps unsurprising given that LMs are known to be very sensitive to the prompt they are provided with in both zeroand few-shot settings (Jiang et al., 2020;Schick and Schütze, 2021a).”
From Toolformer: Language Models Can Teach Themselves to Use Tools (2023) · cites this paper as Jiang et al., 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is some evidence to suggest that formats that occur commonly in the training data will lead to better performance (Jiang et al., 2020).”
From The Prompt Report: A Systematic Survey of Prompt Engineering Techniques (2024) · cites this paper as Jiang et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Jiang et al. [52]'s Mine approach is a mining-based method to automatically find templates given a set of training inputs x and outputs y.”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [52]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“Prompts where the hidden vectors learned through a language modeling objective are then used in downstream language understanding systems (Dai and Le, 2015; Melamud et al., 2016; Peters et al., 2018; Devlin et al., 2019).”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482 · cited 5×
“Similar extensions have been applied to pre-trained LMs like BERT, where contextualized representations are enhancedwith entity embeddings (Zhang et al., 2019; Peters et al., 2019; Pörner et al., 2019).”
MethodInfluentialOpen information extraction from the web
Banko, M., Cafarella, M., Soderland, S. G., Broadhead, M., & Etzioni, O. (2007) · International Joint Conference on Artificial Intelligence · Cited by 1,398
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307
(2019) · doi:10.18653/v1/p19-1356 · Cited by 1,291 · cited 1×
“…a broad suite of NLP tasks are used to analyze both syntactic and semantic properties, providing evidence that contextualized representations encode linguistic knowledge in different layers (Hewitt and Manning, 2019; Tenney et al., 2019a; Tenney et al., 2019b; Jawahar et al., 2019; Goldberg, 2019).”
BackgroundSnowball: extracting relations from large plain-text collections
Agichtein, E., & Gravano, L. (2000) · Proceedings of the fifth ACM conference on Digital libraries · doi:10.1145/336597.336644 · Cited by 1,147
Research works printed but not linked: 7
- Andrew M. Dai and Quoc V. Le. 2015. Semisupervised sequence learning. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada , pages 3079-3087.
- Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversitypromoting objective function for neural conversation models. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016 , pages 110-119.
- Gábor Melis, Chris Dyer, and Phil Blunsom. 2018. On the state of the art of evaluation in neural language models. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings .
- Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018. Regularizing and optimizing LSTM language models. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings .
- 1-6, 2018, Volume 1 (Long Papers) , pages 22272237.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog , 1(8).
- Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. 2019b. What do you learn from context? Probing for sentence structure in contextualized word representations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 .
Cited by
Items 1,001–1,040 of 1,040
Aftab, W., Apostolou, Z., Bouazoune, K., & Straub, T. (2024) · Figshare · doi:10.6084/m9.figshare.26860690
CRIT: Socratic Inquiry for Critical Thinking in LLMs
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749429
A2R: A hybridactivation-attention framework for enhancing large language model reliability
Li, X., Wang, J., Yuan, X., & Wang, W. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130922
PGPCLIP: Harnessing Prompt-Guided Prototypes for Cross-Domain Zero-Shot LearningPreprint
Zheng, X., Chen, R., Xia, H., Li, X., Shao, M., Ding, Z., & Xia, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5954635
Wang, R., & Wang, B. (2025) · Neural Processing Letters · doi:10.1007/s11063-025-11815-0
Auto-prompting with Retrieval Guidance for Frame Detection in Logistics
Do, D. M., Truong, Q. X., Dat, N. T., & Vinh, N. V. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-2597-2_26
Annotating Table Metadata with Knowledge-Enhanced Pre-Trained Language Model
Zhang, Q., Tang, Y., Tian, J., Li, M., & Fang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-4158-4_8
Niftiyev, İ. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-5017-2.ch004
Boosting Active Prompt Learning via Discriminative Self-Training Dual-Curriculum Learning
Tao, S., Liu, J., Zeng, P., Xu, Y., Hu, B., & Zha, Z.-J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02641-x
Novel Zero-Shot Framework for Sanskrit-English Cross-Lingual Information Retrieval
Kuricheti, R., Atharsh, K., & Latha, R. (2025) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3777867.3777871
Enhancing RAPTOR with semantic chunking and adaptive graph clustering
Liu, Y., Xie, X., Wan, X., Pan, Y., & Wang, C. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1710121 · Cited by 2
Tian, F., Wang, X., Lu, S., Gu, J., Zhou, S., Li, P., Wang, Z., Jin, Z., Zhao, F., Yang, Z., et al. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103364
Zhou, X., Liang, X., Du, R., & Wang, Z. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343045
MAKE: Memory-Associated Knowledge Editing
Park, S., Park, S., Kim, J., & Kim, H. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.26 · Cited by 6
Gao, W., & He, W. (2026) · Symmetry · doi:10.3390/sym18020319
A LLaMA-Based Efficient Fine-Tuning Method for Image Captioning Using Multi-Feature Dynamic Prompts
Yin, Y., Cao, H., Zhang, C., Jin, F., Liu, X., & Lin, J. (2026) · Applied Sciences · doi:10.3390/app16041857
MAGNET: Counterfactual samples synthesizing for mitigating hallucination in large language models
Kim, B. S., Kim, B., & Jang, B. (2026) · PLoS ONE · doi:10.1371/journal.pone.0340812 · Cited by 1
Analyzing how pre-trained language models capture factual knowledge using attribution methods
Li, S., Sun, C., Liu, B., Li, X., Shang, L., Dong, Z., Ji, Z., Jiang, X., & Liu, Q. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115553
Context Modeling With Multimodal Prompts for Emotion Recognition in Conversation
Dong, Q., Ren, W., Gao, Y. Q., Liu, J., & Liu, H. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668469
Assessing Factual Knowledge in Large Language Models Using Prime Recall
Çulha, D. (2026) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec69317.2026.11418425
aCat: Automatically Choosing Anchor Tokens in Prompt for Natural Language Understanding
Ye, Z., Kong, L., & Qi, H. (2026) · Neural Processing Letters · doi:10.1007/s11063-026-11837-2
On the Robustness of LLM-Generated Text Detection Against Instruction Diversity
Koike, R., Kaneko, M., & Okazaki, N. (2026) · Journal of Natural Language Processing · doi:10.5715/jnlp.33.158
Developing an Intelligent Question Answering System for EnhancedPreprint
sultan, L., & Gayar, N. E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6442694
吉川, 和., Yoshikawa, H., 岡崎, 直., & Okazaki, N. (2022) · Institutional Repositories DataBase (IRDB)
Implicit Demonstration Augmentation for Robust and Stable In-Context Learning
Zhou, X., Lee, Z., Xie, R., Zou, L., Ye, W., & Zhang, S. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3677885
Enhancing RAPTOR in RAG Systems: Semantic Segmentation for Improved Initial Chunking
Wan, X., Xie, X., Lin, W., Xie, T., Wang, C., Liu, Y., & Pan, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-8420-8_16
Wang, Z., & Chen, W. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_2
Wang, Z., Zhang, G., Shi, N., & Lin, C. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_3
Müller, B. (2022) · theses.fr (ABES) · doi:10.70675/6984dc63z4731z4eabzb3fazaf5d012fe143
Word Meaning Representation in Neural Language Models : Lexical Polysemy and Semantic Relationships
Soler, A. G. (2021) · theses.fr (ABES) · doi:10.70675/99a10411zc05ez4135z8f31z13573e0fd4d1
Prompted Contrastive Learning for Skeleton-Based Action Recognition
Su, T., Liu, Z., & Wang, H. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3680904
Analysis and control of online interactions through neural natural language processing
Laugier, L. (2022) · theses.fr (ABES) · doi:10.70675/4743aa15z209az4addz8f29z95150aa230b8 · Cited by 2
Towards efficient, general and robust entity disambiguation systems
Chen, L. (2023) · theses.fr (ABES) · doi:10.70675/abee8a0cz824dz4e34z917azffda9fe5cd83
Facing Data Scarcity in Dialogues for Discourse Structure Discovery and Prediction
Li, C. (2023) · theses.fr (ABES) · doi:10.70675/c379d1f5z5324z4865z9c13z84c9508836f1 · Cited by 7
Employing Transformers and Humans for Textual-Claim Verification
Saeed, M. (2022) · theses.fr (ABES) · doi:10.70675/3e02a3a6z8281z4d84z8f38ze617eaaf969b · Cited by 2
Prompt-based Relation Extraction for Pharmacovigilance
Yeh, H.-S. (2024) · theses.fr (ABES) · doi:10.70675/762a3598ze739z4c47za72ez62e5a986fd1f · Cited by 1
Genest, P.-Y. (2024) · theses.fr (ABES) · doi:10.70675/88c65002z4b8cz466dz923bz13954d6729bf · Cited by 3
Kim, J., & Oh, U. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798781
Uncertainty-aware synthetic data generation: a systematic review of methods and challenges
Keyhanipour, A. H. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01120-x · Cited by 1
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · arXiv · doi:10.48550/arxiv.2101.00190 · Cited by 289
Metadata
- Authors
- Jiang, Z., Xu, F. F., Araki, J., & Neubig, G.
- Affiliations
- Carnegie Mellon University (US)
- Robert Bosch (United States) (US)
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 423-438
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2991382858
- Cited by
- 1,002
- References
- 52
- Citations by year
- 2026: 76 · 2025: 169 · 2024: 224 · 2023: 285 · 2022: 144 · 2021: 78 · 2020: 25
- FWCI
- 52.35
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1815287)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- prompt generation, ensemble methods
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.