Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.

How Can We Know What Language Models Know?

Open access
Jiang, Zhengbao, Xu, Frank F., Araki, Jun, Neubig, Graham · 2020 · Transactions of the Association for Computational Linguistics

Abstract

Recent work has presented intriguing results examining the knowledge contained in language models (LMs) by having the LM fill in the blanks of prompts such as “ Obama is a __ by profession”. These prompts are usually manually created, and quite possibly sub-optimal; another prompt such as “ Obama worked as a __ ” may result in more accurately predicting the correct profession. Because of this, given an inappropriate prompt, we might fail to retrieve facts that the LM does know, and thus any given prompt only provides a lower bound estimate of the knowledge contained in an LM. In this paper, we attempt to more accurately estimate the knowledge contained in LMs by automatically discovering better prompts to use in this querying process. Specifically, we propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. Extensive experiments on the LAMA benchmark for extracting relational knowledge from LMs demonstrate that our methods can improve accuracy from 31.1% to 39.6%, providing a tighter lower bound on what LMs know. We have released the code and the resulting LM Prompt And Query Archive (LPAQA) at https://github.com/jzbjyb/LPAQA .

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/tacl_a_00324

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

5 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 52 references

Research works printed but not linked: 40

  1. Agichtein Eugene, 2000, Proceedings of the Fifth ACM Conference on Digital Libraries, June 2-7, 2000, San Antonio, TX, USA, 85
  2. Ahn Sungjin, 2016, CoRR
  3. Banko Michele, 2007, IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India, January 6-12, 2007, 2670
  4. Bhagat Rahul, 2008, Proceedings of ACL-08: HLT, 674
  5. Bouraoui Zied, 2020, Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI)
  6. Carpineto Claudio, 2012, ACM, Computing Surveys, 44, 1:1
  7. Dai Andrew M., 2015, Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, 3079
  8. ElSahar Hady, 2018, Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018
  9. Fader Anthony, 2011, Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A meeting of SIGDAT, a Special Interest Group of the ACL, 1535
  10. Gamon Michael, 2005, Proceedings of EAMT, 103
  11. Ghazvininejad Marjan, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 6114
  12. Goldberg Yoav, 2019, CoRR
  13. Hayashi Hiroaki, 2020, Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI)
  14. Hoang Cong Duy Vu, 2017, Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 146
  15. Logan Robert L., 2019, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, 5962
  16. Jawahar Ganesh, 2019, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, 3651
  17. Kingma Diederik P., 2015, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings
  18. Li Jiwei, 2016, NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, 110
  19. Li Jiwei, 2016, CoRR
  20. Mallinson Jonathan, 2017, Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, 881
  21. McCann Bryan, 2018, CoRR
  22. Melamud Oren, 2016, Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August 11-12, 2016, 51
  23. Melis Gábor, 2018, 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings
  24. Merity Stephen, 2018, 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings
  25. Ng Nathan, 2019, Proceedings of the Fourth Conference on Machine Translation, WMT 2019, Florence, Italy, August 1-2, 2019 - Volume 2: Shared Task Papers, Day 1, 314
  26. Peters Matthew E., 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), 2227
  27. Pörner Nina, 2019, CoRR
  28. Radford Alec, 2019, OpenAI Blog, 1
  29. Ravichandran Deepak, 2002, Proceedings of the 40th annual meeting on association for computational linguistics, 41
  30. Romano Lorenza, 2006, 11th Conference of the European Chapter of the Association for Computational Linguistics
  31. Sap Maarten, 2019, Proceedings of the AAAI Conference on Artificial Intelligence, 33, 3027
  32. Sennrich Rico, 2016, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers
  33. Shi Xing, 2016, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1526
  34. Smith Noah A., 2017, Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 1: Long Papers, 1249
  35. Tenney Ian, 2019, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, 4593
  36. Tenney Ian, 2019, 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019
  37. Toutanova Kristina, 2015, Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, 1499
  38. Trinh Trieu H., 2018, CoRR
  39. Yang Zichao, 2017, Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, 1850
  40. Zhang Zhengyan, 2019, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, 1441

Cited by

See all 1,030

Metadata

Authors
Jiang, Z., Xu, F. F., Araki, J., & Neubig, G.
Affiliations
  • Carnegie Mellon University (US)
  • Robert Bosch (United States) (US)
Venue
Transactions of the Association for Computational Linguistics
Publisher
MIT Press
Published
2020-12
Volume
8
Pages
423-438
ISSN
2307-387X (ISSN-L)
OpenAlex
W2991382858
Cited by
1,002
References
52
Citations by year
2026: 76 · 2025: 169 · 2024: 224 · 2023: 285 · 2022: 144 · 2021: 78 · 2020: 25
FWCI
52.35
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
National Science Foundation (1815287)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
prompt generation, ensemble methods

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How Can We Know What Language Models Know? https://app.cochat.ai/papers/10.1162/tacl_a_00324
The paper
Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020). How Can We Know What Language Models Know? Transactions of the Association for Computational Linguistics. https://doi.org/10.1162/tacl_a_00324

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.