Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?
Open accessAbstract
Since the recent prosperity of Large Language Models (LLMs), there have been interleaved discussions regarding how to reduce hallucinations from LLM responses, how to increase the factuality of LLMs, and whether Knowledge Graphs (KGs), which store the world knowledge in a symbolic form, will be replaced with LLMs. In this paper, we try to answer these questions from a new angle: How knowledgeable are LLMs? To answer this question, we constructed Head-to-Tail, a benchmark that consists of 18K question-answer (QA) pairs regarding head, torso, and tail facts in terms of popularity. We designed an automated evaluation method and a set of metrics that closely approximate the knowledge an LLM confidently internalizes. Through a comprehensive evaluation of 16 publicly available LLMs, we show that existing LLMs are still far from being perfect in terms of their grasp of factual knowledge, especially for facts of torso-to-tail entities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLM-based evaluations in the Head-to-Tail structured task are reliable and reproducible.
“Table 5 shows close agreement between our replicated A LM values and the original results, confirming the reliability and reproducibility of LLM-based evaluations in this structured task.”
From Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge (2024) · cites this paper as Sun et al., 2023
Section 2.2 (Metrics; page unavailable) reports 98% agreement between LLM-based judgments and human judgments and calls the metrics reliable. Section 3.5 (Robustness; page unavailable) reports that regenerated ChatGPT answers differed for only 1% of sampled questions with the specified prompt, but it does not establish that the evaluation results can be independently replicated. The close agreement between replicated A_LM values and the originals is the citing authors’ own finding, not one reported by this source. (LLM, conf=0.84)
Experimental — accuracy not yet measuredMost benchmarks utilize multiple choice QA or fill-in-the-blank QA to assess model hallucination.
“Most benchmarks utilize multiple choice QA or fill-in-the-blank QA to assess model hallucination, while some efforts further consider the ratio of partially incorrect answers [241] or instances where the model admits that it cannot answer [156].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [241]
Our replicated A LM values closely agree with the original results reported by Sun et al. (2023).
“Table 5 shows close agreement between our replicated A LM values and the original results, confirming the reliability and reproducibility of LLM-based evaluations in this structured task.”
From Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge (2024) · cites this paper as Sun et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 17
- Akari Asai, Sewon Min, Zexuan Zhong, and Danqi Chen. 2023. Retrieval-based language models and applications. In Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 6: Tutorial Abstracts), pages 41-46, Toronto, Canada. Association for Computational Linguistics.
- Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The Semantic Web, pages 722-735, Berlin, Heidelberg. Springer Berlin Heidelberg.
- Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1533-1544, Seattle, Wash- ington, USA. Association for Computational Linguis- tics.
- Sebastian Borgeaud, Arthur Mensch, and etc. Jor- dan Hoffmann †. 2022. Improving language models by retrieving from trillions of tokens. arXiv.
- Ludovic Denoyer and Patrick Gallinari. 2006. The Wikipedia XML corpus. SIGIR Forum, 40(1):64- 69.
- Nan Hu, Yike Wu, Guilin Qi, Dehai Min, Jiaoyan Chen, Jeff Z Pan, and Zafar Ali. 2023. An empirical study of pre-trained language models in simple knowledge graph question answering. World Wide Web, pages 1-32.
- Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 15696-15707.
- PMLR.
- Youngmin Kim, Rohan Kumar, Sunitha Ravi, Haitian Sun, Christos Faloutsos, Ruslan Salakhutdinov, and Minji Yoon. 2023. Automatic question-answer gen- eration for long-tail knowledge. In Second Workshop on Knowledge Augmented Methods for Natural Lan- guage Processing (KDD-KnowledgeNLP).
- Chin-Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
- Ye Liu, Yao Wan, Lifang He, Hao Peng, and Philip S. Yu. 2021. Kg-bart: Knowledge graph-augmented bart for generative commonsense reasoning. In AAAI.
- Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2022. Webgpt: Browser- assisted question-answering with human feedback. arXiv.
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744.
- Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. 2023. Replug: Retrieval-augmented black-box language models. arXiv.
- Priyansh Trivedi, Gaurav Maheshwari, Mohnish Dubey, and Jens Lehmann. 2017. Lc-quad: A corpus for complex question answering over knowledge graphs. In The Semantic Web -ISWC 2017, pages 210-218, Cham. Springer International Publishing.
- Ricardo Usbeck, Ria Hari Gusmita, Axel- Cyrille Ngonga Ngomo, and Muhammad Saleem. 2018. 9th challenge on question answering over linked data (QALD-9). In Joint proceedings of the 4th Workshop on Semantic Deep Learning (SemDeep-4) and NLIWoD4: Natural Language Interfaces for the Web of Data (NLIWOD-4) and 9th Question Answering over Linked Data challenge (QALD-9) co-located with 17th International Semantic Web Conference (ISWC 2018), Monterey, California, United States of America, October 8th - 9th, 2018., pages 58-64.
- Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. 2021. K-adapter: Infusing knowledge into pre-trained models with adapters. In ACL.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
Self-citationHuang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.212 · Cited by 282 · see citing works
Combating misinformation in the age of LLMs: Opportunities and challenges
Chen, C., & Shu, K. (2024) · AI Magazine · doi:10.1002/aaai.12188 · Cited by 148 · see citing works
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2023) · arXiv · doi:10.48550/arxiv.2309.11495 · Cited by 42 · see citing works
“However, even the largest models can still fail, particularly on lesser known torso and tail distribution facts (Sun et al., 2023a), i.e. those that occur relatively rarely in the training corpora.”
BackgroundShah, S. B., Thapa, S., Acharya, A., Rauniyar, K., Poudel, S., Jain, S., Masood, A., & Naseem, U. (2024) · IEEE Access · doi:10.1109/access.2024.3406644 · Cited by 41 · see citing works
Indoor and Outdoor 3D Scene Graph Generation Via Language-Enabled Spatial Ontologies
Strader, J., Hughes, N., Chen, W., Alberto Speranzon, B., & Carlone, L. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3384084 · Cited by 35 · see citing works
Systematic Assessment of Factual Knowledge in Large Language Models
Luo, L., Vu, T.-T., Phung, D., & Haf, R. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.885 · Cited by 21 · see citing works
Using LLMs to Extract Food Entities from Cooking Recipes
Pitsilou, V., Papadakis, G., & Skoutas, D. (2024) · 2024 IEEE 40th International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew61823.2024.00008 · Cited by 11 · see citing works
Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge
Schroeder, K., & Wood-Doughty, Z. (2024) · arXiv · doi:10.48550/arxiv.2412.12509 · Cited by 8 · see citing works
Emptying the Ocean with a Spoon: Should We Edit Models?
Pinter, Y., & Elhadad, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1012 · Cited by 10 · see citing works
Towards semantically enriched embeddings for knowledge graph completion
Alam, M., van Harmelen, F., & Acosta, M. (2024) · Neurosymbolic Artificial Intelligence · doi:10.3233/nai-240731 · Cited by 5 · see citing works
L3X: Long Object List Extraction from Long Documents
Singhania, S., Razniewski, S., & Weikum, G. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761460 · Cited by 2 · see citing works
Halucynacje chatbotów a prawda: główne nurty debaty i ich interpretacje
Kreft, J., Boguszewicz-Kreft, M., & Cyrek, B. (2024) · Roczniki Nauk Społecznych · doi:10.18290/rns2024.0009 · Cited by 2 · see citing works
ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models
Barmettler, J., Bernstein, A., & Rossetto, L. (2025) · arXiv · doi:10.48550/arxiv.2504.07624
Zero-shot Probing of Pretrained Language Models for Geography Knowledge
Ramrakhiyani, N., Varma, V., Palshikar, G. K., & Pawar, S. S. (2023) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2023.eval4nlp-1.5
Multi-task Instruction Tuning for Temporal Question Answering over Knowledge Graphs
Su, M., Li, Z., Jin, X., & Guo, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1710-4_8
ConceptFormer: Towards Graph-Native Grounding of Large Language Models via Latent Concept Injection
Barmettler, J., Bernstein, A., & Rossetto, L. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3794653
From Illusion to Insight: A Taxonomic Survey of Hallucination Mitigation Techniques in LLMs
Kazlaris, I., Antoniou, E., Diamantaras, K., & Bratsas, C. (2025) · doi:10.20944/preprints202508.1942.v1 · Cited by 13 · see citing works
- Authors
- Sun, K., Xu, Y. E., Zha, H., Liu, Y., & Dong, X. L.
- Affiliations
- University of Surrey (GB)
- University of Cambridge (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2308.10168
- OpenAlex
- W4386081161
- Cited by
- 13
- References
- 0
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 7 · 2023: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, knowledge graphs, factual knowledge, question-answer pairs, hallucinations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
