CoChat
Cited by 54

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

KnowledgeNavigator: leveraging large language models for enhanced reasoning over knowledge graph

Guo, Tiezheng, Yang, Qingwen, Wang, Chen, Liu, Yanyi, Li, Pan, Tang, Jiawei, Li, Dapeng, Wen, Yingyou · 2024 · Complex & Intelligent Systems

Abstract

Large language models have achieved outstanding performance on various downstream tasks with their advanced understanding of natural language and zero-shot capability. However, they struggle with knowledge constraints, particularly in tasks requiring complex reasoning or extended logical sequences. These limitations can affect their performance in question answering by leading to inaccuracies and hallucinations. This paper proposes a novel framework called KnowledgeNavigator that leverages large language models on knowledge graphs to achieve accurate and interpretable multi-hop reasoning. Especially with an analysis-retrieval-reasoning process, KnowledgeNavigator searches the optimal path iteratively to retrieve external knowledge and guide the reasoning to reliable answers. KnowledgeNavigator treats knowledge graphs and large language models as flexible components that can be switched between different tasks without additional costs. Experiments on three benchmarks demonstrate that KnowledgeNavigator significantly improves the performance of large language models in question answering and outperforms all large language models-based baselines.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s40747-024-01527-8

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Research works printed but not linked: 35

  1. Bai J, Bai S, Chu Y, Cui Z, Dang K, Deng X, Fan Y, Ge W, Han Y, Huang F et al (2023) Qwen technical report. arXiv preprint arXiv:2309.16609. Accessed 07 Dec 2023 · arXiv
  2. Martino A, Iannelli M, Truong C (2023) Knowledge injection to counter large language model (llm) hallucination. European Semantic Web Conference. Springer, New York, pp 182–185
  3. Chen W, Yan-yi L, Tie-zheng G, Da-peng L, Tao H, Zhi L, Qing-wen Y, Hui-han W, Ying-you W (2024) Systems engineering issues for industry applications of large language model. Appl Soft Comput 151:111165 · DOI
  4. Creswell A, Shanahan M, Higgins I (2022) Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712. Accessed 10 June 2023 · arXiv
  5. Zhu X, Li Z, Wang X, Jiang X, Sun P, Wang X, Xiao Y, Yuan NJ (2024) Multi-modal knowledge graph construction and application: a survey. IEEE Trans Knowl Data Eng 36(02):715–735 · DOI · Paper page
  6. Dong G, Zhao J, Hui T, Guo D, Wang W, Feng B, Qiu Y, Gongque Z, He K, Wang Z et al (2023) Revisit input perturbation problems for llms: a unified robustness evaluation framework for noisy slot filling task. CCF International Conference on Natural Language Processing and Chinese Computing. Springer, New York, pp 682–694
  7. Moiseev F, Dong Z, Alfonseca E, Jaggi M (2022) Skill: structured knowledge infusion for large language models. arXiv preprint arXiv:2205.08184. Accessed 10 Oct 2023 · DOI · Paper page
  8. Tian L, Zhou X, Wu Y-P, Zhou W-T, Zhang J-H, Zhang T-S (2022) Knowledge graph and knowledge reasoning: A systematic review. Journal of Electronic Science and Technology 20(2):100159 · DOI
  9. Chakraborty N, Lukovnikov D, Maheshwari G, Trivedi P, Lehmann J, Fischer A (2021) Introduction to neural network-based question answering over knowledge graphs. Wiley Interdiscip Rev Data Min Knowl Discov 11(3):1389 · DOI
  10. Berant J, Chou A, Frostig R, Liang P (2013) Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. pp. 1533–1544
  11. Yih W-t, He X, Meek C (2014) Semantic parsing for single-relation question answering. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 643–648 · DOI · Paper page
  12. Krötzsch M, Marx M, Ozaki A, Thost V (2018) Attributed description logics: Reasoning on knowledge graphs. In: IJCAI, pp. 5309–5313 · DOI
  13. Xiao Z, Xing H, Qu R, Feng L, Luo S, Dai P, Zhao B, Dai Y (2024) Densely knowledge-aware network for multivariate time series classification. IEEE Trans Syst, Man, Cybern Syst 54(4):2192–2204 · DOI · Paper page
  14. Xiao Z, Xing H, Zhao B, Qu R, Luo S, Dai P, Li K, Zhu Z (2023) Deep contrastive representation learning with self-distillation. IEEE Trans Emerg Topics Comput Intell 8(1):3–15 · DOI
  15. Huang X, Zhang J, Li D, Li P (2019) Knowledge graph embedding based question answering. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. pp. 105–113 · DOI · Paper page
  16. Liu W, Zhao Q, Piao S, Wang C, Kong Q, An T (2017) Endo-sirna deficiency results in oocyte maturation failure and apoptosis in porcine oocytes. Reprod Fertil Dev 29(11):2168–2174 · doi:10.1071/rd16498
  17. Xiong W, Yu M, Chang S, Guo X, Wang WY (2019) Improving question answering over incomplete kbs with knowledge-aware reader. arXiv preprint arXiv:1905.07098. Accessed 07 Jan 2023 · DOI
  18. Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A (2017) Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv preprint arXiv:1711.05851. Accessed 09 June 2023 · arXiv
  19. Cincovic J, Jovanovic L, Nikolic B, Bacanin N (2024) Neurodegenerative condition detection using modified metaheuristic for attention based recurrent neural networks and extreme gradient boosting tuning. IEEE Access 12:26719–26734 · DOI
  20. Pavlov-Kagadejev M, Jovanovic L, Bacanin N, Deveci M, Zivkovic M, Tuba M, Strumberger I, Pedrycz W (2024) Optimizing long-short-term memory models via metaheuristics for decomposition aided wind energy generation forecasting. Artif Intell Rev 57(3):45 · DOI
  21. Pan S, Luo L, Wang Y, Chen C, Wang J, Wu X (2023) Unifying large language models and knowledge graphs: a roadmap. arXiv preprint arXiv:2306.08302. Accessed 14 July 2023 · arXiv
  22. Feng S, Balachandran V, Bai Y, Tsvetkov Y (2023) Factkb: Generalizable factuality evaluation using language models enhanced with factual knowledge. arXiv preprint arXiv:2305.08281 · DOI
  23. Yu D, Zhu C, Yang Y, Zeng M (2022) Jaket: joint pre-training of knowledge graph and language understanding. Proc AAAI Conf Artif Intell. 36:11630–11638
  24. Zhang Y, Dai H, Kozareva Z, Smola A, Song L (2018) Variational reasoning for question answering with knowledge graph. Proc AAAI Conf Artif Intell. 32: 6069–6076 · DOI · Paper page
  25. Yih W-t, Richardson M, Meek C, Chang M-W, Suh J (2016) The value of semantic parse labeling for knowledge base question answering. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 201–206 · DOI · Paper page
  26. Xu K, Lai Y, Feng Y, Wang Z (2019) Enhancing key-value memory neural networks for knowledge based question answering. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp 2937–2947 · DOI · Paper page
  27. Sun H, Dhingra B, Zaheer M, Mazaitis K, Salakhutdinov R, Cohen WW (2018) Open domain question answering using early fusion of knowledge bases and text. arXiv preprint arXiv:1809.00782. Accessed 09 Sept 2023 · DOI · Paper page
  28. Baek J, Aji AF, Saffari A (2023) Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. arXiv preprint arXiv:2306.04136. Accessed 10 Aug 2023 · DOI · Paper page
  29. Sun J, Xu C, Tang L, Wang S, Lin C, Gong Y, Shum H-Y, Guo J (2023) Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. arXiv preprint arXiv:2307.07697. Accessed 11 Oct 2023 · arXiv
  30. Saxena A, Tripathi A, Talukdar P (2020) Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 4498–4507 · DOI
  31. He G, Lan Y, Jiang J, Zhao WX, Wen J-R (2021) Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. pp. 553–561 · DOI · Paper page
  32. Shi J, Cao S, Hou L, Li J, Zhang H (2021) Transfernet: An effective and transparent framework for multi-hop question answering over relation graph. arXiv preprint arXiv:2104.07302 · DOI
  33. Das R, Godbole A, Naik A, Tower E, Zaheer M, Hajishirzi H, Jia R, McCallum A (2022) Knowledge base question answering by case-based reasoning over subgraphs. In: International Conference on Machine Learning, pp. 4777–4793. PMLR. Accessed 21 Oct 2023
  34. Jiang J, Zhou K, Dong Z, Ye K, Zhao WX, Wen J-R (2023) Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645 · DOI
  35. Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, Bashlykov N, Batra S, Bhargava P, Bhosale S, et al (2023) Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 · arXiv

Cited by

See all 54

Metadata

Authors
Guo, T., Yang, Q., Wang, C., Liu, Y., Li, P., Tang, J., Li, D., & Wen, Y.
Venue
Complex & Intelligent Systems
Published
2024-07-02
Volume
10
Issue
5
Pages
7063-7076
ISSN
  • 2199-4536
  • 2198-6053
Cited by
54
References
10
Language
en
Licence
Funders
  • Shenyang Science and Technology Plan Project (RC210469)
  • Liaoning Provincial Science and Technology Innovation Project in the Field of Artificial Intelligence (2023JH26/10100005)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: KnowledgeNavigator: leveraging large language models for enhanced reasoning over knowledge graph. https://app.cochat.ai/papers/10.1007/s40747-024-01527-8
The paper
Guo, T., Yang, Q., Wang, C., Liu, Y., Li, P., Tang, J., Li, D., & Wen, Y. (2024). KnowledgeNavigator: leveraging large language models for enhanced reasoning over knowledge graph. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-024-01527-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.