Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA
Open accessAbstract
Retrieval-Augmented Generation (RAG) is widely used to inject external non-parametric knowledge into large language models (LLMs).Recent works suggest that Knowledge Graphs (KGs) contain valuable external knowledge for LLMs.Retrieving information from KGs differs from extracting it from document sets.Most existing approaches seek to directly retrieve relevant subgraphs, thereby eliminating the need for extensive SPARQL annotations, traditionally required by semantic parsing methods.In this paper, we model the subgraph retrieval task as a conditional generation task handled by small language models.Specifically, we define a subgraph identifier as a sequence of relations, each represented as a special token stored in the language models.Our base generative subgraph retrieval model, consisting of only 220M parameters, achieves competitive retrieval performance compared to state-of-the-art models relying on 7B parameters, demonstrating that small language models are capable of performing the subgraph retrieval task.Furthermore, our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks.Our model and data will be made available online:
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Combining EEMs for subgraph-extraction with OSAR for path-retrieval is effective, a strategy yet unexplored by existing methods like GSR.
“This highlights the effectiveness of combining EEMs for subgraph-extraction with OSAR for path-retrieval, a strategy yet unexplored by existing methods like GSR [45], which pairs EEMs with structure-based path-retrieval.”
From LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration (2025) · cites this paper as [45]
The cited paper does not describe GSR as pairing EEMs with structure-based path retrieval. Section 3 (Problem Statement; page unavailable) assumes the topic entity is already known, while Sections 2.2 (Generative Retrieval) and 4.3 (Inference GSR; pages unavailable) describe generating relation chains and retrieving them with beam search instead. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredExisting GraphRAG studies have demonstrated the potential of integrating graph data management techniques with GraphRAG.
“Existing GraphRAG studies [22, 45, 50, 65, 71, 72, 74, 98] have demonstrated the potential of integrating graph data management techniques with GraphRAG.”
From LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration (2025) · cites this paper as [45]
Domain-specific fine-tuning can further enhance these models' performance, though it comes at an additional cost.
“Domain-specific fine-tuning can further enhance these models' performance [45, 65, 71, 72], though it comes at an additional cost.”
From LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration (2025) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 10
- Zeyuan Allen-Zhu and Yuanzhi Li. 2024. Physics of language models: Part 3.1, knowledge storage and extraction. In Forty-first International Conference on Machine Learning.
- Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetun- ing of quantized LLMs. In Thirty-seventh Confer- ence on Neural Information Processing Systems.
- Xixin Hu, Xuan Wu, Yiheng Shu, and Yuzhong Qu. 2022. Logical form generation via multi-task learn- ing for complex question answering over knowledge bases. In Proceedings of the 29th International Con- ference on Computational Linguistics, pages 1687- 1696, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Wenyu Huang, Guancheng Zhou, Mirella Lapata, Pav- los Vougiouklis, Sébastien Montella, and Jeff Z. Pan. 2024. Prompting large language models with knowl- edge graphs for question answering involving long- tail facts. CoRR, abs/2405.06524. · DOI
- J.Z. Pan, D. Calvanese, T. Eiter, I. Horrocks, M. Kifer, F. Lin, and Y. Zhao, editors. 2017b. Reasoning Web: Logical Foundation of Knowledge Graph Construc- tion and Querying Answering. Springer. · DOI
- Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel Ni, Heung- Yeung Shum, and Jian Guo. 2024. Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. In The Twelfth Interna- tional Conference on Learning Representations.
- Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Prakash Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. 2022. Transformer memory as a differentiable search index. In Advances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 -December 9, 2022.
- Junjie Wang, Mingyang Chen, Binbin Hu, Dan Yang, Ziqi Liu, Yue Shen, Peng Wei, Zhiqiang Zhang, Jin- jie Gu, Jun Zhou, Jeff Z Pan, Wen Zhang, and Hua- jun Chen. 2024a. Learning to Plan for Retrieval- Augmented Large Language Models from Knowl- edge Graphs. In Proc. of Empirical Methods in Natu- ral Language Processing.
- Yike Wu, Yi Huang, Nan Hu, Yuncheng Hua, Guilin Qi, Jiaoyan Chen, and Jeff Z. Pan. 2024. CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering. In Proc. of Empirical Methods in Natural Language Processing.
- Donghan Yu, Sheng Zhang, Patrick Ng, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Yiqun Hu, William Yang Wang, Zhiguo Wang, and Bing Xiang. 2023. DecAF: Joint decoding of answers and log- ical forms for question answering over knowledge bases. In The Eleventh International Conference on Learning Representations.
Cited by
Sorted by citations
Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768165 · Cited by 106 · see citing works
(2025) · doi:10.1145/3833415 · Cited by 18 · see citing works
IKG-Agent: Intent-driven Knowledge Graph Agent for Adaptive Workflow Reasoning
Liang, Y., Tao, S., Teng, H., Yao, H., Wang, X., Guo, X., Wang, L., & Ji, X. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228190
- Authors
- Huang, W., Zhou, G., Wang, H., Vougiouklis, P., Lapata, M., & Pan, J. Z.
- Affiliations
- University of Edinburgh (GB)
- Chinese University of Hong Kong (HK)
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Published
- 2024
- Pages
- 15787-15803
- OpenAlex
- W4404792882
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 1 · 2025: 1
- FWCI
- 0.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- UK Research and Innovation (EP/S022481/1)
- University of Edinburgh
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- subgraph retrieval, knowledge graph question answering, multi-hop question answering, small language models, retrieval-augmented generation, conditional generation, WebQSP
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
