The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation
Open accessAbstract
Despite the significant progress of large language models (LLMs) in various tasks, they often produce factual errors due to their limited internal knowledge. Retrieval-Augmented Generation (RAG), which enhances LLMs with external knowledge sources, offers a promising solution. However, these methods can be misled by irrelevant paragraphs in retrieved documents. Due to the inherent uncertainty in LLM generation, inputting the entire document may introduce off-topic information, causing the model to deviate from the central topic and affecting the relevance of the generated content. To address these issues, we propose the Retrieve-Plan-Generation (RPG) framework. RPG generates plan tokens to guide subsequent generation in the plan stage. In the answer stage, the model selects relevant fine-grained paragraphs based on the plan and uses them for further answer generation. This plan-answer process is repeated iteratively until completion, enhancing generation relevance by focusing on specific topics. To implement this framework efficiently, we utilize a simple but effective multi-task prompt-tuning method, enabling the existing LLMs to handle both planning and answering. We comprehensively compare RPG with baselines across 5 knowledge-intensive generation tasks, demonstrating the effectiveness of our approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Leveraging external knowledge bases to mitigate plan errors is time-consuming, and many tasks lack effective knowledge bases.
“While leveraging external knowledge bases can help mitigate these errors (Lyu et al., 2024; Zhu et al., 2024), accessing useful information from them is time-consuming, and many tasks lack effective knowledge bases altogether.”
From iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning (2025) · cites this paper as Lyu et al., 2024
The Introduction (page not provided) says that retrieving external information helps address LLM factual errors, supporting the citation-attached point that external knowledge can mitigate errors. It does not establish that retrieval mitigates errors in plans specifically. The sentence’s claims about access time and tasks lacking knowledge bases fall after the citation and are not attributed to this source. (LLM, conf=0.79)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“With the persistent scaling up of training parameters and datasets (Kaplan et al., 2020), large language models (LLMs) (Touvron et al., 2023; Jiang et al., 2023a; Bai et al., 2023; Achiam et al., 2023) have made remarkable advancements, becoming the cornerstone of many Natural Language Processing…”
Flagged first, then by citations
Research works printed but not linked: 34
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
- Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511. · arXiv
- Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609. · arXiv
- Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. 2023. Hallucination detection: Robustly discerning reliable answers in large lan- guage models. In Proceedings of the 32nd ACM International Conference on Information and Knowl- edge Management, pages 245-255.
- Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zong- han Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2022. Delta tuning: A comprehensive study of parameter efficient meth- ods for pre-trained language models. arXiv preprint arXiv:2203.06904. · arXiv
- Shahul Es, Jithin James, Luis Espinosa-Anke, and Steven Schockaert. 2023. Ragas: Automated eval- uation of retrieval augmented generation. arXiv preprint arXiv:2309.15217. · arXiv
- Angela Fan, Yacine Jernite, Ethan Perez, David Grang- ier, Jason Weston, and Michael Auli. 2019. ELI5: Long form question answering. In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 3558-3567, Florence, Italy. Association for Computational Linguistics.
- Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling large language models to generate text with citations. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 6465-6488, Singapore. Associa- tion for Computational Linguistics.
- Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International confer- ence on machine learning, pages 3929-3938. PMLR.
- Hangfeng He, Hongming Zhang, and Dan Roth. 2022. Rethinking with retrieval: Faithful large language model inference. arXiv preprint arXiv:2301.00303. · arXiv
- Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060. · arXiv
- Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adap- tation of large language models. arXiv preprint arXiv:2106.09685. · arXiv
- Gautier Izacard, Mathilde Caron, Lucas Hosseini, Se- bastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense in- formation retrieval with contrastive learning. arXiv preprint arXiv:2112.09118. · arXiv
- Albert Q Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. 2023a. Mistral 7b. arXiv preprint arXiv:2310.06825. · arXiv
- Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023b. Ac- tive retrieval augmented generation. arXiv preprint arXiv:2305.06983. · arXiv
- Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024. Sure: Summarizing re- trievals using answer candidates for open-domain qa of llms. arXiv preprint arXiv:2404.13081. · arXiv
- Yunshi Lan and Jing Jiang. 2021. Modeling transitions of focal entities for conversational knowledge base question answering. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 3288-3297.
- Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691. · arXiv
- Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
- Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. P- tuning v2: Prompt tuning can be comparable to fine- tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602. · arXiv
- Yuanjie Lyu, Zhiyu Li, Simin Niu, Feiyu Xiong, Bo Tang, Wenjin Wang, Hao Wu, Huanyong Liu, Tong Xu, and Enhong Chen. 2024. Crud-rag: A comprehensive chinese benchmark for retrieval- augmented generation of large language models. arXiv preprint arXiv:2401.17043. · arXiv
- Yuanjie Lyu, Chen Zhu, Tong Xu, Zikai Yin, and En- hong Chen. 2022. Faithful abstractive summarization via fact-aware consistency-constrained transformer. In Proceedings of the 31st ACM International Con- ference on Information & Knowledge Management, pages 1410-1419.
- Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. When not to trust language models: Investigating effectiveness of parametric and non-parametric mem- ories. arXiv preprint arXiv:2212.10511. · arXiv
- Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gus- tavo Hernández Ábrego, Ji Ma, Vincent Y Zhao, Yi Luan, Keith B Hall, Ming-Wei Chang, et al. 2021. Large dual encoders are generalizable retriev- ers. arXiv preprint arXiv:2112.07899. · arXiv
- Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap be- tween neural text and human text using divergence frontiers. Advances in Neural Information Process- ing Systems, 34:4816-4828.
- Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023. In chatgpt we trust? measuring and characterizing the reliability of chatgpt. arXiv preprint arXiv:2304.08979. · arXiv
- Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming- Wei Chang. 2022. ASQA: Factoid questions meet long-form answers. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 8273-8288, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
- Hao Sun, Hengyi Cai, Bo Wang, Yingyan Hou, Xi- aochi Wei, Shuaiqiang Wang, Yan Zhang, and Dawei Yin. 2023. Towards verifiable text generation with evolving memory and self-reflection. arXiv preprint arXiv:2312.09075. · arXiv
- Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971. · arXiv
- Zhen Wang, Rameswar Panda, Leonid Karlinsky, Roge- rio Feris, Huan Sun, and Yoon Kim. 2023. Multitask prompt tuning enables parameter-efficient transfer learning. arXiv preprint arXiv:2303.02861. · arXiv
- Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. Re- comp: Improving retrieval-augmented lms with com- pression and selective augmentation. arXiv preprint arXiv:2310.04408. · arXiv
- Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answer- ing. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
- Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2023. Making retrieval-augmented language models robust to irrelevant context. arXiv preprint arXiv:2310.01558. · arXiv
- Tianhua Zhang, Hongyin Luo, Yung-Sung Chuang, Wei Fang, Luc Gaitskell, Thomas Hartvigsen, Xixin Wu, Danny Fox, Helen Meng, and James Glass. 2023. In- terpretable unified language checking. arXiv preprint arXiv:2304.03728. · arXiv
Cited by
Sorted by citations
Large language models for generative information extraction: a survey
Xu, D., Chen, W., Peng, W., Zhang, C., Xu, T., Zhao, X., Wu, X., Zheng, Y., Wang, Y., & Chen, E. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40555-y · Cited by 326
“Pretraining LLMs using auto-regressive prediction allows them to capture the inherent patterns and semantic knowledge within text corpora [13, 14, 15, 16, 17, 18, 19].”
Self-citationiCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning
Chen, S., & Niu, D. (2025) · arXiv · doi:10.48550/arxiv.2512.24014
“Similar to our approach of learning latent plans for CoT rationales, LaRS (Xu et al., 2024) proposes constructing a latent space of rationales via unsupervised learning, enabling LLMs to retrieve latent rationales for given questions.”
Influential
Metadata
- Authors
- Lyu, Y., Niu, Z., Xie, Z., Zhang, C., Xu, T., Wang, Y., & Chen, E.
- Affiliations
- University of Science and Technology of China (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 4683-4702
- arXiv
- 2406.14979
- OpenAlex
- W4404784206
- Cited by
- 8
- References
- 1
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 2
- FWCI
- 2.31
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- retrieval-augmented generation, iterative planning, answer generation, factual errors, LLM hallucination, document relevance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
