Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 2

The DOI matches its DataCite record; 4 of the 4 references it cites were found in a registry or an index.

DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learning

Jiang, Pengcheng, Lin, Jiacheng, Cao, Lang, Tian, Runchu, Kang, SeongKu, Wang, Zifeng, Sun, Jimeng, Han, Jiawei·2025·arXiv
DOI 10.48550/arxiv.2503.00223arXiv 2503.00223
Open this paper in CoChat — free account

Abstract

Information retrieval systems are crucial for enabling effective access to large document collections. Recent approaches have leveraged Large Language Models (LLMs) to enhance retrieval performance through query augmentation, but often rely on expensive supervised learning or distillation techniques that require significant computational resources and hand-labeled data. We introduce DeepRetrieval, a reinforcement learning (RL) approach that trains LLMs for query generation through trial and error without supervised data (reference query). Using retrieval metrics as rewards, our system generates queries that maximize retrieval performance. DeepRetrieval outperforms leading methods on literature search with 65.07% (vs. previous SOTA 24.68%) recall for publication search and 63.18% (vs. previous SOTA 32.11%) recall for trial search using real-world search engines. DeepRetrieval also dominates in evidence-seeking retrieval, classic information retrieval and SQL database search. With only 3B parameters, it outperforms industry-leading models like GPT-4o and Claude-3.5-Sonnet on 11/13 datasets. These results demonstrate that our RL approach offers a more efficient and effective paradigm for information retrieval. Our data and code are available at: https://github.com/pat-jj/DeepRetrieval.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.00223

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Performance Overview of DeepRetrieval. (a) Literature Search Performance (Recall@3K) on Real-World Search Engines (PubMed and ClinicalTrials.gov); (b) Performance on Classic Information Retrieval and SQL database search Tasks.
Figure 1 Performance Overview of DeepRetrieval. (a) Literature Search Performance (Recall@3K) on Real-World Search Engines (PubMed and ClinicalTrials.gov); (b) Performance on Classic Information Retrieval and SQL database search Tasks.CC-BY · From the PDF
Figure 2 Abstractive Overview of DeepRetrieval : Based on an input user query, the LLM generates an augmented query, which is used to retrieve documents. Format reward and retrieval reward are both computed as the feedback to update the model.
Figure 2 Abstractive Overview of DeepRetrieval : Based on an input user query, the LLM generates an augmented query, which is used to retrieve documents. Format reward and retrieval reward are both computed as the feedback to update the model.CC-BY · From the PDF
Figure 3 Knowledge Injection Study on Evidence-Seeking Retrieval . LLMs may input answer information to the generated queries based on their prior knowledge. We implement a postprocessing method (Appendix E) where an auxiliary LLM analyzes the query, identifies knowledge injections, and removes the injections to see the contribution of this factor.
Figure 3 Knowledge Injection Study on Evidence-Seeking Retrieval . LLMs may input answer information to the generated queries based on their prior knowledge. We implement a postprocessing method (Appendix E) where an auxiliary LLM analyzes the query, identifies knowledge injections, and removes the injections to see the contribution of this factor.CC-BY · From the PDF
Qwen (No Think) suffered from excessive decoding time, making training and testing over 8 × slower than Qwen and Llama. For instance, testing at Step 1000 took ∼ 10.5 hours, compared to just 1 hour for models with < think > . Thus, we halted the training early.
Qwen (No Think) suffered from excessive decoding time, making training and testing over 8 × slower than Qwen and Llama. For instance, testing at Step 1000 took ∼ 10.5 hours, compared to just 1 hour for models with < think > . Thus, we halted the training early.CC-BY · From the PDF
Figure 12 Query Generation Examples. We select the most common rewarded query patterns at step 400 and step 600 for all the models. 'No Think' denotes no reasoning.
Figure 12 Query Generation Examples. We select the most common rewarded query patterns at step 400 and step 600 for all the models. 'No Think' denotes no reasoning.CC-BY · From the PDF
Figure 13 Prompt used for literature search.
Figure 13 Prompt used for literature search.CC-BY · From the PDF
Figure 14 Prompt used for evidence-seeking retrieval.
Figure 14 Prompt used for evidence-seeking retrieval.CC-BY · From the PDF
Figure 15 Prompt used for classic sparse document retrieval.
Figure 15 Prompt used for classic sparse document retrieval.CC-BY · From the PDF
Figure 16 Prompt used for classic dense document retrieval.
Figure 16 Prompt used for classic dense document retrieval.CC-BY · From the PDF
Figure 17 Prompt used for SQL database search.
Figure 17 Prompt used for SQL database search.CC-BY · From the PDF
Figure 18 Prompt used for knowledge injection check and clean.
Figure 18 Prompt used for knowledge injection check and clean.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 109

  1. Nasreen Abdul-Jaleel, James Allan, W Bruce Croft, Fernando Diaz, Leah Larkey, Xiaoyan Li, Mark D Smucker, and Courtney Wade. Umass at trec 2004: Novelty and hard. In Proceedings of TREC-13, pp. 715-725, 2004. Anthropic. Introducing claude 3.5 sonnet. https://www.anthropic.com/news/ claude-3-5-sonnet, 2024. Accessed: 2025-03-23.
  2. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learn- ing to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2023.
  3. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calan- driello, Michal Valko, and Rémi Munos. A general theoretical paradigm to under- stand learning from human preferences. ArXiv, abs/2310.12036, 2023. URL https: //api.semanticscholar.org/CorpusID:264288854. · arXiv
  4. Ricardo Baeza-Yates, Berthier Ribeiro-Neto, et al. Modern information retrieval, volume 463. ACM press New York, 1999.
  5. Nicholas J Belkin, Robert N Oddy, and Helen M Brooks. Ask for information retrieval: Part i. background and theory. Journal of documentation, 38(2):61-71, 1982.
  6. Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, and Alexander Kolesnikov. Knowledge distillation: A good teacher is patient and consistent. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10925-10934, 2022.
  7. Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. Inpars: Data augmentation for information retrieval using large language models. arXiv preprint arXiv:2202.05144, 2022. · arXiv
  8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
  9. Paul Francis Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. ArXiv, abs/1706.03741, 2017. URL https://api.semanticscholar.org/CorpusID:4787508. · arXiv
  10. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025. · arXiv
  11. W Bruce Croft, Donald Metzler, and Trevor Strohman. Search engines: Information retrieval in practice, volume 520. Addison-Wesley Reading, 2010.
  12. Benoit Dageville, Thierry Cruanes, Marcin Zukowski, Vadim Antonov, Artin Avanes, Jon Bock, Jonathan Claybaugh, Daniel Engovatov, Martin Hentschel, Jiansheng Huang, et al. The snowflake elastic data warehouse. In Proceedings of the 2016 International Conference on Management of Data, pp. 215-226, 2016.
  13. Zhuyun Dai and Jamie Callan. Deeper text understanding for ir with contextual neural language modeling. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp. 985-988, 2019.
  14. Jeffrey Dalton, Laura Dietz, and James Allan. Entity query feature expansion using knowl- edge base links. Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, 2014. URL https://api.semanticscholar.org/ CorpusID:12620927.
  15. Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023. · arXiv
  16. Naihao Deng, Yulong Chen, and Yue Zhang. Recent advances in text-to-SQL: A survey of what we have and what we expect. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung- Hoon Na (eds.), Proceedings of the 29th International Conference on Computational Linguistics, pp. 2166-2187, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics. URL https://aclanthology.org/2022.coling-1.190/.
  17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019. URL https://api.semanticscholar. org/CorpusID:52967399.
  18. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre- Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. arXiv preprint arXiv:2401.08281, 2024. · arXiv
  19. Mette Brandt Eriksen and Tove Faber Frandsen. The impact of patient, intervention, com- parison, outcome (pico) as a search strategy tool on literature search quality: a systematic review. Journal of the Medical Library Association: JMLA, 106(4):420, 2018.
  20. Wenqi Fan, Yujuan Ding, Liang bo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Knowledge Discovery and Data Mining, 2024. URL https://api. semanticscholar.org/CorpusID:269740933.
  21. Fernando Ferraretto, Thiago Soares Laitz, Roberto de Alencar Lotufo, and Rodrigo Nogueira. Exaranker: Explanation-augmented neural ranker. ArXiv, abs/2301.10521, 2023. URL https://api.semanticscholar.org/CorpusID:256231455. · arXiv
  22. George W. Furnas, Thomas K. Landauer, Louis M. Gomez, and Susan T. Dumais. The vocabulary problem in human-system communication. Communications of the ACM, 30 (11):964-971, 1987.
  23. Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. Text-to-sql empowered by large language models: A benchmark evaluation. arXiv preprint arXiv:2308.15363, 2023a. · arXiv
  24. Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. Text-to-sql empowered by large language models: A benchmark evaluation, 2023b. URL https://arxiv.org/abs/2308.15363. · arXiv
  25. Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. Complement lexical retrieval model with semantic residual embeddings. In Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28-April 1, 2021, Proceedings, Part I 43, pp. 146-160. Springer, 2021.
  26. Luyu Gao, Xueguang Ma, Jimmy J. Lin, and Jamie Callan. Precise zero-shot dense retrieval without relevance labels. ArXiv, abs/2212.10496, 2022. URL https://api. semanticscholar.org/CorpusID:254877046. · arXiv
  27. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. · arXiv
  28. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. · arXiv
  29. J. A. Hartigan and M. A. Wong. Algorithm as 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1):100-108, 1979. ISSN 00359254, 14679876. URL http://www.jstor.org/stable/2346830.
  30. Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. Next-generation database interfaces: A survey of llm-based text-to-sql, 2025. URL https://arxiv.org/abs/2406.08426. · arXiv
  31. Sheryl Hsu, Omar Khattab, Chelsea Finn, and Archit Sharma. Grounding by trying: Llms with reinforcement learning-enhanced retrieval. arXiv preprint arXiv:2410.23214, 2024. · arXiv
  32. Jian Hu. Reinforce++: A simple and efficient approach for aligning large language mod- els. ArXiv, abs/2501.03262, 2025. URL https://api.semanticscholar.org/CorpusID: 275342265. · arXiv
  33. Xiaoli Huang, Jimmy Lin, and Dina Demner-Fushman. Evaluation of pico as a knowledge representation for clinical questions. In AMIA annual symposium proceedings, volume 2006, pp. 359, 2006.
  34. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. · arXiv
  35. Peter Ingwersen. Cognitive perspectives of information retrieval interaction: elements of a cognitive ir theory. Journal of documentation, 52(1):3-50, 1996.
  36. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Ar- mand Joulin, and Edouard Grave. Unsupervised dense information retrieval with con- trastive learning. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https://openreview.net/forum?id=jKN1pXi7b0.
  37. Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. Query expansion by prompting large language models. arXiv preprint arXiv:2305.03653, 2023. · arXiv
  38. Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422-446, 2002.
  39. Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: A survey. J. Artif. Intell. Res., 4:237-285, 1996. URL https://api.semanticscholar.org/ CorpusID:1708582.
  40. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answer- ing. In EMNLP (1), pp. 6769-6781, 2020.
  41. O. Khattab and Matei A. Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020. URL https://api. semanticscholar.org/CorpusID:216553223.
  42. Omar Khattab, Christopher Potts, and Matei Zaharia. Relevance-guided supervision for openqa with colbert. Transactions of the association for computational linguistics, 9:929-944, 2021.
  43. Robert Krovetz and W Bruce Croft. Lexical ambiguity and information retrieval. ACM Transactions on Information Systems (TOIS), 10(2):115-141, 1992.
  44. Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh- Thang Luong, and Orhan Firat. Beyond distillation: Task-level mixture-of-experts for efficient inference. arXiv preprint arXiv:2110.03742, 2021. · arXiv
  45. Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Salman Khan, and Fa- had Shahbaz Khan. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321, 2025. · arXiv
  46. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466, 2019.
  47. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
  48. Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye, Ziyue Li, Rui Zhao, and Hangyu Mao. Pet-sql: A prompt-enhanced two-round refinement of text-to-sql with cross-consistency, 2024. URL https://arxiv.org/abs/2403. 09732.
  49. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
  50. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summa- rization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computa- tional Linguistics. URL https://aclanthology.org/W04-1013/.
  51. Jiacheng Lin, Kun Qian, Haoyu Han, Nurendra Choudhary, Tianxin Wei, Zhongruo Wang, Sahika Genc, Edward W Huang, Sheng Wang, Karthik Subbian, et al. Unleashing the power of llms as multi-modal encoders for text and graph-structured data. arXiv preprint arXiv:2410.11235, 2024. · arXiv
  52. Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2356-2362, 2021.
  53. Sheng-Chieh Lin, Jheng-Hong Yang, Rodrigo Nogueira, Ming-Feng Tsai, Chuan-Ju Wang, and Jimmy J. Lin. Conversational question reformulation via sequence-to-sequence architectures and pretrained language models. ArXiv, abs/2004.01909, 2020. URL https: //api.semanticscholar.org/CorpusID:214802570. · arXiv
  54. Linqing Liu, Minghan Li, Jimmy J. Lin, Sebastian Riedel, and Pontus Stenetorp. Query expansion using contextual clue sampling with language models. ArXiv, abs/2210.07093, 2022. URL https://api.semanticscholar.org/CorpusID:252872857. · arXiv
  55. Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. Query rewriting in retrieval-augmented large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 5303-5315, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.322. URL https://aclanthology.org/ 2023.emnlp-main.322/. · DOI
  56. Xueguang Ma, Kai Sun, Ronak Pradeep, and Jimmy Lin. A replication study of dense passage retriever. arXiv preprint arXiv:2104.05740, 2021. · arXiv
  57. Iain Mackie, Shubham Chatterjee, and Jeffrey Dalton. Generative relevance feedback with large language models. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, pp. 2026-2031, 2023.
  58. Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. Generation-augmented retrieval for open-domain question answering. arXiv preprint arXiv:2009.08553, 2020. · arXiv
  59. Edgar Meij, Dolf Trieschnigg, M. de Rijke, and Wessel Kraaij. Conceptual language models for domain-specific retrieval. Inf. Process. Manag., 46:448-469, 2010. URL https://api. semanticscholar.org/CorpusID:10929365.
  60. Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. ArXiv, abs/2405.14734, 2024. URL https://api.semanticscholar. org/CorpusID:269983560. · arXiv
  61. Donald Metzler and W Bruce Croft. A markov random field model for term dependencies. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 472-479, 2005.
  62. Donald Metzler and W Bruce Croft. Latent concept expansion using markov random fields. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 311-318, 2007.
  63. Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, et al. Ray: A distributed framework for emerging {AI} applications. In 13th USENIX symposium on operating systems design and implementation (OSDI 18), pp. 561-577, 2018.
  64. Rodrigo Nogueira and Kyunghyun Cho. Passage re-ranking with bert. arXiv preprint arXiv:1901.04085, 2019. · arXiv
  65. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155, 2022. URL https: //api.semanticscholar.org/CorpusID:246426909. · arXiv
  66. Jayr Alencar Pereira, Robson do Nascimento Fidalgo, Roberto de Alencar Lotufo, and Rodrigo Nogueira. Visconde: Multi-document qa with gpt-3 and neural reranking. ArXiv, abs/2212.09656, 2022. URL https://api.semanticscholar.org/CorpusID:254854129. · arXiv
  67. Mohammadreza Pourreza and Davood Rafiei. Din-sql: Decomposed in-context learning of text-to-sql with self-correction, 2023. URL https://arxiv.org/abs/2304.11015. · arXiv
  68. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. ArXiv, abs/2305.18290, 2023. URL https://api.semanticscholar.org/CorpusID: 258959321. · arXiv
  69. Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67, 2019. URL https://api.semanticscholar.org/CorpusID:204838007.
  70. John J Riva, Keshena MP Malik, Stephen J Burnie, Andrea R Endicott, and Jason W Busse. What is your research question? an introduction to the picot format for clinicians. The Journal of the Canadian Chiropractic Association, 56(3):167, 2012.
  71. Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333-389, 2009.
  72. John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High- dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. · arXiv
  73. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. ArXiv, abs/1707.06347, 2017a. URL https: //api.semanticscholar.org/CorpusID:28695052. · arXiv
  74. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017b. · arXiv
  75. Hinrich Sch ütze, Christopher D Manning, and Prabhakar Raghavan. Introduction to informa- tion retrieval, volume 39. Cambridge University Press Cambridge, 2008.
  76. Candy Schwartz. Web search engines. Journal of the American Society for Information Science, 49(11):973-982, 1998.
  77. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024. · arXiv
  78. Amit Singhal et al. Modern information retrieval: A brief overview. IEEE Data Eng. Bull., 24 (4):35-43, 2001.
  79. Charles Burton Snell, Dan Klein, and Ruiqi Zhong. Learning by distilling context. ArXiv, abs/2209.15189, 2022. URL https://api.semanticscholar.org/CorpusID:252668389. · arXiv
  80. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan J. Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feed- back. ArXiv, abs/2009.01325, 2020. URL https://api.semanticscholar.org/CorpusID: 221665105. · arXiv
  81. Robert S Taylor. Question-negotiation and information seeking in libraries. College & research libraries, 29(3):178-194, 1968.
  82. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology, 2024. URL https: //arxiv.org/abs/2403.08295.
  83. Nandan Thakur, Nils Reimers, Andreas R ücklé, Abhishek Srivastava, and Iryna Gurevych. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663, 2021. · arXiv
  84. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. ArXiv, abs/2307.09288, 2023. URL https://api.semanticscholar.org/CorpusID:259950998. · arXiv
  85. Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural In- formation Processing Systems, 2017. URL https://api.semanticscholar.org/CorpusID: 13756489.
  86. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Ran- gan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. ArXiv, abs/2212.03533, 2022a. URL https://api.semanticscholar.org/ CorpusID:254366618. · arXiv
  87. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672, 2024a. · arXiv
  88. Xiao Wang, Craig Macdonald, Nicola Tonellotto, and Iadh Ounis. Pseudo-relevance feedback for multiple representation dense retrieval. Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval, 2021. URL https://api.semanticscholar. org/CorpusID:235490586.
  89. Xiao Wang, Craig Macdonald, Nicola Tonellotto, and Iadh Ounis. Colbert-prf: Semantic pseudo-relevance feedback for dense passage and document retrieval. ACM Transac- tions on the Web, 17:1 -39, 2022b. URL https://api.semanticscholar.org/CorpusID: 253763641.
  90. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560, 2022c. · arXiv
  91. Yujing Wang, Hainan Zhang, Liang Pang, Binghui Guo, Hongwei Zheng, and Zhiming Zheng. Maferw: Query rewriting with multi-aspect feedbacks for retrieval-augmented large language models, 2024b. URL https://arxiv.org/abs/2408.17072. · arXiv
  92. Zifeng Wang, Lang Cao, Qiao Jin, Joey Chan, Nicholas Wan, Behdad Afzali, Hyun-Jin Cho, Chang-In Choi, Mehdi Emamverdi, Manjot K. Gill, Sun-Hyung Kim, Yijia Li, Yi Liu, Hanley Ong, Justin Rousseau, Irfan Sheikh, Jenny J. Wei, Ziyang Xu, Christopher M. Zallek, Kyungsang Kim, Yifan Peng, Zhiyong Lu, and Jimeng Sun. A foundation model for human-ai collaboration in medical literature mining, 2025. URL https://arxiv.org/ abs/2501.16255. · arXiv
  93. Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in llms, 2025. URL https://arxiv.org/abs/2502. 07266.
  94. Zeqiu Wu, Yi Luan, Hannah Rashkin, D. Reitter, and Gaurav Singh Tomar. Conqrr: Conversa- tional query rewriting for retrieval with reinforcement learning. In Conference on Empirical Methods in Natural Language Processing, 2021. URL https://api.semanticscholar.org/ CorpusID:245218563.
  95. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. C-pack: Packed resources for general chinese embeddings. In Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, pp. 641-649, 2024.
  96. Chenyan Xiong and Jamie Callan. Query expansion with freebase. Proceedings of the 2015 International Conference on The Theory of Information Retrieval, 2015. URL https: //api.semanticscholar.org/CorpusID:7311615.
  97. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weiling Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. Is dpo superior to ppo for llm alignment? a comprehensive study. ArXiv, abs/2404.10719, 2024. URL https://api.semanticscholar.org/CorpusID:269157140. · arXiv
  98. Yang Xu, G. Jones, and Bin Wang. Query dependent pseudo-relevance feedback based on wikipedia. Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, 2009. URL https://api.semanticscholar.org/ CorpusID:566812.
  99. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. · arXiv
  100. Andrew Yates, Rodrigo Nogueira, and Jimmy Lin. Pretrained transformers for text ranking: Bert and beyond. In Proceedings of the 14th ACM International Conference on web search and data mining, pp. 1154-1156, 2021.
  101. Fanghua Ye, Meng Fang, Shenghui Li, and Emine Yilmaz. Enhancing conversational search: Large language model-aided informative query rewriting. ArXiv, abs/2310.09716, 2023. URL https://api.semanticscholar.org/CorpusID:264145935. · arXiv
  102. HongChien Yu, Chenyan Xiong, and Jamie Callan. Improving query representations for dense retrieval with pseudo relevance feedback. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021. URL https: //api.semanticscholar.org/CorpusID:237363901.
  103. Shih Yuan Yu, Jiahua Liu, Jingqin Yang, Chenyan Xiong, Paul N. Bennett, Jianfeng Gao, and Zhiyuan Liu. Few-shot generative conversational query rewriting. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020. URL https://api.semanticscholar.org/CorpusID:219559295.
  104. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887, 2018. · arXiv
  105. Torsten Zesch, Christof M üller, and Iryna Gurevych. Extracting lexical semantic knowledge from Wikipedia and Wiktionary. In Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, and Daniel Tapias (eds.), Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08), Marrakech, Morocco, May 2008. European Language Resources Association (ELRA). URL https: //aclanthology.org/L08-1139/.
  106. Chengxiang Zhai and John Lafferty. Model-based feedback in the language modeling approach to information retrieval. In Proceedings of the tenth international conference on Information and knowledge management, pp. 403-410, 2001.
  107. Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K. Qiu, and Lili Qiu. Retrieval augmented generation (rag) and beyond: A comprehensive survey on how to make your llms use external data more wisely. ArXiv, abs/2409.14924, 2024. URL https: //api.semanticscholar.org/CorpusID:272827955. · arXiv
  108. Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning, 2017. URL https://arxiv.org/abs/ 1709.00103.
  109. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36:55006-55021, 2023.

Cited by

See all 2

Metadata

Authors
Jiang, P., Lin, J., Cao, L., Tian, R., Kang, S., Wang, Z., Sun, J., & Han, J.
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2503.00223
arXiv
2503.00223
References
4
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.