CoChat
Sign Up
PreprintCited by 2Topic ModelingInformation Retrieval and Search BehaviorMultimodal Machine Learning Applications

The DOI matches its DataCite record.

MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation

Open access
Chang, Chia-Yuan, Jiang, Zhimeng, Rakesh, Vineeth, Pan, Menghai, Yeh, Chin-Chia Michael, Wang, Guanchu, Hu, Mingzhi, Xu, Zhichao, Zheng, Yan, Das, Mahashweta, Zou, Na·2025·arXiv
DOI 10.48550/arxiv.2501.00332arXiv 2501.00332
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) are becoming essential tools for various natural language processing tasks but often suffer from generating outdated or incorrect information. Retrieval-Augmented Generation (RAG) addresses this issue by incorporating external, real-time information retrieval to ground LLM responses. However, the existing RAG systems frequently struggle with the quality of retrieval documents, as irrelevant or noisy documents degrade performance, increase computational overhead, and undermine response reliability. To tackle this problem, we propose Multi-Agent Filtering Retrieval-Augmented Generation (MAIN-RAG), a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents. Specifically, MAIN-RAG introduces an adaptive filtering mechanism that dynamically adjusts the relevance filtering threshold based on score distributions, effectively minimizing noise while maintaining high recall of relevant documents. The proposed approach leverages inter-agent consensus to ensure robust document selection without requiring additional training data or fine-tuning. Experimental results across four QA benchmarks demonstrate that MAIN-RAG consistently outperforms traditional RAG approaches, achieving a 2-11% improvement in answer accuracy while reducing the number of irrelevant retrieved documents. Quantitative analysis further reveals that our approach achieves superior response consistency and answer accuracy over baseline methods, offering a competitive and practical alternative to training-based solutions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2501.00332

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization.

    “Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization.”

    From Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents (2026) · cites this paper as Chang et al., 2025

    The Introduction and Related Works discuss retrieval-augmented generation and document filtering, not separate long-term and short-term memory components. Searches for LTM/STM and for heuristic or auxiliary controllers found no passage addressing the claimed limitation; page numbers were unavailable in our copy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 55

  1. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D'arcy, et al. 2024a. Openscholar: Synthesizing scientific litera- ture with retrieval-augmented lms. arXiv preprint arXiv:2411.14199. · arXiv
  2. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024b. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.
  3. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosse- lut, Emma Brunskill, et al. 2021. On the opportuni- ties and risks of foundation models. arXiv preprint arXiv:2108.07258. · arXiv
  4. Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165. · arXiv
  5. Chia-Yuan Chang, Yu-Neng Chuang, Zhimeng Jiang, Kwei-Herng Lai, Anxiao Jiang, and Na Zou. 2023a. Coda: Temporal domain generalization via concept drift simulator. arXiv preprint arXiv:2310.01508. · arXiv
  6. Chia-Yuan Chang, Yu-Neng Chuang, Guanchu Wang, Mengnan Du, and Na Zou. 2023b. Dispel: Domain generalization via domain-specific liberating. arXiv preprint arXiv:2307.07181. · arXiv
  7. Chia-Yuan Chang, Jiayi Yuan, Sirui Ding, Qiaoyu Tan, Kai Zhang, Xiaoqian Jiang, Xia Hu, and Na Zou. 2024. Towards fair patient-trial matching via patient- criterion level fairness constraint. In AMIA Annual Symposium Proceedings, volume 2023, page 884.
  8. Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pages 17754-17762.
  9. Yu-Neng Chuang, Songchen Li, Jiayi Yuan, Guanchu Wang, Kwei-Herng Lai, Leisheng Yu, Sirui Ding, Chia-Yuan Chang, Qiaoyu Tan, Daochen Zha, et al. 2024a. Understanding different design choices in training large time series models. arXiv preprint arXiv:2406.14045. · arXiv
  10. Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Fan Yang, Mengnan Du, Xuant- ing Cai, and Xia Hu. 2024b. Large language models as faithful explainers. arXiv preprint arXiv:2402.04678. · arXiv
  11. Yu-Neng Chuang, Tianwei Xing, Chia-Yuan Chang, Zirui Liu, Xun Chen, and Xia Hu. 2024c. Learning to compress prompt in natural language formats. arXiv preprint arXiv:2402.18700. · arXiv
  12. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question an- swering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. · arXiv
  13. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. · arXiv
  14. Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. 2024. Alpacafarm: A simulation framework for methods that learn from human feedback. Advances in Neural Information Processing Systems, 36.
  15. Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling large language models to generate text with citations. arXiv preprint arXiv:2305.14627. · arXiv
  16. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International confer- ence on machine learning, pages 3929-3938.
  17. PMLR.
  18. Mingzhi Hu, Xin Zhang, Yanhua Li, Yiqun Xie, Xi- aowei Jia, Xun Zhou, and Jun Luo. 2024. Only attending what matter within trajectories-memory- efficient trajectory attention. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pages 481-489.
  19. Mingzhi Hu, Xin Zhang, Yanhua Li, Xun Zhou, and Jun Luo. 2023a. St-ifgsm: Enhancing robustness of human mobility signature identification model via spatial-temporal iterative fgsm. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 764-774.
  20. Mingzhi Hu, Zhuoyun Zhong, Xin Zhang, Yanhua Li, Yiqun Xie, Xiaowei Jia, Xun Zhou, and Jun Luo. 2023b. Self-supervised pre-training for ro- bust and generic spatial-temporal representations. In 2023 IEEE International Conference on Data Mining (ICDM), pages 150-159. IEEE.
  21. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Se- bastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense in- formation retrieval with contrastive learning. arXiv preprint arXiv:2112.09118. · arXiv
  22. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
  23. Albert Q Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. 2023a. Mistral 7b. arXiv preprint arXiv:2310.06825. · arXiv
  24. Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023b. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pages 7969-7992, Singapore. As- sociation for Computational Linguistics.
  25. Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. 2024. Llm maybe longlm: Selfextend llm context window without tuning. In Forty-first International Conference on Machine Learning.
  26. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehen- sion. arXiv preprint arXiv:1705.03551. · arXiv
  27. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906. · arXiv
  28. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettle- moyer, and Mike Lewis. 2023a. Contrastive decod- ing: Open-ended text generation as optimization. In Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 12286-12312, Toronto, Canada. Association for Computational Linguistics.
  29. Xiaonan Li, Changtai Zhu, Linyang Li, Zhangyue Yin, Tianxiang Sun, and Xipeng Qiu. 2023b. Llatrieval: Llm-verified retrieval for verifiable generation. arXiv preprint arXiv:2311.07838. · arXiv
  30. Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Col- lier. 2024. Prompt compression for large language models: A survey. arXiv preprint arXiv:2410.12388. · arXiv
  31. Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2022a. Pretrained transformers for text ranking: Bert and beyond. Springer Nature.
  32. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022b. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214-3252, Dublin, Ireland. Association for Computational Linguistics.
  33. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paran- jape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language mod- els use long contexts. Transactions of the Association for Computational Linguistics, 11:157-173.
  34. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. When not to trust language models: Investigating effectiveness of parametric and non-parametric mem- ories. arXiv preprint arXiv:2212.10511. · arXiv
  35. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap be- tween neural text and human text using divergence frontiers. Advances in Neural Information Process- ing Systems, 34:4816-4828.
  36. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented lan- guage models. Transactions of the Association for Computational Linguistics, 11:1316-1331.
  37. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettle- moyer, Nicola Cancedda, and Thomas Scialom. 2024. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Pro- cessing Systems, 36.
  38. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. Trusting your evidence: Hallucinate less with context- aware decoding. In Proceedings of the 2024 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies (Volume 2: Short Papers), pages 783-791, Mexico City, Mexico. Association for Com- putational Linguistics.
  39. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. Asqa: Factoid ques- tions meet long-form answers. arXiv preprint arXiv:2204.06092. · arXiv
  40. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288. · arXiv
  41. A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems.
  42. Guanchu Wang, Yu-Neng Chuang, Ruixiang Tang, Shaochen Zhong, Jiayi Yuan, Hongye Jin, Zirui Liu, Vipin Chaudhary, Shuai Xu, James Caverlee, et al. 2024a. Taylor unswift: Secured weight release for large language models via taylor expansion. In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pages 6928- 6941.
  43. Guanchu Wang, Yu-Neng Chuang, Fan Yang, Mengnan Du, Chia-Yuan Chang, Shaochen Zhong, Zirui Liu, Zhaozhuo Xu, Kaixiong Zhou, Xuanting Cai, et al. 2024b. Tve: Learning meta-attribution for transfer- able vision explainer. In Forty-first International Conference on Machine Learning.
  44. Guanchu Wang, Junhao Ran, Ruixiang Tang, Chia- Yuan Chang, Yu-Neng Chuang, Zirui Liu, Vladimir Braverman, Zhandong Liu, and Xia Hu. 2024c. As- sessing and enhancing large language models in rare disease question-answering. arXiv preprint arXiv:2408.08422. · arXiv
  45. Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023. Self-knowledge guided retrieval augmen- tation for large language models. arXiv preprint arXiv:2310.05002. · arXiv
  46. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoc V Le. 2021. Finetuned lan- guage models are zero-shot learners. arXiv preprint arXiv:2109.01652. · arXiv
  47. Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2024a. Re- comp: Improving retrieval-augmented lms with con- text compression and selective augmentation. In The Twelfth International Conference on Learning Repre- sentations.
  48. Zhichao Xu. 2023. Context-aware decoding reduces hallucination in query-focused summarization. arXiv preprint arXiv:2312.14335. · arXiv
  49. Zhichao Xu. 2024. Rankmamba: Benchmarking mamba's document ranking performance in the era of transformers. arXiv preprint arXiv:2403.18276. · arXiv
  50. Zhichao Xu, Daniel Cohen, Bei Wang, and Vivek Sriku- mar. 2024b. In-context example ordering guided by label distributions. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2623-2640, Mexico City, Mexico. Association for Computational Linguistics.
  51. Zhichao Xu, Jinghua Yan, Ashim Gupta, and Vivek Srikumar. 2024c. State space models are strong text rerankers. arXiv preprint arXiv:2412.14354. · arXiv
  52. Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. RankRAG: Unifying context rank- ing with retrieval-augmented generation in LLMs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  53. Jiayi Yuan, Hongyi Liu, Shaochen Zhong, Yu-Neng Chuang, Songchen Li, Guanchu Wang, Duy Le, Hongye Jin, Vipin Chaudhary, Zhaozhuo Xu, et al. 2024. Kv cache compression, but what must we give in return? a comprehensive benchmark of long context capable approaches. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2024, pages 4623-4648.
  54. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems, 32.
  55. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. Siren's song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Chang, C.-Y., Jiang, Z., Rakesh, V., Pan, M., Yeh, C.-C. M., Wang, G., Hu, M., Xu, Z., Zheng, Y., Das, M., & Zou, N.
Affiliations
  • Texas A&M University (US)
  • Visa (United Kingdom) (GB)
  • Worcester Polytechnic Institute (US)
  • University of Utah (US)
  • University of Houston (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2501.00332
arXiv
2501.00332
OpenAlex
W4406031118
Cited by
1
References
0
Citations by year
2025: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
retrieval-augmented generation, multi-agent systems, document filtering, relevance scoring, question answering, response consistency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review