FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
With the advent of large language models (LLMs) and multimodal large language models (MLLMs), the potential of retrieval-augmented generation (RAG) has attracted considerable research attention. However, the absence of a standardized framework for implementation, coupled with the inherently complex RAG process, makes it challenging and time-consuming for researchers to compare and evaluate these approaches in a consistent environment. In response to this challenge, we develop FlashRAG, an efficient and modular open-source toolkit designed to assist researchers in reproducing and comparing existing RAG methods and developing their own algorithms within a unified framework. Our toolkit has implemented 16 advanced RAG methods and gathered and organized 38 benchmark datasets. It has various features, including a customizable modular framework, a rich collection of pre-implemented RAG works, comprehensive datasets, efficient auxiliary pre-processing scripts, and extensive and standard evaluation metrics. Our toolkit and resources are available at https://github.com/RUC-NLPIR/FlashRAG.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“FlashRAG Toolkit: Implements multiple RAG methods and benchmark datasets to support efficient and standardized evaluation [66].”
From Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG (2025) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2021) · arXiv · doi:10.48550/arxiv.2109.07958 · Cited by 899
Zero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) · doi:10.18653/v1/k17-1034 · Cited by 482
Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3463238 · Cited by 360
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.200 · Cited by 357
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.15294 · Cited by 200
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · arXiv · Cited by 168
Active Retrieval Augmented Generation
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · arXiv · doi:10.48550/arxiv.2305.06983 · Cited by 18
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
Jiang, H., Wu, Q., Luo, X., Li, D., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · arXiv · Cited by 12
Research works printed but not linked: 26
- A. Asai Z. Wu Y. Wang A. Sil and H. Hajishirzi. 2023. Self-RAG: Learning to Retrieve Generate and Critique through Self-Reflection. ArXiv preprint (2023).
- J. Fang, Z. Meng, and C. Macdonald. 2024. TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation. ArXiv preprint (2024).
- Z. Feng X. Feng D. Zhao M. Yang and B. Qin. 2023. Retrieval-Generation Synergy Augmented Large Language Models. ArXiv preprint (2023).
- Y. Gao Y. Xiong X. Gao K. Jia J. Pan Y. Bi Y. Dai J. Sun M. Wang and H. Wang. 2023. Retrieval-Augmented Generation for Large Language Models: A Survey. ArXiv preprint (2023).
- K. Guu, K. Lee, Z. Tung, P. Pasupat, and M.-W. Chang. 2020. REALM: Retrieval-Augmented Language Model Pre-Training. ArXiv preprint (2020).
- P. Izsak M. Berchansky D. Fleischer and R. Laperdon. 20f. fastRAG: Efficient Retrieval Augmentation and Generation Framework. ArXiv preprint (20f).
- S. Jeong, J. Baek, S. Cho, S. Hwang, and J. Park. 2024. Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. In Proc. of NAACL-HLT.
- H. Jiang Q. Wu X. Luo D. Li C.-Y. Lin Y. Yang and L. Qiu. 2023b. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenario. ArXiv preprint (2023).
- D. Kim B. Kim D. Han and M. Eibich. 2024a. AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline. ArXiv preprint (2024).
- J. Kim, J. Nam, S. Mo, J. Park, S.-W. Lee, M. Seo, J.-W. Ha, and J. Shin. 2024b. SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs. In Proc. of ICLR.
- W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proc. of SOSP.
- Y. Li. 2023. Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering. ArXiv preprint (2023).
- T. Nguyen, M. Rosenberg, X. Song, J. Gao, S. Tiwary, R. Majumder, and L. Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Proc. of NeurIPS Workshop), Barcelona, Spain, December 9, 2016 (CEUR Workshop Proceedings).
- A. Radford, J. Wook Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proc. of ICML (Proceedings of Machine Learning Research).
- S. E. Robertson and H. Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr. 4 (2009). · DOI · Paper page
- W. Shi, S. Min, M. Yasunaga, M. Seo, R. James, M. Lewis, L. Zettlemoyer, and W. Yih. 2024. REPLUG: Retrieval-Augmented Black-Box Language Models. In Proc. of NAACL-HLT.
- H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal. 2023. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In Proc. of ACL.
- Y. Wang, P. Li, M. Sun, and Y. Liu. 2023. Self-Knowledge Guided Retrieval Augmentation for Large Language Models. In Proc. of EMNLP 2023 Findings.
- T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proc. of EMNLP.
- F. Xu, W. Shi, and E. Choi. 2023. RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation. ArXiv preprint (2023).
- O. Yoran, T. Wolfson, O. Ram, and J. Berant. 2023. Making Retrieval-Augmented Language Models Robust to Irrelevant Context. ArXiv preprint (2023).
- X. Yu Y. Lu and Z. Yu. 2024. LocalRQA: From Generating Data to Locally Training Testing and Deployin. ArXiv preprint (2024). · DOI
- Z. Yu, C. Xiong, S. Yu, and Z. Liu. 2023. Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In. In Proc. of ACL.
- X. Zhang, Y. Song, Y. Wang, S. Tang, X. Li, Z. Zeng, Z. Wu, W. Ye, W. Xu, Y. Zhang, X. Dai, S. Zhang, and Q. Wen. 2024. RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation. In Proc. of the EMNLP: Demo, Delia Irazu Hernandez Farias, Tom Hope, and Manling Li (Eds.).
- L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proc. of NeurIPS.
- Y. Zhu, Z. Huang, Z. Dou, and J.-R. Wen. 2024. One Token Can Help! Learning Scalable and Pluggable Virtual Token. ArXiv preprint (2024).
Cited by
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
Self-citationAgentic Retrieval-Augmented Generation: A Survey on Agentic RAG
Singh, A., Ehtesham, A., Kumar, S., Khoei, T. T., & Vasilakos, A. V. (2025) · arXiv · doi:10.48550/arxiv.2501.09136 · Cited by 36
Metadata
- Authors
- Jin, J., Zhu, Y., Dou, Z., Dong, G., Yang, X., Zhang, C., Zhao, T., Yang, Z., & Wen, J.-R.
- Affiliations
- Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
- ORCID
- Venue
- Companion Proceedings of the ACM on Web Conference 2025
- Publisher
- ACM
- Published
- 2025-05-08
- Pages
- 737-740
- Cited by
- 53
- References
- 8
- Funders
- the National Natural Science Foundation of China award (62272467)
- Beijing Natural Science Foundation award (L233008)
- Beijing Municipal Science and Technology Project award (Z231100010323009)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research. https://app.cochat.ai/papers/10.1145/3701716.3715313
- The paper
- Jin, J., Zhu, Y., Dou, Z., Dong, G., Yang, X., Zhang, C., Zhao, T., Yang, Z., & Wen, J.-R. (2025). FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research. Companion Proceedings of the ACM on Web Conference 2025. https://doi.org/10.1145/3701716.3715313
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.