CoChat

FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research

Jin, Jiajie, Zhu, Yutao, Dou, Zhicheng, Dong, Guanting, Yang, Xinyu, Zhang, Chenghao, Zhao, Tong, Yang, Zhao, Wen, Ji-Rong · 2025 · Companion Proceedings of the ACM on Web Conference 2025
Cited by 53

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Abstract

With the advent of large language models (LLMs) and multimodal large language models (MLLMs), the potential of retrieval-augmented generation (RAG) has attracted considerable research attention. However, the absence of a standardized framework for implementation, coupled with the inherently complex RAG process, makes it challenging and time-consuming for researchers to compare and evaluate these approaches in a consistent environment. In response to this challenge, we develop FlashRAG, an efficient and modular open-source toolkit designed to assist researchers in reproducing and comparing existing RAG methods and developing their own algorithms within a unified framework. Our toolkit has implemented 16 advanced RAG methods and gathered and organized 38 benchmark datasets. It has various features, including a customizable modular framework, a rich collection of pre-implemented RAG works, comprehensive datasets, efficient auxiliary pre-processing scripts, and extensive and standard evaluation metrics. Our toolkit and resources are available at https://github.com/RUC-NLPIR/FlashRAG.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3701716.3715313

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 8 references

Research works printed but not linked: 26

  1. A. Asai Z. Wu Y. Wang A. Sil and H. Hajishirzi. 2023. Self-RAG: Learning to Retrieve Generate and Critique through Self-Reflection. ArXiv preprint (2023).
  2. J. Fang, Z. Meng, and C. Macdonald. 2024. TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation. ArXiv preprint (2024).
  3. Z. Feng X. Feng D. Zhao M. Yang and B. Qin. 2023. Retrieval-Generation Synergy Augmented Large Language Models. ArXiv preprint (2023).
  4. Y. Gao Y. Xiong X. Gao K. Jia J. Pan Y. Bi Y. Dai J. Sun M. Wang and H. Wang. 2023. Retrieval-Augmented Generation for Large Language Models: A Survey. ArXiv preprint (2023).
  5. K. Guu, K. Lee, Z. Tung, P. Pasupat, and M.-W. Chang. 2020. REALM: Retrieval-Augmented Language Model Pre-Training. ArXiv preprint (2020).
  6. P. Izsak M. Berchansky D. Fleischer and R. Laperdon. 20f. fastRAG: Efficient Retrieval Augmentation and Generation Framework. ArXiv preprint (20f).
  7. S. Jeong, J. Baek, S. Cho, S. Hwang, and J. Park. 2024. Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. In Proc. of NAACL-HLT.
  8. H. Jiang Q. Wu X. Luo D. Li C.-Y. Lin Y. Yang and L. Qiu. 2023b. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenario. ArXiv preprint (2023).
  9. D. Kim B. Kim D. Han and M. Eibich. 2024a. AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline. ArXiv preprint (2024).
  10. J. Kim, J. Nam, S. Mo, J. Park, S.-W. Lee, M. Seo, J.-W. Ha, and J. Shin. 2024b. SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs. In Proc. of ICLR.
  11. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proc. of SOSP.
  12. Y. Li. 2023. Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering. ArXiv preprint (2023).
  13. T. Nguyen, M. Rosenberg, X. Song, J. Gao, S. Tiwary, R. Majumder, and L. Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Proc. of NeurIPS Workshop), Barcelona, Spain, December 9, 2016 (CEUR Workshop Proceedings).
  14. A. Radford, J. Wook Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proc. of ICML (Proceedings of Machine Learning Research).
  15. S. E. Robertson and H. Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr. 4 (2009). · DOI · Paper page
  16. W. Shi, S. Min, M. Yasunaga, M. Seo, R. James, M. Lewis, L. Zettlemoyer, and W. Yih. 2024. REPLUG: Retrieval-Augmented Black-Box Language Models. In Proc. of NAACL-HLT.
  17. H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal. 2023. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In Proc. of ACL.
  18. Y. Wang, P. Li, M. Sun, and Y. Liu. 2023. Self-Knowledge Guided Retrieval Augmentation for Large Language Models. In Proc. of EMNLP 2023 Findings.
  19. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proc. of EMNLP.
  20. F. Xu, W. Shi, and E. Choi. 2023. RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation. ArXiv preprint (2023).
  21. O. Yoran, T. Wolfson, O. Ram, and J. Berant. 2023. Making Retrieval-Augmented Language Models Robust to Irrelevant Context. ArXiv preprint (2023).
  22. X. Yu Y. Lu and Z. Yu. 2024. LocalRQA: From Generating Data to Locally Training Testing and Deployin. ArXiv preprint (2024). · DOI
  23. Z. Yu, C. Xiong, S. Yu, and Z. Liu. 2023. Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In. In Proc. of ACL.
  24. X. Zhang, Y. Song, Y. Wang, S. Tang, X. Li, Z. Zeng, Z. Wu, W. Ye, W. Xu, Y. Zhang, X. Dai, S. Zhang, and Q. Wen. 2024. RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation. In Proc. of the EMNLP: Demo, Delia Irazu Hernandez Farias, Tom Hope, and Manling Li (Eds.).
  25. L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proc. of NeurIPS.
  26. Y. Zhu, Z. Huang, Z. Dou, and J.-R. Wen. 2024. One Token Can Help! Learning Scalable and Pluggable Virtual Token. ArXiv preprint (2024).

Cited by

See all 53

Metadata

Authors
Jin, J., Zhu, Y., Dou, Z., Dong, G., Yang, X., Zhang, C., Zhao, T., Yang, Z., & Wen, J.-R.
Affiliations
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Venue
Companion Proceedings of the ACM on Web Conference 2025
Publisher
ACM
Published
2025-05-08
Pages
737-740
Cited by
53
References
8
Funders
  • the National Natural Science Foundation of China award (62272467)
  • Beijing Natural Science Foundation award (L233008)
  • Beijing Municipal Science and Technology Project award (Z231100010323009)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research. https://app.cochat.ai/papers/10.1145/3701716.3715313
The paper
Jin, J., Zhu, Y., Dou, Z., Dong, G., Yang, X., Zhang, C., Zhao, T., Yang, Z., & Wen, J.-R. (2025). FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research. Companion Proceedings of the ACM on Web Conference 2025. https://doi.org/10.1145/3701716.3715313

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.