CoChat
Sign Up
PreprintCited by 12Topic ModelingInformation Retrieval and Search BehaviorNatural Language Processing Techniques

The DOI matches its DataCite record; the references it cites are not checked yet.

Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks

Open access
Gao, Yunfan, Xiong, Yun, Wang, Meng, Wang, Haofen·2024·arXiv
DOI 10.48550/arxiv.2407.21059arXiv 2407.21059
Open this paper in CoChat — free account

Abstract

Retrieval-augmented Generation (RAG) has markedly enhanced the capabilities of Large Language Models (LLMs) in tackling knowledge-intensive tasks. The increasing demands of application scenarios have driven the evolution of RAG, leading to the integration of advanced retrievers, LLMs and other complementary technologies, which in turn has amplified the intricacy of RAG systems. However, the rapid advancements are outpacing the foundational RAG paradigm, with many methods struggling to be unified under the process of "retrieve-then-generate". In this context, this paper examines the limitations of the existing RAG paradigm and introduces the modular RAG framework. By decomposing complex RAG systems into independent modules and specialized operators, it facilitates a highly reconfigurable framework. Modular RAG transcends the traditional linear architecture, embracing a more advanced design that integrates routing, scheduling, and fusion mechanisms. Drawing on extensive research, this paper further identifies prevalent RAG patterns-linear, conditional, branching, and looping-and offers a comprehensive analysis of their respective implementation nuances. Modular RAG presents innovative opportunities for the conceptualization and deployment of RAG systems. Finally, the paper explores the potential emergence of new operators and paradigms, establishing a solid theoretical foundation and a practical roadmap for the continued evolution and practical deployment of RAG technologies.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2407.21059

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Modular RAG separates training of the retriever and generator, which improves modularity and flexibility but limits retrieval adaptation during generation.

    “Unlike Naïve or Advanced RAG, Modular RAG separates the training of the retriever and generator. The retriever, a dense dual encoder, is trained using contrastive loss, and the generator is fine-tuned independently on topk retrieved documents using maximum likelihood estimation. Although this improves modularity and flexibility, it limits retrieval adaptation during generation.”

    From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [23]

    The Introduction (p. 3) says Modular RAG uses independent modules and improves flexibility. But the Tuning Pattern section (p. 14) explicitly includes simultaneous fine-tuning of the retriever and generator, so separate training is not a defining feature. Our copy also does not say modularity limits retrieval adaptation during generation. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Modular RAG [28] modularizes general RAG but falls short of addressing GraphRAG's workflows and needs.

    “While efforts like Modular RAG [28] have modularized general RAG, they fall short of addressing the GraphRAG's workflows and needs.”

    From LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration (2025) · cites this paper as [28]

    The Introduction (p. 3) describes a modular framework for RAG, supporting the first part of the claim. The paper also discusses knowledge-graph indexing (p. 5) and graph-data queries (p. 6), but our copy does not say that Modular RAG falls short of GraphRAG’s workflows or needs. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Modular RAG [28] neither dissects the core retrieval process of GraphRAG nor clarifies its distinction from general RAG.

    “Moreover, while Modular RAG [28] modularizes general RAG workflows, it neither dissects the core retrieval process of GraphRAG nor clarifies its distinction from general RAG.”

    From LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration (2025) · cites this paper as [28]

    The Introduction (p. 3) describes a modular framework for general RAG processes, supporting the first part of the sentence. The Indexing section (p. 5) discusses knowledge-graph indexes, but the retrieved passages do not analyze GraphRAG’s core retrieval process or explicitly compare GraphRAG with general RAG. The cited paper therefore supports the description of its scope, while the citing authors’ negative assessment is not directly established by a passage in our copy. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

  • Modular RAG extends capabilities of advanced RAG by decomposing the architecture into independently configurable modules.

    “Modular RAG extends the capabilities of advanced RAG by decomposing the architecture into independently configurable modules [23].”

    From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [23]

  • Modular RAG [28] and related surveys [24, 35, 86, 126] focus on conceptual overviews and lack implementation or evaluation support for GraphRAG.

    “However, Modular RAG [28] and related surveys [24, 35, 86, 126] focus on conceptual overviews and lack implementation or evaluation support for GraphRAG.”

    From LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration (2025) · cites this paper as [28]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists haven’t been fetched yet

Cited by 12

Research works printed but not linked: 54

  1. [1][1] Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen et al. , 'Siren's song in the ai ocean: A survey on hallucination in large language models,' arXiv preprint arXiv:2309.01219 , 2023. - · arXiv
  2. [2][2] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang, 'Retrieval-augmented generation for large language models: A survey,' arXiv preprint arXiv:2312.10997 , 2023. - · arXiv
  3. [3][3] Z. Xu, M. J. Cruz, M. Guevara, T. Wang, M. Deshpande, X. Wang, and Z. Li, 'Retrieval-augmented generation with knowledge graphs for customer service question answering,' in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , 2024, pp. 2905-2909. -
  4. [4][4] C. Zhang, S. Wu, H. Zhang, T. Xu, Y. Gao, Y. Hu, and E. Chen, 'Notellm: A retrievable large language model for note recommendation,' in Companion Proceedings of the ACM on Web Conference 2024 , 2024, pp. 170-179. -
  5. [5][5] R. Anantha, T. Bethi, D. Vodianik, and S. Chappidi, 'Context tuning for retrieval augmented generation,' arXiv preprint arXiv:2312.05708 , 2023. - · arXiv
  6. [6][6] Y. Gao, T. Sheng, Y. Xiang, Y. Xiong, H. Wang, and J. Zhang, 'Chatrec: Towards interactive and explainable llms-augmented recommender system,' arXiv preprint arXiv:2303.14524 , 2023. - · arXiv
  7. [8][8] D. S. Asudani, N. K. Nagwani, and P. Singh, 'Impact of word embedding models on text analytics in deep learning environment: a review,' Artificial intelligence review , vol. 56, no. 9, pp. 10 345-10 425, 2023. -
  8. [9][9] F. Cuconasu, G. Trappolini, F. Siciliano, S. Filice, C. Campagnano, Y. Maarek, N. Tonellotto, and F. Silvestri, 'The power of noise: Redefining retrieval for rag systems,' arXiv preprint arXiv:2401.14887 , 2024. - · arXiv
  9. [10][10] W. Peng, G. Li, Y. Jiang, Z. Wang, D. Ou, X. Zeng, E. Chen et al. , 'Large language model based long-tail query rewriting in taobao search,' arXiv preprint arXiv:2311.03758 , 2023. - · arXiv
  10. [11][11] Y. Xi, J. Lin, W. Liu, X. Dai, W. Zhang, R. Zhang, R. Tang, and Y. Yu, 'A bird's-eye view of reranking: from list level to page level,' in Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining , 2023, pp. 1075-1083. -
  11. [12][12] Z. Feng, X. Feng, D. Zhao, M. Yang, and B. Qin, 'Retrievalgeneration synergy augmented large language models,' arXiv preprint arXiv:2310.05149 , 2023. - · arXiv
  12. [13][13] G. Kim, S. Kim, B. Jeon, J. Park, and J. Kang, 'Tree of clarifications: Answering ambiguous questions with retrieval-augmented large language models,' arXiv preprint arXiv:2310.14696 , 2023. - · arXiv
  13. [14][14] Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y. Yang, J. Callan, and G. Neubig, 'Active retrieval augmented generation,' arXiv preprint arXiv:2305.06983 , 2023. - · arXiv
  14. [15][15] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson, 'From local to global: A graph rag approach to queryfocused summarization,' arXiv preprint arXiv:2404.16130 , 2024. - · arXiv
  15. [17][17] X. Wang, Z. Wang, X. Gao, F. Zhang, Y. Wu, Z. Xu, T. Shi, Z. Wang, S. Li, Q. Qian et al. , 'Searching for best practices in retrieval-augmented generation,' arXiv preprint arXiv:2407.01219 , 2024. - · arXiv
  16. [18][18] M. Lee, S. An, and M.-S. Kim, 'Planrag: A plan-then-retrieval augmented generation for generative large language models as decision makers,' arXiv preprint arXiv:2406.12430 , 2024. - · arXiv
  17. [19][19] D. Arora, A. Kini, S. R. Chowdhury, N. Natarajan, G. Sinha, and A. Sharma, 'Gar-meets-rag paradigm for zero-shot information retrieval,' arXiv preprint arXiv:2310.20158 , 2023. - · arXiv
  18. [20][20] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschel et al. , 'Retrievalaugmented generation for knowledge-intensive nlp tasks,' Advances in Neural Information Processing Systems , vol. 33, pp. 9459-9474, 2020. -
  19. [21][21] S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark et al. , 'Improving language models by retrieving from trillions of tokens,' in International conference on machine learning . PMLR, 2022, pp. 22062240. -
  20. [22][22] G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, F. Petroni, T. Schick, J. Dwivedi-Yu, A. Joulin, S. Riedel, and E. Grave, 'Few-shot learning with retrieval augmented language models,' arXiv preprint arXiv:2208.03299 , 2022. - · arXiv
  21. [23][23] H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal, 'Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions,' arXiv preprint arXiv:2212.10509 , 2022. - · arXiv
  22. [24][24] X. Ma, Y. Gong, P. He, H. Zhao, and N. Duan, 'Query rewriting for retrieval-augmented large language models,' arXiv preprint arXiv:2305.14283 , 2023. - · arXiv
  23. [25][25] N. Anderson, C. Wilson, and S. D. Richardson, 'Lingua: Addressing scenarios for live interpretation and automatic dubbing,' in Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas (Volume 2: Users and Providers Track and Government Track) , J. Campbell, S. Larocca, J. Marciano, K. Savenkov, and A. Yanishevsky, Eds. Orlando, USA: Association for Machine Translation in the Americas, Sep. 2022, pp. 202-209. [Online]. Available: https://aclanthology.org/2022.amta-upg.14 -
  24. [26][26] L. Luo, Y.-F. Li, G. Haffari, and S. Pan, 'Reasoning on graphs: Faithful and interpretable large language model reasoning,' arXiv preprint arXiv:2310.01061 , 2023. - · arXiv
  25. [27][27] X. V. Lin, X. Chen, M. Chen, W. Shi, M. Lomeli, R. James, P. Rodriguez, J. Kahn, G. Szilvasy, M. Lewis et al. , 'Ra-dit: Retrieval-augmented dual instruction tuning,' arXiv preprint arXiv:2310.01352 , 2023. - · arXiv
  26. [28][28] A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, 'Self-rag: Learning to retrieve, generate, and critique through self-reflection,' arXiv preprint arXiv:2310.11511 , 2023. - · arXiv
  27. [29][29] Y. Huang and J. Huang, 'A survey on retrieval-augmented text generation for large language models,' arXiv preprint arXiv:2404.10981 , 2024. - · arXiv
  28. [30][30] Y. Hu and Y. Lu, 'Rag and rau: A survey on retrieval-augmented language model in natural language processing,' arXiv preprint arXiv:2404.19543 , 2024. - · arXiv
  29. [31][31] Y. Ding, W. Fan, L. Ning, S. Wang, H. Li, D. Yin, T.-S. Chua, and Q. Li, 'A survey on rag meets llms: Towards retrieval-augmented large language models,' arXiv preprint arXiv:2405.06211 , 2024. - · arXiv
  30. [32][32] P. Zhao, H. Zhang, Q. Yu, Z. Wang, Y. Geng, F. Fu, L. Yang, W. Zhang, and B. Cui, 'Retrieval-augmented generation for ai-generated content: A survey,' arXiv preprint arXiv:2402.19473 , 2024. - · arXiv
  31. [34][34] Y. Wang, N. Lipka, R. A. Rossi, A. Siu, R. Zhang, and T. Derr, 'Knowledge graph prompting for multi-document question answering,' arXiv preprint arXiv:2308.11730 , 2023. - · arXiv
  32. [35][35] D. Zhou, N. Sch¨ arli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le et al. , 'Least-to-most prompting enables complex reasoning in large language models,' arXiv preprint arXiv:2205.10625 , 2022. - · arXiv
  33. [36][36] S. Dhuliawala, M. Komeili, J. Xu, R. Raileanu, X. Li, A. Celikyilmaz, and J. Weston, 'Chain-of-verification reduces hallucination in large language models,' arXiv preprint arXiv:2309.11495 , 2023. - · arXiv
  34. [37][37] L. Gao, X. Ma, J. Lin, and J. Callan, 'Precise zero-shot dense retrieval without relevance labels,' arXiv preprint arXiv:2212.10496 , 2022. - · arXiv
  35. [38][38] H. S. Zheng, S. Mishra, X. Chen, H.-T. Cheng, E. H. Chi, Q. V. Le, and D. Zhou, 'Take a step back: Evoking reasoning via abstraction in large language models,' arXiv preprint arXiv:2310.06117 , 2023. - · arXiv
  36. [39][39] H. Cao, 'Recent advances in text embedding: A comprehensive review of top-performing methods on the mteb benchmark,' arXiv preprint arXiv:2406.01607 , 2024. - · arXiv
  37. [41][41] Z. Li, X. Zhang, Y. Zhang, D. Long, P. Xie, and M. Zhang, 'Towards general text embeddings with multi-stage contrastive learning,' arXiv preprint arXiv:2308.03281 , 2023. - · arXiv
  38. [42][42] H. Yang, Z. Li, Y. Zhang, J. Wang, N. Cheng, M. Li, and J. Xiao, 'Prca: Fitting black-box large language models for retrieval question answering via pluggable reward-driven contextual adapter,' arXiv preprint arXiv:2310.18347 , 2023. - · arXiv
  39. [43][43] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, 'Lost in the middle: How language models use long contexts,' arXiv preprint arXiv:2307.03172 , 2023. - · arXiv
  40. [44][44] Y. Lyu, Z. Li, S. Niu, F. Xiong, B. Tang, W. Wang, H. Wu, H. Liu, T. Xu, and E. Chen, 'Crud-rag: A comprehensive chinese benchmark for retrieval-augmented generation of large language models,' arXiv preprint arXiv:2401.17043 , 2024. - · arXiv
  41. [45][45] L. Xia, J. Xu, Y. Lan, J. Guo, and X. Cheng, 'Learning maximal marginal relevance model via directly optimizing diversity evaluation measures,' in Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval , 2015, pp. 113-122. -
  42. [48][48] R. Litman, O. Anschel, S. Tsiper, R. Litman, S. Mazor, and R. Manmatha, 'Scatter: selective context attentional scene text recognizer,' in proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 11 962-11 972. -
  43. [49][49] J. Cui, Z. Li, Y. Yan, B. Chen, and L. Yuan, 'Chatlaw: Open-source legal large language model with integrated external knowledge bases,' arXiv preprint arXiv:2306.16092 , 2023. - · arXiv
  44. [50][50] T. Schick, J. Dwivedi-Yu, R. Dess` ı, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom, 'Toolformer: Language models can teach themselves to use tools,' arXiv preprint arXiv:2302.04761 , 2023. - · arXiv
  45. [51][51] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , 'Training language models to follow instructions with human feedback,' Advances in neural information processing systems , vol. 35, pp. 27 730-27 744, 2022. -
  46. [52][52] S. J. Semnani, V. Z. Yao, H. C. Zhang, and M. S. Lam, 'Wikichat: Stopping the hallucination of large language model chatbots by fewshot grounding on wikipedia,' arXiv preprint arXiv:2305.14292 , 2023. - · arXiv
  47. [53][53] J. Baek, S. Jeong, M. Kang, J. C. Park, and S. J. Hwang, 'Knowledge-augmented language model verification,' arXiv preprint arXiv:2310.12836 , 2023. - · arXiv
  48. [54][54] G. V. Cormack, C. L. Clarke, and S. Buettcher, 'Reciprocal rank fusion outperforms condorcet and individual rank learning methods,' in Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval , 2009, pp. 758-759. -
  49. [55][55] W. Shi, S. Min, M. Yasunaga, M. Seo, R. James, M. Lewis, L. Zettlemoyer, and W.-t. Yih, 'Replug: Retrieval-augmented black-box language models,' arXiv preprint arXiv:2301.12652 , 2023. - · arXiv
  50. [56][56] Z. Shao, Y. Gong, Y. Shen, M. Huang, N. Duan, and W. Chen, 'Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy,' arXiv preprint arXiv:2305.15294 , 2023. - · arXiv
  51. [57][57] S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al. , 'Metagpt: Meta programming for multi-agent collaborative framework,' arXiv preprint arXiv:2308.00352 , 2023. - · arXiv
  52. [58][58] Z. Yu, C. Xiong, S. Yu, and Z. Liu, 'Augmentation-adapted retriever improves generalization of language models as generic plug-in,' arXiv preprint arXiv:2305.17331 , 2023. - · arXiv
  53. [59][59] Z. Hei, W. Wei, W. Ou, J. Qiao, J. Jiao, Z. Zhu, and G. Song, 'Dr-rag: Applying dynamic document relevance to retrieval-augmented generation for question-answering,' arXiv preprint arXiv:2406.07348 , 2024. - · arXiv
  54. [60][60] M. Besta, A. Kubicek, R. Niggli, R. Gerstenberger, L. Weitzendorf, M. Chi, P. Iff, J. Gajda, P. Nyczyk, J. M¨ uller et al. , 'Multihead rag: Solving multi-aspect problems with llms,' arXiv preprint arXiv:2406.05085 , 2024. · arXiv

Metadata

Authors
Gao, Y., Xiong, Y., Wang, M., & Wang, H.
Affiliations
  • Tongji University (CN)
  • Fudan University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
44
Issue
7
Pages
1-53
ISSN
1046-8188 (ISSN-L)
DOI
10.48550/arxiv.2407.21059
arXiv
2407.21059
OpenAlex
W4401306278
Cited by
12
References
0
Citations by year
2026: 3 · 2025: 8 · 2024: 1
FWCI
10.39
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Transactions on Information Systems (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
retrieval-augmented generation, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review