CoChat
Sign Up
PreprintCited by 1Topic ModelingInformation Retrieval and Search BehaviorAI-based Problem Solving and Planning

SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions

Open access
Mishra, Saroj, Niroula, Suman, Yadav, Umesh, Thakur, Dilip, Gyawali, Srijan, Gaire, Shiva·2026·arXiv
DOI 10.48550/arxiv.2603.07379arXiv 2603.07379
Open this paper in CoChat — free accountOpen PDF

Abstract

Retrieval-Augmented Generation (RAG) systems are increasingly evolving into agentic architectures where large language models autonomously coordinate multi-step reasoning, dynamic memory management, and iterative retrieval strategies. Despite rapid industrial adoption, current research lacks a systematic understanding of Agentic RAG as a sequential decision-making system, leading to highly fragmented architectures, inconsistent evaluation methodologies, and unresolved reliability risks. This Systematization of Knowledge (SoK) paper provides the first unified framework for understanding these autonomous systems. We formalize agentic retrieval-generation loops as finite-horizon partially observable Markov decision processes, explicitly modeling their control policies and state transitions. Building upon this formalization, we develop a comprehensive taxonomy and modular architectural decomposition that categorizes systems by their planning mechanisms, retrieval orchestration, memory paradigms, and tool-invocation behaviors. We further analyze the critical limitations of traditional static evaluation practices and identify severe systemic risks inherent to autonomous loops, including compounding hallucination propagation, memory poisoning, retrieval misalignment, and cascading tool-execution vulnerabilities. Finally, we outline key doctoral-scale research directions spanning stable adaptive retrieval, cost-aware orchestration, formal trajectory evaluation, and oversight mechanisms, providing a definitive roadmap for building reliable, controllable, and scalable agentic retrieval systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2603.07379

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most current benchmarks evaluate agents primarily through task completion under fixed prompts and fixed model settings.

    “Most current benchmarks evaluate agents primarily through task completion under fixed prompts and fixed model settings [Zhu et al., 2025, Mishra et al., 2026].”

    From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Mishra et al., 2026

    Section VII, “Evaluation and Benchmarking” (page unavailable), says current methods primarily assess final-answer quality rather than agents’ multi-step behavior. Section B, “From Static Benchmarks to Evaluation Frameworks” (page unavailable), describes existing RAG benchmarks as heavily focused on static, one-shot evaluation. Neither passage establishes that benchmarks use fixed prompts and fixed model settings. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Every retrieved artifact competes for limited context budget, and excessive context loading can degrade performance through information overload or the 'lost in the middle' phenomenon.

    “Every retrieved artifact competes for limited context budget, and excessive context loading can degrade performance through information overload or the 'lost in the middle' phenomenon [Corallo and Papotti, 2026, Mishra et al., 2026, Esmi et al., 2025].”

    From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Mishra et al., 2026

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 100

  1. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, "Retrieval-augmented generation for knowledge-intensive NLP tasks," in Advances in Neural Information Processing Systems (NeurIPS), 2020. [Online]. Available: https://arxiv.org/abs/2005.11401 · arXiv
  2. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih, "Dense passage retrieval for open-domain question answering," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020. [Online]. Available: https://aclanthology.org/2020.emnlp-main.550/
  3. Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning, "Hotpotqa: A dataset for diverse, explainable multi-hop question answering," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018. [Online]. Available: https://aclanthology.org/D18-1259/
  4. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, "Lost in the middle: How language models use long contexts," Transactions of the Association for Computational Linguistics, vol. 12, pp. 157-173, 2024. [Online]. Available: https://aclanthology.org/2024.tacl-1.9/
  5. Z. Shao, Y. Gong, Y. Shen, M. Huang, N. Duan, and W. Chen, "Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy," in Findings of the Association for Computational Linguistics: EMNLP 2023, 2023. [Online]. Available: https://aclanthology.org/2023.findings-emnlp.620/
  6. A. Asai et al., "Self-rag: Learning to retrieve, generate, and critique through self-reflection," in International Conference on Learning Representations (ICLR), 2024. [Online]. Available: https: //arxiv.org/abs/2310.11511
  7. Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y. Yang, J. Callan, and G. Neubig, "Active retrieval augmented generation," arXiv preprint arXiv:2305.06983, 2023. [Online]. Available: https: //arxiv.org/abs/2305.06983 · arXiv
  8. Q. Cheng, X. Li, S. Li, Q. Zhu, Z. Yin, Y. Shao, L. Li, T. Sun, H. Yan, and X. Qiu, "Unified active retrieval for retrieval augmented generation," in Findings of the Association for Computational Linguistics: EMNLP 2024, 2024. [Online]. Available: https://aclanthology.org/2024.findings-emnlp.999/
  9. H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal, "Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions," in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), Long Papers, 2023. [Online]. Available: https://aclanthology.org/2023. acl-long.557/
  10. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom, "Toolformer: Language models can teach themselves to use tools," arXiv preprint arXiv:2302.04761, 2023. [Online]. Available: https://arxiv.org/abs/ 2302.04761 · arXiv
  11. E. Karpas, A. Singer, J. Ainslie, E. Omer, A. Petrović et al., "MRKL systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning," arXiv preprint arXiv:2205.00445, 2022. [Online]. Available: https://arxiv.org/abs/2205.00445 · arXiv
  12. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, "ReAct: Synergizing reasoning and acting in language models," in International Conference on Learning Representations (ICLR), 2023. [Online]. Available: https://arxiv.org/abs/2210.03629 · arXiv
  13. N. Shinn, B. Labash, A. Gopinath, K. Narasimhan, and S. Yao, "Reflexion: Language agents with verbal reinforcement learning," arXiv preprint arXiv:2303.11366, 2023. [Online]. Available: https: //arxiv.org/abs/2303.11366 · arXiv
  14. B. Bohnet, V. Q. Tran, P. Verga, R. Aharoni, D. Andor, L. Baldini Soares, M. Ciaramita, J. Eisenstein, K. Ganchev, J. Herzig, K. Hui, T. Kwiatkowski, J. Ma, J. Ni, L. Sestorain Saralegui, T. Schuster, W. W. Cohen, M. Collins, D. Das, D. Metzler, S. Petrov, and K. Webster, "Attributed question answering: Evaluation and modeling for attributed large language models," arXiv preprint arXiv:2212.08037, 2022. [Online]. Available: https://arxiv.org/abs/ 2212.08037 · arXiv
  15. R. Nakano, J. Hilton, S. Balaji, J. Wu, P. Abbeel et al., "WebGPT: Browser-assisted question-answering with human feedback," arXiv preprint arXiv:2112.09332, 2021. [Online]. Available: https://arxiv. org/abs/2112.09332 · arXiv
  16. A. Singh, A. Ehtesham, S. Kumar, and T. T. Khoei, "Agentic retrieval- augmented generation: A survey on agentic rag," arXiv preprint arXiv:2501.09136, 2025. · arXiv
  17. P. Ferrazzi, M. Cvjeticanin, A. Piraccini, and D. Giannuzzi, "Is agentic rag worth it? an experimental comparison of rag approaches," arXiv preprint arXiv:2601.07711, 2026. [Online]. Available: https://arxiv.org/abs/2601.07711 · arXiv
  18. M. Du, B. Xu, C. Zhu, S. Wang, P. Wang, X. Wang, and Z. Mao, "A-rag: Scaling agentic retrieval-augmented generation via hierarchical retrieval interfaces," arXiv preprint arXiv:2602.03442, 2026. · arXiv
  19. W. M. P. van der Aalst, A. H. M. ter Hofstede et al., "Workflow patterns: On the expressive power of petri-net-based workflow languages," in Proceedings of the Fourth Workshop on the Practical Use of Coloured Petri Nets and CPN Tools (CPN 2002), 2002. [Online]. Available: https://research.tue.nl/en/publications/ workflow-patterns-on-the-expressive-power-of-petri-net-based-work/
  20. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017. [Online]. Available: https://arxiv.org/abs/1706.03762 · arXiv
  21. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., "Language models are few-shot learners," in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020. [Online]. Available: https://arxiv.org/abs/2005.14165 · arXiv
  22. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, "Chain-of-thought prompting elicits reasoning in large language models," in Advances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022. [Online]. Available: https://arxiv.org/abs/2201.11903 · arXiv
  23. L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu, "A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions," arXiv preprint arXiv:2311.05232, 2023. [Online]. Available: https://arxiv.org/abs/2311.05232 · arXiv
  24. G. Izacard and E. Grave, "Leveraging passage retrieval with generative models for open domain question answering," in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume (EACL), 2021. [Online]. Available: https://aclanthology.org/2021.eacl-main.74/
  25. L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, W. X. Zhao, Z. Wei, and J.-R. Wen, "A survey on large language model based autonomous agents," Frontiers of Computer Science, vol. 18, no. 6, p. 186345, 2024. [Online]. Available: https://arxiv.org/abs/2308.11432 · arXiv
  26. D. Zhou et al., "Least-to-most prompting enables complex reasoning in large language models," arXiv preprint arXiv:2205.10625, 2022. [Online]. Available: https://arxiv.org/abs/2205.10625 · arXiv
  27. L. Wang et al., "Plan-and-solve prompting: Improving zero- shot chain-of-thought reasoning by large language models," in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), Long Papers, 2023. [Online]. Available: https://aclanthology.org/2023.acl-long.147/
  28. O. Press et al., "Measuring and narrowing the compositionality gap in language models," arXiv preprint arXiv:2210.03350, 2022. [Online]. Available: https://arxiv.org/abs/2210.03350 · arXiv
  29. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan, "Tree of thoughts: Deliberate problem solving with large language models," in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2024. [Online]. Available: https://arxiv.org/abs/2305.10601 · arXiv
  30. J. S. Park, J. C. O'Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, "Generative agents: Interactive simulacra of human behavior," in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST), 2023. [Online]. Available: https://arxiv.org/abs/2304.03442 · arXiv
  31. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, "Lost in the middle: How language models use long con- texts," Transactions of the Association for Computational Linguistics, vol. 12, pp. 157-173, 2024.
  32. H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal, "Interleav- ing retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions," in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 10 014-10 037.
  33. O. Khattab, C. Potts, and M. Zaharia, "Baleen: Robust multi-hop reasoning at scale via condensed retrieval," in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 27 670-27 682.
  34. W. Shi, M. Xia, A. R. Fabbri, L. Zettlemoyer, and R. Das, "Trusting your evidence: Hallucinate less with context-aware decoding," in Pro- ceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics, 2024, pp. 1234-1249.
  35. T. Khot, H. Trivedi, M. Finlayson, Y. Fu, K. Richardson, P. Clark, and A. Sabharwal, "MuSiQue: Multihop questions via single-hop question generation," Transactions of the Association for Computational Lin- guistics, vol. 9, pp. 537-554, 2021.
  36. T. Gao, H. Yen, J. Yu, and D. Chen, "Enabling large language models to generate text with citations," in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 6465- 6488.
  37. Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang, "Retrieval-augmented generation for large language models: A survey," 2023. [Online]. Available: https://arxiv.org/abs/2312.10997 · arXiv
  38. W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T.-S. Chua, and Q. Li, "A survey on RAG meeting LLMs: Towards retrieval- augmented large language models," 2024. [Online]. Available: https://arxiv.org/abs/2405.06211 · arXiv
  39. Y. Shen et al., "Hugginggpt: Solving AI tasks with chatgpt and its friends in hugging face," 2023. [Online]. Available: https://arxiv.org/abs/2303.17580 · arXiv
  40. L. E. Erdogan et al., "Improving planning of agents for long-horizon tasks," 2025. [Online]. Available: https://arxiv.org/abs/2503.09572 · arXiv
  41. Q. Wu et al., "Autogen: Enabling next-gen llm applications via multi-agent conversation," arXiv preprint arXiv:2308.08155, 2023. [Online]. Available: https://arxiv.org/abs/2308.08155 · arXiv
  42. L. Gao, Z. Dai, P. Pasupat, A. Chen, A. T. Chaganty, Y. Fan, V. Zhao, N. Lao, H. Lee, D.-C. Juan, and K. Guu, "RARR: Researching and revising what language models say, using language models," in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023, pp. 16 477-16 508. [Online]. Available: https://aclanthology.org/2023.acl-long.910/
  43. Z. Xu, Z. Liu, Y. Yan, S. Wang, S. Yu, Z. Zeng, C. Xiao, Z. Liu, G. Yu, and C. Xiong, "Activerag: Autonomously knowledge assimilation and accommodation through retrieval-augmented agents," 2024. [Online]. Available: https://arxiv.org/abs/2402.13547 · arXiv
  44. Y. Yu, L. Yao, Y. Xie, Q. Tan, J. Feng, Y. Li, and L. Wu, "Agentic memory: Learning unified long-term and short-term memory management for large language model agents," 2026. [Online]. Available: https://arxiv.org/abs/2601.01885 · arXiv
  45. Z. Wang, J. Araki, Z. Jiang, M. R. Parvez, and G. Neubig, "Learning to filter context for retrieval-augmented generation," 2023. [Online]. Available: https://arxiv.org/abs/2311.08377 · arXiv
  46. N. Chirkova, T. Formal, V. Nikoulina, and S. Clinchant, "Provence: efficient and robust context pruning for retrieval-augmented generation," 2025. [Online]. Available: https://arxiv.org/abs/2501.16214 · arXiv
  47. J. S. Park, J. C. O'Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, "Generative agents: Interactive simulacra of human behavior," 2023. [Online]. Available: https://arxiv.org/abs/2304.03442 · arXiv
  48. W. Zhong, L. Guo et al., "Enhancing large language models with long-term memory," 2023. [Online]. Available: https://arxiv.org/abs/ 2305.10250
  49. C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez, "Memgpt: Towards llms as operating systems," 2024. [Online]. Available: https://arxiv.org/abs/2310.08560 · arXiv
  50. T. Nguyen, P. Chin, and Y.-W. Tai, "Ma-rag: Multi-agent retrieval- augmented generation via collaborative chain-of-thought reasoning," arXiv preprint arXiv:2505.20096, 2025. · arXiv
  51. Y. Chen, E. Zhang, T. Hu, S. Wang, Z. Yang, M. Zhong, and J. Mao, "Jade: Bridging the strategic-operational gap in dynamic agentic rag," arXiv preprint arXiv:2601.21916, 2026. · arXiv
  52. A. Aavani, "Capturing p: On the expressive power and efficient eval- uation of boolean retrieval," arXiv preprint arXiv:2601.18747, 2026. · arXiv
  53. S. Xu, W. Hao, and T. Lu, "Ka-rag: Integrating knowledge graphs and agentic retrieval-augmented generation for an intelligent educational question-answering model," Applied Sciences, vol. 15, no. 12547, 2025.
  54. X. Shi, M. Zheng, and Q. Lou, "Learning latency-aware orchestration for parallel multi-agent systems," arXiv preprint arXiv:2601.10560, 2026. · arXiv
  55. A. Maharjan and U. Yadav, "Chunking, retrieval, and re-ranking: An empirical evaluation of rag architectures for policy document question answering," arXiv preprint arXiv:2601.15457, 2026. [Online]. Available: https://arxiv.org/abs/2601.15457 · arXiv
  56. K. Mukherjee et al., "Llm-driven provenance forensics for threat investigation and detection," arXiv preprint arXiv:2508.21323, 2025. · arXiv
  57. J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. R. Narasimhan, and O. Press, "SWE-agent: Agent-computer interfaces enable au- tomated software engineering," in Advances in Neural Information Processing Systems, vol. 37, 2024.
  58. J. Logan, "Continuum memory architectures for long-horizon llm agents," arXiv preprint arXiv:2601.09913, 2026. · arXiv
  59. Agentic Memory Authors, "Agentic memory systems," arXiv preprint arXiv:2502.12110, 2025. · arXiv
  60. H. Huang, Z. Lin, Z. Wang, X. Chen, K. Ding, and J. Zhao, "Towards llm-powered verilog rtl assistant: Self-verification and self-correction," arXiv preprint arXiv:2406.00115, 2024. · arXiv
  61. S. Min, V. Zhong, L. Zettlemoyer, and H. Hajishirzi, "Multi- hop reading comprehension through question decomposition and rescoring," in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019. [Online]. Available: https://aclanthology.org/P19-1613/
  62. X. Ma, Y. Gong, P. He, H. Zhao, and N. Duan, "Query rewriting for retrieval-augmented large language models," arXiv preprint arXiv:2305.14283, 2023. [Online]. Available: https://arxiv.org/abs/ 2305.14283 · arXiv
  63. W. Zhang et al., "Dllm-searcher: Diffusion large language models for search and reasoning," arXiv preprint arXiv:2404.XXXXX, 2024. [Online]. Available: https://arxiv.org/abs/2404.XXXXX
  64. B. Xu, Z. Peng, B. Lei, S. Mukherjee, Y. Liu, and D. Xu, "Rewoo: Decoupling reasoning from observations for efficient augmented language models," arXiv preprint arXiv:2305.18323, 2023. [Online]. Available: https://arxiv.org/abs/2305.18323 · arXiv
  65. Z. Gou et al., "Critic: Large language models can self-correct with tool-interactive critiquing," arXiv preprint arXiv:2305.11738, 2023. [Online]. Available: https://arxiv.org/abs/2305.11738 · arXiv
  66. G. Li et al., "Camel: Communicative agents for "mind" exploration of large language model society," arXiv preprint arXiv:2303.17760, 2023. [Online]. Available: https://arxiv.org/abs/2303.17760 · arXiv
  67. C. Qian et al., "Chatdev: Communicative agents for software development," in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), Long Papers, 2024. [Online]. Available: https://aclanthology.org/2024.acl-long.810/
  68. S. Hong et al., "Metagpt: Meta programming for a multi- agent collaborative framework," in International Conference on Learning Representations (ICLR), 2024. [Online]. Available: https: //openreview.net/forum
  69. S. Dhuliawala et al., "Chain-of-verification reduces hallucination in large language models," arXiv preprint arXiv:2309.11495, 2023. [Online]. Available: https://arxiv.org/abs/2309.11495 · arXiv
  70. M. Li et al., "Search-r2: Search-augmented reasoning and refinement for large language models," arXiv preprint arXiv:2405.XXXXX, 2024. [Online]. Available: https://arxiv.org/abs/2405.XXXXX
  71. J. Menick et al., "Teaching language models to support answers with verified quotes," arXiv preprint arXiv:2203.11147, 2022. [Online]. Available: https://arxiv.org/abs/2203.11147 · arXiv
  72. T. Gao et al., "Enabling large language models to generate text with citations," arXiv preprint arXiv:2305.14627, 2023. [Online]. Available: https://arxiv.org/abs/2305.14627 · arXiv
  73. M. Mohammadi, Y. Li, J. Lo, and W. Yip, "Evaluation and benchmarking of LLM agents: A survey," 2025. [Online]. Available: https://arxiv.org/abs/2507.21504 · arXiv
  74. A. Yehudai, L. Eden, A. Li, G. Uziel, Y. Zhao, R. Bar-Haim, A. Cohan, and M. Shmueli-Scheuer, "Survey on evaluation of LLM-based agents," 2025. [Online]. Available: https://arxiv.org/abs/2503.16416 · arXiv
  75. B. Malin, T. Kalganova, and N. Boulgouris, "A review of faithfulness metrics for hallucination assessment in large language models," 2025. [Online]. Available: https://arxiv.org/abs/2501.00269 · arXiv
  76. T. Wei, T.-W. Li, Z. Liu, X. Ning, Z. Yang, J. Zou, Z. Zeng, R. Qiu, X. Lin, D. Fu, Z. Li, M. Ai, D. Zhou, W. Bao, Y. Li, G. Li, C. Qian, Y. Wang, X. Tang, and Y. Xiao, "Agentic reasoning for large language models," 2026. [Online]. Available: https://arxiv.org/abs/2601.12538 · arXiv
  77. K. Zhu, Y. Luo, D. Xu, Y. Yan, Z. Liu, S. Yu, R. Wang, Y. Li, N. Zhang, X. Han, Z. Liu, and M. Sun, "RAGEval: Scenario specific RAG evaluation dataset generation framework," in Association for Computational Linguistics, Vienna, Austria, 2025.
  78. G. Xiong, Q. Jin, Z. Lu, and A. Zhang, "Benchmarking retrieval- augmented generation for medicine," in Association for Computational Linguistics, Bangkok, Thailand, 2024.
  79. R. Friel, M. Belyi, and A. Sanyal, "RAGBench: Explainable benchmark for retrieval-augmented generation systems," 2025. [Online]. Available: https://arxiv.org/abs/2407.11005 · arXiv
  80. Y. Ming, S. Purushwalkam, S. Pandit, Z. Ke, X.-P. Nguyen, C. Xiong, and S. Joty, "Faitheval: Can your language model stay faithful to context, even if "the moon is made of marshmallows"," 2024. [Online]. Available: https://arxiv.org/abs/2410.03727 · arXiv
  81. A. Costarelli, M. Allen, R. Hauksson, G. Sodunke, S. Hariharan, C. Cheng, W. Li, and A. Yadav, "GAMEBENCH: Evaluating strategic reasoning abilities of LLM agents," arXiv, 2024. [Online]. Available: https://arxiv.org/html/2406.06613v1#S1
  82. J. Chen, H. Lin, X. Han, and L. Sun, "Benchmarking large language models in retrieval-augmented generation," 2023. [Online]. Available: https://arxiv.org/abs/2309.01431 · arXiv
  83. Y. Chen et al., "Draco: Diagnostic reasoning for comprehensive agent evaluation," arXiv preprint arXiv:2403.XXXXX, 2024. [Online]. Available: https://arxiv.org/abs/2403.XXXXX
  84. H. Wang et al., "Cl-bench: A contamination-aware context learning benchmark for rag," arXiv preprint arXiv:2406.XXXXX, 2024. [Online]. Available: https://arxiv.org/abs/2406.XXXXX
  85. Z. Zhao, Y. Dong, A. Liu, L. Zheng, D. Yin et al., "Tura: Tool- augmented unified retrieval agent for ai search," arXiv preprint arXiv:2508.04604, 2025. · arXiv
  86. B. Bhusal, M. Acharya, R. Kaur, C. Samplawski, A. Roy, A. D. Cobb, R. Chadha, and S. Jha, "Privacy preserving in-context- learning framework for large language models," arXiv preprint arXiv:2509.13625, 2025. [Online]. Available: https://arxiv.org/abs/ 2509.13625 · arXiv
  87. S. Mishra and H. Reza, "A face recognition method using deep learning to identify mask and unmask objects," in 2022 IEEE World AI IoT Congress (AIIoT), 2022, pp. 091-099.
  88. M. D. Skarlinski, S. Cox, J. M. Laurent, J. D. Braza, M. M. Hinks, M. J. Hammerling, M. Ponnapati, S. G. Rodriques, and A. D. White, "Language agents achieve superhuman synthesis of scientific knowl- edge," arXiv preprint arXiv:2409.13740, 2024. · arXiv
  89. M. Alenezi, "From prompt-response to goal-directed systems: The evolution of agentic ai software architecture," arXiv preprint arXiv:2602.10479, 2026. · arXiv
  90. S. Gaire, S. Gyawali, S. Mishra, S. Niroula, D. Thakur, and U. Yadav, "Systematization of knowledge: Security and safety in the model context protocol ecosystem," arXiv preprint arXiv:2512.08290, 2025. [Online]. Available: https://arxiv.org/abs/2512.08290 · arXiv
  91. V. Magesh, F. Surani, M. Dahl, M. Suzgun, C. D. Manning, and D. E. Ho, "Hallucination-free? assessing the reliability of leading AI legal research tools," Journal of Empirical Legal Studies, 2025. [Online]. Available: https://dho.stanford.edu/wp-content/uploads/ Legal RAG Hallucinations.pdf
  92. U. Yadav, S. Niroula, G. K. Gupta, and B. Yadav, "Exploring secure machine learning through payload injection and fgsm attacks on resnet- 50," in 2025 IEEE Silicon Valley Cybersecurity Conference (SVCC), 2025.
  93. K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, "Not what you've signed up for: Compromising real- world LLM-integrated applications with indirect prompt injection," arXiv preprint arXiv:2302.12173, 2023. [Online]. Available: https: //arxiv.org/abs/2302.12173 · arXiv
  94. W. Zou, R. Geng, B. Wang, and J. Jia, "PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models," in USENIX Security Symposium, 2025. [Online]. Available: https://arxiv.org/abs/2402.07867 · arXiv
  95. D. Pasquini et al., "Securing AI agents against prompt injection attacks," arXiv preprint arXiv:2511.15759, 2024. [Online]. Available: https://arxiv.org/abs/2511.15759 · arXiv
  96. S. Cohen, R. Bitton, and B. Nassi, "Here comes the AI worm: Unleashing zero-click worms that target GenAI-powered applications," arXiv preprint arXiv:2403.02817, 2024. [Online]. Available: https://arxiv.org/abs/2403.02817 · arXiv
  97. L. Zheng, W.-L. Chiang, Y. Sheng, S. Hao, Z. Wu, S. Ba, E. Zhuang, Y. Lin, Z. Li, D. Weng, X. Xing, J. E. Gonzalez, I. Stoica, and E. P. Xing, "Judging LLM-as-a-judge with MT-Bench and chatbot arena," in Advances in Neural Information Processing Systems, vol. 36, 2023.
  98. G. K. Gupta, P. Pande, N. Acharya, A. K. Singh, and S. Niroula, "Llms in disease diagnosis: A comparative study of deepseek-r1 and o3 mini across chronic health conditions," arXiv preprint arXiv:2503.10486, 2025. [Online]. Available: https://arxiv.org/abs/2503.10486 · arXiv
  99. K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, "More than you've asked for: A comprehensive analysis of novel prompt injection threats to application-integrated large language models," in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023.
  100. O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts, "DSPy: Compiling declarative language model calls into state-of-the-art pipelines," in International Conference on Learning Representations (ICLR), 2024.

Cited by

Sorted by citations

Related

Authors
Mishra, S., Niroula, S., Yadav, U., Thakur, D., Gyawali, S., & Gaire, S.
Affiliations
  • University of North Dakota (US)
  • Youngstown State University (US)
  • University of Toledo (US)
  • University of Missouri (US)
  • Tribhuvan University (NP)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2603.07379
arXiv
2603.07379
OpenAlex
W7134814111
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
agentic retrieval-augmented generation, large language models, multi-step reasoning, partially observable Markov decision processes, memory management, tool invocation, memory poisoning, autonomous decision-making, oversight mechanisms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review