CoChat

From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review

Ferrag, Mohamed Amine, Tihanyi, Norbert, Debbah, Mérouane · 2026 · IEEE Access
Cited by 50

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2026.3698694

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 38 references

Research works printed but not linked: 204

  1. Jaech, 2024, OpenAI o1 system card, arXiv:2412.16720 · arXiv
  2. Xu, 2025, Qwen2.5-omni technical report
  3. Guo, 2025, DeepSeek-r1: Incentivizing reasoning capability in LLMs via reinforcement learning, arXiv:2501.12948 · arXiv
  4. Grattafiori, 2024, The llama 3 herd of models, arXiv:2407.21783 · arXiv
  5. Huang, 2024, Understanding the planning of LLM agents: A survey, arXiv:2402.02716 · arXiv
  6. 10.1145/3733799.3762964 · DOI
  7. 10.18653/v1/2025.emnlp-main.464 · DOI
  8. Li, 2024, Benchmarking multimodal retrieval augmented generation with dynamic VQA dataset and self-adaptive planning agent, arXiv:2411.02937 · arXiv
  9. Qing Yu, 2025, RAG-KG-IL: A multi-agent hybrid framework for reducing hallucinations and enhancing LLM reasoning through RAG and incremental knowledge graph learning integration, arXiv:2503.13514 · arXiv
  10. Ateia, 2024, BioRAGent: A retrieval-augmented generation system for showcasing generative query expansion and domain-specific search for scientific Q&A, arXiv:2412.12358 · arXiv
  11. 10.1109/GCCE65946.2025.11275306 · DOI
  12. Xiong, 2025, RAG-gym: Optimizing reasoning and search agents with process supervision, arXiv:2502.13957 · arXiv
  13. 10.1016/j.icte.2025.09.003 · DOI
  14. Achiam, 2023, GPT-4 technical report
  15. Team, 2023, Gemini: A family of highly capable multimodal models, arXiv:2312.11805 · arXiv
  16. 10.18653/v1/2025.findings-emnlp.320 · DOI · Paper page
  17. 10.18653/v1/2024.emnlp-main.840 · DOI · Paper page
  18. 10.18653/v1/2025.findings-emnlp.303 · DOI
  19. 10.18653/v1/2025.naacl-long.342 · DOI · Paper page
  20. Gridach, 2025, Agentic AI for scientific discovery: A survey of progress, challenges, and future directions, arXiv:2503.08979 · arXiv
  21. 10.18653/v1/2024.findings-acl.557 · DOI
  22. Li, 2024, Agent hospital: A simulacrum of hospital with evolvable medical agents, arXiv:2405.02957 · arXiv
  23. Durante, 2024, Agent AI: Surveying the horizons of multimodal interaction, arXiv:2401.03568 · arXiv
  24. 10.18653/v1/2025.findings-naacl.205 · DOI
  25. 10.18653/v1/2024.acl-long.371 · DOI
  26. Mukherjee, 2024, Polaris: A safety-focused LLM constellation architecture for healthcare, arXiv:2403.13313 · arXiv
  27. 10.18653/v1/2024.findings-emnlp.79 · DOI · Paper page
  28. Yan, 2025, The application of large language models in primary healthcare services and the challenges, Chin. Gen. Pract., 28, 1
  29. Yu, 2024, AIPatient: Simulating patients with EHRS and LLM powered agentic workflow, arXiv:2409.18924 · arXiv
  30. 10.18653/v1/2025.findings-acl.539 · DOI · Paper page
  31. Wang, Executable code actions elicit better LLM agents, 50208
  32. 10.52202/075280-0377 · doi:10.52202/075280-0377 · Paper page
  33. Yao, REACT: Synergizing reasoning and acting in language models
  34. Su, 2025, Learn-by-interact: A data-centric framework for self-adaptive agents in realistic environments, arXiv:2501.10893 · arXiv
  35. Hu, 2024, AgentGen: Enhancing planning abilities for large language model based agent via environment and task generation, arXiv:2408.00764 · arXiv
  36. 10.18653/v1/2024.findings-acl.181 · DOI · Paper page
  37. Gulcehre, 2023, Reinforced self-training (ReST) for language modeling, arXiv:2308.08998 · arXiv
  38. Aksitov, 2023, ReST meets ReAct: Self-improvement for multi-step reasoning LLM agent, arXiv:2312.10003 · arXiv
  39. Goldie, 2025, Synthetic data generation & multi-step RL for reasoning & tool use, arXiv:2504.04736 · arXiv
  40. Hong, 2023, MetaGPT: Meta programming for multi-agent collaborative framework, arXiv:2308.00352 · arXiv
  41. 10.1109/ICRA57147.2024.10610855 · DOI · Paper page
  42. 10.1145/3586183.3606763 · DOI · Paper page
  43. Wang, 2023, Avalon’s game of thoughts: Battle against deception through recursive contemplation, arXiv:2310.01320 · arXiv
  44. Wang, 2024, Agents in software engineering: Survey, landscape, and vision, arXiv:2409.09030 · arXiv
  45. Jin, 2024, From LLMs to LLM-based agents for software engineering: A survey of current, challenges and future, arXiv:2408.02479 · arXiv
  46. Yehudai, 2025, Survey on evaluation of LLM-based agents, arXiv:2503.16416 · arXiv
  47. Yan, 2025, Beyond self-talk: A communication-centric survey of LLM-based multi-agent systems, arXiv:2502.14321 · arXiv
  48. Feng, 2024, A survey on large language model-based social agents in game-theoretic scenarios, arXiv:2412.03920 · arXiv
  49. Zhang, 2024, Large language model-brained GUI agents: A survey, arXiv:2411.18279 · arXiv
  50. Li, 2024, Personal LLM agents: Insights and survey about the capability, efficiency and security, arXiv:2401.05459 · arXiv
  51. Wang, 2025, EnigmaEval: A benchmark of long multimodal reasoning challenges, arXiv:2502.08859 · arXiv
  52. Hendrycks, 2020, Measuring massive multitask language understanding, arXiv:2009.03300 · arXiv
  53. Zhong, 2025, ComplexFuncBench: Exploring multi-step and constrained function calling under long-context scenario, arXiv:2501.10132 · arXiv
  54. 2023, Facts & Grounding: A New Benchmark for Evaluating the Factuality of Large Language Models
  55. Zheng, 2024, ProcessBench: Identifying process errors in mathematical reasoning, arXiv:2412.06559 · arXiv
  56. Zhuge, 2024, Agent-as-a-judge: Evaluate agents with agents, arXiv:2410.10934 · arXiv
  57. 2024, Introducing Simpleqa
  58. 2024, Fine Tasks
  59. 10.18653/v1/2025.naacl-long.243 · DOI
  60. 2025, Dabstep
  61. Mao, 2024, BFCL V2 Live
  62. Miserendino, 2025, SWE-lancer: Can frontier LLMs earn $1 million from real world freelance software engineering?, arXiv:2502.12115 · arXiv
  63. Yang, 2024, CRAG—Comprehensive RAG benchmark, arXiv:2406.04744 · arXiv
  64. Kouremetis, 2025, OCCULT: Evaluating large language models for offensive cyber operation capabilities, arXiv:2502.15797 · arXiv
  65. 10.1109/BigData62323.2024.10825051 · DOI
  66. 10.1109/CSR61664.2024.10679494 · DOI
  67. 10.18653/v1/2025.acl-long.421 · DOI · Paper page
  68. Mialon, GAIA: A benchmark for general AI assistants, 9025
  69. Dubniczky, 2025, CASTLE: Benchmarking dataset for static code analyzers and LLMs towards CWE detection, arXiv:2503.09433 · arXiv
  70. Yao, SPIN-bench: How well do LLMs plan strategically and reason socially?, arXiv:2503.12349 · arXiv
  71. Yao, 2024, A benchmark for tool-agent-user interaction in real-world domains, arXiv:2406.12045 · arXiv
  72. Hendrycks, 2021, Measuring mathematical problem solving with the math dataset, arXiv:2103.03874 · arXiv
  73. Chen, 2021, Evaluating large language models trained on code, arXiv:2107.03374 · arXiv
  74. Shi, 2022, Language models are multilingual chain-of-thought reasoners, arXiv:2210.03057 · arXiv
  75. 10.18653/v1/2025.findings-emnlp.368 · DOI
  76. Ye, 2024, MIRAI: Evaluating LLM agents for event forecasting, arXiv:2407.01231 · arXiv
  77. 10.18653/v1/2024.acl-long.850 · DOI · Paper page
  78. Liu, 2024, VisualAgentBench: Towards large multimodal models as visual foundation agents, arXiv:2408.06327 · arXiv
  79. Chen, 2024, ScienceAgentBench: Toward rigorous assessment of language agents for data-driven scientific discovery, arXiv:2410.05080 · arXiv
  80. Majumder, 2024, DiscoveryBench: Towards data-driven discovery with large language models, arXiv:2407.01725 · arXiv
  81. 10.18653/v1/2024.findings-emnlp.815 · DOI
  82. Liu, 2024, Medchain: Bridging the gap between LLM agents and clinical practice through interactive sequential benchmarking, arXiv:2412.01605 · arXiv
  83. Long, 2024, TeamCraft: A benchmark for multi-modal multi-agent systems in minecraft, arXiv:2412.05255 · arXiv
  84. 10.18653/v1/2025.acl-long.116 · DOI
  85. Rein, GPQA: A graduate-level Google-proof Q&A benchmark
  86. Tang, 2025, MedAgentsBench: Benchmarking thinking models and agent frameworks for complex medical reasoning, arXiv:2503.07459 · arXiv
  87. Cheng, 2025, EmbodiedEval: Evaluate multimodal LLMs as embodied agents, arXiv:2501.11858 · arXiv
  88. 10.52202/079017-0607 · DOI
  89. Xiang, 2025, SciReplicate-bench: Benchmarking LLMs in agent-driven algorithmic reproduction from research papers, arXiv:2504.00255 · arXiv
  90. Fish, 2025, EconEvals: Benchmarks and litmus tests for LLM agents in unknown environments, arXiv:2503.18825 · arXiv
  91. Sung, 2025, VeriLA: A human-centered evaluation framework for interpretable verification of LLM agent failures, arXiv:2503.12651 · arXiv
  92. Li, 2025, AgentOrca: A dual-system framework to evaluate language agents on operational routine and constraint adherence, arXiv:2503.08669 · arXiv
  93. 10.18653/v1/2025.findings-acl.1036 · DOI
  94. Gautam, 2025, RefactorBench: Evaluating stateful reasoning in language agents through code, arXiv:2503.07832 · arXiv
  95. Gonzalez-Pumariega, 2025, Robotouille: An asynchronous planning benchmark for LLM agents, arXiv:2502.05227 · arXiv
  96. Tang, 2025, DSGBench: A diverse strategic game benchmark for evaluating LLM-based agents in complex decision-making environments, arXiv:2503.06047 · arXiv
  97. Ku, 2025, TheoremExplainAgent: Towards multimodal explanations for LLM theorem understanding, arXiv:2502.19400 · arXiv
  98. 10.1109/TASLPRO.2025.3626926 · DOI
  99. Zhang, 2025, DataSciBench: An LLM agent benchmark for data science, arXiv:2502.13897 · arXiv
  100. Wei, 2025, BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
  101. Chan, 2025, MLE-bench: Evaluating machine learning agents on machine learning engineering, arXiv:2410.07095 · arXiv
  102. Rashid, 2025, SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents, arXiv:2504. 08703
  103. Zan, 2025, Multi-SWE-bench: A multilingual benchmark for issue resolving, arXiv:2504.02605 · arXiv
  104. Srivastava, 2022, Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, arXiv:2206.04615 · arXiv
  105. 10.18653/v1/2023.findings-acl.824 · DOI · Paper page
  106. 2025, Langchain Agents Tutorial
  107. 2025, Building a Basic Agent
  108. 2025, Crewai
  109. 2024, Swarm
  110. Wu, 2025, Agentic reasoning: Reasoning LLMs with tools for the deep research, arXiv:2502.04644 · arXiv
  111. Lu, 2025, OctoTools: An agentic framework with extensible tools for complex reasoning, arXiv:2502.11271 · arXiv
  112. 2025, Agents SDK
  113. Wang, 2024, StarWhisper telescope: Agent-based observation assistant system to approach AI astrophysicist, arXiv:2412. 06412
  114. 10.18653/v1/2024.findings-emnlp.192 · DOI
  115. Wang, 2024, GeneAgent: Self-verification language agent for gene set knowledge discovery using domain databases, arXiv:2405.16205 · arXiv
  116. 10.1038/s44387-025-00003-z · DOI
  117. Liang, 2025, SurveyX: Academic survey automation via large language models, arXiv:2502.14776 · arXiv
  118. Li, 2024, Chain of ideas: Revolutionizing research via novel idea development with LLM agents, arXiv:2410.13185 · arXiv
  119. Mitra, 2024, AgentInstruct: Toward generative teaching with agentic flows, arXiv:2407.03502 · arXiv
  120. Tang, 2025, ChemAgent: Self-updating library in large language models improves chemical reasoning, arXiv:2501.06590 · arXiv
  121. 10.1145/3701716.3717527 · DOI
  122. 10.1016/j.jss.2024.112278 · DOI · Paper page
  123. 2025, How to Design an Agent for Production
  124. Sun, 2024, OS-genesis: Automating GUI agent trajectory construction via reverse task synthesis, arXiv:2412.19723 · arXiv
  125. 10.18653/v1/2025.findings-acl.740 · DOI · Paper page
  126. Zhou, 2024, Zodiac: A cardiologist-level LLM framework for multi-agent diagnostics, arXiv:2410.02026 · arXiv
  127. Wang, 2025, MedAgent-pro: Towards multi-modal evidence-based medical diagnosis via reasoning agentic workflow, arXiv:2503.18968 · arXiv
  128. 10.1145/3706598.3714014 · DOI
  129. Feng, 2025, M³builder: A multi-agent system for automated machine learning in medical imaging, arXiv:2502.20301 · arXiv
  130. 10.18653/v1/2025.acl-long.677 · DOI · Paper page
  131. 10.1109/ICCV51701.2025.02175 · DOI
  132. 10.26615/978-954-452-106-6-001 · doi:10.26615/978-954-452-106-6-001
  133. 10.18653/v1/2025.acl-long.1024 · DOI
  134. Xu, 2025, AutoCBT: An autonomous multi-agent framework for cognitive behavioral therapy in psychological counseling, arXiv:2501.09426 · arXiv
  135. Lee, 2025, Psyche: A multi-faceted patient simulation framework for evaluation of psychiatric assessment conversational agents, arXiv:2501.01594 · arXiv
  136. Zhang, 2024, PsyDraw: A multi-agent multimodal system for mental health screening in left-behind children, arXiv:2412.14769 · arXiv
  137. 10.18653/v1/2025.acl-long.846 · DOI
  138. Wasenmüller, 2024, Script-based dialog policy planning for LLM-powered conversational agents: A basic architecture for an, arXiv:2412.15242 · arXiv
  139. 10.18653/v1/2025.emnlp-main.603 · DOI
  140. Wang, 2024, PatentAgent: Intelligent agent for automated pharmaceutical patent analysis, arXiv:2410.21312 · arXiv
  141. Inoue, 2024, DrugAgent: Explainable drug repurposing agent with large language model-based reasoning, arXiv:2408.13378 · arXiv
  142. Chen, 2025, MAP: Evaluation and multi-agent enhancement of large language models for inpatient pathways, arXiv:2503.13205 · arXiv
  143. 10.18653/v1/2025.findings-acl.729 · DOI
  144. Lin, 2024, BioKGBench: A knowledge graph checking benchmark of AI agent for biomedical science, arXiv:2407.00466 · arXiv
  145. Schmidgall, 2025, AgentRxiv: Towards collaborative autonomous research, arXiv:2503.18102 · arXiv
  146. 10.18653/v1/2025.findings-acl.1016 · DOI
  147. Dong, 2025, The Ann Arbor architecture for agent-oriented programming, arXiv:2502.09903 · arXiv
  148. Jain, 2025, R2E-gym: Procedural environments and hybrid verifiers for scaling open-weights SWE agents, arXiv:2504.07164 · arXiv
  149. 10.18653/v1/2025.findings-acl.642 · DOI
  150. Chen, 2025, Verbal process supervision elicits better coding agents, arXiv:2503.18494 · arXiv
  151. 10.18653/v1/2025.acl-long.973 · DOI
  152. 10.18653/v1/2025.acl-long.426 · DOI
  153. Gholamzadeh Khoee, 2025, GateLens: A reasoning-enhanced LLM agent for automotive software release analytics, arXiv:2503
  154. Hu, 2025, An LLM-based agent for reliable Docker environment configuration, arXiv:2502.13681 · arXiv
  155. 10.1145/3706599.3719729 · DOI
  156. Pan, 2024, Training software engineering agents and verifiers with SWE-gym, arXiv:2412.21139 · arXiv
  157. Yang, 2025, Multi-agent collaboration for multilingual code instruction tuning, arXiv:2502.07487 · arXiv
  158. Guo, 2025, SyncMind: Measuring agent out-of-sync recovery in collaborative software engineering, arXiv:2502.06994 · arXiv
  159. 10.18653/v1/2025.findings-naacl.285 · DOI
  160. Wan, 2024, Enhancing the efficiency and accuracy of underlying asset reviews in structured finance: The application of multi-agent framework, arXiv:2405.04294 · arXiv
  161. Yang, 2025, TwinMarket: A scalable behavioral and socialsimulation for financial markets, arXiv:2502.01506 · arXiv
  162. 10.52202/079017-4354 · DOI
  163. Lin, 2024, Strategic collusion of LLM agents: Market division in multi-commodity competitions, arXiv:2410.00031 · arXiv
  164. 10.1145/3677052.3698686 · DOI
  165. 10.1145/3677052.3698645 · DOI
  166. Han, 2025, FinSphere: A conversational stock analysis agent equipped with quantitative tools based on real-time database, arXiv:2501.12399 · arXiv
  167. 10.1109/ICDMW69685.2025.00105 · DOI
  168. Zeng, 2025, Cite before you speak: Enhancing context-response grounding in e-commerce conversational LLM-agents, arXiv:2503.04830 · arXiv
  169. 10.18653/v1/2025.emnlp-industry.191 · DOI
  170. 10.18653/v1/2025.findings-naacl.420 · DOI
  171. Lei, 2024, MACM: Utilizing a multi-agent system for condition mining in solving complex mathematical problems, arXiv:2404.04735 · arXiv
  172. Xie, 2024, MathLearner: A large language model agent framework for learning to solve mathematical problems, arXiv:2408.01779 · arXiv
  173. Lee, 2024, Expanding search space with diverse prompting agents: An efficient sampling approach for LLM mathematical reasoning, arXiv:2410.09780 · arXiv
  174. Deng, 2024, Flow-DPO: Improving LLM mathematical reasoning through online multi-agent learning, arXiv:2410.22304 · arXiv
  175. Li, 2025, Automating mathematical proof generation using large language model agents and knowledge graphs, arXiv:2503.11657 · arXiv
  176. Wang, 2025, MA-LoT: Multi-agent lean-based long chain-of-thought reasoning enhances formal theorem proving, arXiv:2503.03205 · arXiv
  177. Yue, 2024, MathVC: An LLM-simulated multi-character virtual classroom for mathematics education, arXiv:2404.06711 · arXiv
  178. Ma, 2025, LLM knows geometry better than algebra: Numerical understanding of LLM-based agents in a trading arena, arXiv:2502.17967 · arXiv
  179. Yu, 2024, MineAgent: Towards remote-sensing mineral exploration with multimodal large language models, arXiv:2412.17339 · arXiv
  180. 10.1080/17538947.2025.2458688 · DOI
  181. Xu, 2025, FilmAgent: A multi-agent framework for end-to-end film automation in virtual 3D spaces, arXiv:2501.12909 · arXiv
  182. Wang, 2024, AesopAgent: Agent-driven evolutionary system on story-to-video production, arXiv:2403.07952 · arXiv
  183. 10.18653/v1/2024.acl-long.88 · DOI
  184. Maronikolakis, 2024, What should I wear to a party in a Greek taverna? Evaluation for conversational agents in the fashion domain, arXiv:2408.08907 · arXiv
  185. Deng, 2024, ComposerX: Multi-agent symbolic music composition with LLMs, arXiv:2404.18081 · arXiv
  186. 10.18653/v1/2023.emnlp-demo.21 · doi:10.18653/v1/2023.emnlp-demo.21
  187. Zhang, 2024, LLM-based multi-agent poetry generation in non-cooperative environments, arXiv:2409.03659 · arXiv
  188. 10.1109/O-COCOSDA64382.2024.10800588 · DOI
  189. 2025, Introduction to MCP
  190. 2025, BEEAI Now Has Multiple Agents, and a Standardized Way for Them to Talk
  191. 2025, A2A: A New Era of Agent Interoperability
  192. 10.1145/3796519 · DOI · Paper page
  193. Yuan, 2025, NaturalReasoning: Reasoning in the wild with 2.8M challenging questions, arXiv:2502.13124 · arXiv
  194. Penedo, 2024, FINEWEB2: A Sparkling Update With 1000s of Languages
  195. 2024, MAGPIE Ultra V0.1
  196. Perrot, 2025, Implement Human-in-the-Loop Confirmation With Amazon Bedrock Agents
  197. Wang, 2025, Talk structurally, act hierarchically: A collaborative framework for LLM multi-agent systems, arXiv:2502.11098 · arXiv
  198. 10.18653/v1/2024.findings-naacl.112 · DOI
  199. Costello, 2025, Think, prune, train, improve: Scaling reasoning without scaling models, arXiv:2504.18116 · arXiv
  200. Xiang, 2025, Towards system 2 reasoning in LLMs: Learning how to think with meta chain-of-though, arXiv:2501.04682 · arXiv

200 of 204 shown

Cited by

Metadata

Authors
Ferrag, M. A., Tihanyi, N., & Debbah, M.
Affiliations
  • Department of Computer and Network Engineering, United Arab Emirates University, Al Ain, United Arab Emirates
  • Technology Innovation Institute, Abu Dhabi, United Arab Emirates
  • Research Institute for Digital Future, Khalifa University, Abu Dhabi, United Arab Emirates
Venue
IEEE Access
Published
2026
Volume
14
Pages
84237-84285
ISSN
2169-3536
Cited by
50
References
38
Funders
  • United Arab Emirates University (G00005769)
  • Khalifa University of Science and Technology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.