The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey

“Solutions that use multiple agent personas to solve problems also show significant improvements compared to Chain-of-Thought (CoT) prompting where the model is asked to break down its plans step by step [28, 29].”

“Language Models have been known to exhibit bias both in terms of evaluation as well as in social or fairness terms [5].”

“For example, Plan Like a Graph (PLaG) is an approach that represents plans as directed graphs, with multiple steps being executed in parallel [15, 33].”

“Reflexion is a single-agent pattern that uses self-reflection through linguistic feedback [23].”

“In this paper, we follow the definition of agent that consists of “brain, perception, and action” [31].”

“We find that successful goal execution by agents is contingent upon proper planning and self-correction [32, 16, 23, 1].”

“Many language model benchmarks are designed to be solved in a single iteration, with no tool calls, such as MMLU or GSM8K [3, 10].”

“Many multi-agent architectures work in stages where teams of agents are created and reorganized dynamically for each planning, execution, and evaluation phase [2, 9, 18].”

“[6], [17] propose general agent frameworks and classify existing architectures.”

“Following prior work (Wei et al., 2023; Yao et al., 2023; Masterman et al., 2024), the model is instructed to think through whether any tools are relevant, then output each available tool within its response along with a YES or NO determination.”

“In these works, LLM s are often used to help an agent plan its actions [4], [13], [14], despite evidence that they may not be capable of reliably planning [15].”

“This is operationalized as either a single agent or as part of a multi-agent architecture in which specialized sub-agents handle distinct sub-tasks [71].”

“We also discuss its application for foundation model-based AI agents , given their widespread adoption in this area [69].”

“Advances in multimodal large language models [37] have markedly improved interaction competence, task understanding, and coverage of application ecosystems, indicating strong potential for intelligent interaction and everyday automation [22].”

“This architecture is evolving toward multi-agent systems 58 , where multiple domain-specific LLM-powered agents collaborate on complex tasks.”

“We will also explore the main Agents Design Patterns applied in industry to boost their effectiveness [3] and their respective trade-offs related to scalability and communication bottlenecks, computational efficiency.”

“Recent advances in large language models (LLMs) have fundamentally reshaped this paradigm, enabling agentic systems that are far more fl exible and context-aware [23, 27].”

“In November 2024, Accenture reported that one in three companies were adopting agentic AI systems to “ enhance productivity, decision-making, and operational efficiency ” by delegating real-world tasks to autonomous agents (11).”

“This has motivated the development of meth-ods in which an LLM invokes tools and external computation engines for deriving information from structured data (Schick et al., 2024; Masterman et al., 2024).”

“So, such systems are best used in applications that need complex workflows and applications that need to solve problems in real time [4].”

“High-privilege LLM agents deployed in software installation workflows operate in elevated privilege contexts, granted terminal access, filesystem control, and outbound network connectivity [1], [2].”

“Tool-calling based agent system design has become prevalent in both general-purpose and domain-specific applications [22, 60], driven by the capabilities of large language models (LLMs) to perform complex tasks through tool utilization [25, 30, 58].”

“Recent advances in large language models (LLMs) have fundamentally reshaped this paradigm, enabling agentic systems that are far more flexible and context-aware [21], [22].”

“Just as CoT prompts provide a rough outline for reasoning steps, the plan phase outlines actionable steps guiding an agent iteratively through execution, a critical factor for successful goal completion [22].”

“Agentic RAG extends the RAG paradigm by enabling models to plan, select, and invoke tools dynamically in response to user queries [Masterman et al., 2024].”

“Or should we also develop their understanding of company processes, teamwork skills, project execution capabilities, and interpersonal communication skills?”

“Recently, there have also been efforts to explore collaboration between LLMs to enhance their reasoning abilities (Zhang et al., 2025b; Putta et al., 2024; Masterman et al., 2024; Yan et al., 2025b; Chu et al., 2025).”

“…al., 2024b; Nakano et al., 2021) or calculators (Schick et al., 2023; Gao et al., 2023), Figure 1: A Visualization of Single and Multi-agent Architectures (Masterman et al., 2024). which enable them to act as intelligent agents using tools and enhance their ability to solve specific complex tasks.”

“Agentic AI systems increasingly act on behalf of users by invoking tools and composing API calls to achieve user specified goals [1, 2, 3, 4].”

“Besides reasoning and fact-checking, questioning plays a major role in many other areas (Masterman et al. 2024), like encouraging creativity (Wang et al. 2024a), stimulating discussion, and driving innovation (Si, Yang, and Hashimoto 2024; Ghafarollahi and Buehler 2024).”

“These advancements enable the construction of LLM-based Agent systems designed to tackle increasingly complex tasks [15, 8, 34].”

“A long line of work (Maes, 1994; Wooldridge & Jennings, 1995; Masterman et al., 2024) has proposed general architectures and conceptual frameworks for organizing intelligent agents.”

“However, such systems exhibit clear limitations when applied to tasks requiring deep domain expertise [50, 51].”

“In modern agent systems, the LLMs serve as the ‘brain; for reasoning and decision making, and are equipped with external tools to extend their capabilities [24].”

“Prior surveys of agent architectures (Masterman et al. 2024) reveal the diversity of reasoning, planning, and tool use, yet current observability approaches lack unified models to analyze vulnerabilities across these architectures.”

“Consequently, “capability-first” pipelines often achieve strong task performance yet degrade interaction quality due to silence, interruptions, and inconsistent persona, whereas “responsiveness-first” conversational setups preserve fluency but struggle with reliable tool use and long-horizon task…”

“Unlike conventional rule-based systems, Agentic AI exhibits deep contextual reasoning, self-directed planning, and robust adaptability, enabling it to operate effectively in dynamic and uncertain environments [14].”

“The output generated by the execution of the external tool is subsequently fed back to the LLM to be incorporated into the final response of the LLM, thereby creating a dynamic and iterative problem-solving loop (Masterman et al., 2024).”

“The advent of content-aware, context-specific Large Language Model (LLM) agents made intelligent interactions and automated management of sub-tasks possible [15, 16, 17].”

“A prominent research direction involves leveraging LLMs to automate tasks by deploying agents to operate on the user’s behalf [41, 67, 70].”

“A central capability of such agents is effective tool use, where they invoke external tools to solve diverse tasks (Masterman et al., 2024; Xu et al., 2025).”

“LLM [Xie et al., 2025, 2026] agents interleave reasoning, tool use, and environment feedback across turns [Yao et al., 2023, Masterman et al., 2024, Zhang et al., 2025b].”

“P3 (Sequential workflow with optional loops): Scaling collaborating agents requires careful orchestration [15] and a communication strategy that sustains rising volume [9], [11], [20].”

“Today the choice is forced by the framework rather than made by the engineer, and the design patterns literature records the same split without resolving it [31, 17, 34].”

“Large language models (LLMs) and LLM-based agents have rapidly expanded the range of tasks that can be handled with natural-language interfaces, including question answering, planning, tool use, and interactive knowledge work [1, 2, 3].”

“This pattern has enabled powerful tool-using systems [9, 19], but it treats the hard part of agent integration as endpoint access.”

“Agentic systems vary in architecture, including single-agent and multi-agent designs with different role structures and coordination patterns, which may influence where failures appear [45].”

“In fully dynamic systems, agent roles, relationships, and even the number of participating agents can evolve over time [1, 2, 86].”

“An agentic LLM system composes four functional stages in a recursive loop (Yao et al., 2022; Masterman et al., 2024): a perception function P that processes external inputs, a reasoning (behavior) function B that plans actions using the current input and persistent memory m t , an execution…”

“An open-source, extensible sys-tem for controlled quantitative framework comparison—unlike qualitative surveys [12] or failure taxonomies [5].”

“Within the harness, LLM-based reasoning agents are equipped with tool-use capabilities (Masterman et al., 2024) and translate intents, plan multi-step workflows, select tools, and esca-late uncertain cases to human operators.”

“Unlike supervised fine-tuning approaches (Zhu et al., 2025; Masterman et al., 2024; Liu et al., 2024; Lin et al., 2024) that suffer from "parametric rigidity," SGA-MCTS adopts a non-parametric, training-free paradigm via retrievable experience, enabling zero-shot adaptation without models’…”

“Unlike traditional software with fixed functionality, LLM-based agents dynamically execute actions through reasoning [16, 28], creating authorization challenges that fundamentally differ from those of conventional mobile or web applications [35].”

“More recent work [Masterman et al., 2024] further classifies agents by interaction topology, such as chain, star, mesh, and explicit work-flow graphs, highlighting the growing diversity of multi-agent designs.”

“Tool Augmentation The practice of augmenting AI agents with external functions for specialized tasks has emerged as an effective means of extending agent capabilities beyond pure language generation [20].”

“These agents are increasingly adopted in domains such as enterprise automation, customer service, and software engineering (Masterman et al. 2024).”

“The agent’s intelligence is an emergent property of these components working in synergy: the brain plans, calls a tool, perceives the result, updates its memory, and repeats the cycle until the goal is achieved [31].”

“In contrast to the ongoing practice of a firm posing a series of “prompting questions” based on “prompting engineering frame-works” (Masterman, Besen, Sawtell & Chao, (2024), Short, & Short (2023), the above proposed framework is a simple heuristic based on a thoroughly thought-out, logically-and…”

“FinanceQA [19], a dataset released in January 2025, is the benchmark selected for evaluation in this study.”

“…Large Language Models (LLMs), catalyzed by foundational innovations like the Transformer architecture (Vaswani et al., 2017), has ushered in the ‘Age of the Agent’—a new paradigm in software where autonomous systems can reason, plan, and act to achieve complex goals (Masterman et al., 2024).”

“Consequently, a successful robotic agent must integrate visual perception, task-level planning, and low-level motor execution in a way that aligns with the interactive, context-dependent nature of embodied cognition [7], [8].”

“Techniques such as Chain-of-Thought (CoT) prompting [1] and agentic reasoning architectures [2] have enabled LLMs to perform complex reasoning tasks, including mathematical problem solving.”

“Today’s AI using Large Language Models is increasingly adopting Agentic Architectures (Masterman et al., 2024).”

“These rationales are appealing since they are interpretable and can provide auxiliary supervision to improve task performance (Wang et al., 2022, 2025; Masterman et al., 2024).”

“Complex workflows, such as the US Open 2024 match reports, are built on top of agentic architectures where multiple agents work together on a common task [6, 7].”

“Besides reasoning and fact-checking, questioning plays a major role in many other areas (Masterman et al. 2024), like encouraging creativity (Wang et al. 2024a), stimulating discussion, and driving innovation (Si, Yang, and Hashimoto 2024; Ghafarollahi and Buehler 2024).”

“…with external context from retrieved documents or tool-calling feedback, LLMs can perform complex downstream tasks such as reasoning Giadikiaroglou et al. (2024); Chen et al. (2025), question answering Kamalloo et al. (2023); Yue (2025), and tool calling Li (2025); Masterman et al. (2024).”

“Research on LLM-based agents has gained significant prominence in recent times due to the impressive reasoning performance of LLMs (Cheng et al., 2024; Liu et al., 2024; Masterman et al., 2024; Li et al., 2023; Crispino et al., 2023; Xi et al., 2025; Fırat and Kuleli, 2023).”

“Agentic software powered by large language models (LLMs) [6, 23, 36] (i.e., Agentware [19]) is increasingly being adopted across a variety of domains such as autonomous software engineering [26, 51], customer support [18], and data analytics [17].”

“Surveys of LLM agents highlight mechanisms such as task decomposition, plan revision, and tool routing (Huang et al., 2024; Masterman et al., 2024).”

“These capabilities have motivated agent-based generative systems that coordinate specialized tools for complex generation and editing tasks (Durante et al. 2024; Masterman et al. 2024).”

“Masterman et al. and Zhou et al. classify agent systems as single-versus multi-agent and hierarchical versus decentralized architecture, but draw their examples from research prototypes [7], [8].”

“In multi-agent systems, the classification criteria include agent roles, hierarchy, communication protocols, task distribution, and coordination mechanisms [47–49].”

“Recent advances in autonomous AI agents [3, 16, 4]—systems capable of multi-step reasoning, iterative self-correction, and dynamic tool use—offer a transformative opportunity to harmonize bioactivity metadata retrospectively from primary literature at scale.”

“Some agents also contain a memory component that allows them to save and load information outside their messages and prompts (Masterman et al., 2024).”

“LLM-based agents can retrieve external data and use it in iterative test log analysis, where evidence may be distributed across several devices [15].”

“Recent studies on LLM-based agents describe reasoning, planning, memory, tool calling, and autonomous task execution as central parts of agentic AI systems [1], [2].”

“…has evolved to support elastic, scalable, and composable services across cloud and edge environments [24], [28], the emergence of AI agents introduces new opportunities and challenges for goal-driven, self-choreography of business processes to solve real-world tasks under uncertainty [29].”

“Single-agent systems Single-agent systems consist of one LLM instance that handles all reasoning, decision making, and execution towards a goal [82].”

“Such systems initiate and execute multi-step workflows with limited human involvement (Kolt, 2024; Masterman et al., 2024; Schreyer et al., 2024).”

“Ut over å etab le re en ty de lig grup pe le der, har ty de li ge re rol le de fi ni sjo ner og rol le spe sia li se ring også blitt fore slått som må ter å for bed re LLM­agen ters team­ pre sta sjo ner og re du se re kom plek si tet på (Masterman et al., 2024).”

“9 RAG - processing and chunking the documents into embeddings According to [19], "In the context of AI agents, ’tools’ represent any functions that the model can call.”