The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey

“Solutions that use multiple agent personas to solve problems also show significant improvements compared to Chain-of-Thought (CoT) prompting where the model is asked to break down its plans step by step [28, 29].”

“For example, Plan Like a Graph (PLaG) is an approach that represents plans as directed graphs, with multiple steps being executed in parallel [15, 33].”

“Reflexion is a single-agent pattern that uses self-reflection through linguistic feedback [23].”

“In this paper, we follow the definition of agent that consists of “brain, perception, and action” [31].”

“Language Models have been known to exhibit bias both in terms of evaluation as well as in social or fairness terms [5].”

“We find that successful goal execution by agents is contingent upon proper planning and self-correction [32, 16, 23, 1].”

“Many language model benchmarks are designed to be solved in a single iteration, with no tool calls, such as MMLU or GSM8K [3, 10].”

“Many multi-agent architectures work in stages where teams of agents are created and reorganized dynamically for each planning, execution, and evaluation phase [2, 9, 18].”

“Many language model benchmarks are designed to be solved in a single iteration, with no tool calls, such as MMLU or GSM8K [3, 10].”

“Language models also exhibit sycophantic behavior, where they “tend to mirror the user’s stance, even if it means forgoing the presentation of an impartial or balanced viewpoint” [20].”

“Crucially, Wang et. al finds that multi-agent patterns perform better than single agents in scenarios when no examples are provided [26].”

“StrategyQA improves upon this by assessing models’ reasoning abilities over multiple steps, but the answers are limited to Yes/No responses [7].”

“MetaGPT addresses the issue of unproductive chatter amongst agents by requiring agents to generate structured outputs like documents and diagrams instead of sharing unstructured chat messages [11].”

“Researchers have found that “shaped personality verifiably influences Large Language Model (LLM) behavior in common downstream (i.e. subsequent) tasks, such as writing social media posts” [21].”

“Emerging research indicates that there is significant data contamination in the model’s training data, supported by the observation that a model’s performance significantly worsens when benchmark questions are modified [8, 38, 37].”

“Researchers have also explored the idea of generating an entirely synthetic benchmark based on a user’s specific environment or use case [14, 27].”

“Benchmarks like AgentBench and SmartPlay introduce objective evaluation metrics designed to evaluate the implementation’s success rate, output similarity to human responses, and overall efficiency [17, 30].”

“This is partially because language models tend to commit to an answer earlier in their response, which can cause a ‘snowball effect’ of increasing diversion from their goal state [34] .”

“Solutions that use multiple agent personas to solve problems also show significant improvements compared to Chain-of-Thought (CoT) prompting where the model is asked to break down its plans step by step [28, 29].”

“Other works focused on advancing agent patterns highlight that while breaking a larger problem into smaller subproblems can be effective at solving complex tasks, single agent patterns often struggle to complete the long sequence required [22, 6].”

“For example, SWE-bench is a benchmark that uses a set of real-world issues raised on GitHub for software engineering tasks in Python [13].”

“Emerging research indicates that there is significant data contamination in the model’s training data, supported by the observation that a model’s performance significantly worsens when benchmark questions are modified [8, 38, 37].”

“As of today, including human validation and feedback in the agent architecture yields more reliable and trustworthy results [4, 9].”

“For example, Plan Like a Graph (PLaG) is an approach that represents plans as directed graphs, with multiple steps being executed in parallel [15, 33].”

“Planning, which requires strong reasoning abilities, commonly falls into one of five major approaches: task decomposition, multi-plan selection, external module-aided planning, reflection and refinement and memory-augmented planning [12].”

“Other research has found “a tendency for LLM agents to conform to the model’s inherent social biases despite being directed to debate from certain political perspectives” [24].”

“Beyond team structure, the paper emphasizes the importance of employing a “criticize-reflect” step for generating plans, evaluating performance, providing feedback, and re-organizing the team [9].”

“Horizontal architectures are generally used for tasks where collaboration, feedback and group discussion are key to the overall success of the task [2].”

“AUTOGPT + P. AutoGPT + P (Planning) is a method that addresses reasoning limitations for agents that command robots in natural language [1].”

“Emerging research indicates that there is significant data contamination in the model’s training data, supported by the observation that a model’s performance significantly worsens when benchmark questions are modified [8, 38, 37].”

“Researchers have also explored the idea of generating an entirely synthetic benchmark based on a user’s specific environment or use case [14, 27].”

“Language Agent Tree Search (LATS) is a single-agent method that synergizes planning, acting, and reasoning by using trees [36].”

“Moreover, agents have specifically been shown to be “less robust, prone to more harmful behaviors, and capable of generating stealthier content than LLMs, highlighting significant safety challenges” [25].”

“Figure 3: A diagram showing the RAISE method [16] specific and relevant feedback to the agent.”

“Benchmarks like AgentBench and SmartPlay introduce objective evaluation metrics designed to evaluate the implementation’s success rate, output similarity to human responses, and overall efficiency [17, 30].”

“Other works focused on advancing agent patterns highlight that while breaking a larger problem into smaller subproblems can be effective at solving complex tasks, single agent patterns often struggle to complete the long sequence required [22, 6].”

“This dual-axis taxonomy provides the unified lens necessary to rigorously analyze the field’s theoretical underpinnings, architectural innovations, and practical deployments.”

“The literature [105] summarized the current AI agent architecture.”

“…faculties, encompassing perception (Driess et al., 2023; Wang et al., 2024; Zheng et al., 2023; Wei et al., 2024), planning (Zhu et al., 2024; Erdogan et al., 2025; Huang et al., 2024), reasoning (Putta et al., 2024; Masterman et al., 2024), and action (Li et al., 2024; Yang et al., 2024).”

“An example of prompt parsing process of an instruction-response pair { ( I i , R i ) } .”

“…of artificial intelligence, LLM-driven agents have exhibited unprecedented prowess in perception [2, 3, 4, 5], planning [6, 7, 8], reasoning [9, 10], and action [11, 12], which have catalyzed remarkable progress across diverse downstream domains, including code generation [13, 14], data…”

“Recent surveys on LLM-based agents (Xi et al., 2025; Zhao et al., 2023; Masterman et al., 2024) have outlined a generalizable agent framework comprising three core components: (1) a reasoning engine driven by LLMs, (2) perceptual modules that process multimodal inputs, and (3) action mechanisms…”

“The successful execution of GUI Agents relies on two critical components: precise planning and robust verification mechanisms [33, 61, 162].”

“We use the multi-agent paradigm for the design of our agent system to allow for an easily extendable and modular system [80, 46, 26, 82, 70, 93, 81, 27, 43, 96, 45, 11, 90, 111, 67, 37, 39, 18, 5, 30].”

“These agent frameworks typically consist of several components, including reasoning, tool usage, memory, and multi-agent debate (Masterman et al., 2024), many of which can be seamlessly integrated into fact-checking pipelines to enhance the performance of traditional fact-checking systems.”

“Second, these systems heavily depend on a predefined set of tools or APIs and cannot access additional tools or functions on demand, limiting their ability to address unforeseen problems requiring operations beyond those anticipated by their designers [198].”

“These architectures range from simple single-agent setups, which work well for clearly defined tasks but struggle with flexibility, to more complex multi-agent systems that can scale better but require sophisticated coordination mechanisms [20], [21].”

“Recent progress in large language models (LLMs) has significantly changed the AI agent system, driving advances in automation and human-AI collaboration [3]–[9].”

“Despite these efforts, the development of o1-like multimodal systems is still in its stages (Chen et al., 2025c; Masterman et al., 2024; Xu et al., 2025), with significant problems such as overthinking (Cuadron et al., 2025; Yang et al., 2025a), safety (Chen et al., 2025a; Huo et al., 2025; Zhao et…”

“AI agents are entities powered by language models that can plan and perform actions across multiple steps to achieve a goal [29].”

“The emergence of LLMs introduces new paradigms for agents (Chu et al., 2023; Wang et al., 2024a; Masterman et al., 2024), demonstrating significant intelligence in reasoning (Kojima et al., 2022; Wei et al., 2023; Wang et al., 2023b), planning (Yao et al., 2023b,a; Besta et al., 2024),…”

“LLMs have demonstrated great capabilities in reasoning, tool usage, coding, multi-step planning and decision making (Masterman et al., 2024; Wu et al., 2023a).”

“LLM-Geo [21] has shown great potential in the study of autonomous GIS.”

“The profile module defines the agent ’ s role, personality, and social context through tailored prompts (Masterman et al., 2024; Wang et al., 2024a).”

“Masterman et al. (2024) reviewed emerging AI agent architectures, focusing on modularity and scalability, which are central to this framework’s refinement approach.”

“To make the study more challenging and account for the fact that surveys often contain more references than regular papers, we selected five surveys in the MAS field [61–65].”

“…tools like vector search engines, web searcher, computation and reasoning tools like calculators, code interpreters, knowledge and query tools like databases, Q&A systems, APIs and function calling Interfaces like OpenAI function calling, toolbox Interface (e.g., AutoGPT ToolManager) [11].”

“The communication structure of LLM-based multi-agent systems varies across different studies to address specific challenges [23, 34, 24].”

“Intermediate results are shared among agents to refine outputs, ensuring coherence and accuracy (Masterman et al., 2024 ).”

“Moreover, existing MAD frameworks lack the structured process of fact-checking, where claims are systematically examined through evidence collection, counterargument analysis, and multi-dimensional evaluation conducted by domain experts (Masterman et al., 2024; Slonim et al., 2021).”

“Frameworks like the Model-Controller-Program (MCP) approach (Krishnan, 2025; Hou et al., 2025; Ray, 2025) provide a structured way for agents to leverage external tools (Masterman et al., 2024), enabling them to interact with the environment and access specialized functionalities.”

“On the other hand, the few methods that do incorporate tool interaction [24] are heavily reliant on massive, proprietary models.”

“This design follows a “plan-and-execute” paradigm [9], [10], [11], in which the planner explicitly “thinks through” all required steps in advance and the executor focuses on carrying them out.”

“The development of effective LLM-based agents critically depends on high-quality training data that could capture the complexity of multi-step reasoning, tool usage, and environmental interaction (Yao et al., 2022b; Schick et al., 2023; Deng et al., 2023; Masterman et al., 2024).”

“Studies show that multi-agent systems with a leader complete tasks nearly 10% faster than those without [Masterman et al., 2024].”

“By keeping workflow logic minimal and delegating most cognitive tasks to LLMs and agents, the system remains adaptable to new models, new tools, and new runtime environments without requiring architectural rewrites [26].”

“Masterman et al. [8] provide an overview of single-agent and multi-agent architectures in industrial projects and present the insights and limitations of existing research.”

“(3) We outline future sociotechnical research directions, provide design implications to address developers’ immediate oversight challenges, and highlight broader implications for the software engineering practice.”

“…(Driess et al., 2023; Zheng et al., 2023; Wei et al., 2024), planning (Zhu et al., 2024; Erdogan et al., 2025), reasoning (Putta et al., 2024; Masterman et al., 2024), and action (Li et al., 2024; Yang et al., 2024), LLM-driven agentic AI systems have achieved remarkable performance across a…”

“While several recent surveys have addressed aspects of LLM-based agents [46, 50, 65, 69, 104, 122, 148], they fall short in several key areas.”

“Furthermore, agentic systems possess planning capabilities [77], [78], enabling them to decompose complex user goals or queries into a series of manageable subtasks.”

“Agentic software powered by large language models (LLMs) [6, 23, 36] (i.e., Agentware [19]) is increasingly being adopted across a variety of domains such as autonomous software engineering [26, 51], customer support [18], and data analytics [17].”

“The safety of AI behavior is particularly important as the Internet of Ev-erything becomes reality (Han, 2024), especially with recent developments in AI agents that can interact with the world through actions (Masterman et al., 2024).”

“AI agents address this gap by coupling a foundation model with an execution loop that can observe an environment, plan, call tools, update memory, and verify outcomes [10, 31].”

“Such isolated evaluations fail to capture the operational reality of modern agents, whose behaviors emerge from iterative workflows that tightly couple three functional stages: Planning for action orchestration, Memory for context retrieval, and Tools for environmental interaction [26, 33, 45].”

“The tools component enables the agent to interact with external systems, thereby performing operations that extend beyond text generation [57].”

“The power of these agents is greatly amplified by their ability to use external tools (Masterman et al. 2024).”

“However, REST APIs are typically not designed with LLM agents in mind, creating challenges for seamless integration due to complex input schema and verbose responses.”

“Beyond these intrinsic strengths,LLMscan serve asthe cognitivecoreof agenticsystemsthat integratememory, planning, toolkits, andexecutablework fl ows 27 .”

“Tools, which can be regarded as functions that take specific inputs and produce corresponding outputs through predefined operations, offer a promising solution by bridging this gap (Masterman et al., 2024; Yao et al., 2023).”

“Traditional AI systems in hospitality and tourism can be broadly classified as reflexive, goal-driven, or utility-based agents (Masterman et al., 2024; Wooldridge & Jennings, 1995).”

“Metrics such as scores (accuracy, precision, recall, and F1) and success rates across tasks, output similarity to human responses, memory persistence, efficiency (e.g., response time or resource usage) and cost are commonly used [52, 71].”

“LLM-based tool calling (generating text as parsable function calls) (Masterman et al., 2024; Qu et al., 2025) has emerged as a viable solution to real-world environment interaction, allowing them to solve a variety of complex tasks from mathematics (Das et al., 2024) to molecular structure…”

“LLMs, with their ability to process and generate natural language, can interpret user instructions, orchestrate tasks, and integrate domain-specific tools to perform specialized computations[45, 46].”

“Large-model AI agents—autonomous systems built around large-scale models such as LLMs—introduce qualitatively new security risks compared to both traditional AI systems and standalone language models [53], [67], [68].”

“Recently, large language model (LLM) powered AI agents have demonstrated remarkable capabilities and are increasingly adopted for real-world tasks (Xu et al., 2024; Masterman et al., 2024).”

“…can range from fairly simple prompting strategies (Arora et al., 2022; Wang et al., 2023), to highly complex control-flows that involve tool use (Masterman et al., 2024; Qin et al., 2024; Ruan et al., 2023), coding and code execution (Guo et al., 2024; Huang et al., 2023; Islam et al., 2024;…”

“In particular, multi-agent systems have demonstrated superior performance in scenarios requiring collaboration and multiple execution paths [10], making them well-suited for addressing the intricate and complex challenges in technical education, which normally requires nuanced solutions and…”

“These foundation-model-based agent systems have the potential to revolutionize industries by enhancing efficiencies, enabling automation, and facilitating complex decision-making processes [5].”

“Such permissions systems have traditionally been built on foundational security principles like least privilege (grant each entity only the minimum necessary permissions) and failing closed (when in doubt, deny permission) [47, 51].”

“As described in Algorithm 1, It first identifies public or external entry functions, then verifies the presence of access-control checks (e.g., Modifiers ), and finally retains variables relevant to authority (e.g., owner ).”

“Beyond the fundamental operation of agents, this module involves container scheduling (Xiao et al., 2024), hybrid on-chain and off-chain data storage (Tsang et al., 2024), invocation of various tools (Masterman et al., 2024), and generation of proofs (Čapko et al., 2022).”

“However, these agents were limited by their inability to plan or learn from past experiences, reducing their effectiveness in dynamic environments [13, 48].”

“Recent approaches have focused on enhancing LLMs by integrating external tools to improve their problem-solving capabilities (Masterman et al., 2024; Qu et al., 2024; Qian et al., 2023; Yuan et al., 2024a).”

“The 1990s introduced reactive agents [8], which responded to environmental stimuli without internal models.”

“• Inter-agent collaboration patterns: leader/planner, layered/nested,sequentialhierarchical,dynamic, chainofagents, etc. [9, 29, 35] • Orchestration patterns: planning, critic and/or reflection, group planning, and tool use [35, 44] 3.3.3 Agentic Frameworks.”

“…users during reasoning (Ma et al., 2024; Li et al., 2025; Guan et al., 2025), and (II) tool use, which allows models to invoke ex-ternal tools to support reasoning and substantially expands their capabilities beyond pure text generation (Wang et al., 2024; Masterman et al., 2024; Li et al., 2026a).”

“Reasoning-and-acting frameworks [2] and surveys of agent architectures and agentic programming [4, 43, 44] show that models can plan tool use across multi-step workflows, while work on long chain-of-thought and reasoning boundaries [19, 45, 46] studies when more inference-time reasoning helps or…”

“We have built agents capable of reasoning, planning and negotiation [66, 92, 62], but have largely studied them in terms of individual task completion or as independent units contributing to a collective objective.”

“Modern agent frameworks Corresponding author: Yutao Hu. commonly support goal decomposition, context management, long-term memory, external tool invocation, environment interaction, and multi-agent collaboration [2].”

“Large language models enable agentic systems (Masterman et al., 2024) that write and execute code, invoke external tools (Patil et al., 2025), perform multi-step reasoning (Kim et al., 2024; Aksitov et al., 2023), maintain memory (Hu et al., 2025; Kang et al., 2025), and interact with external…”

“Concurrently, using the LangGraph framework (29, 30), we designed a dynamic agentic workflow (31, 32).”

“To overcome this limit, LLMs are increasingly augmented with external tools that provide access to real-time information and domain-specific services, such as APIs, calculators, and Python interpreters [4]–[6].”

“…agen-tic workflows of today, where model planning is instrumental in addressing a myriad of complex tasks, including question answering (QA) (Mavi et al., 2022), embodied interaction (Yao et al., 2022), tool invocation (Masterman et al., 2024), and long-form text generation (Jiang et al., 2024).”

“The capabilities of LLMs have transcended simple single-round inference, evolving into sophisticated agentic work-flows that integrate interleaved reasoning with ex-ternal tool execution (Masterman et al., 2024; Plaat et al., 2025; Wang et al., 2024, 2025d; Wang et al.).”

“Language models are increasingly deployed as agents that plan, call tools, and interact with external systems over multiple steps (Wang et al., 2024; Masterman et al., 2024; Lumer et al., 2025).”

“While the definition of the properties that characterize AI agents has evolved (Masterman et al., 2024”

“By incorporating capabilities such as memory (Xu et al., 2025), tool usage (Masterman et al., 2024), and advanced planning (Huang et al., 2024), these agents can solve complex tasks in diverse domains.”

“Best practices to create such structures are still actively discussed and the approaches range from “quantity over quality” to elaborate architectures governing agent planning and cooperation [76, 77].”

“In our dataset, even all questions are known, asking all the questions simultaneously in a single-state setting may cause the agent to get overly focused on one question while overlooking others [62], [63].”

“LLM-based agents [8, 25] have emerged as a powerful paradigm capable of understanding complex contexts and generating human-level intelligent responses.”

“More recently, AI agents can permit more complex user interactions, using planning, loops, reflection, and other control devices [82].”

“Structure design User interaction Drag-and-drop workshop emphasizes enhancing agent performance through modules such as profile, memory, planning, and action [Masterman et al., 2024].”

“In this section, we describe the methodology for building LLM-based agents ontologies in specialized domains.”

“Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day” D.5: Shavit et al. (2023) 94 “Agentic AI systems are characterized by the ability to take…”

“Recent surveys identify reasoning, planning, tool use, memory, and reflection as core capabilities of such systems [45,77].”

“Surveys of software agents describe components (memory, tools, reasoning loop) without consolidating what constitutes the harness as a unit [17, 24, 29]; work on the agent-computer interface treats one facet, tools, and stops there [55]; context engineering, now a topic of its own, covers another…”

“Large language models (LLMs) have evolved from text generators into autonomous agents capable of executing consequential actions across systems [6], [10].”

“…2025; Cui et al., 2025a), web environments pose greater challenges due to their diverse, context-dependent action choices/function calling (e.g., click , type , scroll ) (Masterman et al., 2024; Qu et al., 2025; Cui et al., 2025b) and partial observability (Ning et al., 2025; Gao et al., 2025).”

“As detailed by Masterman et al. 28 , modern AI agent frameworks can be categorized into single-agent and multi-agent systems, with varying levels of autonomy and reasoning capabilities, incorporating planning, execution, and reflection phases to iteratively solve problems, often using complex reasoning loops and tool-…”

“[6], [17] propose general agent frameworks and classify existing architectures.”