ReAct: Synergizing Reasoning and Acting in Language Models

ReAct would require 16-20 LLM calls for this task, while MACLA reduces this to 2 calls, representing an >85% reduction in LLM inference overhead.

“ReAct would require 16-20 LLM calls for this task: reasoning before each action (8 actions × 2 calls/action for 'thought' and 'action'), plus initial planning and reflection. MACLA reduces this to 2 calls by retrieving learned procedures, representing an > 85% reduction in LLM inference overhead.”

LM programs typically contain multiple LLM calls interspersed with control flow.

“All of these new use cases require multiple, often dependent, LLM generation calls, showing a trend of using multi-call structures to complete complex tasks [57, 21].”

Recent work has equipped dialogue agents with the ability to use tools such as calculators and calendars, and to consult external websites.

“Recent work has equipped dialogue agents with the ability to use tools such as calculators and calendars, and to consult external websites 24,25 .”

A common approach in long-horizon LLM systems is to append all past observations, actions, and thoughts to the context at every turn.

“In systems designed for long-horizon settings, a common approach is to append all past observations, actions, and thoughts to the context at every turn [55, 61].”

Early LLM agents lack persistent 'how-to' procedures.

“Early LLM agents used prompt-based planning [26] and selfcritique [15], but lack persistent 'how-to' procedures - when tasks are similar but not identical, agents must re-plan from scratch, increasing cost and latency.”

Existing works adopted either CoT or ReACT for model reasoning, which cannot fully elicit LLM capabilities and fail to handle complex instructions.

“(3) inferior planning and reasoning : existing works adopted either CoT (Wei et al., 2023) or ReACT (Yao et al., 2022) for model reasoning, which cannot fully elicit the capabilities stored in LLMs and thus fail to handle complex instructions.”

LLM-based approaches may assist threat attribution by summarizing large-scale interaction logs, extracting structured event representations, or generating hypotheses about plausible attack paths.

“LLM-based approaches may assist by summarizing large-scale interaction logs, extracting structured event representations, or generating hypotheses about plausible attack paths; recent work on tool-augmented language models and automated log analysis suggests their usefulness in triage and forensic workflows (Schick et al., 2023;Solomon et al., 2026;Yao et al., 2022).”

Incorporating user profiles and memory modules into agent systems allows them to display a range of behaviors similar to humans, based on their current observations.

“Incorporating user profiles and memory modules into agent systems allows them to display a range of behaviors similar to humans, based on their current observations [33].”

Most modern agentic workloads follow the ReAct-agent loop, alternating between a reasoning step where the LLM interprets context and outputs thoughts, and an action step where it invokes external tools.

“Most modern agentic workloads follow the ReAct -agent loop [61], alternating between a reasoning step where the LLM interprets context and outputs thoughts, and an action step where it invokes external tools.”

ReAct (Yao et al., 2022) is a multi-step agent for tool use, implemented as a built-in module in DSPy.

“To that end, we first test ReAct (Yao et al., 2022), a multi-step agent for tool use, which is implemented as a built-in module in DSPy.”

The ReAct prompt from Yao et al. (2022) uses 593 words and 3889 characters.

“| Prompt 3: ReAct | Yao et al. (2022) | 593 | 3889 |”

Yao et al. (2022) achieve 27.4% EM using ReAct with PaLM-540B and 30.8% with text-davinci-002, and push their PaLM results to 35.1% EM by applying an additional CoT step with self-consistency.

“Yao et al. (2022) achieve 27.4% EM using ReAct with PaLM-540B and 30.8 with text-davinci-002 , with a tool giving it the ability for search using a Wikipedia API. They push their PaLM results to 35.1% EM by applying an additional CoT step with selfconsistency, which may resemble our ensemble approach in the sense of aggregating multiple answers.”

LLM agents communicate and coordinate with one another and interface with external resources such as tools, APIs, and structured data sources.

“These agents not only communicate and coordinate with one another but also interface with external resources such as tools, APIs, and structured data sources [24, 16].”

Prompting LLMs to generate explicit plans using internal knowledge is limited by errors arising from LLM hallucinations.

“One preliminary approach to achieving this goal involves prompting LLMs to generate explicit plans using their internal knowledge (Yao et al., 2022; Wang et al., 2023a; Sun et al., 2023). However, this method is limited by errors in the plans, which arise from the inevitable hallucinations of LLMs.”

Guiding LLM thought generation with explicit plans improves practical usage and reliability of LLMs.

“Guiding thought generation with explicit plans (Yao et al., 2022), which are step-wise and question-specific reasoning instructions, is crucial for improving practical usage and reliability of LLMs (Huang et al., 2024; 2022).”

Current efforts such as ReACT and PlaSma that rely on explicit plans face a key limitation: specific and question-oriented instructions often fail to generalize well across diverse problems, and the detailed content involved is prone to errors, making it difficult for the LLM to learn effectively.

“Current efforts such as ReACT (Yao et al., 2022) and PlaSma (Brahman et al., 2024) that rely on explicit plans face a key limitation: specific and question-oriented instructions often fail to generalize well across diverse problems, and the detailed content involved is prone to errors, making it difficult for the LLM to learn effectively.”

LLM-based agents increasingly rely on external tools to complete tasks beyond text generation.

“Large Language Model (LLM)-based agents increasingly rely on external tools to complete tasks beyond text generation (Yao et al., 2022).”

ReAct Agent maintains full interaction history with a ReAct-style reasoning-acting loop.

“ReAct Agent (Yao et al., 2022).Maintains full interaction history with a ReAct-style reasoning-acting loop.”

Yao et al. have shown success in long trajectory decision-making using explicit intermediate thoughts.

“We use ReAct [30] as the action generator as Yao et al. [30] has shown success in long trajectory decisionmaking using explicit intermediate thoughts.”

ReAct uses chain-of-thought prompting by generating both reasoning traces and action plans with LLMs.

“ReAct [29] uses chain-of-thought prompting [46] by generating both reasoning traces and action plans with LLMs.”

A promising design paradigm for AI agents is to combine an LLM with tools that interact with a broader environment.

“A promising design paradigm for AI agents [65] is to combine an LLM with tools that interact with a broader environment [14, 35, 40, 47, 51, 53, 55, 69].”

Frameworks integrating reasoning with action selection have shown that models can iteratively parse complex goals, call appropriate tools, and interpret results to inform further reasoning.

“Frameworks integrating reasoning with action selection have shown that models can iteratively parse complex goals, call appropriate tools, and interpret results to inform further reasoning [9].”

ReAct interleaves language reasoning with environment actions.

“ReAct interleaves language reasoning with environment actions [16], while Toolformer demonstrates self-supervised learning of when and how to call external APIs [17].”

LLM advances in reasoning and decision-making have greatly increased agent autonomy.

“advancements in reasoning and decision-making in LLMs have significantly enhanced agent autonomy (52; 58; 36; 45).”

Early LLM agents used prompt-based planning and self-critique.

“Early LLM agents used prompt-based planning [26] and selfcritique [15], but lack persistent 'how-to' procedures - when tasks are similar but not identical, agents must re-plan from scratch, increasing cost and latency.”

Early frameworks such as ReAct and Reflexion integrate reasoning and acting within the same loop.

“Early frameworks such as ReAct[26] and Reflexion[14] integrate reasoning and acting within the same loop, while trajectory-tuning methods [2, 22] fine-tune models using expert demonstrations.”

LLM agents can solve complex, interactive tasks such as web shopping and embodied AI housekeeping by transforming natural-language instructions into sequences of environment actions.

“Large language model (LLM) agents can solve complex, interactive tasks such as web shopping [25] and embodied AI housekeeping [9], by transforming natural-language instructions into sequences of environment actions [26].”

System memory can strengthen the system's reasoning ability, such as in approaches like CoT (Chain-of-Thought) and ReAct.

“Personal memory helps the system improve its understanding of user behavior and enhances its personalization capabilities, while system memory can strengthen the system's reasoning ability, such as in approaches like CoT (Chain-of-Thought) [23] and ReAct [24].”

These agents typically operate in an iterative loop, at each turn they reason about the current state, devise a plan, and execute a tool.

“These agents, powered by LLMs, typically operate in an iterative loop [39, 29], at each turn they reason about the current state, devise a plan, and execute a tool (e.g., read a file, run tests).”

ReAct prompts LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, fostering synergy between the two processes.

“ReAct (Yao et al., 2022) is a novel approach that prompts LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, fostering synergy between the two processes.”

In ReAct, by leveraging an intuitive prompt, LLMs learn to generate both reasoning paths and task-specific actions alternately when solving a specific task, with execution delegated to tools and external feedback used to validate and guide reasoning.

“In ReAct [230], by leveraging an intuitive prompt, LLMs learn to generate both reasoning paths and task-specific actions alternately when solving a specific task. The execution of specific actions is delegated to corresponding tools, and external feedback obtained from these tools is utilized to validate and guide the reasoning process further.”

Subsequent studies like ToT, Plan-and-Solve, and ReAct have proposed more advanced techniques for prompt design to assist in guiding LLM to engage in deeper thinking and tool planning.

“Subsequent studies like ToT [228], Plan-and-Solve [229] and ReAct [230] have proposed more advanced techniques for prompt design to assist in guiding LLM to engage in deeper thinking and tool planning.”

ReACT integrates reasoning with acting by allowing LLMs to give a reason for an action and incorporating environmental feedback for reasoning.

“ReACT (Yao et al., 2022) integrates reasoning with acting by allowing LLMs to give a proper reason for an action and incorporating environmental feedback for reasoning.”

ReAct's reasoning-and-acting traces double as short-horizon working memory and achieve a 34% absolute gain on ALFWorld.

“Reasoning-and-acting traces double as short-horizon working memory; 34% absolute gain on ALFWorld.”

Recent studies on tool-assisted reasoning have shown how AI systems can intertwine reasoning with action selection, allowing agents to select appropriate tools, interpret returned information, and update subsequent steps accordingly.

“Recent studies on tool-assisted reasoning have shown how AI systems can intertwine reasoning with action selection, allowing agents to select appropriate tools, interpret the returned information, and update subsequent steps accordingly [9].”

More sophisticated RAG methods are able to integrate information across new or updated knowledge elements.

“More sophisticated RAG methods, which perform multiple steps of retrieval and generation from an LLM, are even able to integrate information across new or updated knowledge elements[38, 64, 72, 78, 88, 90, 92], another crucial aspect of long-term memory systems.”

Thinking (considering evidence) has been shown to improve the quality of reasoning and future actions.

“which has been shown to improve the quality of reasoning and future actions71,72”

Both modular-public and triframe/duet incorporate principles from the ReAct framework, which interleaves reasoning traces with actions.

“Both modular-public and triframe/duet incorporate principles from the ReAct framework [46], which interleaves reasoning traces with actions.”

Fine-tuning LLMs on synthesized samples with explicit plans still struggles to achieve better performance because plans within and across tasks are vast and highly diverse.

“More promising recent efforts (Yao et al., 2022; Jiao et al., 2024; Qiao et al., 2024b; Brahman et al., 2024) focus on finetuning LLMs on automatically or manually synthesized samples with explicit plans. Unfortunately, LLMs fine-tuned in this manner still struggle to achieve better performance because the plans required by problems within a single task, as well as across tasks, are vast in number”

In ReAct, agents answer questions about general knowledge by multiple reasoning steps; they can utilize Wikipedia APIs to obtain external knowledge if they lack information during these steps.

“In ReAct [104], the agents are required to answer questions about general knowledge by multiple reasoning steps. They can utilize Wikipedia APIs to obtain external knowledge if they lack information during these steps.”

Accessing external knowledge through Application Programming Interface (API) is a common practice.

“For instance, accessing external knowledge through Application Programming Interface (API) is a common practice [104, 5].”

Previous work introduced non-executable code review and self-reflection but still faces challenges in ensuring code executability and runtime correctness.

“Previous work introduced non-executable code review and self-reflection (Zhao et al., 2023; Yao et al., 2022; Shinn et al., 2023; Dong et al., 2023). However, they still face challenges in ensuring code executability and runtime correctness.”

The agent remains in an experimental stage, limiting clinical applicability.

“although equipped with a broad array of tools compared to other frameworks17, it remains in an experimental stage, thus limiting clinical applicability.”

Equipping an LLM with a suite of tools, such as calculators or web search, has proven superiority in tasks that require multistep reasoning and planning.

“Equipping an LLM with a suite of tools, such as calculators or web search, has proven superiority in tasks that require multistep reasoning and planning17,18.”