Toolformer: Language Models Can Teach Themselves to Use Tools

Reinforcement learning can improve complex behavior sequences in large language models.

“These approaches demonstrate that RL can improve complex behavior sequences in LLMs.”

AI Agents have significant theoretical and practical limitations that inhibit their reliability, generalization, and long-term autonomy.

“While AI Agents have garnered considerable attention for their ability to automate structured tasks using LLMs and interfaces to specific tools, the literature highlights significant theoretical and practical limitations that inhibit their reliability, generalization, and long-term autonomy [136,162].”

LLMs have inherent limitations in their enclosed reasoning, particularly when addressing complex situations that extend beyond textual boundaries.

“Despite this, LLMs have inherent limitations in their enclosed reasoning, particularly when addressing complex situations that extend beyond textual boundaries (Schick et al., 2023).”

Research on language agent decision-making has progressed from simple action proposal to intermediate reasoning/retrieval to complex iterative proposal and evaluation.

“Empirically, many early language agents simply use LLMs to propose an action (Schick et al., 2023), a sequence of actions (Huang et al., 2022b), or evaluate a fixed set of actions (Ahn et al., 2022) without intermediate reasoning or retrieval. Followup work (Yao et al., 2022b; Shinn et al., 2023; Xu et al., 2023b; Lin et al., 2023; Wang et al., 2023a; Park et al., 2023) has exploited intermediate”

Recent work has equipped dialogue agents with the ability to use tools such as calculators and calendars, and to consult external websites.

“Recent work has equipped dialogue agents with the ability to use tools such as calculators and calendars, and to consult external websites 24,25 .”

LLM-based approaches may assist threat attribution by summarizing large-scale interaction logs, extracting structured event representations, or generating hypotheses about plausible attack paths.

“LLM-based approaches may assist by summarizing large-scale interaction logs, extracting structured event representations, or generating hypotheses about plausible attack paths; recent work on tool-augmented language models and automated log analysis suggests their usefulness in triage and forensic workflows (Schick et al., 2023;Solomon et al., 2026;Yao et al., 2022).”

LLM agents communicate and coordinate with one another and interface with external resources such as tools, APIs, and structured data sources.

“These agents not only communicate and coordinate with one another but also interface with external resources such as tools, APIs, and structured data sources [24, 16].”

This separation has been shown to improve model performance on a range of logical and mathematical reasoning tasks.

“This separation has been shown to improve model performance on a range of logical and mathematical reasoning tasks (Chen et al., 2022;Gao et al., 2023;Schick et al., 2023;Wang et al., 2022;Zhou et al., 2022;Cobbe et al., 2021).”

LaMDA, Toolformer, and Mind's Eye are examples of augmented language models.

“Notable examples of such systems include the likes of LaMDA (Thoppilan et al., 2022), Toolformer (Schick et al., 2023), and Mind's Eye (Liu et al., 2022).”

A promising design paradigm for AI agents is to combine an LLM with tools that interact with a broader environment.

“A promising design paradigm for AI agents [65] is to combine an LLM with tools that interact with a broader environment [14, 35, 40, 47, 51, 53, 55, 69].”

Toolformer demonstrates self-supervised learning of when and how to call external APIs.

“ReAct interleaves language reasoning with environment actions [16], while Toolformer demonstrates self-supervised learning of when and how to call external APIs [17].”

Methods such as Toolformer showed that LLMs can learn to call external APIs to aid in problem-solving.

“methods such as Toolformer showed that LLMs can learn to call external APIs to aid in problem-solving [18].”

Toolformer combines diverse tools within a single model, providing flexible decision-making abilities and improved generalization capabilities through a self-supervised method.

“The motivation behind Toolformer [246] aligns closely with ReAct; however, it goes a step further by combining diverse tools within a single model. This integration provides the model with flexible decision-making abilities and improved generalization capabilities, achieved through a simple yet effective self-supervised method.”

Agentic AI systems inherit the modularity of AI Agents but extend their architecture to support distributed intelligence, inter-agent communication, and iterative planning.

“As discussed before, Agentic AI systems inherit the modularity of AI Agents but extend their architecture to support distributed intelligence, inter-agent communication, and iterative planning.”

Agents can implement a Tree-of-Thoughts strategy, collectively exploring different reasoning branches in parallel and exchanging information to decide on the best path forward.

“Agents can also implement a Tree-of-Thoughts strategy, collectively exploring different reasoning branches in parallel and exchanging information to decide on the best path forward [18,20].”

Equipping an LLM with a suite of tools, such as calculators or web search, has proven superiority in tasks that require multistep reasoning and planning.

“Equipping an LLM with a suite of tools, such as calculators or web search, has proven superiority in tasks that require multistep reasoning and planning17,18.”

Toolformer employs a training approach to determine optimal usage of specific APIs and integrates results into subsequent token predictions.

“Toolformer employs a training approach to determine the optimal usage of specific APIs and integrates the obtained results into subsequent token predictions.”