The DOI matches its DataCite record; 6 of the 42 references it cites were found in a registry or an index; 12 of the 18 sources its claims cite were checked for support.
Toolformer: Language Models Can Teach Themselves to Use Tools
Open accessAbstract
Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q\&A system, two different search engines, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Toolformer is a 6.7B GPT-J model trained self-supervised to insert API calls for question answering, Wikipedia search, a calculator, machine translation, and a calendar, deciding when and how to call them based on whether calls reduce future-token loss. It improves zero-shot downstream performance, often beating much larger GPT-3/OPT models, without increasing perplexity when API calls are disabled.
Key findings
- On LAMA subsets, Toolformer improves over the best baseline by 11.7, 5.2, and 18.6 points on SQuAD, Google-RE, and T-REx, respectively, and uses the question answering tool in 98.1% of cases; it outperforms OPT (66B) and GPT-3 (175B).
- On math benchmarks, Toolformer reaches ASDiv 40.4, SVAMP 29.4, and MAWPS 44.0, versus GPT-J’s 7.5, 5.2, and 9.9, using the calculator for 97.9% of examples.
- On question answering, Toolformer scores WebQS 26.3, NQ 17.7, and TriviaQA 48.8, mostly using Wikipedia search (99.3%), but still lags GPT-3 (175B) at 29.0, 22.6, and 65.9.
- On temporal datasets, Toolformer reaches DATESET 27.3 versus GPT-J 3.9, with the calendar used for 54.8% of examples; on TEMPLAMA it scores 16.3, but the calendar is used for only 0.2%.
- Language modeling perplexity is not degraded: Toolformer (disabled) has WikiText 10.3 and CCNet 10.5, matching GPT-J + CC. Tool use emerges around 775M parameters.
Methods
- Self-supervised pipeline: sample candidate API calls with an LM, execute them, filter calls by whether they reduce the weighted loss over future tokens by at least τ_f, merge useful calls into a CCNet subset, and finetune GPT-J on the augmented dataset.
- Tools: Atlas for QA, BM25 retriever over Wikipedia for search, a Python calculator for basic arithmetic, 600M NLLB for machine translation, and a calendar API; up to 25k examples per API, batch size 128, learning rate 1e-4.
- Evaluation: zero-shot downstream tasks with greedy decoding modified to allow `<API>` if among top k=10 tokens, at most one API call per input; baselines include GPT-J, GPT-J + CC, Toolformer (disabled), OPT (66B), and GPT-3 (175B).
Limitations
- Stated: Toolformer cannot chain tools, cannot use tools interactively, is sensitive to input wording, is sample-inefficient for some tools (e.g., over a million documents yield only a few thousand useful calculator calls), and does not account for tool-dependent computational cost.
- Evident from the text: the one-API-call-per-example restriction and independent sampling prevent chained calls; TEMPLAMA gains are not attributable to the calendar tool; MLQA does not consistently outperform GPT-J because CCNet finetuning can cause distribution shift; QA lags GPT-3 partly because the search engine is simple and Toolformer cannot reformulate queries or browse multiple results.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Own interpretation · 18 Own result · 17 No basis given · 6 Own method · 5 Partly supported · 3 Supported · 3 Sources not found · 1 Mixed support · 1 Not checked yet · accuracy not yet measured
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“or limit tool use to task-specific settings only (e.g., Gao et al., 2022;Parisi et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Disputedcites Gao et al., 2022 sourceExperimental — accuracy not yet measured
Section 7 (Related Work; page unavailable) explicitly contrasts PAL with task-specific tools, saying its Python interpreter handles both arithmetic and dates without specialized modules. Section 4 (Experimental Setup; page unavailable) also describes evaluations across mathematical, symbolic, and algorithmic tasks. The paper therefore does not limit PAL’s tool use to task-specific settings only. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Partly supportedcites Parisi et al., 2022 sourceExperimental — accuracy not yet measured
The Results (p. 3) evaluate TALM on two specific tasks, Natural Questions and MathQA, with tools suited to each. But the Conclusion (p. 4) describes TALM as a framework for arbitrary tools that makes exploring additional tools and tasks possible. The paper’s evaluations are task-specific; its approach is not limited to task-specific settings only. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“These limitations include an inability to access up-to-date information on recent events (Komeili et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Komeili et al., 2022 source
The Introduction says static language models cannot account for new information because their knowledge is frozen at training, and contrasts them with an internet-augmented model that can use the latest information. The Conclusions likewise describe models limited to knowledge in their fixed training set. This supports the claim as a limitation of LMs without dynamic access to outside information. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“existing approaches either rely on large amounts of human annotations (Komeili et al., 2022;Thoppilan et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Komeili et al., 2022 source
Section 3.2 says the internet-search model's modules require supervised context–search-query and context–response pairs, supplied by the data collected for this work. Section 4.1 reports a substantial crowdsourced dataset: 9,633 dialogues and 82,952 training utterances. This supports the claim for Komeili et al.'s approach, without establishing that every external-tool approach has the same requirement. (LLM, conf=0.91)
- Supportedcites Thoppilan et al., 2022 source
Section 6.2 (p. 8) describes LaMDA learning to consult external tools using 40,000 annotated dialog turns and 9,000 turns with human-assigned labels. This supports the claim that this tool-using approach relies on substantial human annotation, although the Discussion (p. 15) calls the amount modest relative to pre-training data. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“a lack of mathematical skills to perform precise calculations (Patel et al., 2021)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Partly supportedcites Patel et al., 2021 sourceExperimental — accuracy not yet measured
The Results on SVAMP report that the best-performing model achieved 43.8% accuracy, supporting a limitation in solving elementary arithmetic word problems. But the Problem Formulation section says the models generate expressions that are then evaluated to calculate answers; the paper does not establish a general inability of language models to perform precise calculations. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This is due to the fact that API calls for each tool are generated independently; as a consequence, there are no examples of chained tool use in the finetuning dataset.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“increasing k leads to the model doing API calls for more examples -from 40.3% and 8.5% with k = 1 (i.e., regular greedy decoding) to 98.1% and 100% for k = 10.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 5 Analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Toolformer achieves stronger results even when API calls are disabled.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Depending on the tool, our method is also very sample-inefficient; for example, processing more than a million documents results in only a few thousand examples of useful calls to the calculator API.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plaincorrelationalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Beyond this, we found models trained with Toolformer to often be sensitive to the exact wording of their input when deciding whether or not to call an API; this is perhaps unsurprising given that LMs are known to be very sensitive to the prompt they are provided with in both zeroand few-shot settings (Jiang et al., 2020;Schick and Schütze, 2021a).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the considerable improvement of Toolformer compared to other models can be fully accredited to the calendar tool, which it makes use of for 54.8% of all examples.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.5 Temporal Datasets
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 22
- The quote only partly states this claim. 3
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and the related tendency to hallucinate facts (Maynez et al., 2020;Ji et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Maynez et al., 2020 source
Section 2.1 says a decoder functioning like an open-ended language model is prone to extrinsic hallucinations. Section 5.3 reports that over 90% of the hallucinations produced by one summarization model were erroneous. The paper thus supports the broad claim, though its empirical results concern abstractive summarization models. (LLM, conf=0.88)
- Not matchedcites Ji et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Large language models achieve impressive zeroand few-shot results on a variety of natural language processing tasks (Brown et al., 2020;Chowdhery et al., 2022, i.a.)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Not matchedcites Brown et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Chowdhery et al., 2022 sourceExperimental — accuracy not yet measured
The Conclusion (p. 50) reports outstanding few-shot performance across English NLP tasks, and Section 6.1 (p. 11) describes results on 29 benchmarks. Section 6.8 (p. 33) evaluates zero-shot performance on five tasks, but the passages do not establish the claimed impressive zero-shot results across a variety of tasks. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Beyond this, we found models trained with Toolformer to often be sensitive to the exact wording of their input when deciding whether or not to call an API; this is perhaps unsurprising given that LMs are known to be very sensitive to the prompt they are provided with in both zeroand few-shot settings (Jiang et al., 2020;Schick and Schütze, 2021a).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
What the sources say
- Partly supportedcites Jiang et al., 2020 sourceExperimental — accuracy not yet measured
The Evaluation Results (p. 5) show that changing prompt wording substantially changed BERT’s accuracy, supporting prompt sensitivity in the paper’s knowledge-retrieval task. The paper does not establish the claim specifically for both zero-shot and few-shot settings; its prompt-selection method uses training examples (Top-1 Prompt Selection, p. 4). (LLM, conf=0.86)
- Supportedcites Schick and Schütze, 2021a source
Section 5 (Analysis; page unavailable) reports a large performance gap between the best and worst prompt patterns even after few-shot finetuning. This supports the prompt-sensitivity point for which Schick and Schütze are cited; the sentence also cites Jiang et al. for the broader zero-shot context. (LLM, conf=0.90)
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“and an unawareness of the progression of time (Dhingra et al., 2022).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Dhingra et al., 2022 source
The Introduction says standard LM training generally ignores temporal metadata and that standard LMs have no knowledge of what is “new” or “old.” This supports the claim as a broad description of standard LMs, while the paper distinguishes them from models trained with temporal context. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“TALM (Parisi et al., 2022), an approach that uses a similar self-supervised objective for teaching a model to use a calculator and a search engine, but explores this only in settings where a model is finetuned for downstream tasks.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 6 Related Work
What the sources say
- Partly supportedcites Parisi et al., 2022 sourceExperimental — accuracy not yet measured
Sections 3.1 and 3.2 (p. 3) describe retrieval and calculation tools for downstream question-answering and math tasks; §2.2 (p. 2) says TALM is fine-tuned during iterative self-play. But §2.2 describes supervised input–target pairs and a small set of tool-use demonstrations, not a self-supervised objective. The main retrieval experiment uses a BM25 index; a public search engine is used for an out-of-distribution test (§3.4, p. 4). (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the ability to leverage the provided tools only emerges at around 775M parameters: smaller models achieve similar performance both with and without tools.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.4 Scaling Laws
plaindirectn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“improvements on TEMPLAMA can not be attributed to the calendar tool, which is only used for 0.2% of all examples, but mostly to the Wikipedia search and question answering tools”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.5 Temporal Datasets
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“high values of L - i -L + i typically correspond to useful API calls, whereas low values correspond to API calls that do not provide any information that is useful for predicting future tokens.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > Data Quality
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“OPT and GPT-3 perform surprisingly weak across all languages, mostly because they fail to provide an answer in English despite being instructed to do so.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, when deciding whether or not to make an API call, Toolformer currently does not take into account the tool-dependent, computational cost incurred from making an API call.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Depending on the language, this tool is used for 63.8% to 94.9% of all examples; the only exception to this is Hindi, for which the machine translation tool is used in only 7.3% of cases.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As one would expect, finetuning on CCNet leads to slightly improved performance on a different CC-Net subset, but it slightly deteriorates performance on WikiText, presumably because the original pre-”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Toolformer, which is based on a pretrained GPT-J model (Wang and Komatsuzaki, 2021) with 6.7B parameters, achieves much stronger zero-shot results, clearly outperforming a much larger GPT-3 model (Brown et al., 2020) and several other baselines on various tasks.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Not matchedcites Brown et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“for 97.9% of all examples the model decides to ask the calculator tool for help.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Toolformer clearly outperforms all other models based on GPT-J, this time mostly relying on the Wikipedia search API (99.3%) to find relevant information.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.3 Question Answering
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the Wikipedia search tool for most examples, Toolformer clearly outperforms baselines of the same size, but falls short of GPT-3 (175B).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Toolformer outperforms all baselines for both TEM-PLAMA and DATESET.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.5 Temporal Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nonetheless, allowing the model to make API calls more than doubles performance for all tasks, and also clearly outperforms the much larger OPT and GPT-3 models.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Toolformer outperforms all baselines, but does not make use of the calendar tool for TEMPLAMA.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the model independently decides to ask the question answering tool for the required information in almost all cases (98.1%); for only very few examples, it uses a different tool (0.7%) or no tool at all (1.2%).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.1 LAMA
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“improving upon the best baseline by 11.7, 5.2 and 18.6 points, respectively.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.1 LAMA
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“A potential reason for GPT-J not suffering from this problem is that it was trained on more multilingual data than both OPT and GPT-3, including the EuroParl corpus (Koehn, 2005;Gao et al., 2020).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
What the sources say
- Not matchedcites Koehn, 2005
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Gao et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we prevent Toolformer from using the Wikipedia Search API to avoid giving it an unfair advantage.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.1 LAMA
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In a similar spirit to these approaches, Toolformer is trained on its own predictions after applying a perplexity-based filtering step.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 6 Related Work
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: The remaining examples after filtering are close enough to the original distribution so that the model's language modeling abilities remain unaffected.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“some amount of noise in the API calls that are not filtered can actually be useful as it forces the model finetuned on C * to not always blindly follow the results of each call it makes.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > Data Quality
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This ensures that the model does not lose any of its generality and language modeling abilities.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We surmise that this is because the model is finetuned on many examples of API calls and their results, improving its own mathematical capabilities.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, Toolformer learns for itself to explicitly asks for the right information.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 6 Related Work
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For Toolformer, we disable the question answering tool as this would make solving the tasks trivial”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.3 Question Answering
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we only at most one API call per input to make sure the model does not get stuck in a loop where it constantly calls APIs without producing any actual output.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2 Downstream Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While during data generation the model can look ahead to generate API calls, this is not possible at inference time, so we want to dissuade the model from calling the API in such cases.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.1 Implementation Question Answering
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“difficulties in understanding low-resource languages (Lin et al., 2021)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Not matchedcites Lin et al., 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This filtering mechanism allows us to generate data more efficiently by focusing our API call generations in places where the MT tool is likely to be helpful.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.1 Implementation Question Answering
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is mainly because for some languages, finetuning on CCNet deteriorates performance; this might be due to a distribution shift compared to GPT-J's original pretraining data.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We do not evaluate the perplexity of Toolformer with API calls enabled as computing the probability pM (xt | x1, . . ., xt-1) of token xt given x1, . . ., xt-1 would require marginalizing over all potential API calls that the model could make at position t, which is intractable.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is likely due to both the simplicity of our search engine (in many cases, it returns results that are clearly not a good match for a given query) and the inability of Toolformer to interact with it, e.g., by reformulating its query if results are not helpful or by browsing through multiple of the top results.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.3 Question Answering
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
What humans find useful may be different from what a model finds useful.
“what humans find useful may be different from what a model finds useful.”
The authors state this as an assumption.
An API call is helpful to the model if providing both its input and output makes it easier for the model to predict future tokens, compared to not receiving the API call or receiving only its input.
“Intuitively, an API call is helpful to M if providing it with both the input and the output of this call makes it easier for the model to predict future tokens, compared to not receiving the API call at all, or receiving only its input.”
The authors state this as an assumption.
For creating the calendar dataset, the calendar date in such cases should be the date that the document was created.
“we operate under the assumption that the calendar date in such cases should be the date that the document was created.”
The authors state this as an assumption.
The remaining examples after filtering are close enough to the original distribution so that the model's language modeling abilities remain unaffected.
“While this filtering alters the distribution of training examples, we assume that the remaining examples are close enough to the original distribution so that M 's language modeling abilities remain unaffected.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“These approaches demonstrate that RL can improve complex behavior sequences in LLMs.”
From Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2025) · cites this paper as Schick et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Earlier efforts in this line of work rely on supervised fine-tuning (SFT), which trains the LLM on high-quality tool-use trajectories to explicitly demonstrate how tools should be invoked and integrated into task execution (Schick et al., 2023; Du et al., 2024; Liu et al., 2025g; Wang et al., 2025e).”
From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Schick et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While AI Agents have garnered considerable attention for their ability to automate structured tasks using LLMs and interfaces to specific tools, the literature highlights significant theoretical and practical limitations that inhibit their reliability, generalization, and long-term autonomy [136,162].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [162]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Despite this, LLMs have inherent limitations in their enclosed reasoning, particularly when addressing complex situations that extend beyond textual boundaries (Schick et al., 2023).”
From Scaling Large Language Model-based Multi-Agent Collaboration (2024) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Empirically, many early language agents simply use LLMs to propose an action (Schick et al., 2023), a sequence of actions (Huang et al., 2022b), or evaluate a fixed set of actions (Ahn et al., 2022) without intermediate reasoning or retrieval. Followup work (Yao et al., 2022b; Shinn et al., 2023; Xu et al., 2023b; Lin et al., 2023; Wang et al., 2023a; Park et al., 2023) has exploited intermediate”
From Cognitive Architectures for Language Agents (2023) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent work has equipped dialogue agents with the ability to use tools such as calculators and calendars, and to consult external websites 24,25 .”
From Role play with large language models (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Modern LLM agents can select tools, maintain state across steps, and modify the plan as new evidence emerges.[13], [14].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reliability comes mainly from typed tool interfaces, provenance-aware retrieval, explicit budgets, independent gates, and rollback-aware execution, not from model fluency alone [12], [37], [13], [75], [76].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“General agent frameworks motivate the separation between reasoning and acting, but operations demands that the separation is enforceable and auditable rather than merely prompt-stated [14], [13], [159], [161].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Context engineering primarily addresses the structural organization of the interaction interface between LLMs and their operational environment. This includes optimizing tool-integrated reasoning and selection pipelines (Qin et al., 2024a; Schick et al., 2023; Jia and Li, 2025) and standardizing communication protocols, such as MCP (Qiu et al., 2025c).”
From Memory in the Age of AI Agents (2025) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“language models have further acquired capabilities such as retrieval, calculation, and program invocation, extending multi-step task solving from pure text generation to the joint process of reasoning and execution [3]”
From A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving (2026) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Beyond conversational history, tool-based agents must retain tool inputs, execution outputs, and intermediate states, many of which are repeatedly referenced in subsequent reasoning steps (Schick et al., 2023).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, another critical feature of LLMs is their capability to use tools for automated online search, task design, and experiment execution [203–205], which exhibits their further capabilities in automatically collecting and preprocessing biomedical datasets, designing bioinformatic analyses to identify disease-related molecular regulations, conducting literature reviews for result inte”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [203]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLM-based approaches may assist by summarizing large-scale interaction logs, extracting structured event representations, or generating hypotheses about plausible attack paths; recent work on tool-augmented language models and automated log analysis suggests their usefulness in triage and forensic workflows (Schick et al., 2023;Solomon et al., 2026;Yao et al., 2022).”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the literature highlights significant theoretical and practical limitations that inhibit their reliability, generalization, and long-term autonomy [138], [164].”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [164]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These agents not only communicate and coordinate with one another but also interface with external resources such as tools, APIs, and structured data sources [24, 16].”
From Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This separation has been shown to improve model performance on a range of logical and mathematical reasoning tasks (Chen et al., 2022;Gao et al., 2023;Schick et al., 2023;Wang et al., 2022;Zhou et al., 2022;Cobbe et al., 2021).”
From Reliable Reasoning Beyond Natural Language (2024) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notable examples of such systems include the likes of LaMDA (Thoppilan et al., 2022), Toolformer (Schick et al., 2023), and Mind's Eye (Liu et al., 2022).”
From ChatGPT, extended: large language models and the extended mind (2025) · cites this paper as Schick et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“A promising design paradigm for AI agents [65] is to combine an LLM with tools that interact with a broader environment [14, 35, 40, 47, 51, 53, 55, 69].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [51]
The cited source says what the claim attributes to it.
In the citing paper
“ReAct interleaves language reasoning with environment actions [16], while Toolformer demonstrates self-supervised learning of when and how to call external APIs [17].”
From HELIX: Model-Harness Co-evolution for Recursive Self-Improvement (2026) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“methods such as Toolformer showed that LLMs can learn to call external APIs to aid in problem-solving [18].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Toolformer employs a training approach to determine the optimal usage of specific APIs and integrates the obtained results into subsequent token predictions.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [172]
The cited source says what the claim attributes to it.
In the citing paper
“Skill-related work has expanded from narrow function calling and tool learning toward reusable capability packages, registries, and progressive disclosure mechanisms [OpenAI, 2023a, Schick et al., 2023, Wang et al., 2023a, Anthropic, 2025, 2026, Jiang et al., 2026b].”
From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Schick et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“The motivation behind Toolformer [246] aligns closely with ReAct; however, it goes a step further by combining diverse tools within a single model. This integration provides the model with flexible decision-making abilities and improved generalization capabilities, achieved through a simple yet effective self-supervised method.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [246]
The cited source says what the claim attributes to it.
In the citing paper
“This motivates an established alternative-the tool-use paradigm [9, 10]-in which, rather than perform every reasoning step internally, a model delegates well-specified tasks to external tools.”
From PrologMCP: A Standardized Prolog Tool Interface for LLM Agents (2026) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“Various studies (Liu et al., 2023;Parisi et al., 2022;Schick et al., 2023;Shen et al., 2023;Lu et al., 2023;Paranjape et al., 2023;Liang et al., 2023) have demonstrated that supplementing LLMs with tools, such as calculators, search engines, translation systems, calendars, or even API calls on other models, can help solve tasks that are not easily addressed by LLMs alone.”
From Large language models as tool makers (2023) · cites this paper as Schick et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“As discussed before, Agentic AI systems inherit the modularity of AI Agents but extend their architecture to support distributed intelligence, inter-agent communication, and iterative planning.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [162]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Agents can also implement a Tree-of-Thoughts strategy, collectively exploring different reasoning branches in parallel and exchanging information to decide on the best path forward [18,20].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Indirect prompting could be leveraged to leak users' chat sessions [52] or exfiltrate users' data (e.g., credentials, personal information).”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [52]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Agentic AI systems consist of multiple agents, each assigned a specialized function or task (e.g., a summarizer, a retriever, or a planner).”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [164]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Agentic AI systems inherit the modularity of AI Agents but extend their architecture to support distributed intelligence, inter-agent communication, and iterative planning.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [164]
Experimental — accuracy not yet measuredIn the citing paper
“Equipping an LLM with a suite of tools, such as calculators or web search, has proven superiority in tasks that require multistep reasoning and planning17,18.”
From Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology (2025) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.2 Prompts
“This is three times less than the 2169 goals last year.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.2 Prompts
“This is twenty goals more than the [Calculator(723 -20)] 703 goals last year.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 12
- Ricardo Baeza-Yates, Berthier Ribeiro-Neto, et al. 1999. Modern information retrieval, volume 463. ACM press New York.
- Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1533-1544, Seattle, Wash- ington, USA. Association for Computational Lin- guistics.
- Sergey Brin. 1999. Extracting patterns and relations from the world wide web. In The World Wide Web and Databases, pages 172-183, Berlin, Heidelberg. Springer Berlin Heidelberg.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Junxian He, Jiatao Gu, Jiajun Shen, and Marc'Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations.
- Gautier Izacard and Edouard Grave. 2021. Distilling knowledge from reader to retriever for question an- swering. In International Conference on Learning Representations.
- Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of machine translation summit x: papers, pages 79-86.
- David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Pro- ceedings of the Human Language Technology Con- ference of the NAACL, Main Conference, pages 152- 159, New York City, USA. Association for Compu- tational Linguistics.
- Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture mod- els. In International Conference on Learning Repre- sentations.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Lan- guage models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Roi Reichart and Ari Rappoport. 2007. Self-training for enhancement and domain adaptation of statisti- cal parsers trained on small datasets. In Proceed- ings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 616-623, Prague, Czech Republic. Association for Computational Lin- guistics.
- Guillaume Wenzek, Marie-Anne Lachaux, Alexis Con- neau, Vishrav Chaudhary, Francisco Guzmán, Ar- mand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 4003-4012, Marseille, France. European Language Resources Association.
Cited by
Sorted by citations
Items 1,801–1,900 of 3,209
Sivia: Learning to Draw, Learning to LearnPreprint
Yuan, F., Gao, Y., 孙自俊, Li, H., Hua, M., Li, X., & Gao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33386 · Cited by 1
ControlScope: Workflow Revision and Reliability in LLM AgentsPreprint
Ning, J., Li, X., Kong, Y., & Li, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34313 · Cited by 1
Distilling Agentic Systems: A Roadmap across Models, Artifacts, and HarnessesPreprint
Luo, Z., Wang, S., Li, C., Yin, J., Yan, H., Cheng, M., Wang, C., Liu, S., Zhang, L., Ye, Q., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36630 · Cited by 1
Heng, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.02036 · Cited by 1
Agent Behavior as Code: Efficient and Robust LLM Agents with Programmatic SpecificationsPreprint
Qi, P., Lyu, C., Li, G., Chan, F., Chang, C., Cases, I., & Lu, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.04824 · Cited by 1
From Uncertainty to Action: Learning to Steer LLM AgentsPreprint
Li, H., Duan, J., Zhu, G., Lu, J., Shen, B., Yuan, C., & Xu, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.09115 · Cited by 1
Scaling Laws for Tool-Use Agents via Controllable Environment SynthesisPreprint
Szot, A., Abnar, S., Attia, O., Kazerouni, A., Kirchhof, M., Hsieh, C.-Y., Vemulapalli, R., & Toshev, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.37648 · Cited by 1
Chen, S. (2026) · arXiv · doi:10.48550/arxiv.2606.22528 · Cited by 21
“As LLM agents move from single-turn assistants to autonomous systems that reason and act over long, multi-step tool-use trajectories [24, 27], their histories outgrow the context window, and the dominant response is context compaction : an LLM-based summarization or eviction step that periodically…”
SE-Blackboard: A Shared-State Architecture for Multi-Agent Software Engineering Pipelines
Liu, E., Zhu, Q., & Dong, X. (2026) · IEEE Access · doi:10.1109/access.2026.3695435
Agent-Dice: Disentangling Knowledge Updates via Geometric Consensus for Agent Continual Learning
Wu, Z., Lou, X., Ma, X., Li, Y., Liu, W., Zhang, W., Wang, J., & Zhang, Z. (2026) · arXiv · doi:10.48550/arxiv.2601.03641
“…important direction is tool-use agents, which extend LLM capabilities by integrating external tools and APIs to perform complex reasoning and execution (Schick et al., 2023; Qin et al., 2023; Liu et al., 2024, 2025a; Zhang et al., 2025c; Patil et al., 2025; Barres et al., 2025; Chen et al., 2025).”
Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents
Dao, Q., Kathalkar, P., & Eaton, K. (2026) · arXiv · doi:10.48550/arxiv.2608.20631 · Cited by 1
HELIX: Model-Harness Co-evolution for Recursive Self-Improvement
Fan, T., & Huang, C. (2026) · arXiv
“ReAct interleaves language reasoning with environment actions [16], while Toolformer demonstrates self-supervised learning of when and how to call external APIs [17].”
PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
Woo, S., Kim, H., Shim, S., Jo, M., Jeong, H., Lee, J., Kim, J., Lee, S., Ryu, J., Park, B., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.12029
“Building on these advances, recent systems increasingly explore agentic work-flows, where models interact with tools, intermediate memory states, and external resources to solve complex problems (Yao et al., 2022; Schick et al., 2023; Shinn et al., 2023).”
On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
Liu, T., Qian, C., Cief, M., He, Y., Dan, D., Aletras, N., & Kazai, G. (2026) · arXiv
“Subsequent work improves reliability through post-training, including supervised fine-tuning (SFT) on tool-use traces (Schick et al., 2023; Qin et al., 2023; Patil et al., 2024; Zhang et al., 2025a) and, more recently, reinforcement learning (RL) for tool-calling and agent behavior (Qian et al.,…”
MAPLE-Guard: Memory-Aware Link Enforcement Against Memory-Link Poisoning in Multi-Agent Systems
Xiong, W., Zhou, Y., Wang, J., Gu, S., Tang, B., Li, Z., Xiong, F., Wen, Y., & Wen, M. (2026) · arXiv · doi:10.48550/arxiv.2608.00426 · Cited by 2
“Large language model (LLM)-based multi-agent systems (MAS) increasingly plan, communicate, use tools, and coordinate over extended horizons [15, 22, 34, 35, 49, 53].”
MemGym: a Long-Horizon Memory Environment for LLM Agents
Xu, W., Wang, Y., Mei, K., Liang, K., Wang, Z., Jin, M., Zhang, H., Zhang, S.-X., Hua, W., Sahu, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.20833 · Cited by 11
Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
Li, H., He, R., Mang, Q., Zhang, Q., Mao, H., Chen, X., Zhou, H., Zhang, H., Cheung, A., Gonzalez, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.02230
“This limits these work from optimizing for the wide variety of agents in practice [9, 33, 56].”
PAACE: A Plan-Aware Automated Agent Context Engineering Framework
Yuksel, K. A. (2025) · arXiv · doi:10.48550/arxiv.2512.16970
“Systems such as ReAct (Yao et al., 2023a), Toolformer (Schick et al., 2023a), AutoGPT, Devin, We-bArena (Zhou et al., 2024), and AgentBench (Liu et al., 2023) highlight the promise and challenges of designing agents with reasoning, planning, and tool-use capabilities.”
Rahgozar, A., & Mortezaagha, P. (2026) · arXiv · doi:10.48550/arxiv.2608.14551
AgentSHAP: Interpreting LLM Agent Tool Importance with Monte Carlo Shapley Value Estimation
Horovicz, M. (2025) · arXiv · doi:10.48550/arxiv.2512.12597
“Tool-former [12] pioneered self-supervised tool learning, while ReAct [20] introduced interleaved reasoning and acting.”
Shared Selective Persistent Memory for Agentic LLM Systems
Pedada, S., Dhavala, A., & Patil, N. (2026) · arXiv · doi:10.48550/arxiv.2607.09493
“We observe that this re-specification is both the primary source of user friction and the primary source of wasted tokens.”
Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control
Rezazadeh, A., Li, Z., Lou, A., Zhao, Y., Wei, W., & Bao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.18279
“These agents not only communicate and coordinate with one another but also interface with external resources such as tools, APIs, and structured data sources [24, 16].”
Accurate and Efficient Long-Term Memory for LLM Agents
Zhao, Z., Guo, X., Lv, L., Wang, M., Li, M., & Li, S. (2026) · arXiv · doi:10.48550/arxiv.2607.16211
“Tool-augmented approaches [Schick et al., 2023] extend agents with external capabilities but do not address the fundamental memory organization problem.”
Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval
Chen, Y., Yoon, J., Sachan, D. S., Wang, Q., Cohen-Addad, V., Bateni, M., Lee, C.-Y., & Pfister, T. (2024) · arXiv · doi:10.48550/arxiv.2408.01875
“Schick et al. (2023) proposes a self-supervised training method with API demonstrations.”
Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming
Chen, Z., & Ma, M. (2026) · arXiv · doi:10.48550/arxiv.2607.23019
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
Zhou, P., Leon, B., Ying, X., Zhang, C., Shao, Y., Ye, Q., Chong, D., Jin, Z., Xie, C., Cao, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.19314
“Representative works such as WebGPT Nakano et al. (2021), Toolformer Schick et al. (2023), and ReAct Yao et al. (2023) explore how LLMs leverage search engines, tool usage, and reasoning strategies to tackle complex question-answering tasks.”
Finding the Right Fit: Model-Harness Interactions across Agent Tasks
Li, Y., Zhou, Y., Teng, Y. L., Yang, F., Deng, Y., Lyu, Z., Dong, X., Chen, F., & An, B. (2026) · arXiv
Enrich-Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routing
Sorathiya, N., Zhang, D., & Akhbari, B. (2026) · arXiv · doi:10.48550/arxiv.2608.22695 · Cited by 1
Locality-aware Fair Scheduling in LLM Serving
Cao, S., Wang, Y., Mao, Z., Hsu, P.-L., Yin, L., Xia, T., Li, D., Liu, S., Zhang, Y., Zhou, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.14312
SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills
Zheng, B., Fatemi, M. Y., Jin, X., Wang, Z. Z., Gandhi, A., Song, Y., Gu, Y., Srinivasa, J., Liu, G., Neubig, G., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.07079
Oracle Agent Memory as an Enterprise Memory Substrate for Long-Horizon AI Agents
Alake, R., Bernardis, C., Cayet, P., Engel, L., Hilloulin, D., Hong, S., Hosler, A., Kavantzas, N., Kossyk, I., Le, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.13157
Design and Implementation of a Lightweight Neural Controller for LLM Agent Systems
Langay, B. B. (2026) · International Journal of Latest Technology in Engineering Management & Applied Science · doi:10.51583/ijltemas.2026.150500172
Li, H., Feng, L., & Wang, D. (2026) · Electronics · doi:10.3390/electronics15102011
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052 · Cited by 70
Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.07952
Barbaste, P., Darrigol, T., Vu, G., & Wiltberger, T. (2026) · arXiv · doi:10.48550/arxiv.2609.00006
Synapse: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation
Jiang, H., Chen, J., Pan, Y., Chen, L., You, W., Zhou, Y., Zhang, R., Abate, Y., & Liu, T. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1108
Are Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
Potamitis, N., & Arora, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12951 · Cited by 2
HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety
Bai, Y., Duan, J., Peng, J., Wu, X., Liu, S., Wang, S., & Chen, T. (2026) · arXiv · doi:10.48550/arxiv.2608.17597
Zhang, J., Zhao, B., Yang, W., Foerster, J., Clune, J., Jiang, M., Devlin, S., & Shavrina, T. (2026) · arXiv · doi:10.48550/arxiv.2603.19461
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Chen, Z., Sun, Z., Shi, Y., Lo, D., & Jiang, L. (2026) · arXiv · doi:10.48550/arxiv.2607.01211 · Cited by 9
Mendel Gödel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution
Liu, C., Liu, Y., Yan, S., Tresp, V., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2608.07645
Recursive Self-Evolving Agents via Held-Out Selection
Nguyen, M., Nguyen, Q., & Vuong, P. (2026) · arXiv · doi:10.48550/arxiv.2606.28374
On The Statistical Limits of Self-Improving Agents
Wang, C. L., Dorchen, K., & Jin, P. (2025) · arXiv · doi:10.48550/arxiv.2510.04399
PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
Chang, M., Chhablani, G., Clegg, A., Cote, M. D., Desai, R., Hlavac, M., Karashchuk, V., Krantz, J., Mottaghi, R., Parashar, P., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.00081
Mitigating Context Interference for Reliable and Efficient Search Agents
Xue, B., Wu, B., Qiao, S., Wang, S., Wang, R., Du, Y., Wang, H., Pan, J. Z., Yilmaz, E., Wong, K.-F., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.160
Shopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer
Wadi, D., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2609.28372
A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
Khan, S., Madhavan, S., & Natarajan, K. (2025) · arXiv · doi:10.48550/arxiv.2506.18957
Can an LLM Induce a Graph? Investigating Memory Drift and Context Length
Yousuf, R. B., Khatri, A., Xu, S., Sharma, M., & Ramakrishnan, N. (2025) · 2025 IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg66886.2025.00056
Arithmetic Reasoning with LLM: Prolog Generation & Permutation
Yang, X., Chen, B., & Tam, Y.-C. (2024) · arXiv · doi:10.48550/arxiv.2405.17893
“A line of work has focused on augmenting language models with deterministic computation resources like a calculator (Schick et al., 2023) or program-based tools (Gao et al., 2023; Gou et al., 2023).”
BackgroundZaninello, A., & Magnini, B. (2023) · NLRSE · doi:10.18653/v1/2023.nlrse-1.3
Using Planning to Improve Semantic Parsing of Instructional Texts
Cohen, V., & Mooney, R. J. (2023) · doi:10.18653/v1/2023.nlrse-1.5
Research on "MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting"
Inaba, T. (2023) · Journal of Natural Language Processing · doi:10.5715/jnlp.30.1100
Stepping Stones for Self-Learning
Hai-Jew, S. (2023) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0074-9.ch005
End-to-End Autoregressive Retrieval via Bootstrapping for Smart Reply Systems
Towle, B., & Zhou, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.510
Building Knowledge Graph for Products at Web Scale
Agarwal, M. (2024) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3632410.3633292
Augmenting Large Language Models for Enhanced Interaction with Government Data RepositoriesPreprint
Paul Trust, Omala, K., & Minghim, R. (2024) · Research Square · doi:10.21203/rs.3.rs-3897706/v1
Srivastava, N., & Kim, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4790013
ProgAI: Enhancing Code Generation with LLMs For Real World Challenges
Afsal Ahamad, A., Harshad, D., & Nivedha, M. S. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem36242
Pixel Logo Attack: Embedding Attacks as Logo-Like Pixels
Zhu, J., Zhao, H., Yu, H. L., & Liu, J. (2024) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3638529.3654231 · Cited by 1
Dorka, F., Otmani, K. E., Hentsch, M., Künster, N., & Palm, D. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-65400-8_40
CIQA: A Coding Inspired Question Answering Model
Arraf, M., & Radinsky, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657830
Personal Manifold: Management of Personal Data in the Age of Large Language Models
Halevy, A. Y., Li, Y., & Tan, W.-C. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00440
Shot Selection to Determine Legality of Actions in a Rule-Heavy Environment
Licato, J., Fields, L., Steinle, S., & Hollis, B. (2024) · IEEE Conference on Games (CoG) · doi:10.1109/cog60054.2024.10645668
Targeted training for numerical reasoning with large language models
Xiao, L., Liu, S., Zhu, Y., & Gong, C. B. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02216-1
FastLearn: A Rapid Learning Agent for Chat Models to Acquire Latest Knowledge
Fu, C., Wang, G., Lu, R., & Tang, S. (2024) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr62202.2024.00035 · Cited by 1
Assessing the Potential of Large Language Models for Chemical Engineering Applications
Xuan, L., Haoxiang, Z., Baozheng, J., & You, L. (2024) · doi:10.1145/3698385.3699878
Contrastive Learning-optimized recommendation data to Construct a Language Model RecommendationPreprint
Ka, Z., Zhao, P., & Zhang, B. (2024) · Research Square · doi:10.21203/rs.3.rs-5304404/v1
Tomorrow’s patient management: LLMs empowered by external tools
Szolnoky, K., Nordström, T., & Eklund, M. (2024) · Nature Reviews Urology · doi:10.1038/s41585-024-00965-w
Learning to Plan by Updating Natural Language
Guo, Y., Liang, Y., Wu, C., Wu, W., Zhao, D., & Duan, N. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.589
Empowering Comprehensive Biomedical Information Analysis with Large Language Models
Zhao, Y., Chen, J., Wu, N., & Wang, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0840-9_17
Addressing Past Tense Temporal Attacks in Large Language Models through Multiple Methods
Zhai, N.-J., Tuo, X., & Jin, Z. (2024) · International Conference on E-Society, E-Education and E-Technology (ICSET) · doi:10.1145/3704217.3705008
Kinder, L., & Käfer, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78955-7_21
Saranya, V., Devi., T., & Deepa, N. (2024) · International Conference on Ubiquitous Computing and Intelligent Information Systems (ICUIS) · doi:10.1109/icuis64676.2024.10867238
Collaborating LLMs and PLMs for Medical Tasks
Xu, G., Meng, Y., Wang, R., & Qi, G. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00060
Research Agenda for Sociotechnical Approaches to AI SafetyPreprint
Curtis, S., Iyer, R., Kirk‐Giannini, C. D., Krakovna, V., Krueger, D., Lambert, N., Marnette, B., McKenzie, C., Michael, J., Miyazono, E., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5097286
Li, L. (2025) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2025-0349
Recall Mechanism and Multi-Head Attention for Numerical Reasoning
Lai, L. F., Tan, T.-P., & Zeng, B. (2025) · Applied Sciences · doi:10.3390/app15073528
Zeng, Y., Guo, B., Yao, J., Wang, H., Ding, Y., Liang, Y., & Yu, Z. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00194
KI in der Wirbelsäulenchirurgie: Die Macht der Vorhersage
Hegewald, A. A. (2025) · Die Wirbelsäule · doi:10.1055/a-2279-4590
Cooperative Agentic Framework for Enhanced Function Calling
Jiang, W., Ma, J., Zhang, Y., Wang, Y., Ji, W., & Zhang, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4756-9_15
Web Agents and Benchmarks - A Survey Focusing on Algorithmic Strategies and Performance RatingsPreprint
Krupp, L., Geißler, D., Lukowicz, P., & Karolus, J. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/vhn2c_v1
ChatGPT: A Review of Capabilities, Challenges, and Future Perspectives
Jain, L., & Puri, D. M. (2025) · International Journal of Research Publication and Reviews · doi:10.55248/gengpi.6.0425.1694
A Method for Detecting Knowledge Conflicts in Chinese Intelligent Agent Interactions
Cheng, H., Gu, Y., Xi, M., Zhong, Q., & Liu, W. (2025) · Interdisciplinary Journal of Information Knowledge and Management · doi:10.28945/5497
Programmable NLU Powered 3-in-1 Bot :Video, Voice, Text
Alladi, R. (2025) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem49760
A Middleware System for Detecting and Mitigating Unsafe Tool Use in Large Language ModelsPreprint
Amos, F., Emmanuel, S., & Chukwuemeka, J. (2025) · Preprints.org · doi:10.20944/preprints202506.1398.v1
Advancing Retrieval-Augmented Generation for Enhanced QA Performance: A Multi-Agent ChatPDF Approach
(2025) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets79964
Blueprint for Intelligence : Multi-Agent LLM Model Design
(2025) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets79973
A Novel Visual-Enhanced Dual Stream Long-Term Decision Framework for Large Language Model Agents
Wang, X., Zhao, J., Shi, J., Li, L., & Zeng, D. D. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6599-0_23
ВИКОРИСТАННЯ ВЕЛИКИХ МОВНИХ МОДЕЛЕЙ ДЛЯ КЕРУВАННЯ СИСТЕМОЮ КОНТРОЛЮ КЛІМАТИЧНИХ УМОВ У ЗЕРНОСХОВИЩІ
Добряк, О., & Донченко, Є. (2025) · Наука і техніка сьогодні · doi:10.52058/2786-6025-2025-6(47)-1112-1132
Mast, J., Geiß, C., & Taubenböck, H. (2025) · Joint Urban Remote Sensing Event (JURSE) · doi:10.1109/jurse60372.2025.11076073
Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning
Jain, K., Kate, K., Tsay, J., Goues, C. L., & Hirzel, M. J. (2025) · doi:10.1109/ast66626.2025.00016
AutoBackend: Agent-Driven Framework for Intelligent Backend Development
Lin, R., Hu, B., & Zhang, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9986-5_35
Agentic Approach in the Quest for AGI
Basavaraj, M. (2025) · Indian Journal of Computer Science and Technology · doi:10.59256/indjcst.20250402043
Fusing Highly Specialized Language Models for Comprehensive Expertise
Ding, N., Chen, Y., Cui, G., Lv, X., Zhao, W., Zhang, K., Xie, R., Zhou, B., Liu, Z., & Sun, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.42
Self-play through Computational Runtimes improves Chart Reasoning
Misiūnas, T., Mansoor, H., Uijlings, J., Riva, O., & Cărbune, V. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.559
Metadata
- Authors
- Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T.
- ORCID
- Zettlemoyer, Luke
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 68539-68551
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2302.04761
- OpenAlex
- W7133214447
- Cited by
- 1,150
- References
- 0
- Citations by year
- 2026: 730 · 2025: 136 · 2024: 165 · 2023: 119
- FWCI
- 492.67
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- tool use, language models, self-supervised learning, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
