The DOI matches its DataCite record; 6 of the 42 references it cites were found in a registry or an index; 12 of the 18 sources its claims cite were checked for support.
Toolformer: Language Models Can Teach Themselves to Use Tools
Open accessAbstract
Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q\&A system, two different search engines, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Toolformer is a 6.7B GPT-J model trained self-supervised to insert API calls for question answering, Wikipedia search, a calculator, machine translation, and a calendar, deciding when and how to call them based on whether calls reduce future-token loss. It improves zero-shot downstream performance, often beating much larger GPT-3/OPT models, without increasing perplexity when API calls are disabled.
Key findings
- On LAMA subsets, Toolformer improves over the best baseline by 11.7, 5.2, and 18.6 points on SQuAD, Google-RE, and T-REx, respectively, and uses the question answering tool in 98.1% of cases; it outperforms OPT (66B) and GPT-3 (175B).
- On math benchmarks, Toolformer reaches ASDiv 40.4, SVAMP 29.4, and MAWPS 44.0, versus GPT-J’s 7.5, 5.2, and 9.9, using the calculator for 97.9% of examples.
- On question answering, Toolformer scores WebQS 26.3, NQ 17.7, and TriviaQA 48.8, mostly using Wikipedia search (99.3%), but still lags GPT-3 (175B) at 29.0, 22.6, and 65.9.
- On temporal datasets, Toolformer reaches DATESET 27.3 versus GPT-J 3.9, with the calendar used for 54.8% of examples; on TEMPLAMA it scores 16.3, but the calendar is used for only 0.2%.
- Language modeling perplexity is not degraded: Toolformer (disabled) has WikiText 10.3 and CCNet 10.5, matching GPT-J + CC. Tool use emerges around 775M parameters.
Methods
- Self-supervised pipeline: sample candidate API calls with an LM, execute them, filter calls by whether they reduce the weighted loss over future tokens by at least τ_f, merge useful calls into a CCNet subset, and finetune GPT-J on the augmented dataset.
- Tools: Atlas for QA, BM25 retriever over Wikipedia for search, a Python calculator for basic arithmetic, 600M NLLB for machine translation, and a calendar API; up to 25k examples per API, batch size 128, learning rate 1e-4.
- Evaluation: zero-shot downstream tasks with greedy decoding modified to allow `<API>` if among top k=10 tokens, at most one API call per input; baselines include GPT-J, GPT-J + CC, Toolformer (disabled), OPT (66B), and GPT-3 (175B).
Limitations
- Stated: Toolformer cannot chain tools, cannot use tools interactively, is sensitive to input wording, is sample-inefficient for some tools (e.g., over a million documents yield only a few thousand useful calculator calls), and does not account for tool-dependent computational cost.
- Evident from the text: the one-API-call-per-example restriction and independent sampling prevent chained calls; TEMPLAMA gains are not attributable to the calendar tool; MLQA does not consistently outperform GPT-J because CCNet finetuning can cause distribution shift; QA lags GPT-3 partly because the search engine is simple and Toolformer cannot reformulate queries or browse multiple results.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Own interpretation · 18 Own result · 17 No basis given · 6 Own method · 5 Partly supported · 3 Supported · 3 Sources not found · 1 Mixed support · 1 Not checked yet · accuracy not yet measured
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“or limit tool use to task-specific settings only (e.g., Gao et al., 2022;Parisi et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Disputedcites Gao et al., 2022 sourceExperimental — accuracy not yet measured
Section 7 (Related Work; page unavailable) explicitly contrasts PAL with task-specific tools, saying its Python interpreter handles both arithmetic and dates without specialized modules. Section 4 (Experimental Setup; page unavailable) also describes evaluations across mathematical, symbolic, and algorithmic tasks. The paper therefore does not limit PAL’s tool use to task-specific settings only. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Partly supportedcites Parisi et al., 2022 sourceExperimental — accuracy not yet measured
The Results (p. 3) evaluate TALM on two specific tasks, Natural Questions and MathQA, with tools suited to each. But the Conclusion (p. 4) describes TALM as a framework for arbitrary tools that makes exploring additional tools and tasks possible. The paper’s evaluations are task-specific; its approach is not limited to task-specific settings only. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“These limitations include an inability to access up-to-date information on recent events (Komeili et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Komeili et al., 2022 source
The Introduction says static language models cannot account for new information because their knowledge is frozen at training, and contrasts them with an internet-augmented model that can use the latest information. The Conclusions likewise describe models limited to knowledge in their fixed training set. This supports the claim as a limitation of LMs without dynamic access to outside information. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“existing approaches either rely on large amounts of human annotations (Komeili et al., 2022;Thoppilan et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Komeili et al., 2022 source
Section 3.2 says the internet-search model's modules require supervised context–search-query and context–response pairs, supplied by the data collected for this work. Section 4.1 reports a substantial crowdsourced dataset: 9,633 dialogues and 82,952 training utterances. This supports the claim for Komeili et al.'s approach, without establishing that every external-tool approach has the same requirement. (LLM, conf=0.91)
- Supportedcites Thoppilan et al., 2022 source
Section 6.2 (p. 8) describes LaMDA learning to consult external tools using 40,000 annotated dialog turns and 9,000 turns with human-assigned labels. This supports the claim that this tool-using approach relies on substantial human annotation, although the Discussion (p. 15) calls the amount modest relative to pre-training data. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“a lack of mathematical skills to perform precise calculations (Patel et al., 2021)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Partly supportedcites Patel et al., 2021 sourceExperimental — accuracy not yet measured
The Results on SVAMP report that the best-performing model achieved 43.8% accuracy, supporting a limitation in solving elementary arithmetic word problems. But the Problem Formulation section says the models generate expressions that are then evaluated to calculate answers; the paper does not establish a general inability of language models to perform precise calculations. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This is due to the fact that API calls for each tool are generated independently; as a consequence, there are no examples of chained tool use in the finetuning dataset.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“increasing k leads to the model doing API calls for more examples -from 40.3% and 8.5% with k = 1 (i.e., regular greedy decoding) to 98.1% and 100% for k = 10.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 5 Analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Toolformer achieves stronger results even when API calls are disabled.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Depending on the tool, our method is also very sample-inefficient; for example, processing more than a million documents results in only a few thousand examples of useful calls to the calculator API.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plaincorrelationalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Beyond this, we found models trained with Toolformer to often be sensitive to the exact wording of their input when deciding whether or not to call an API; this is perhaps unsurprising given that LMs are known to be very sensitive to the prompt they are provided with in both zeroand few-shot settings (Jiang et al., 2020;Schick and Schütze, 2021a).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the considerable improvement of Toolformer compared to other models can be fully accredited to the calendar tool, which it makes use of for 54.8% of all examples.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.5 Temporal Datasets
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 22
- The quote only partly states this claim. 3
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and the related tendency to hallucinate facts (Maynez et al., 2020;Ji et al., 2022)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Maynez et al., 2020 source
Section 2.1 says a decoder functioning like an open-ended language model is prone to extrinsic hallucinations. Section 5.3 reports that over 90% of the hallucinations produced by one summarization model were erroneous. The paper thus supports the broad claim, though its empirical results concern abstractive summarization models. (LLM, conf=0.88)
- Not matchedcites Ji et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Large language models achieve impressive zeroand few-shot results on a variety of natural language processing tasks (Brown et al., 2020;Chowdhery et al., 2022, i.a.)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Not matchedcites Brown et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Chowdhery et al., 2022 sourceExperimental — accuracy not yet measured
The Conclusion (p. 50) reports outstanding few-shot performance across English NLP tasks, and Section 6.1 (p. 11) describes results on 29 benchmarks. Section 6.8 (p. 33) evaluates zero-shot performance on five tasks, but the passages do not establish the claimed impressive zero-shot results across a variety of tasks. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Beyond this, we found models trained with Toolformer to often be sensitive to the exact wording of their input when deciding whether or not to call an API; this is perhaps unsurprising given that LMs are known to be very sensitive to the prompt they are provided with in both zeroand few-shot settings (Jiang et al., 2020;Schick and Schütze, 2021a).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
What the sources say
- Partly supportedcites Jiang et al., 2020 sourceExperimental — accuracy not yet measured
The Evaluation Results (p. 5) show that changing prompt wording substantially changed BERT’s accuracy, supporting prompt sensitivity in the paper’s knowledge-retrieval task. The paper does not establish the claim specifically for both zero-shot and few-shot settings; its prompt-selection method uses training examples (Top-1 Prompt Selection, p. 4). (LLM, conf=0.86)
- Supportedcites Schick and Schütze, 2021a source
Section 5 (Analysis; page unavailable) reports a large performance gap between the best and worst prompt patterns even after few-shot finetuning. This supports the prompt-sensitivity point for which Schick and Schütze are cited; the sentence also cites Jiang et al. for the broader zero-shot context. (LLM, conf=0.90)
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“and an unawareness of the progression of time (Dhingra et al., 2022).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Supportedcites Dhingra et al., 2022 source
The Introduction says standard LM training generally ignores temporal metadata and that standard LMs have no knowledge of what is “new” or “old.” This supports the claim as a broad description of standard LMs, while the paper distinguishes them from models trained with temporal context. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“TALM (Parisi et al., 2022), an approach that uses a similar self-supervised objective for teaching a model to use a calculator and a search engine, but explores this only in settings where a model is finetuned for downstream tasks.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 6 Related Work
What the sources say
- Partly supportedcites Parisi et al., 2022 sourceExperimental — accuracy not yet measured
Sections 3.1 and 3.2 (p. 3) describe retrieval and calculation tools for downstream question-answering and math tasks; §2.2 (p. 2) says TALM is fine-tuned during iterative self-play. But §2.2 describes supervised input–target pairs and a small set of tool-use demonstrations, not a self-supervised objective. The main retrieval experiment uses a BM25 index; a public search engine is used for an out-of-distribution test (§3.4, p. 4). (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the ability to leverage the provided tools only emerges at around 775M parameters: smaller models achieve similar performance both with and without tools.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.4 Scaling Laws
plaindirectn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“improvements on TEMPLAMA can not be attributed to the calendar tool, which is only used for 0.2% of all examples, but mostly to the Wikipedia search and question answering tools”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.5 Temporal Datasets
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“high values of L - i -L + i typically correspond to useful API calls, whereas low values correspond to API calls that do not provide any information that is useful for predicting future tokens.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > Data Quality
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“OPT and GPT-3 perform surprisingly weak across all languages, mostly because they fail to provide an answer in English despite being instructed to do so.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, when deciding whether or not to make an API call, Toolformer currently does not take into account the tool-dependent, computational cost incurred from making an API call.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 7 Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Depending on the language, this tool is used for 63.8% to 94.9% of all examples; the only exception to this is Hindi, for which the machine translation tool is used in only 7.3% of cases.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As one would expect, finetuning on CCNet leads to slightly improved performance on a different CC-Net subset, but it slightly deteriorates performance on WikiText, presumably because the original pre-”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Toolformer, which is based on a pretrained GPT-J model (Wang and Komatsuzaki, 2021) with 6.7B parameters, achieves much stronger zero-shot results, clearly outperforming a much larger GPT-3 model (Brown et al., 2020) and several other baselines on various tasks.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Not matchedcites Brown et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“for 97.9% of all examples the model decides to ask the calculator tool for help.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Toolformer clearly outperforms all other models based on GPT-J, this time mostly relying on the Wikipedia search API (99.3%) to find relevant information.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.3 Question Answering
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the Wikipedia search tool for most examples, Toolformer clearly outperforms baselines of the same size, but falls short of GPT-3 (175B).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Toolformer outperforms all baselines for both TEM-PLAMA and DATESET.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.5 Temporal Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nonetheless, allowing the model to make API calls more than doubles performance for all tasks, and also clearly outperforms the much larger OPT and GPT-3 models.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Toolformer outperforms all baselines, but does not make use of the calendar tool for TEMPLAMA.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the model independently decides to ask the question answering tool for the required information in almost all cases (98.1%); for only very few examples, it uses a different tool (0.7%) or no tool at all (1.2%).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.1 LAMA
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“improving upon the best baseline by 11.7, 5.2 and 18.6 points, respectively.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.1 LAMA
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“A potential reason for GPT-J not suffering from this problem is that it was trained on more multilingual data than both OPT and GPT-3, including the EuroParl corpus (Koehn, 2005;Gao et al., 2020).”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
What the sources say
- Not matchedcites Koehn, 2005
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Gao et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we prevent Toolformer from using the Wikipedia Search API to avoid giving it an unfair advantage.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.1 LAMA
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In a similar spirit to these approaches, Toolformer is trained on its own predictions after applying a perplexity-based filtering step.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 6 Related Work
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: The remaining examples after filtering are close enough to the original distribution so that the model's language modeling abilities remain unaffected.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“some amount of noise in the API calls that are not filtered can actually be useful as it forces the model finetuned on C * to not always blindly follow the results of each call it makes.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > Data Quality
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This ensures that the model does not lose any of its generality and language modeling abilities.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We surmise that this is because the model is finetuned on many examples of API calls and their results, improving its own mathematical capabilities.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.2 Math Datasets
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, Toolformer learns for itself to explicitly asks for the right information.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 6 Related Work
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For Toolformer, we disable the question answering tool as this would make solving the tasks trivial”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.3 Question Answering
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we only at most one API call per input to make sure the model does not get stuck in a loop where it constantly calls APIs without producing any actual output.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2 Downstream Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While during data generation the model can look ahead to generate API calls, this is not possible at inference time, so we want to dissuade the model from calling the API in such cases.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.1 Implementation Question Answering
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“difficulties in understanding low-resource languages (Lin et al., 2021)”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 1 Introduction
What the sources say
- Not matchedcites Lin et al., 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This filtering mechanism allows us to generate data more efficiently by focusing our API call generations in places where the MT tool is likely to be helpful.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.1 Implementation Question Answering
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is mainly because for some languages, finetuning on CCNet deteriorates performance; this might be due to a distribution shift compared to GPT-J's original pretraining data.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.4 Multilingual Question Answering
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We do not evaluate the perplexity of Toolformer with API calls enabled as computing the probability pM (xt | x1, . . ., xt-1) of token xt given x1, . . ., xt-1 would require marginalizing over all potential API calls that the model could make at position t, which is intractable.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > D DATESET
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is likely due to both the simplicity of our search engine (in many cases, it returns results that are clearly not a good match for a given query) and the inability of Toolformer to interact with it, e.g., by reformulating its query if results are not helpful or by browsing through multiple of the top results.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > 4.2.3 Question Answering
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
What humans find useful may be different from what a model finds useful.
“what humans find useful may be different from what a model finds useful.”
The authors state this as an assumption.
An API call is helpful to the model if providing both its input and output makes it easier for the model to predict future tokens, compared to not receiving the API call or receiving only its input.
“Intuitively, an API call is helpful to M if providing it with both the input and the output of this call makes it easier for the model to predict future tokens, compared to not receiving the API call at all, or receiving only its input.”
The authors state this as an assumption.
For creating the calendar dataset, the calendar date in such cases should be the date that the document was created.
“we operate under the assumption that the calendar date in such cases should be the date that the document was created.”
The authors state this as an assumption.
The remaining examples after filtering are close enough to the original distribution so that the model's language modeling abilities remain unaffected.
“While this filtering alters the distribution of training examples, we assume that the remaining examples are close enough to the original distribution so that M 's language modeling abilities remain unaffected.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“These approaches demonstrate that RL can improve complex behavior sequences in LLMs.”
From Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2025) · cites this paper as Schick et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Earlier efforts in this line of work rely on supervised fine-tuning (SFT), which trains the LLM on high-quality tool-use trajectories to explicitly demonstrate how tools should be invoked and integrated into task execution (Schick et al., 2023; Du et al., 2024; Liu et al., 2025g; Wang et al., 2025e).”
From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Schick et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While AI Agents have garnered considerable attention for their ability to automate structured tasks using LLMs and interfaces to specific tools, the literature highlights significant theoretical and practical limitations that inhibit their reliability, generalization, and long-term autonomy [136,162].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [162]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Despite this, LLMs have inherent limitations in their enclosed reasoning, particularly when addressing complex situations that extend beyond textual boundaries (Schick et al., 2023).”
From Scaling Large Language Model-based Multi-Agent Collaboration (2024) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Empirically, many early language agents simply use LLMs to propose an action (Schick et al., 2023), a sequence of actions (Huang et al., 2022b), or evaluate a fixed set of actions (Ahn et al., 2022) without intermediate reasoning or retrieval. Followup work (Yao et al., 2022b; Shinn et al., 2023; Xu et al., 2023b; Lin et al., 2023; Wang et al., 2023a; Park et al., 2023) has exploited intermediate”
From Cognitive Architectures for Language Agents (2023) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent work has equipped dialogue agents with the ability to use tools such as calculators and calendars, and to consult external websites 24,25 .”
From Role play with large language models (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Modern LLM agents can select tools, maintain state across steps, and modify the plan as new evidence emerges.[13], [14].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reliability comes mainly from typed tool interfaces, provenance-aware retrieval, explicit budgets, independent gates, and rollback-aware execution, not from model fluency alone [12], [37], [13], [75], [76].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“General agent frameworks motivate the separation between reasoning and acting, but operations demands that the separation is enforceable and auditable rather than merely prompt-stated [14], [13], [159], [161].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Context engineering primarily addresses the structural organization of the interaction interface between LLMs and their operational environment. This includes optimizing tool-integrated reasoning and selection pipelines (Qin et al., 2024a; Schick et al., 2023; Jia and Li, 2025) and standardizing communication protocols, such as MCP (Qiu et al., 2025c).”
From Memory in the Age of AI Agents (2025) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“language models have further acquired capabilities such as retrieval, calculation, and program invocation, extending multi-step task solving from pure text generation to the joint process of reasoning and execution [3]”
From A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving (2026) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Beyond conversational history, tool-based agents must retain tool inputs, execution outputs, and intermediate states, many of which are repeatedly referenced in subsequent reasoning steps (Schick et al., 2023).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, another critical feature of LLMs is their capability to use tools for automated online search, task design, and experiment execution [203–205], which exhibits their further capabilities in automatically collecting and preprocessing biomedical datasets, designing bioinformatic analyses to identify disease-related molecular regulations, conducting literature reviews for result inte”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [203]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLM-based approaches may assist by summarizing large-scale interaction logs, extracting structured event representations, or generating hypotheses about plausible attack paths; recent work on tool-augmented language models and automated log analysis suggests their usefulness in triage and forensic workflows (Schick et al., 2023;Solomon et al., 2026;Yao et al., 2022).”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the literature highlights significant theoretical and practical limitations that inhibit their reliability, generalization, and long-term autonomy [138], [164].”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [164]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These agents not only communicate and coordinate with one another but also interface with external resources such as tools, APIs, and structured data sources [24, 16].”
From Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This separation has been shown to improve model performance on a range of logical and mathematical reasoning tasks (Chen et al., 2022;Gao et al., 2023;Schick et al., 2023;Wang et al., 2022;Zhou et al., 2022;Cobbe et al., 2021).”
From Reliable Reasoning Beyond Natural Language (2024) · cites this paper as Schick et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notable examples of such systems include the likes of LaMDA (Thoppilan et al., 2022), Toolformer (Schick et al., 2023), and Mind's Eye (Liu et al., 2022).”
From ChatGPT, extended: large language models and the extended mind (2025) · cites this paper as Schick et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“A promising design paradigm for AI agents [65] is to combine an LLM with tools that interact with a broader environment [14, 35, 40, 47, 51, 53, 55, 69].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [51]
The cited source says what the claim attributes to it.
In the citing paper
“ReAct interleaves language reasoning with environment actions [16], while Toolformer demonstrates self-supervised learning of when and how to call external APIs [17].”
From HELIX: Model-Harness Co-evolution for Recursive Self-Improvement (2026) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“methods such as Toolformer showed that LLMs can learn to call external APIs to aid in problem-solving [18].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Toolformer employs a training approach to determine the optimal usage of specific APIs and integrates the obtained results into subsequent token predictions.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [172]
The cited source says what the claim attributes to it.
In the citing paper
“Skill-related work has expanded from narrow function calling and tool learning toward reusable capability packages, registries, and progressive disclosure mechanisms [OpenAI, 2023a, Schick et al., 2023, Wang et al., 2023a, Anthropic, 2025, 2026, Jiang et al., 2026b].”
From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Schick et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“The motivation behind Toolformer [246] aligns closely with ReAct; however, it goes a step further by combining diverse tools within a single model. This integration provides the model with flexible decision-making abilities and improved generalization capabilities, achieved through a simple yet effective self-supervised method.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [246]
The cited source says what the claim attributes to it.
In the citing paper
“This motivates an established alternative-the tool-use paradigm [9, 10]-in which, rather than perform every reasoning step internally, a model delegates well-specified tasks to external tools.”
From PrologMCP: A Standardized Prolog Tool Interface for LLM Agents (2026) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“Various studies (Liu et al., 2023;Parisi et al., 2022;Schick et al., 2023;Shen et al., 2023;Lu et al., 2023;Paranjape et al., 2023;Liang et al., 2023) have demonstrated that supplementing LLMs with tools, such as calculators, search engines, translation systems, calendars, or even API calls on other models, can help solve tasks that are not easily addressed by LLMs alone.”
From Large language models as tool makers (2023) · cites this paper as Schick et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“As discussed before, Agentic AI systems inherit the modularity of AI Agents but extend their architecture to support distributed intelligence, inter-agent communication, and iterative planning.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [162]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Agents can also implement a Tree-of-Thoughts strategy, collectively exploring different reasoning branches in parallel and exchanging information to decide on the best path forward [18,20].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Indirect prompting could be leveraged to leak users' chat sessions [52] or exfiltrate users' data (e.g., credentials, personal information).”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [52]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Agentic AI systems consist of multiple agents, each assigned a specialized function or task (e.g., a summarizer, a retriever, or a planner).”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [164]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Agentic AI systems inherit the modularity of AI Agents but extend their architecture to support distributed intelligence, inter-agent communication, and iterative planning.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [164]
Experimental — accuracy not yet measuredIn the citing paper
“Equipping an LLM with a suite of tools, such as calculators or web search, has proven superiority in tasks that require multistep reasoning and planning17,18.”
From Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology (2025) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.2 Prompts
“This is three times less than the 2169 goals last year.”
§ Toolformer: Language Models Can Teach Themselves to Use Tools > A.2 Prompts
“This is twenty goals more than the [Calculator(723 -20)] 703 goals last year.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 12
- Ricardo Baeza-Yates, Berthier Ribeiro-Neto, et al. 1999. Modern information retrieval, volume 463. ACM press New York.
- Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1533-1544, Seattle, Wash- ington, USA. Association for Computational Lin- guistics.
- Sergey Brin. 1999. Extracting patterns and relations from the world wide web. In The World Wide Web and Databases, pages 172-183, Berlin, Heidelberg. Springer Berlin Heidelberg.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Junxian He, Jiatao Gu, Jiajun Shen, and Marc'Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations.
- Gautier Izacard and Edouard Grave. 2021. Distilling knowledge from reader to retriever for question an- swering. In International Conference on Learning Representations.
- Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of machine translation summit x: papers, pages 79-86.
- David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Pro- ceedings of the Human Language Technology Con- ference of the NAACL, Main Conference, pages 152- 159, New York City, USA. Association for Compu- tational Linguistics.
- Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture mod- els. In International Conference on Learning Repre- sentations.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Lan- guage models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Roi Reichart and Ari Rappoport. 2007. Self-training for enhancement and domain adaptation of statisti- cal parsers trained on small datasets. In Proceed- ings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 616-623, Prague, Czech Republic. Association for Computational Lin- guistics.
- Guillaume Wenzek, Marie-Anne Lachaux, Alexis Con- neau, Vishrav Chaudhary, Francisco Guzmán, Ar- mand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 4003-4012, Marseille, France. European Language Resources Association.
Cited by
Sorted by citations
Items 1,201–1,300 of 3,208
Beyond decomposition: Hierarchical dependency management in multi‐document question answering
Zheng, X., Li, Z., Chen, Q., & Zhang, Y. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24971 · Cited by 3
The 2nd Futuredial Challenge: Dialog Systems With Retrieval Augmented Generation (Futuredial-RAG)
Cai, Y., Chen, S., Wu, Y., Huang, Y., Feng, J., & Ou, Z. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832299 · Cited by 3
MedicalGLM: A Pediatric Medical Question Answering Model with a quality evaluation mechanism
Wang, X., Sun, Z., Wang, P., & Wei, B. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104793 · Cited by 3
SOLAR: Illuminating LLM performance in API discovery and service ranking for edge AI and IoT
Al‐Masri, E., & Subramanian, I. N. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101630 · Cited by 3
Data driven discovery of escape phenomena in stochastic systems
Liu, J., Zhao, J., Yi, M., Duan, J., & Chen, X. (2025) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0264403 · Cited by 3
Zhang, H. (2024) · Journal of Advanced Computing Systems · doi:10.69987/jacs.2024.40305 · Cited by 3
Braga, V. (2023) · Journal of Engineering Science · doi:10.52326/jes.utm.2023.30(3).07 · Cited by 3
Tool Augmented LLMs for Big Data Analysis
Essabri, M. A., Jamal, R., & Erradi, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-67321-4_7 · Cited by 3
RRdE: A Decision Making Framework for Language Agents in Interactive Environments
Zhou, X., Li, L., & Zeng, D. D. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650094 · Cited by 3
Growing a Tail: Increasing Output Diversity in Large Language Models
Shur-Ofry, M., Horowitz-Amsalem, B., Rahamim, A., & Belinkov, Y. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.100951 · Cited by 3
Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets
Azime, I. A., Tonja, A. L., Belay, T. D., Fuge, M. Y., Wassie, A. K., Jada, E. S., Chanie, Y., Sewunetie, W. T., & Yimam, S. M. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.25 · Cited by 3
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
Dutta, S., Pandey, I., Singh, J., Manchanda, S., Chakrabarti, S., & Chakraborty, T. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29750 · Cited by 4
Xia, S.-J., Zhang, H., Jiang, Y., Chen, X., Chen, Y., Li, Z., Wan, Z., & Sun, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5731427 · Cited by 3
Suppressing echo cascades in language-model agents with multi-critic plan selectionPreprint
Li, R., Cao, S., Liu, R., & Duprey, A. (2026) · Research Square · doi:10.21203/rs.3.rs-8769125/v1 · Cited by 3
A high-level programming language for generative biology with ProtoPreprint
Merchant, A. T., Guo, D., Viggiano, B., Brennan-Almaraz, L., Hur, E., Mai, T., Yin, P., King, S. H., Ashley, E. A., & Hie, B. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.22.733870 · Cited by 5
CyGPT: Knowledge Graph-Based Enhancement Techniques for Large Language Models in Cybersecurity
Ou, L., Ni, X., Wu, W., & Tian, Z. (2024) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc63484.2024.00036 · Cited by 3
Shao, Y., You, W., Zheng, Z., Lu, Y., Yang, C., & Zhou, Z. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2470285 · Cited by 3
LLMAGENTNET: A COLLABORATIVE NETWORK OF AUTONOMOUS AI AGENTS FOR COMPLEX TASK EXECUTION
Бідочко, А., & Виклюк, Я. І. (2025) · Науковий вісник НЛТУ України · doi:10.36930/40350412 · Cited by 3
Big Loop and Atomization: A Holistic Review on the Expansion Capabilities of Large Language Models
Hu, Z., Huang, Y., Feng, J., & Deng, C. (2025) · Applied Sciences · doi:10.3390/app15179466 · Cited by 3
Collaborating with LLMs Through a Voice and Graphical User Interface
Sorokin, L., Huynh, K., Eiband, M., Stappen, L., & Dillmann, J. (2025) · International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI Adjunct) · doi:10.1145/3737821.3749555 · Cited by 3
Yi, J., Du, F., Nie, Y., Liang, W., Zhou, X., Chen, J., Li, G., Liu, M., Lv, Y., Zhao, W., et al. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104651 · Cited by 3
Why Text Prevails: Vision May Undermine Multimodal Medical Decision Making
Dai, S., Li, L., Zhao, K., Lila, E., Crane, P. K., Huang, H., Xu, D., Tang, H., & Zhan, L. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00245 · Cited by 3
From Scanning Guidelines to Action: A Robotic Ultrasound Agent with LLM-Based Reasoning
Bi, Y., Zhou, Y., Liu, P., Li, F., Jiang, Z., & Navab, N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38236-8_21 · Cited by 3
Think-Augmented Function Calling: Improving LLM Parameter Accuracy through Embedded Reasoning
Wei, L., Ou, J., Peng, X., & Wang, B. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462757 · Cited by 3
TIC: Translate-Infer-Compile for Accurate “Text to Plan” Using LLMs and Logical Representations
Agarwal, S., & Sreepathy, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-71170-1_19 · Cited by 3
FaceOracle: Chat with a Face Image Oracle
Kabbani, W., Raja, K. B., Ramachandra, R., & Busch, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92089-9_14 · Cited by 3
Yin, K., Ji, X., Ding, Z., Hou, S., & Wang, Z. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122601 · Cited by 3
Iterative improvements from feedback for language modelsPreprint
Li, Y. (2023) · doi:10.14293/pr2199.000220.v1 · Cited by 3
PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play
Fang, W., Zhang, Y., Qian, K., Glass, J. R., & Zhu, Y. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1347 · Cited by 3
SoK: Measuring what Matters for Closed-Loop Security Agents
Khurana, M., & Jain, R. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00151 · Cited by 4
Co-Disclosing the Computer: LLM-Mediated Computing through Reflective Conversation
Rost, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791769 · Cited by 3
Tian, W., & Zhou, Y. (2026) · Expert Systems · doi:10.1111/exsy.70341 · Cited by 3
When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents
Liu, S., Tang, X., Yang, X., Liang, L. Y., Zhou, B., Xiao, W., & Liu, W. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582512 · Cited by 3
Querying Structured Data Through Natural Language Using Language Models
Micu-Hontan, V., Bunea, A.-A., Tantaroudas, N. D., & Popovici, D.-M. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-35043-5_10 · Cited by 3
A lifecycle-oriented survey of LLM-based agents: Workflow, graph augmentation, and future directions
Kang, C., & Li, K. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134838 · Cited by 3
Concise and Precise Context Compression for Tool-Using Language Models
Xu, Y., Feng, Y., Mu, H., Hou, Y., Li, Y., Wang, X., Zhong, W., Li, Z., Tu, D., Zhu, Q., et al. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.974 · Cited by 3
Beyond Text: Unveiling Multimodal Proficiency of Large Language Models with MultiAPI Benchmark
Liu, X., Lin, J., & Zhang, J. (2024) · doi:10.18653/v1/2024.knowllm-1.4 · Cited by 3
Evolutionary Contrastive Distillation for Language Model Alignment
Katz-Samuels, J., Li, Z., Yun, H., Nigam, P., Xu, Y., Petricek, V., Yin, B., & Chilimbi, T. M. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.307 · Cited by 3
Creative Planning with Language Models: Practice, Evaluation and Applications
Spangher, A., Huang, T., Laban, P., & Peng, N. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-tutorial.1 · Cited by 3
Sfeir, G., Nova, G., Hess, S., & van Cranenburgh, S. (2026) · Journal of Choice Modelling · doi:10.1016/j.jocm.2026.100623 · Cited by 3
EpiK-Eval: Evaluation for Language Models as Epistemic Models
Prato, G., Huang, J., Parthasarathi, P., Sodhani, S., & Chandar, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.593 · Cited by 3
TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition
Vellaisamy, P., Tripathi, S., Natarajan, V., Thenarasu, S. S., Blanton, S., & Shen, J. P. (2026) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass69572.2026.00014 · Cited by 3
Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models
Pan, Z., Luo, H., Li, M., & Liu, H. (2024) · arXiv · doi:10.48550/arxiv.2403.17359 · Cited by 3
Agent Alignment in Evolving Social Norms
Li, S., Sun, T., Cheng, Q., & Qiu, X. (2024) · arXiv · doi:10.48550/arxiv.2401.04620 · Cited by 3
TSGuard: Automated User-Centric Incident Diagnosis for AI Workloads in the Cloud
Yang, Y., Deng, Y., Xiong, Y., Li, B., Xu, H. L., & Cheng, P. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797149 · Cited by 3
OpenDerisk: An Industrial Framework for AI-Driven SRE, with Design, Implementation, and Case Studies
Di, P., Chen, F., Bai, X., Yang, H., Li, Q., Wei, G., Mou, J., Shi, F., Chen, K., Tang, P., et al. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786864 · Cited by 3
Li, H., Guo, W., Wu, H., Wu, M., Zhang, J., Zhu, Q., Liu, Y., Huang, X., & Tao, C. (2026) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2026.3695749 · Cited by 3
Teaching Language Models to Reason with Tools
Li, C. N., Tang, Z., Li, Z., Xue, M., Bao, K., Ding, T., Sun, R., Wang, B., Wang, X., Lin, J., et al. (2025) · neural information processing systems · doi:10.52202/085713-2242 · Cited by 3
Cost-Aware Model Orchestration for LLM-based Systems
Smirnova, D., Nasiri, H., Adamska, M., Yu, Z., & Garraghan, P. (2026) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3805621.3807624 · Cited by 3
Architectures for Building Agentic AI
Nowaczyk, S. (2026) · Studies in computational intelligence · doi:10.1007/978-3-032-18585-3_3 · Cited by 3
Skeleton-of-thought: Prompting LLMs for efficient parallel generation
Ning, X., Lin, Z., Zhou, Z., Wang, Z., Yang, H., & Wang, Y. (2024) · Lirias · Cited by 3
KORAL: Knowledge Graph Guided LLM Reasoning for SSD Operational Analysis
Akewar, M., Madireddy, S., Luo, D., & Bhimani, J. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00085 · Cited by 3
When to Trust Tools? Adaptive Tool Trust Calibration For Tool-Integrated Math Reasoning
Xu, R., Ji, Y., Luo, Y., Li, J., Li, D., Li, P., Li, J., & Zhang, M. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.877 · Cited by 3
Intelligence as Managed Autonomy: Failure, Escalation, and Governance for Agentic AI Systems
Ramaswamy, S. (2026) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-026-02413-w · Cited by 3
Autonomous Knowledge Graph Exploration with Adaptive Breadth-Depth Retrieval
Polonuer, J., Vittor, L., Arango, I., Noori, A., Clifton, D. A., Corro, L. D., & Žitnik, M. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.714 · Cited by 3
Bringing Agentic Search to Earth Observation Data Discovery
Yu, M., Sun, Y., Yi, C., Wen, Y., & Yang, H. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3841109 · Cited by 3
Agent-Enhanced Large Language Models for Researching Political Institutions
Loffredo, J., & Yun, S. (2025) · Journal of Political Institutions and Political Economy · doi:10.1561/113.00000125 · Cited by 3
LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrieval
Zhuang, L., Zhang, Q., Zhou, H., Zhang, Y., & Huang, X. X. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792336 · Cited by 3
Lecture I: Governing the Algorithmic City
Lazar, S. (2026) · Oxford University Press eBooks · doi:10.1093/9780197787977.003.0002 · Cited by 3
What Is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?
Triaridis, K., Kaliosis, P., Nguyen, E.-R., Xu, J., Samaras, D., & Le, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36987-1_4 · Cited by 3
Skeleton-of-thought: Large language models can do parallel decoding
Ning, X., Lin, Z., Zhou, Z., Wang, Z., Yang, H., & Wang, Y. (2023) · Lirias · Cited by 3
Udías, Á., Alonso‐Ayuso, A., Sanchez, I., Hernandez, S., Castellanos, M. E., Díez, R. M., & Cano, E. L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05686 · Cited by 3
Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM ResponsesPreprint
Kim, J., Lee, J., Chang, Y., Choi, C., Kim, J., & Sohn, J.-Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.17097 · Cited by 3
InteraRec: Screenshot Based Recommendations Using Multimodal Large Language ModelsPreprint
Karra, S. R., & Tulabandhula, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.00822 · Cited by 3
Logic Agent: Enhancing Validity with Logic Rule InvocationPreprint
Liu, H., Teng, Z., Zhang, C., & Zhang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.18130 · Cited by 3
Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM GenerationPreprint
Chen, L., Feng, Z., Liu, Z. Q., Jiang, W., Xu, J., Chen, Y., & Wang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.12411 · Cited by 3
MARCO: Multi-Agent Real-time Chat OrchestrationPreprint
Shrimal, A., Kanagaraj, S., Biswas, K., Raghuraman, S., Nediyanchath, A., Zhang, Y., & Yenigalla, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.21784 · Cited by 3
AgentRec: Agent Recommendation Using Sentence Embeddings Aligned to Human FeedbackPreprint
Park, J., & Zhang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.13333 · Cited by 3
OmniNova:A General Multimodal Agent FrameworkPreprint
Du, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.20028 · Cited by 3
CoMind: Towards Community-Driven Agents for Machine Learning EngineeringPreprint
Li, S., Sun, W., Li, S., Talwalkar, A., & Yang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.20640 · Cited by 3
AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level EvaluationPreprint
Huang, T., Cao, R., Zhang, Y., Kang, Z., Wang, Z., Wang, C., Luo, Y., Zheng, H., Qian, L., Lü, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.16952 · Cited by 3
Governing Automated Strategic IntelligencePreprint
Kruus, N., Thakur, M., Khoja, A., Nagel, L., Nicholson, M., Sharma, A., Hausenloy, J., KoTafoya, A., Mukhanova, A., Katila-Miikkulainen, A., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.17087 · Cited by 3
Visualizing Thought: Conceptual Diagrams Enable Robust Planning in LMMsPreprint
Borazjanizadeh, N., Herzig, R., Oks, E., Darrell, T., Feris, R. S., & Karlinsky, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.11790 · Cited by 3
AppSelectBench: Application-Level Tool Selection BenchmarkPreprint
Chen, T., Solodko, M., Wang, S., Ko, J., Hao, J., Banbury, C., Abdali, S., Amizadeh, S., Xiao, Q., Li, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.19957 · Cited by 3
Cheng-Hong, F., & Du, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.08308 · Cited by 3
Inducing Causal World Models in LLMs for Zero-Shot Physical ReasoningPreprint
Sharma, A., Gupta, A., Wang, C., Adebayo, C., & Kowalski, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.19855 · Cited by 3
Multi Agent based Medical Assistant for Edge DevicesPreprint
Gawade, S., Akhouri, S., Kulkarni, C., Samant, J. S., Sahu, P. P., Aastik, Pahal, J., & Meher, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.05397 · Cited by 3
Estimating the Empowerment of Language Model AgentsPreprint
Song, J., Gore, J., & Max Kleiman-weiner, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.22504 · Cited by 3
AI-driven ionic liquid discovery with unified chemical intelligencePreprint
Yin, Y., Fu, Y., Wang, S., Huo, L., Sun, P., Wang, H., Wang, X., Pan, C., Li, Z., Liu, Z., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.11257 · Cited by 3
AWARE-US: Preference-Aware Infeasibility Resolution in Tool-Calling AgentsPreprint
Kurmaz, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.02643 · Cited by 3
AgentCompress: Task-Aware Compression for Affordable Large Language Model AgentsPreprint
Taha, Z. A. K., Uddin, M. M., & Alam, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.05191 · Cited by 3
From Perception to Action: Spatial AI Agents and World ModelsPreprint
Felicia, G., Bryant, N., Putra, H., Gazali, A., Lobo, E., & Rojas, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.01644 · Cited by 3
Architecting AgentOS: From Token-Level Context to Emergent System-Level IntelligencePreprint
Li, C., Liu, X., Meng, X., & Zhao, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.20934 · Cited by 3
Enhancing Tool Calling in LLMs with the International Tool Calling DatasetPreprint
Zhang, Z., & Zhu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.05515 · Cited by 3
Agent Privilege Separation in OpenClaw: A Structural Defense Against Prompt InjectionPreprint
Cheng, D., & Tsao, W.-K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.13424 · Cited by 3
Guardrails as Infrastructure: Policy-First Control for Tool-Orchestrated WorkflowsPreprint
Sigdel, A., & Baral, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.18059 · Cited by 3
Uncertainty-Guided Latent Diagnostic Trajectory Learning for Sequential Clinical DiagnosisPreprint
Shen, X., Liu, H., Song, D., & Min, M. R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.05116 · Cited by 3
Can Coding Agents Be General Agents?Preprint
Ivanov, M., Rana, A., & Prabhakaran, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.13107 · Cited by 3
Bounded Autonomy for Enterprise AI: Typed Action Contracts and Consumer-Side ExecutionPreprint
Sohail, S., & Haider, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.14723 · Cited by 3
Can Current Agents Close the Discovery-to-Application Gap? A Case Study in MinecraftPreprint
Ziheng, Z., Tang, H., Zhang, J., Lin, H., Yang, B., Long, Q., Sun, F., Sun, Y., Liang, Y., Wu, Y. N., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.24697 · Cited by 3
When Agents Evolve, Institutions FollowPreprint
Fei, C., Guo, H., & Xiao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.27691 · Cited by 3
Wu, W., Zhu, S., Zhou, K., & Huang, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.02168 · Cited by 3
MemFlow: Intent-Driven Memory Orchestration for Small Language Model AgentsPreprint
Chen, J., Li, Y., & Wang, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.03312 · Cited by 3
When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent MemoryPreprint
Shao, J., Lu, Y., Zhang, Y., & Luo, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.07313 · Cited by 3
CrackMeBench: Binary Reverse Engineering for AgentsPreprint
David, I., & Gervais, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.10597 · Cited by 3
LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time ScalingPreprint
Cao, Q., Wang, Y., Qin, P., Zhang, S., & Xie, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.14186 · Cited by 3
Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge GraphsPreprint
Ding, W., Li, Z., Bai, L., Chen, Z., Su, K., Wang, F., Jin, X., Zhang, J., Guo, J., & Cheng, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.03705 · Cited by 3
Semantic Quorum Assurance: Collective Certification for Non-Deterministic AI InfrastructurePreprint
He, J., & Yu, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.08021 · Cited by 3
Strategic Decision Support for AI AgentsPreprint
Kiyani, S., Noorani, S., Pappas, G. J., & Hassani, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.12587 · Cited by 3
Metadata
- Authors
- Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T.
- ORCID
- Zettlemoyer, Luke
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 68539-68551
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2302.04761
- OpenAlex
- W7133214447
- Cited by
- 1,150
- References
- 0
- Citations by year
- 2026: 730 · 2025: 136 · 2024: 165 · 2023: 119
- FWCI
- 492.67
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- tool use, language models, self-supervised learning, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
