CoChat
Sign Up
Cited by 1,150Robot Manipulation and LearningAI-based Problem Solving and PlanningMultimodal Machine Learning Applications

Toolformer: language models can teach themselves to use tools

Open access
Schick, Timo, Dwivedi-Yu, Jane, Dessi, Roberto, Raileanu, Roberta, Lomeli, Maria, Hambro, Eric, Zettlemoyer, Luke, Cancedda, Nicola +1 more·2023·Advances in Neural Information Processing Systems 36
DOI 10.52202/075280-2997
Open this paper in CoChat — free accountOpen PDF

Abstract

Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q\&A system, two different search engines, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/075280-2997

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Modern LLM agents can select tools, maintain state across steps, and modify the plan as new evidence emerges.

    “Modern LLM agents can select tools, maintain state across steps, and modify the plan as new evidence emerges.[13], [14].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]

    The Introduction and Experiments sections support the tool-selection part: Toolformer learns to choose which tool to use, when, and how. The Limitations section says it cannot chain tool uses or interactively refine a search, so this source does not support maintaining state across steps or modifying a plan as new evidence emerges. Page numbers are not provided in our copy. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Reliability comes mainly from typed tool interfaces, provenance-aware retrieval, explicit budgets, independent gates, and rollback-aware execution, not from model fluency alone.

    “Reliability comes mainly from typed tool interfaces, provenance-aware retrieval, explicit budgets, independent gates, and rollback-aware execution, not from model fluency alone [12], [37], [13], [75], [76].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]

    The Abstract describes using external tools through APIs to improve performance, and §4.2 imposes a one-call-per-input limit to prevent loops. The paper does not establish typed interfaces, provenance-aware retrieval, independent gates, or rollback-aware execution as the main sources of reliability; §7 says Toolformer does not account for tool-dependent computational costs. Page numbers are unavailable in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • General agent frameworks motivate the separation between reasoning and acting, but operations demands that the separation is enforceable and auditable rather than merely prompt-stated.

    “General agent frameworks motivate the separation between reasoning and acting, but operations demands that the separation is enforceable and auditable rather than merely prompt-stated [14], [13], [159], [161].”

    From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [13]

    In “Sampling API Calls” (page unavailable), Toolformer distinguishes model-generated text from external action: decoding pauses when the model requests an API call, then resumes after the tool responds. This supports using the paper as an example of a model that acts through tools, but it does not establish a general separation between reasoning and acting or discuss the citing authors’ requirement that the separation be enforceable and auditable rather than prompt-stated. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Language models have acquired capabilities such as retrieval, calculation, and program invocation, extending multi-step task solving from pure text generation to the joint process of reasoning and execution.

    “language models have further acquired capabilities such as retrieval, calculation, and program invocation, extending multi-step task solving from pure text generation to the joint process of reasoning and execution [3]”

    From A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving (2026) · cites this paper as [3]

    The Abstract and §3 (Tools) support retrieval, calculation, and API-based tool use alongside text generation. The Sampling API Calls section also describes executing calls and incorporating their results. But §4.2 (Downstream Tasks) says Toolformer makes at most one API call per input, so this paper does not establish the claimed extension to multi-step reasoning-and-execution workflows. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Beyond conversational history, tool-based agents must retain tool inputs, execution outputs, and intermediate states, many of which are repeatedly referenced in subsequent reasoning steps.

    “Beyond conversational history, tool-based agents must retain tool inputs, execution outputs, and intermediate states, many of which are repeatedly referenced in subsequent reasoning steps (Schick et al., 2023).”

    From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Schick et al., 2023

    The Sampling API Calls section says Toolformer inserts API calls and their responses into the text before continuing generation, supporting the point about tool inputs and outputs being available to subsequent text. It does not say agents must retain intermediate states or that these records are repeatedly referenced in later reasoning steps. Section 4.2 also says its evaluation permits at most one API call per input. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 54

  1. Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, and Luke Zettle- moyer. 2021. Htlm: Hyper-text pre-training and prompting of language models. · DOI · Paper page
  2. Eugene Agichtein and Luis Gravano. 2000. Snowball: Extracting relations from large plain-text collections. In Proceedings of the Fifth ACM Conference on Dig- ital Libraries, DL '00, page 85-94, New York, NY, USA. Association for Computing Machinery. · DOI
  3. Ricardo Baeza-Yates, Berthier Ribeiro-Neto, et al. 1999. Modern information retrieval, volume 463. ACM press New York.
  4. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1533-1544, Seattle, Wash- ington, USA. Association for Computational Lin- guistics.
  5. Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hen- nigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Ge- offrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. 2021. Improving language models by retriev- ing from trillions of tokens. · DOI · Paper page
  6. Sergey Brin. 1999. Extracting patterns and relations from the world wide web. In The World Wide Web and Databases, pages 172-183, Berlin, Heidelberg. Springer Berlin Heidelberg.
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
  8. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghe- mawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fe- dus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankara- narayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Bren- nan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language modeling with pathways. · DOI · Paper page
  9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. · arXiv
  10. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. · arXiv
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Associ- ation for Computational Linguistics. · DOI · Paper page
  12. Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2022. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257-273. · DOI
  13. Leo Gao, Stella Biderman, Sid Black, Laurence Gold- ing, Travis Hoppe, Charles Foster, Jason Phang, Ho- race He, Anish Thite, Noa Nabeshima, et al. 2020. The pile: An 800gb dataset of diverse text for lan- guage modeling. arXiv preprint arXiv:2101.00027. · arXiv
  14. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Gra- ham Neubig. 2022. Pal: Program-aided language models. · DOI · Paper page
  15. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming-Wei Chang. 2020. Realm: Retrieval- augmented language model pre-training. · DOI · Paper page
  16. Junxian He, Jiatao Gu, Jiajun Shen, and Marc'Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations.
  17. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022. Unnatural instructions: Tuning lan- guage models with (almost) no human labor. · DOI
  18. Gautier Izacard and Edouard Grave. 2021. Distilling knowledge from reader to retriever for question an- swering. In International Conference on Learning Representations.
  19. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Atlas: Few-shot learning with retrieval augmented language models. · DOI · Paper page
  20. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022. Survey of hallu- cination in natural language generation. ACM Com- puting Surveys. · DOI · Paper page
  21. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438. · DOI · Paper page
  22. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale dis- tantly supervised challenge dataset for reading com- prehension. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601-1611, Van- couver, Canada. Association for Computational Lin- guistics. · DOI · Paper page
  23. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612.03651. · arXiv
  24. Nitish Shirish Keskar, Bryan McCann, Lav R. Varsh- ney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. · DOI · Paper page
  25. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of machine translation summit x: papers, pages 79-86.
  26. Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2022. Internet-augmented dialogue generation. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 8460-8478, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
  27. Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016. MAWPS: A math word problem repository. In Pro- ceedings of the 2016 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1152-1157, San Diego, California. Association for Computational Linguistics. · DOI
  28. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Al- berti, Danielle Epstein, Illia Polosukhin, Jacob De- vlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question an- swering research. Transactions of the Association for Computational Linguistics, 7:452-466. · DOI · Paper page
  29. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. 2022. Internet- augmented language models through few-shot prompting for open-domain question answering. arXiv preprint arXiv:2203.05115. · arXiv
  30. Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2019. Mlqa: Eval- uating cross-lingual extractive question answering. arXiv preprint arXiv:1910.07475. · arXiv
  31. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Na- man Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettle- moyer, Zornitsa Kozareva, Mona Diab, Veselin Stoy- anov, and Xian Li. 2021. Few-shot learning with multilingual language models. · DOI
  32. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factual- ity in abstractive summarization. · DOI · Paper page
  33. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Pro- ceedings of the Human Language Technology Con- ference of the NAACL, Main Conference, pages 152- 159, New York City, USA. Association for Compu- tational Linguistics.
  34. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture mod- els. In International Conference on Learning Repre- sentations.
  35. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020. A diverse corpus for evaluating and develop- ing English math word problem solvers. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975-984, On- line. Association for Computational Linguistics. · DOI · Paper page
  36. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2021. Webgpt: Browser- assisted question-answering with human feedback. · DOI · Paper page
  37. Aaron Parisi, Yao Zhao, and Noah Fiedel. 2022. Talm: Tool augmented language models. · DOI · Paper page
  38. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080-2094, Online. Association for Computational Linguistics. · DOI · Paper page
  39. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 2523-2544, Online. Association for Computational Linguistics. · DOI · Paper page
  40. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowl- edge bases? In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- IJCNLP), pages 2463-2473, Hong Kong, China. As- sociation for Computational Linguistics. · DOI · Paper page
  41. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Lan- guage models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  42. Roi Reichart and Ari Rappoport. 2007. Self-training for enhancement and domain adaptation of statisti- cal parsers trained on small datasets. In Proceed- ings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 616-623, Prague, Czech Republic. Association for Computational Lin- guistics.
  43. Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Se- bastian Riedel. 2022. Peer: A collaborative lan- guage model. · DOI · Paper page
  44. Timo Schick and Hinrich Schütze. 2021a. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the As- sociation for Computational Linguistics: Main Vol- ume, pages 255-269, Online. Association for Com- putational Linguistics. · DOI · Paper page
  45. Timo Schick and Hinrich Schütze. 2021b. Generating datasets with pretrained language models. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 6943- 6951, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI · Paper page
  46. Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, Morteza Behrooz, William Ngan, Spencer Poff, Naman Goyal, Arthur Szlam, Y-Lan Boureau, Melanie Kam- badur, and Jason Weston. 2022. Blenderbot 3: a de- ployed conversational agent that continually learns to responsibly engage. · DOI · Paper page
  47. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, De- hao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinod- kumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman- John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022. Lamda: Lan- guage models for dialog applications. · DOI · Paper page
  48. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Al- isa Liu, Noah A. Smith, Daniel Khashabi, and Han- naneh Hajishirzi. 2022. Self-instruct: Aligning lan- guage model with self generated instructions. · DOI · Paper page
  49. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raf- fel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Met- zler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models. · DOI · Paper page
  50. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Con- neau, Vishrav Chaudhary, Francisco Guzmán, Ar- mand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 4003-4012, Marseille, France. European Language Resources Association.
  51. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. · DOI · Paper page
  52. David Yarowsky. 1995. Unsupervised word sense dis- ambiguation rivaling supervised methods. In 33rd Annual Meeting of the Association for Computa- tional Linguistics, pages 189-196, Cambridge, Mas- sachusetts, USA. Association for Computational Linguistics. · DOI
  53. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. · DOI · Paper page
  54. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre- trained transformer language models. · DOI · Paper page

Cited by

Sorted by citations

Authors
Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T.
ORCID
Zettlemoyer, Luke
Venue
Advances in Neural Information Processing Systems 36
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2023
Pages
68539-68551
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/075280-2997
OpenAlex
W7133214447
Cited by
1,150
References
0
Citations by year
2026: 730 · 2025: 136 · 2024: 165 · 2023: 119
FWCI
492.67
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
tool use, language models, self-supervised learning, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review