Cited by 169Domain Adaptation and Few-Shot LearningTopic ModelingNatural Language Processing Techniques

MetaICL: Learning to Learn In Context

Open access
Min, Sewon, Lewis, Mike, Zettlemoyer, Luke, Hajishirzi, Hannaneh·2022·Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Abstract

Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2022.naacl-main.201

Cites

131 of 131 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Instruction tuning of large language models on natural language tasks improves their usability and general performance.

    “Finetuning large language models (LLMs) on a variety of language tasks explained via instructions (instruction tuning) has been shown to improve model usability and general performance (Wei et al., 2022;Sanh et al., 2022;Min et al., 2022;Ouyang et al., 2022).”

    From OctoPack: Instruction Tuning Code Large Language Models (2023) · cites this paper as Min et al., 2022

    The paper’s instruction ablation reports that multi-task zero-shot models outperform a raw-LM baseline and that adding instructions improves MetaICL’s task performance (section “Number of meta-training tasks”; page unavailable). It does not establish the claimed improvement in model usability, and its strongest performance finding concerns combining instructions with MetaICL rather than instruction tuning alone. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

6 of the 137 references couldn’t be matched to a known paper.

Research works printed but not linked: 15

  1. Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second pascal recognising tex- tual entailment challenge. In Proceedings of the sec- ond PASCAL challenges workshop on recognising textual entailment.
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In NeurIPS.
  3. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ul- far Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX Secu- rity Symposium (USENIX Security 21).
  4. Michael Chen, Mike D'Arcy, Alisa Liu, Jared Fer- nandez, and Doug Downey. 2019. CODAH: An adversarially-authored question answering dataset for common sense. In Proceedings of the 3rd Work- shop on Evaluating Vector Space Representations for NLP.
  5. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-REx: A large scale alignment of natural language with knowledge base triples. In LREC.
  6. Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2019. QASC: A dataset for question answering via sentence compo- sition. In AAAI.
  7. Xin Li and Dan Roth. 2002. Learning question classi- fiers. In COLING.
  8. Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. 2020. Ethos: an on- line hate speech detection dataset. arXiv preprint arXiv:2006.08328. · arXiv
  9. Courtney Napoles, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Pro- ceedings of the Joint Workshop on Automatic Knowl- edge Base Construction and Web-scale Knowledge Extraction (AKBC-WEKEX).
  10. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Ad- versarial NLI: A new benchmark for natural lan- guage understanding. In ACL.
  11. NeurIPS.
  12. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavat- ula, and Yejin Choi. 2020b. Winogrande: An adver- sarial winograd schema challenge at scale. In AAAI.
  13. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019b. Social iqa: Com- monsense reasoning about social interactions. In EMNLP-IJCNLP.
  14. Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018. CARER: Con- textualized affect representations for emotion recog- nition. In EMNLP.
  15. TACL.

Cited by

Sorted by citations

Authors
Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H.
Affiliations
  • University of Washington (US)
  • Meta (United States) (US)
Venue
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Published
2022
Pages
2791-2809
OpenAlex
W3210277894
Cited by
169
References
137
Citations by year
2026: 23 · 2025: 61 · 2024: 44 · 2023: 36 · 2022: 4 · 2021: 1
FWCI
20.04
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
Funders
  • National Science Foundation (2044660)
  • Office of Naval Research (N00014-18-1-2826, N00014)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
in-context learning, prompt engineering, large language models, meta-learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review