The DOI matches its Crossref record; 131 of the 137 references it cites were found in a registry or an index.
MetaICL: Learning to Learn In Context
Open accessAbstract
Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finetuning large language models (LLMs) on a variety of language tasks explained via instructions (instruction tuning) has been shown to improve model usability and general performance (Wei et al., 2022;Sanh et al., 2022;Min et al., 2022;Ouyang et al., 2022).”
From OctoPack: Instruction Tuning Code Large Language Models (2023) · cites this paper as Min et al., 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 4,481
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 3,827
DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia
Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P. N., Hellmann, S., Morsey, M., van Kleef, P., Auer, S., et al. (2015) · Semantic Web · doi:10.3233/sw-140134 · Cited by 3,737
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 3,710
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Pang, B., & Lee, L. (2005) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1219840.1219855 · Cited by 2,213
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2067 · Cited by 2,010
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804
HellaSwag: Can a Machine Really Finish Your Sentence?Preprint
Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07830 · Cited by 1,693
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture notes in computer science · doi:10.1007/11736790_9 · Cited by 1,674
(2018) · International Conference on Learning Representations · Cited by 1,621
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P. E., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1260 · Cited by 1,602
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1160 · Cited by 1,593
Hidden factors and hidden topics
McAuley, J., & Leskovec, J. (2013) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2507157.2507163 · Cited by 1,565
Regularized multi--task learning
Evgeniou, T., & Pontil, M. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014067 · Cited by 1,473
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement LearningPreprint
Zhong, V., Xiong, C., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.00103 · Cited by 1,384
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343
Li, X., & Roth, D. (2002) · International Conference on Computational Linguistics · doi:10.3115/1072228.1072378 · Cited by 1,308
Neural Network Acceptability Judgments
Warstadt, A., Singh, A., & Bowman, S. R. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00290 · Cited by 1,215
Narayan, S., Cohen, S. B., & Lapata, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1206 · Cited by 1,207
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179
A perspective view and survey of meta-learning
Vilalta, R., & Drissi, Y. (2002) · Artificial Intelligence Review · doi:10.1023/a:1019956318069 · Cited by 1,108
Social IQa: Commonsense Reasoning about Social Interactions
Sap, M., Rashkin, H., Chen, D., LeBras, R., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1454 · Cited by 1,106
Automatically Constructing a Corpus of Sentential Paraphrases.
Dolan, W. B., & Brockett, C. (2005) · Cited by 1,101
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., D., M., Châtel, G., Chaumond, J., Rault, T., Voss, C., Wang, F., et al. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3385997 · Cited by 1,057
RACE: Large-scale ReAding Comprehension Dataset From Examinations
Lai, G., Xie, Q., Liu, H., Yang, Y., & Hovy, E. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1082 · Cited by 1,044
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification
Barbieri, F., Camacho-Collados, J., Anke, L. E., & Neves, L. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.148 · Cited by 929
Robust Disambiguation of Named Entities in Text
Hoffart, J., Yosef, M. A., Bordino, I., Fürstenau, H., Pinkal, M., Spaniol, M., Taneva, B., Thater, S., & Weikum, G. (2011) · Cited by 925
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1421 · Cited by 919
Levesque, H. J., Davis, E., & Morgenstern, L. (2011) · Cited by 879
WikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829
CommonsenseQA: A Question Answering Challenge Targeting Commonsense KnowledgePreprint
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00937 · Cited by 822
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Zellers, R., Bisk, Y., Schwartz, R., & Choi, Y. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1009 · Cited by 801
The Third PASCAL Recognizing Textual Entailment Challenge
Giampiccolo, D., Magnini, B., Dagan, I., & Dolan, B. (2007) · ACL-PASCAL@ACL · doi:10.3115/1654536.1654538 · Cited by 801
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C., & Renshaw, E. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 778
Pang, B., & Lee, L. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0506075 · Cited by 744
Language Models as Knowledge Bases?Preprint
Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A., & Riedel, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01066 · Cited by 714
NewsQA: A Machine Comprehension Dataset
Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2623 · Cited by 706
A SICK cure for the evaluation of compositional distributional semantic models
Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Bernardi, R., & Zamparelli, R. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/39qdhuevzbqa · Cited by 693
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
Mathew, B., Saha, P., Yimam, S. M., Biemann, C., Goyal, P., & Mukherjee, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17745 · Cited by 657
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1300 · Cited by 655
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610 · cited 2×
“Moreover, it can have high variance and poor worst-case accuracy (Perez et al., 2021; Lu et al., 2021).”
BackgroundMethodThe NarrativeQA Reading Comprehension Challenge
Kočiský, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., & Grefenstette, E. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00023 · Cited by 596
WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Sakaguchi, K., Le Bras, R., Bhagavatula, C., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6399 · Cited by 579
Crowdsourcing Multiple Choice Science Questions
Welbl, J., Liu, N. F., & Gardner, M. (2017) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w17-4413 · Cited by 574
The Seventh PASCAL Recognizing Textual Entailment Challenge.
Bentivogli, L., Clark, P. E., Dagan, I., & Giampiccolo, D. (2008) · Cited by 533
UNIFIEDQA: Crossing Format Boundaries with a Single QA System
Khashabi, D., Min, S., Khot, T., Sabharwal, A., Tafjord, O., Clark, P., & Hajishirzi, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.171 · Cited by 514
Self-citationGood debt or bad debt: Detecting semantic orientations in economic texts
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., & Takala, P. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23062 · Cited by 513
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513
Zero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1706.04115 · Cited by 482
Self-citationZero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) · doi:10.18653/v1/k17-1034 · Cited by 482
Self-citationThe CommitmentBank: Investigating projection in naturally occurring discourse
de Marneffe, M.-C., Simons, M., & Tonhauser, J. (2019) · Digital Access to Libraries · doi:10.18148/sub/2019.v23i2.601 · Cited by 471
Contributions to the study of SMS spam filtering
Almeida, T. A., Hidalgo, J. M. G., & Yamakami, A. (2011) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/2034691.2034742 · Cited by 466
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462
Gurulingappa, H., Rajput, A. M., Roberts, A., Fluck, J., Hofmann‐Apitius, M., & Toldo, L. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.04.008 · Cited by 447
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Hate Speech Dataset from a White Supremacy Forum
de Gibert, O., Pérez, N., Pablos, A. G., & Cuadros, M. (2018) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w18-5102 · Cited by 428
SciTaiL: A Textual Entailment Dataset from Science Question Answering
Khot, T., Sabharwal, A., & Clark, P. E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12022 · Cited by 419
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1023 · Cited by 407
WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations
Pilehvar, M. T., & Camacho-Collados, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1128 · Cited by 406
Abductive Commonsense ReasoningPreprint
Bhagavatula, C., Bras, R. J. L., Malaviya, C., Sakaguchi, K., Holtzman, A., Rashkin, H., Downey, D., Yih, S., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05739 · Cited by 394
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., & Gardner, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1246 · Cited by 393
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
Dunn, M., Sagun, L., Higgins, M., Guney, V. U., Cirik, V., & Cho, K. (2017) · arXiv · doi:10.48550/arxiv.1704.05179 · Cited by 359
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading ComprehensionPreprint
Zhang, S., Liu, X., Liu, J., Gao, J., Duh, K., & Durme, B. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.12885 · Cited by 339
CARER: Contextualized Affect Representations for Emotion Recognition
Saravia, E., Liu, H.-C. T., Huang, Y.-H., Wu, J., & Chen, Y.-S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1404 · Cited by 337
SemEval-2019 Task 3: EmoContext Contextual Emotion Detection in Text
Chatterjee, A., Narahari, K. N., Joshi, M., & Agrawal, P. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2005 · Cited by 311
PAWS: Paraphrase Adversaries from Word Scrambling
Zhang, Y., Baldridge, J., & He, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1131 · Cited by 306
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
Huang, L., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1243 · Cited by 302
CLIMATE-FEVER: A Dataset for Verification of Real-World Climate ClaimsPreprint
Diggelmann, T., Boyd-Graber, J. L., Bulian, J., Ciaramita, M., & Leippold, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00614 · Cited by 280
8-bit Optimizers via Block-wise QuantizationPreprint
Dettmers, T., Lewis, M., Shleifer, S., & Zettlemoyer, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02861 · Cited by 278
Self-citationCross-Task Generalization via Natural Language Crowdsourcing Instructions
Mishra, S., Khashabi, D., Baral, C., & Hajishirzi, H. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.244 · Cited by 269
Self-citationDREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension
Sun, K., Yu, D., Chen, J., Yu, D., Choi, Y., & Cardie, C. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00264 · Cited by 252
ETHOS: a multi-label hate speech detection dataset
Mollas, I., Chrysopoulou, Z., Karlos, S., & Tsoumakas, G. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00608-2 · Cited by 243
TabFact: A Large-scale Dataset for Table-based Fact Verification
Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., & Wang, W. Y. (2019) · arXiv · doi:10.48550/arxiv.1909.02164 · Cited by 239
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., & Gardner, M. (2019) · arXiv · doi:10.48550/arxiv.1903.00161 · Cited by 238
True Few-Shot Learning with Language Models
Perez, E., Kiela, D., & Cho, K. (2021) · arXiv · doi:10.48550/arxiv.2105.11447 · Cited by 226
PAWS: Paraphrase Adversaries from Word ScramblingPreprint
Zhang, Y., Baldridge, J., & He, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01130 · Cited by 210
Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language
He, L., Lewis, M., & Zettlemoyer, L. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1076 · Cited by 197
Self-citationExplainable Automated Fact-Checking for Public Health Claims
Kotonya, N., & Toni, F. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.623 · Cited by 195
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.08887 · Cited by 193
(2018) · doi:10.63317/2yu5nu76n7bu · Cited by 189
Muppet: Massive Multi-task Representations with Pre-Finetuning
Aghajanyan, A., Gupta, A., Shrivastava, A., Chen, X., Zettlemoyer, L., & Gupta, S. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.468 · Cited by 183
Self-citationNoisy Channel Language Model Prompting for Few-Shot Text Classification
Min, S., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.365 · Cited by 181
Self-citationToward semantics-based answer pinpointing
Hovy, E. H., Gerber, L., Hermjakob, U., Lin, C.-Y., & Ravichandran, D. (2001) · International Conference on Human Language Technology Research · doi:10.3115/1072133.1072221 · Cited by 176
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
Dasigi, P., Liu, N. F., Marasović, A., Smith, N. A., & Gardner, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1606 · Cited by 173
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense ReasoningPreprint
Huang, L., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00277 · Cited by 168
How Context Affects Language Models' Factual Predictions
Petroni, F., Lewis, P., Piktus, A., Rocktäschel, T., Wu, Y., Miller, A. H., & Riedel, S. (2020) · arXiv · doi:10.48550/arxiv.2005.04611 · Cited by 154
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP
Ye, Q., Lin, B. Y., & Ren, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.572 · Cited by 154
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections
Zhong, R., Lee, K., Zhang, Z., & Klein, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.244 · Cited by 152
Meta-learning via Language Model In-context Tuning
Chen, Y., Zhong, R., Zha, S., Karypis, G., & He, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.53 · Cited by 143
Lin, B. Y., Lee, S., Khanna, R., & Ren, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.557 · Cited by 142
Reframing Instructional Prompts to GPTk’s Language
Mishra, S. R., Khashabi, D., Baral, C. R., Choi, Y., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.50 · Cited by 140
Self-citationZhou, B., Khashabi, D., Ning, Q., & Roth, D. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1332 · Cited by 137
Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right
Holtzman, A., West, P., Shwartz, V., Choi, Y., & Zettlemoyer, L. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.564 · Cited by 127
Self-citationOneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification
Vajjala, S., & Lučić, I. (2018) · doi:10.18653/v1/w18-0535 · Cited by 112
Reasoning Over Paragraph Effects in Situations
Lin, K., Tafjord, O., Clark, P. E., & Gardner, M. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5808 · Cited by 110
SemEval-2012 Task 7: Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning
Gordon, A. S., Kozareva, Z., & Roemmele, M. (2012) · Cited by 84
TWEETQA: A Social Media Focused Question Answering Dataset
Xiong, W., Wu, J., Wang, H., Kulkarni, V., Yu, M., Chang, S., Guo, X., & Wang, W. Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1496 · Cited by 79
Getting Closer to AI Complete Question Answering: A Set of Prerequisite Real Tasks
Rogers, A., Kovaleva, O., Downey, M., & Rumshisky, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6398 · Cited by 73
QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions
Tafjord, O., Gardner, M., Lin, K., & Clark, P. E. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1608 · Cited by 72
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning
Boratko, M., Li, X. L., O’Gorman, T., Das, R., Le, D., & McCallum, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.85 · Cited by 56
WiC: 10, 000 Example Pairs for Evaluating Context-Sensitive RepresentationsPreprint
Pilehvar, M. T., & Camacho-Collados, J. (2018) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.1808.09121 · Cited by 50
“I’d rather just go to bed”: Understanding Indirect Answers
Louis, A. P., Roth, D., & Radlinski, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.601 · Cited by 46
FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase
Jiang, K., Wu, D., & Jiang, H. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1028 · Cited by 37
PIQA: Reasoning about Physical Commonsense in Natural Language
Bisk, Y., Zellers, R., Le bras, R., Gao, J., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6239 · Cited by 37
BioMRC: A Dataset for Biomedical Machine Reading Comprehension
Pappas, D., Stavropoulos, P., Androutsopoulos, I., & McDonald, R. (2020) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2020.bionlp-1.15 · Cited by 36
Effective Transfer Learning for Identifying Similar Questions
McCreery, C. H., Katariya, N., Kannan, A., Chablani, M., & Amatriain, X. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3412861 · Cited by 35
Identifying Well-formed Natural Language Questions
Faruqui, M., & Das, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1091 · Cited by 33
Natural Instructions: Benchmarking Generalization to New Tasks from Natural Language InstructionsPreprint
Mishra, S. R., Khashabi, D., Baral, C. R., & Hajishirzi, H. (2021) · arXiv (Cornell University) · Cited by 32
Self-citationMining Discourse Markers for Unsupervised Sentence Representation LearningPreprint
Sileo, D., de Cruys, T. V., Pradel, C., & Müller, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.11850 · Cited by 31
BLiMP: the benchmark of linguistic minimal pairs for English
Warstadt, A., Parrish, A., Liu, H., Mohananey, A., Peng, W., Wang, S.-F., & Bowman, S. R. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00321 · Cited by 28
Investigating Societal Biases in a Poetry Composition SystemPreprint
Sheng, E., & Uthus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02686 · Cited by 26
QASC: A Dataset for Question Answering via Sentence Composition
Khot, T., Clark, P., Guerquin, M., Jansen, P., & Sabharwal, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6319 · Cited by 20
WIQA: A dataset for "What if..." reasoning over procedural textPreprint
Tandon, N., Mishra, B. D., Sakaguchi, K., Bosselut, A., & Clark, P. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04739 · Cited by 17
McCreery, C. H., Katariya, N., Kannan, A., Chablani, M., & Amatriain, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.13546 · Cited by 16
QUAREL: A Dataset and Models for Answering Questions about Qualitative Relationships
Tafjord, O., Clark, P. E., Gardner, M., Yih, W.-T., & Sabharwal, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017063 · Cited by 10
BIOMRC: A Dataset for Biomedical Machine Reading ComprehensionPreprint
Stavropoulos, P., Pappas, D., Androutsopoulos, I., & McDonald, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06376 · Cited by 6
Research works printed but not linked: 15
- Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second pascal recognising tex- tual entailment challenge. In Proceedings of the sec- ond PASCAL challenges workshop on recognising textual entailment.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In NeurIPS.
- Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ul- far Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX Secu- rity Symposium (USENIX Security 21).
- Michael Chen, Mike D'Arcy, Alisa Liu, Jared Fer- nandez, and Doug Downey. 2019. CODAH: An adversarially-authored question answering dataset for common sense. In Proceedings of the 3rd Work- shop on Evaluating Vector Space Representations for NLP.
- Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-REx: A large scale alignment of natural language with knowledge base triples. In LREC.
- Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2019. QASC: A dataset for question answering via sentence compo- sition. In AAAI.
- Xin Li and Dan Roth. 2002. Learning question classi- fiers. In COLING.
- Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. 2020. Ethos: an on- line hate speech detection dataset. arXiv preprint arXiv:2006.08328. · arXiv
- Courtney Napoles, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Pro- ceedings of the Joint Workshop on Automatic Knowl- edge Base Construction and Web-scale Knowledge Extraction (AKBC-WEKEX).
- Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Ad- versarial NLI: A new benchmark for natural lan- guage understanding. In ACL.
- NeurIPS.
- Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavat- ula, and Yejin Choi. 2020b. Winogrande: An adver- sarial winograd schema challenge at scale. In AAAI.
- Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019b. Social iqa: Com- monsense reasoning about social interactions. In EMNLP-IJCNLP.
- Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018. CARER: Con- textualized affect representations for emotion recog- nition. In EMNLP.
- TACL.
Cited by
Sorted by citations
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755
Self-citationA Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.64 · Cited by 649
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2305.14314 · Cited by 3,832
“Approaches and datasets include MetaICL [40], MetaTuning [74], InstructGPT [43], FLAN [63, 12], PromptSource [3], Super-NaturalInstructions [62, 51], Self-instruct [60], UnnaturalInstructions [26], OPT-IML [29], UnifiedSKG[68], OIG/Chip2 [32], Alpaca [56], Vicuna [10], Koala [20], and…”
BackgroundMethodSelf-citationPrecise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.99 · Cited by 362
Metadata
- Authors
- Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H.
- Affiliations
- University of Washington (US)
- Meta (United States) (US)
- Venue
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2022
- Pages
- 2791-2809
- OpenAlex
- W3210277894
- Cited by
- 169
- References
- 137
- Citations by year
- 2026: 23 · 2025: 61 · 2024: 44 · 2023: 36 · 2022: 4
- FWCI
- 20.04
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Funders
- National Science Foundation (2044660)
- Office of Naval Research (N00014-18-1-2826, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- in-context learning, prompt engineering, large language models, meta-learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
