Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Wenpeng Yin, Jamaal Hay, Dan Roth. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is evidenced by LLMs impressive zero-shot and few-shot learning ability [18,132], as well as the fact that in-context learning does not require weight update, which mechanically prevents unfavorable fine-tuning-related behavior like catastrophic forgetting.”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have been shown to outperform previous zero-shot methods [132].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“• Word2Vec 5 (Mikolov et al., 2013): Both the representations of the text and the labels are the addition of word embeddings element-wisely.”
BackgroundGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 11×
“For label-partially-unseen setup in which we intentionally provide annotated data, we first pretrain BERT on the MNLI/FEVER/RTE, then fine-tune on the provided training data.”
BackgroundMethodInfluentialA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 2×
“, 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al.”
BackgroundMethodCharacter-level Convolutional Networks for Text Classification
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 3,205 · cited 1×
“We use the large-scale Yahoo dataset released by Zhang et al. (2015).”
MethodComputing semantic relatedness using Wikipedia-based explicit semantic analysis
Gabrilovich, E., & Markovitch, S. (2007) · Cited by 1,981
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
(2018) · International Conference on Learning Representations · Cited by 1,621
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 1×
“In this work, we make use of the widely-recognized state of the art entailment technique – BERT (Devlin et al., 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al., 2005; Wang et al., 2019) and FEVER 3 (Thorne et al., 2018), respectively.”
Method
Research works printed but not linked: 2
- [8]8. Chenliang Li, Jian Xing, Aixin Sun, and Zongyang Ma. 2016a. Effective document labeling with very few seed words: A topic model approach. In CIKM, pages 85-94.
- [11]11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed repre- sentations of words and phrases and their composi- tionality. In NIPS, pages 3111-3119.
Cited by
Items 101–200 of 867
Lbl2Vec: An Embedding-based Approach for Unsupervised Document Retrieval on Predefined Topics
Schopf, T., Braun, D., & Matthes, F. (2021) · International Conference on Web Information Systems and Technologies · doi:10.5220/0010710300003058 · Cited by 43
“Yin et al. (2019) used various public entailment datasets to train a bidirectional encoder representations from transformers (BERT) model (Devlin et al., 2019) and used the pretrained BERT entailment model to directly classify texts from different datasets.”
MethodMetadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification
Zhang, Y., Shen, Z., Wu, C.-H., Xie, B.-Y., Hao, J., Wang, Y.-X., Wang, K., & Han, J. (2022) · The Web Conference · doi:10.1145/3485447.3512174 · Cited by 34
“Cross-Encoders [13, 61], in contrast, perform self attention within as well as across two text units at the same time.”
BackgroundMethodInfluentialReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval
Yu, Y., Zhuang, Y., Zhang, R., Yu, M., Shen, J., & Zhang, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.748 · Cited by 42
Learning to Generate Task-Specific Adapters from Task Description
Ye, Q., & Ren, X. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-short.82 · Cited by 42
“Zeroshot learning (ZSL) has been explored for various NLP tasks, including text classification (Yin et al., 2019), entity linking (Logeswaran et al., 2019) and entity typing (Obeidat et al., 2019).”
BackgroundHan, G., Liu, W., Huang, X., & Borsari, B. E. (2024) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi61247.2024.00057 · Cited by 40
“To demonstrate the effectiveness of our proposed Chain-of-Interaction prompting method, we compare it with three state-of-the-art prompting techniques: 1) Zero-Shot Prompting [32], [33], 2) Few-Shot Prompting [34], [35], 3) Zero-Shot Chain-of-Thought (ZeroCoT) Prompting [36].”
BackgroundMethodBelief Revision: The Adaptability of Large Language Models Reasoning
Wilie, B., Cahyawijaya, S., Ishii, E., He, J., & Fung, P. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.586 · Cited by 40
Unified Semantic Typing with Meaningful Label Inference
Huang, J. Y., Li, B., Xu, J., & Chen, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.190 · Cited by 40
Capecchi, I., Borghini, T., Bellotti, M., & Bernetti, I. (2025) · Sustainability · doi:10.3390/su17052113 · Cited by 39
Natural Language Processing in Marketing
Hartmann, J., & Netzer, O. (2023) · Artificial Intelligence in Marketing · doi:10.1108/s1548-643520230000020011 · Cited by 38
Evaluating large language models for user stance detection on X (Twitter)
Gambini, M., Senette, C., Fagni, T., & Tesconi, M. (2024) · Machine Learning · doi:10.1007/s10994-024-06587-y · Cited by 38
“Yin et al. (2019) defines ZSC as a Language Model (e.g., BART or DeBERTa) pre-trained on Natural Language Inference tasks that, given a text and a set of labels (e.g., topics), assigns a classification probability score to each label.”
Natural Language Processing for Policymaking
Jin, Z., & Mihalcea, R. F. (2022) · Springer eBooks · doi:10.1007/978-3-031-16624-2_7 · Cited by 37
Event Log Construction from Customer Service Conversations Using Natural Language Inference
Kecht, C., Egger, A., Kratsch, W., & Röglinger, M. (2021) · International Conference on Process Mining (ICPM) · doi:10.1109/icpm53251.2021.9576869 · Cited by 36
Central bank mandates and monetary policy stances: Through the lens of Federal Reserve speeches
Bertsch, C., Hull, I., Lumsdaine, R. L., & Zhang, X. (2025) · Journal of Econometrics · doi:10.1016/j.jeconom.2025.105948 · Cited by 38
Zero-Shot Aspect-Based Sentiment AnalysisPreprint
Shu, L., Hu, X., Liu, B., & Chen, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.01924 · Cited by 36
“Zeroshot learning was introduced for text classification tasks using entailment as the solution (Yin et al., 2019).”
MethodA Unifying Framework for Robust and Efficient Inference with Unstructured DataPreprint
Carlson, J., & Dell, M. (2025) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2505.00282 · Cited by 38
Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference
Li, B., Yin, W., & Chen, M. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00479 · Cited by 35
“Specifically, since pre-trained NLI models ben-efit generalizable logical inference, current literature has also proposed to leverage NLI models to improve prediction tasks with insufficient training labels, including zero-shot and few-shot text classification (Yin et al., 2019).”
MethodSelf-citationDougherty, R. F., Clarke, P., Atli, M., Kuc, J., Schlosser, D. A., Dunlop, B. W., Hellerstein, D. J., Aaronson, S. T., Zisook, S., Young, A. H., et al. (2023) · Psychopharmacology · doi:10.1007/s00213-023-06432-5 · Cited by 35
The Singleton Fallacy: Why Current Critiques of Language Models Miss the Point
Sahlgren, M., & Carlsson, F. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.682578 · Cited by 35
“This is eminently well demonstrated in the recent work on zero-shot learning [33,3].”
BackgroundCurrent Advances in Neural Networks
Gallego, V., & Insua, D. R. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-040220-112019 · Cited by 34
A Zero-Shot Learning Approach to Classifying Requirements: A Preliminary Study
Alhoshan, W., Zhao, L., Ferrari, A., & Letsholo, K. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98464-9_5 · Cited by 33
Asprino, L., Bulla, L., Giorgis, S. D., Gangemi, A., Marinucci, L., & Mongiovì, M. (2022) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2022.deelio-1.4 · Cited by 33
“Starting from the method developed by Yin et al. (2019), we adapt a checkpoint for BART-large trained on the MultiNLI (MNLI) dataset (Kim et al., 2018).”
MethodLiu, H., Zhang, D., Yin, B., & Zhu, X. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.83 · Cited by 33
Composed Variational Natural Language Generation for Few-shot Intents
Xia, C., Xiong, C., Yu, P. S., & Socher, R. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.303 · Cited by 32
Fusing external knowledge resources for natural language understanding techniques: A survey
Wang, Y., Wang, W., Chen, Q., Huang, K., Nguyen, A., De, S., & Hussain, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.025 · Cited by 32
The Benefits of Label-Description Training for Zero-Shot Text Classification
Gao, L., Ghosh, D., & Gimpel, K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.853 · Cited by 32
Prompt-based Zero-shot Relation Extraction with Semantic Knowledge Augmentation
Gong, J., & Eldardiry, H. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2yxn5oemh6ks · Cited by 32
“We also conduct ablation experiments following the zero-shot definition in [53] which is a generalized zero-shot setting where partial labels are unseen.”
BackgroundMethodLMLPA: Language Model Linguistic Personality Assessment
Zheng, J., Wang, X., Hosio, S. J., Xu, X., & Lee, L. (2025) · Computational Linguistics · doi:10.1162/coli_a_00550 · Cited by 31
Continual Learning with Transformers for Image Classification
Ermiş, B., Zappella, G., Wistuba, M., Rawal, A., & Archambeau, C. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00422 · Cited by 31
“Besides, some recent studies [2, 18, 35] showed transformers have a good ability to generalize well”
BackgroundPESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
Wang, Y.-S., Chi, T.-C., Zhang, R., & Yang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.832 · Cited by 31
Ask2Transformers: Zero-Shot Domain labelling with Pretrained Language Models
Sainz, O., & Rigau, G. (2021) · Global WordNet Conference (GWC) · doi:10.18653/v1/2021.gwc-1.6 · Cited by 30
Active Few-Shot Learning with FASL
Müller, T., Pérez-Torró, G., Basile, A., & Franco-Salvador, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08473-7_9 · Cited by 30
Exploring electric vehicle adoption in Indonesia using zero-shot aspect-based sentiment analysis
Adi, N. S., & Widianto, S. (2024) · Sustainable Operations and Computers · doi:10.1016/j.susoc.2024.08.002 · Cited by 30
Understanding NFT Price Moves through Tweets Keywords Analysis
Luo, J., Jia, Y., & Liu, X. (2023) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3582515.3609562 · Cited by 29
“To quantify the categories of the words, we adopt a zero-shot classification pipeline [39] with a BART language comprehension model [20] pretrained on MultiNLI (MNLI) dataset [37] for categorizing the top keywords.”
MethodLLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert RatingsPreprint
Maier, B. F., Aslak, U., Fiaschi, L., Rismal, N., Fletcher, K., Luhmann, C. C., Dow, R., Pappas, K., & Wiecki, T. V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.08338 · Cited by 29
“This approach draws on established methods in NLP (semantic similarity mapping) [16] and survey methodology (anchoring vignettes) [17], but to our knowledge has not been applied in the context of LLMs as survey respondents.”
Evaluating Decision Models for Text Annotation in Computational Social SciencePreprint
Ibrahim, H., & Zaki, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.24574 · Cited by 30
Unified benchmark for zero-shot Turkish text classification
Çelik, E., & Dalyan, T. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103298 · Cited by 28
Describing Differences between Text Distributions with Natural LanguagePreprint
Zhong, R., Snell, C., Klein, D., & Steinhardt, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.12323 · Cited by 29
Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across CorporaPreprint
Plaza-del-Arco, F. M., Valdivia, M. T. M., & Klinger, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.06701 · Cited by 28
“We treat ZSL emotion classification as a textual entailment problem, but represent each label under consideration with multiple prompts, in contrast to Yin et al. (2019).”
MethodResultInfluentialEstimating the Personality of White-Box Language Models
Karra, S. R., Nguyen, S. T., & Tulabandhula, T. (2022) · arXiv · doi:10.48550/arxiv.2204.12000 · Cited by 27
“• We propose a novel estimation scheme for evaluating the personality traits of both the datasets and the language models, using Zero-shot learning (ZSL) (Yin et al., 2019).”
MethodInfluentialWeakly Supervised Multi-Label Classification of Full-Text Scientific Papers
Zhang, Y., Jin, B., Chen, X., Shen, Y., Zhang, Y., Yu, M., & Han, J. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599544 · Cited by 27
“This setting is also called “wild zero-shot” in some previous studies [57, 69].”
BackgroundMethodResultInfluentialZ-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Yang, Y., Yao, W., Zhang, H., Wang, X., Yu, D., & Chen, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.78 · Cited by 27
T.y.s.s, S., Ichim, O., & Grabmair, M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.44 · Cited by 28
LLM-driven Instruction Following: Progresses and Concerns
Yin, W., Ye, Q., Liu, P., Ren, X., & Schütze, H. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-tutorial.4 · Cited by 27
“For instance, some instructions serve to elucidate the output labels in classification tasks, as exemplified by NLI-oriented task instructions (Yin et al., 2019; Xu et al., 2022; Li et al., 2022; Xia et al., 2021; Sainz et al., 2021, 2022).”
BackgroundSelf-citationContent Moderation for Evolving Policies using Binary Question Answering
Mullick, S. S., Bhambhani, M., Sinha, S., Mathur, A., Gupta, S., & Shah, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.54 · Cited by 26
“…Language Models (LLMs) (Caselli et al., 2020; Tan et al., 2020), putting attention on suspicious pieces of text (Pavlopoulos et al., 2017), or reformulating the problem as multi-task learning (Kapil and Ek-bal, 2020) or natural language inference (Yin et al., 2019; Goldzycher and Schneider, 2022).”
BackgroundZero-Shot Text Classification via Self-Supervised Tuning
Liu, C., Zhang, W., Chen, G., Wu, X., Luu, A. T., Chang, C.-H., & Bing, L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.110 · Cited by 26
A weakly supervised textual entailment approach to zero-shot text classification
Pàmies, M., Palao, J. L., Multari, F., Durán-Silva, N., Parra-Rojas, C. A., González-Agirre, A., Massucci, F. A., & Villegas, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.22 · Cited by 26
“Topic Emotion Situation RTE (Yin et al., 2019) 43.8 12.6 37.2 FEVER (Yin et al., 2019) 40.1 24.7 21.0 MNLI (Yin et al., 2019) 37.9 22.3 15.4 NSP (Ma et al., 2021) 50.6 16.5 25.8 NSP (Reverse) (Ma et al., 2021) Figures 3 and 4 corroborate our hypothesis that the performance in the ZSTC task does not…”
BackgroundMethodInfluentialHypothesis Engineering for Zero-Shot Hate Speech DetectionPreprint
Goldzycher, J., & Schneider, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.00910 · Cited by 26
“Yin et al. (2019) proposed to use an NLI model for zero-shot topic classification, by inputting the text to classify as the premise and constructing for each topic a hypothesis of the form “This text is about < topic > ”.”
MethodRey‐Moreno, M., Sánchez‐Franco, M. J., & Rey-Tienda, S. (2023) · Tourism & Management Studies · doi:10.18089/tms.2023.190202 · Cited by 25
“Yin, Hay, and Roth (2019) propose a pre-trained Natural Language Inference (NLI) approach as a ready-made Zero-shot sequence classifier.”
BackgroundLanSER: Language-Model Supported Speech Emotion Recognition
Gong, T., Belanich, J., Somandepalli, K., Nagrani, A., Eoff, B. D., & Jou, B. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1832 · Cited by 25
“To this end, we use textual entailment [12] to generate weak labels that also allows us to constrain the emotion taxonomy apriori.”
MethodInfluentialZero-Shot Multi-Label Topic Inference with Sentence Encoders and LLMs
Sarkar, S., Feng, D., & Santu, S. K. K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1008 · Cited by 25
Troxler, A., & Schelldorfer, J. (2024) · British Actuarial Journal · doi:10.1017/s1357321724000023 · Cited by 26
“Zero-shot classification involves classifying text sequences in an unsupervised way (without having training data in advance and building a model), as studied by Yin et al. (2019).”
BackgroundPublic sentiment and engagement on cybersecurity: Insights from Reddit discussions
Achuthan, K., Khobragade, S., & Kowalski, R. M. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100573 · Cited by 25
GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection?
Jin, Y., Wanner, L., & Shvets, A. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/457md257uym3 · Cited by 25
Improving In-Context Few-Shot Learning via Self-Supervised TrainingPreprint
Chen, M., Du, J.-F., Pasunuru, R., Mihaylov, T., Iyer, S., Stoyanov, V., & Kozareva, Z. (2022) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2205.01703 · Cited by 89
“…on unifying task formats to maximize the benefits of human annotations, including question answering (Zhong et al., 2021), textual entailment (Yin et al., 2019, 2020; Wang et al., 2021a), and many other tasks (McCann et al., 2018; Keskar et al., 2019; Raffel et al., 2020; Bragg et al., 2021).”
BackgroundCitePrompt: Using Prompts to Identify Citation Intent in Scientific Papers
Lahiri, A., Sanyal, D. K., & Mukherjee, I. (2023) · 2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL) · doi:10.1109/jcdl57899.2023.00017 · Cited by 24
“Manual prompts have been used in GPT-2 [4] and other zero-shot text classification problems [25].”
MethodUnsupervised Label Refinement Improves Dataless Text Classification
Chu, Z., Stratos, K., & Gimpel, K. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.365 · Cited by 24
Prompt-based Zero-shot Text Classification with Conceptual Knowledge
Wang, Y., Wang, W., Chen, Q., Huang, K., Nguyen, A., & De, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.4 · Cited by 24
“In order to achieve this goal, it is essential to ensure that the aspect being described in the input text and the meanings of the labels are comprehensible to the framework (Yin et al., 2019).”
InfluentialPSAN: Prompt Semantic Augmented Network for aspect-based sentiment analysis
He, Y., Huang, X., Zou, S., & Zhang, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121632 · Cited by 24
Bareiß, P., Klinger, R., & Barnes, J. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651902 · Cited by 24
“Such zero-shot classification can achieve good results [50], including emotion classification [34].”
BackgroundThe role of feed spacers in membrane technology: 45 years of research
Ibrahim, Y., Aytaç, E., Khanzada, N. K., Khayet, M., & Hilal, N. (2024) · Separation and Purification Technology · doi:10.1016/j.seppur.2024.130109 · Cited by 24
Distributed training of large language models: A survey
Zeng, F., Gan, W., Wang, Y., & Yu, P. S. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100174 · Cited by 24
DSP: Discriminative Soft Prompts for Zero-Shot Entity and Relation Extraction
Lv, B., Liu, X., Dai, S., Liu, N., Yang, F., Luo, P., & Yu, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.339 · Cited by 23
Multi-view enhanced zero-shot node classification
Wang, J., Wu, L., Zhao, H., & Jia, N. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103479 · Cited by 23
Bagdon, C., Karmalkar, P., Gurulingappa, H., & Klinger, R. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.439 · Cited by 23
Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive SurveyPreprint
Chen, J., Geng, Y., Chen, Z., Pan, J. Z., He, Y., Zhang, W., Horrocks, I. R., & Chen, H. (2021) · Research Explorer (The University of Manchester) · doi:10.48550/arxiv.2112.10006 · Cited by 23
“Although there have been some benchmarking studies for ZSL and FSL [26, 192], systematic evaluation and comparison of KG-aware methods is still not enough.”
BackgroundllmNER: (Zero|Few)-Shot Named Entity Recognition, Exploiting the Power of Large Language ModelsPreprint
Villena, F., Miranda, L., & Aracena, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.04528 · Cited by 23
“Even though prompting methods for some NLP tasks, such as text classification [13, 14, 15, 16] or generation [6, 17, 18], prompting methods for NER still need to be explored.”
Few-Shot Tabular Data Enrichment Using Fine-Tuned Transformer Architectures
Harari, A., & Katz, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.111 · Cited by 22
“It should be noted that a similar process of including the class value as part of the input was previously used in the domain of zero-shot Text Classification (Yin et al., 2019), to address the possibility of new classes appearing in mid-training.”
MethodTo tune or not to tune? zero-shot models for legal case entailment
Rosa, G. M., Rodrigues, R. C., Lotufo, R., & Nogueira, R. F. (2021) · International Conference on Artificial Intelligence and Law · doi:10.1145/3462757.3466103 · Cited by 22
How Many Data Samples is an Additional Instruction Worth?
Puri, R. S., Mishra, S. R., Parmar, M., & Baral, C. R. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.77 · Cited by 22
Plaza-del-Arco, F. M., Collado-Montañez, J., Ureña, L. A., & Valdivia, M. T. M. (2022) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2022.wassa-1.23 · Cited by 22
“The NLI model needs to decide if the hypothesis (a prompt which represents the class label) entails the premise (which corresponds to the instance to be classified) or contradict it (Yin et al., 2019).”
BackgroundHow Effective are Generative Large Language Models in Performing Requirements Classification?Preprint
Alhoshan, W., Ferrari, A., & Zhao, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.16768 · Cited by 22
“There are two popular ZSL approaches for LLMs: embedding-based learning [63] and inference-based learning (also known as entailment-based ) [69].”
Jev in the Wild: A Data-Driven Analysis of the Jev Model's Functionality, Applications and EcosystemPreprint
Ling, G., Xue, M., & Ye, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.30216 · Cited by 22
“Zero-shot methods reduce this dependence by using natural-language label descriptions to transfer across categories and tasks (Yin et al., 2019; Halder et al., 2020; Clarke et al., 2023).”
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation
Yuan, X., Wang, T., Wang, Y.-H., Fine, E., Abdelghani, R., Sauzéon, H., & Oudeyer, P.-Y. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.820 · Cited by 21
New User Intent Discovery With Robust Pseudo Label Training and Source Domain Joint Training
An, W., Tian, F., Chen, P., Zheng, Q., & Ding, W. (2023) · IEEE Intelligent Systems · doi:10.1109/mis.2023.3283909 · Cited by 21
An Alien in the Newsroom: AI Anxiety in European and American Newspapers
Sanguinetti, P., & Palomo, B. (2024) · Social Sciences · doi:10.3390/socsci13110608 · Cited by 21
MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information
Zhang, Y., Garg, S., Yu, M., Chen, X., & Han, J. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498384 · Cited by 21
“A pioneering approach is dataless classification [2, 27, 37] which relies on Wikipedia to map labels and documents into the same semantic space and derive their relevance.”
BackgroundAre the BERT family zero-shot learners? A study on their potential and limitations
Wang, Y., Wu, L., Li, J., Liang, X., & Zhang, M. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103953 · Cited by 21
Extraction and linking of motivation, specification and structure of inventions for early design use
Jiang, P., Atherton, M. A., & Sorce, S. (2023) · Journal of Engineering Design · doi:10.1080/09544828.2023.2227934 · Cited by 20
A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classification
Bhambhoria, R. V., Chen, L., & Zhu, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.152 · Cited by 20
Zero-Shot Sentiment Analysis Exploring BART Models
Kyritsis, K., Perikos, I., & Paraskevas, M. (2023) · IEEE/ACIS International Conference on Big Data, Cloud Computing, and Data Science (BCD) · doi:10.1109/bcd57833.2023.10466289 · Cited by 20
GenFlowchart: Parsing and Understanding Flowchart Using Generative AI
Arbaz, A., Fan, H., Ding, J., Qiu, M., & Feng, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5492-2_8 · Cited by 20
Sentiment Analysis of Customer Comments in Banking using BERT-based Approaches
Masarifoglu, M., Tigrak, U., Hakyemez, S., Gul, G., Bozan, E., Büyüklü, A. H., & Özgür, A. (2021) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu53274.2021.9477890 · Cited by 20
Fine-grained Event Classification in News-like Text Snippets - Shared Task 2, CASE 2021
Haneczok, J., Jacquet, G., Piskorski, J., & Stefanovitch, N. (2021) · Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE) · doi:10.18653/v1/2021.case-1.23 · Cited by 20
“We follow the approach proposed in Yin et al. (2019) and take the text snippet as the premise and the descriptive forms of candidate labels as alternative hypotheses .”
MethodEvaluating software user feedback classifier performance on unseen apps, datasets, and metadata
Devine, P., Koh, Y. S., & Blincoe, K. (2023) · Empirical Software Engineering · doi:10.1007/s10664-022-10254-y · Cited by 20
“7 This model was not explicitly trained to classify user feedback, but has been designed to classify text without pre-training on class-labelled training data (hence “zero-shot”) by leveraging the entailment prediction abilities of natural language inference models, as proposed by Yin et al. (2019).”
MethodTyped Decision Models: An Early Evidence Audit and Evaluation ChecklistPreprint
Tang, L., & Zheng, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32160 · Cited by 21
“In instruction-tuned models, first-token option probabilities can also disagree with the answer the model would generate [68].”
Sentiment Analysis of Reviews in Kazakh With Transfer Learning Techniques
Nugumanova, A., Baiburin, Y., & Alimzhanov, Y. (2022) · IEEE International Conference on Smart Information Systems and Technologies (SIST) · doi:10.1109/sist54437.2022.9945811 · Cited by 19
“While most of the zero-shot learning research is centered around Computer Vision, there has been some interesting works in the NLP domain as well ([20], [21]).”
BackgroundSchopf, T., Braun, D., & Matthes, F. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-24197-0_4 · Cited by 19
Zero-shot Micro-video Classification with Neural Variational Inference in Graph Prototype Network
Chen, J., Wang, J., Dai, Z., Wu, H., Wang, M., Zhang, Q., & Wang, H. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611740 · Cited by 19
Literacy and Financial Education: Private Providers, Public Certification and Political Preferences
Guerini, C., Masciandaro, D., & Papini, A. (2024) · Italian Economic Journal · doi:10.1007/s40797-024-00287-1 · Cited by 20
Banerjee, A., Satish, A., Aisyah, F. N., Wörndl, W., & Deldjoo, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730321 · Cited by 19
“This pre-trained Natural Language Inference (NLI) model enables sequence classification without additional training [39] and assigns a probability score indicating how well each POI aligns with the defined travel filters.”
Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning
Luo, H., & Glass, J. R. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.89 · Cited by 19
Crowdsourcing Scholarly Discourse Annotations
Oelen, A., Stocker, M., & Auer, S. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450685 · Cited by 19
“A zero-shot text classifier is able to predict classes for text without requiring training data [58].”
BackgroundRafe, A., & Das, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.00346 · Cited by 22
“Zero-shot classifiers score label descriptions without task training (Yin et al., 2019).”
Chen, H., Alfred, M. C., & Cohen, E. (2025) · JMIR Medical Informatics · doi:10.2196/68955 · Cited by 19
“The textual entailment approach conceptualizes zero-shot text classification as an entailment task [42].”
On the effectiveness of pretrained models for API learning
Hadi, M. A., Yusuf, I. N. B., Thung, F., Luong, K., Jiang, L., Fard, F. H., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527886 · Cited by 18
“Consequently, PTMs are used to transfer the learned knowledge to a new domain or a new task, and in settings where a model has not seen any example during training (known as zero-shot learning) [47].”
BackgroundButcher, B., Zilka, M., Hron, J., Cook, D., & Weller, A. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3652591 · Cited by 18
“We then use a RoBERTa Natural Language Inference (NLI) model (Yin et al., 2019) to assign labels PNLI(label | excerpt).”
MethodExtracting knowledge from customer reviews: an integrated framework for digital platform analytics
Kyriakidis, A., & Tsafarakis, S. (2024) · International Transactions in Operational Research · doi:10.1111/itor.13537 · Cited by 19
IBM MNLP IE at CASE 2021 Task 2: NLI Reranking for Zero-Shot Text Classification
Barker, K., Awasthy, P., Ni, J., & Florian, R. (2021) · Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE) · doi:10.18653/v1/2021.case-1.24 · Cited by 18
“By choosing an appropriate task representation, QA models and NLI models can be used as “pre-tuned” models for few-shot (Schick and Sch¨utze, 2021) or even zero-shot (Yin et al., 2019) text classification.”
Method
Metadata
- Authors
- Yin, W., Hay, J., & Roth, D.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3912-3921
- OpenAlex
- W2970200208
- Cited by
- 537
- References
- 30
- Citations by year
- 2026: 66 · 2025: 86 · 2024: 75 · 2023: 141 · 2022: 94 · 2021: 64 · 2020: 11
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency (HR0011-15-C-0113)
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- zero-shot text classification, benchmarking, evaluation metrics, logical consequence, cross-domain classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

