Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Wenpeng Yin, Jamaal Hay, Dan Roth. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is evidenced by LLMs impressive zero-shot and few-shot learning ability [18,132], as well as the fact that in-context learning does not require weight update, which mechanically prevents unfavorable fine-tuning-related behavior like catastrophic forgetting.”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have been shown to outperform previous zero-shot methods [132].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“• Word2Vec 5 (Mikolov et al., 2013): Both the representations of the text and the labels are the addition of word embeddings element-wisely.”
BackgroundGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 11×
“For label-partially-unseen setup in which we intentionally provide annotated data, we first pretrain BERT on the MNLI/FEVER/RTE, then fine-tune on the provided training data.”
BackgroundMethodInfluentialA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 2×
“, 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al.”
BackgroundMethodCharacter-level Convolutional Networks for Text Classification
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 3,205 · cited 1×
“We use the large-scale Yahoo dataset released by Zhang et al. (2015).”
MethodComputing semantic relatedness using Wikipedia-based explicit semantic analysis
Gabrilovich, E., & Markovitch, S. (2007) · Cited by 1,981
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
(2018) · International Conference on Learning Representations · Cited by 1,621
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 1×
“In this work, we make use of the widely-recognized state of the art entailment technique – BERT (Devlin et al., 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al., 2005; Wang et al., 2019) and FEVER 3 (Thorne et al., 2018), respectively.”
Method
Research works printed but not linked: 2
- [8]8. Chenliang Li, Jian Xing, Aixin Sun, and Zongyang Ma. 2016a. Effective document labeling with very few seed words: A topic model approach. In CIKM, pages 85-94.
- [11]11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed repre- sentations of words and phrases and their composi- tionality. In NIPS, pages 3111-3119.
Cited by
Items 301–400 of 867
Khandaker, F., Senderovich, A., Zhao, J., Cohen, E., Yu, E. S. K., Carbajales, S., & Chan, A. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107450 · Cited by 10
The Contribution of Lyrics and Acoustics to Collaborative Understanding of Mood
Naseri, S., Reddy, S., Correia, J., Jussi Karlgren, C., & Jones, R. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19326 · Cited by 10
LabelPrompt: Effective Prompt-based Learning for Relation ClassificationPreprint
Zhang, W., Song, X., Feng, Z., Xu, T., & Wu, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.08068 · Cited by 10
Utilizing Natural Language Processing to Examine Self-Reflections in Self-Regulated LearningPreprint
Gamieldien, Y., McCord, R. E., & Katz, A. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4487795 · Cited by 10
Emotion-Conditioned Text Generation through Automatic Prompt OptimizationPreprint
Resendiz, Y. M., & Klinger, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.04857 · Cited by 10
“In areas like text classification or structured prediction, prompt optimization has established itself as a zero-or few-shot learning paradigm (Ding et al., 2022; Zhang et al., 2022; Wang et al., 2022), also in emotion analysis (Plaza-del Arco et al., 2022; Zheng et al., 2022; Yin et al., 2019).”
BackgroundCoburn, B., He, J., Rollo, M. E., Dhaliwal, S. S., Kerr, D. A., & Zhu, F. M. (2025) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi67747.2025.11269555 · Cited by 9
“Nutrient content varied widely: 83–1 , 968 kcal (median 622 kcal) of energy, 2–110 g (median 22 g) of fat, 3–95 g (median 29 g) of protein, and 3–332 g (median 71 g) of carbohydrates, with a median of five items per meal spanning 11 cuisines (predicted by BART [29], [30]).”
AI-Powered Text Screening: A Generative Approach to Identifying Sensitive Information
Khan, A. A., & Rajagopalan, S. (2025) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc68654.2025.11437858 · Cited by 9
“Fixed-LM prompt tuning fine-tunes models with static prompts, achieving improved accuracy but with higher computational cost [28], [32].”
Zero-shot approach for news and scholarly article classification
Patadia, D., Kejriwal, S., Mehta, P., & Joshi, A. R. (2021) · International Conference on Advances in Computing, Communication and Control (ICAC3) · doi:10.1109/icac353642.2021.9697327 · Cited by 9
Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing
Wang, Y., Xia, C., Wang, G., & Yu, P. S. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020766 · Cited by 9
“This textual entailment formulation gives our model the ability to easily adapt to new entities [1].”
InfluentialDriving the Technology Value Stream by Analyzing App Reviews
Das, S., Deb, N., Chaki, N., & Cortesi, A. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3270708 · Cited by 9
“[51] on the Yahoo Answers [57] benchmark dataset.”
MethodGoldzycher, J., Preisig, M., Amrhein, C., & Schneider, G. (2023) · doi:10.18653/v1/2023.woah-1.19 · Cited by 9
Enhancing blockchain security through natural language processing and real-time monitoring
Salzano, F., & Pareschi, R. (2023) · International Journal of Parallel Emergent and Distributed Systems · doi:10.1080/17445760.2023.2272280 · Cited by 9
“As illustrated in [48], this methodology classifies text, documents, or sentences without resorting to any previously tagged data by using a NLP model, pre-trained in an environment or domain that may be completely different from the application domain.”
Deep Natural Language Feature Learning for Interpretable Prediction
Urrutia, F., Calderon, C. B., & Barrière, V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.229 · Cited by 9
“…two strong advantages: (i) it leverages the semantic knowledge encoded during pre-training to understand the label, (ii) it can be applied in a zero-shot manner using new labels formulated as natural language binary question (Yin et al., 2019; Vamvas and Sennrich, 2020; Barriere and Jacquet, 2022).”
BackgroundSEGLLM: Topic-Oriented Call Segmentation Via LLM-Based Conversation Synthesis
Malkiel, I., Alon, U., Yehuda, Y., Keren, S., Barkan, O., Ronen, R., & Koenigstein, N. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446156 · Cited by 9
“First, a zero-shot model from [16] predicts the topic.”
BackgroundCost Effective Annotation Framework Using Zero-Shot Text Classification
Kasthuriarachchy, B. H., Chetty, M., Shatte, A. B. R., & Walls, D. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534335 · Cited by 9
“[3] proposed to use a pretrained Multi-Genre Natural Language Inference (MNLI) [4] sequence-pair classifier as an out-of-the-box zero-shot to per-”
MethodPrompt-Based Learning for Aspect-Level Sentiment Classification
Li, G., Lin, F., Chen, W., Dong, D., & Liu, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30111-7_43 · Cited by 9
Towards Weakly-Supervised Hate Speech Classification Across Datasets
Jin, Y., Wanner, L., Kadam, V., & Shvets, A. (2023) · doi:10.18653/v1/2023.woah-1.4 · Cited by 9
Automatic Classification of Legal Violations in Cookie Banner Texts
Hofslot, M. V., Salah, A. A. A., Albert Gatt, A., & Santos, C. T. (2022) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2022.nllp-1.27 · Cited by 9
“Zero-shot (ZS) classification in NLP has been used to classify text on which a model is not specifically trained (Sarkar et al., 2021; Yin et al., 2019a; Ye et al., 2020).”
MethodMeaeQ: Mount Model Extraction Attacks with Efficient Queries
Dai, C., Lv, M., Li, K., & Wei, Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.781 · Cited by 9
Automated Taxonomy Discovery and Exploration
Shen, J., & Han, J. (2022) · Springer eBooks · doi:10.1007/978-3-031-11405-2 · Cited by 9
LIME: Weakly-Supervised Text Classification Without SeedsPreprint
Park, S., & Lee, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.06720 · Cited by 9
“Inspired by recent advances in prompt-based text classification (Yin et al., 2019, 2020; Schick and Schütze, 2021), we replace the keywordbased pseudo-labeling step with a more streamlined entailment-based approach.”
BackgroundMethodA Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event ExtractionPreprint
Cai, E., & O'Connor, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15051 · Cited by 9
“…one approach that has been studied in supervised EE (Du and Cardie, 2020; Liu et al., 2020), zero-shot relation extraction (RE) (Levy et al., 2017), and zero-shot text classification (TC) (Yin et al., 2019) queries over text spans to detect events and extract arguments (Lyu et al., 2021).”
BackgroundMethodInfluentialWhat People Think of Machines as Doctors: Unveiling the Value of Gen-AI for e-HealthPreprint
Ozdemir, D. Y., Ayvaci, M., & Zentner, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4765222 · Cited by 9
Cognitive Agency Surrender: Defending Epistemic Sovereignty via Scaffolded AI FrictionPreprint
Xu, K., Shen, Y., Yan, L., & Ren, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.21735 · Cited by 10
“To empirically quantify this trajectory, we deployed a zero-shot semantic classification pipeline [7] to parse high-impact HCI literature from 2023 through the first quarter of 2026 (Fig 1).”
Chausson, S., Fourcade, M., Harding, D. J., Roß, B., & Renard, G. (2025) · Sociological Methods & Research · doi:10.1177/00491241251326819 · Cited by 8
“Our reliance on a small sample of annotated data for threshold-tuning means that we do not use NLI models in a “ zero-shot ” way as suggested by Yin, Hay, and Roth (2019).”
InfluentialText Classification Under Class Distribution Shift: A Survey
Costache, A. V., Gheorghe, S.-F., Poesina, E., Irofti, P., & Ionescu, R. T. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.189 · Cited by 8
“A possible methodology for zero-shot testing is proposed by Yin et al. (2019), alongside a few suitable datasets.”
#creativity: Exploring Lay Conceptualizations of Creativity with Twitter Hashtags
Ceh, S. M., Christensen, A. P., Lebuda, I., & Benedek, M. (2023) · Creativity Research Journal · doi:10.1080/10400419.2023.2214472 · Cited by 8
“To check the goodness of fit for the manually chosen community labels, we ran zero-shot classification (Yin, Hay, & Roth, 2019) using a pre-trained model ( facebook/bart-large-mnli ).”
Resinas, M., del–Río–Ortega, A., & van der Aa, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41620-0_16 · Cited by 8
“simbert, which uses pre-trained natural language inference models as zero shot text classifiers [22].”
The Importance of Multi-Lingual Sentiment Analysis in Digital Marketing
Biswas, P., Neogi, S. G., Arockiam, D., & Mitra, A. (2024) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito61523.2024.10522115 · Cited by 8
“To effectively engage various audiences online, businesses require integrated capabilities spanning global content, social listening, and analytical insights [40].”
BackgroundPhilippy, F., Guo, S., Lothritz, C., Klein, J., & Bissyandé, T. F. (2025) · doi:10.18653/v1/2025.lm4uc-1.8 · Cited by 8
Heinrich, P., Blombach, A., Dang, B. M. D., Zilio, L., Havenstein, L., Dykes, N., Evert, S., & Schäfer, F. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5pdoao8iyoyd · Cited by 8
“An early benchmark for zero-shot text classification based on natural language inference (NLI) was proposed by Yin et al. (2019).”
BackgroundMethodBorazio, F., Croce, D., Gambosi, G., Basili, R., Margiotta, D., Scaiella, A., Manso, M. D., Petrone, D., Cannone, A., Mateo‐Urdiales, A., et al. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/23y9zw783r7h · Cited by 8
“This approach, grounded in natural language inference (Yin et al., 2019), exploits logical alignment between text and topic-defining assertions in the form of prompts.”
BackgroundMethodAccuracy is Not Agreement: Expert-Aligned Evaluation of Crash Narrative Classification Models
Bhagat, S. R., Shihab, I. F., Wood, J., & Sharma, A. (2025) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc60802.2025.11423032 · Cited by 8
“BERT with Zero-shot Text Classification (BZS) [18] adapts pre-trained BERT knowledge to our specific classification task without extensive domain-specific training.”
Zero-Shot Topic Labeling for Hazard Classification
Rondinelli, A., Bongiovanni, L., & Basile, V. (2022) · Information · doi:10.3390/info13100444 · Cited by 8
“The number of different approaches proposed in the literature is huge, and with many of them reaching very good performances; in Yin et al. [6] an extensive description of the problem with different solutions and benchmark results is given, while Xian et al. [7] presents a review of the current…”
Transfer Learning via Representation Learning
Rostami, M. G., He, H., Chen, M., & Roth, D. (2022) · Adaptation, learning, and optimization · doi:10.1007/978-3-031-11748-0_10 · Cited by 8
Self-citationReducing Negative Effects of the Biases of Language Models in Zero-Shot Setting
Wang, X., Xiong, Y., Kang, B., Zhang, Y., Yu, P. S., & Zhu, Y. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570382 · Cited by 8
“…stimulate the knowledge of PLMs that benefits downstream NLP tasks by leveraging natural-language prompts to formalize downstream tasks as language modeling problems, and show that such prompts induce better performances for PLMs on few-shot [15, 17, 26, 27] and zero-shot settings [23, 31, 34– 36].”
BackgroundIndirectly Supervised Natural Language Processing
Yin, W., Chen, M., Zhou, B., Qiang, N., Chang, K.-W., & Roth, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-tutorials.5 · Cited by 8
“In specific, we will introduce typical work that treats different topics (Yin et al., 2019), stances (Xu et al., 2022), entity types (Li et al., 2022a; Du et al., 2023), event types (Lyu et al., 2021), entity relations (Xia et al., 2021; Sainz et al., 2021, 2022), and question-answer (Yin et al.,…”
BackgroundSelf-citationSeed-Guided Fine-Grained Entity Typing in Science and Engineering Domains
Zhang, Y., Zhang, Y., Shen, Y., Deng, Y., Popa, L. C., Shwartz, L., Zhai, C., & Han, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29933 · Cited by 8
A Prompt-Based Hierarchical Pipeline for Cross-Domain Slot Filling
Wei, X., Li, Y., Si, Y., Wang, L., Wang, X., & Dang, J. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3410877 · Cited by 8
“Similar to the template, existing work on verbalizer includes manual design [20], [43] and automatic design [44].”
MethodReboud, A., Harrando, I., Lisena, P., & Troncy, R. (2023) · Multimedia Systems · doi:10.1007/s00530-022-01040-3 · Cited by 8
“They proposed a standardized evaluation on “conceptually different and diverse aspects”: topics, emotions and situations.”
BackgroundMethodInfluentialAspect-oriented Consumer Health Answer SummarizationPreprint
Chaturvedi, R., Bhattacharya, A., & Yadav, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.06295 · Cited by 8
“We begin with a zero-shot pipeline (ZS) using a prompt and predict paradigm (Yin et al., 2019) based on NLI.”
MethodUnderstanding the Effects of Explaining Predictive but Unintuitive Features in Human-XAI Interaction
Qu, J., Arguello, J., & Wang, Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732021 · Cited by 8
Adapting Emotion Detection to Analyze Influence Campaigns on Social Media
Bhaumik, A., Bernhardt, A., Katsios, G., Sa, N., & Strzalkowski, T. (2023) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2023.wassa-1.38 · Cited by 7
“Recent studies in zero-shot emotion detection use text entailment approaches, wherein target labels generate hypotheses for the model (Yin et al., 2019; Basile et al., 2021).”
BackgroundMethodVerbal reports as data revisited: Using natural language models to validate cognitive models.
Mechera‐Ostrovsky, T., & Newell, B. R. (2024) · Decision · doi:10.1037/dec0000243 · Cited by 7
A Comparative Analysis of Transformer Models in Zero-Shot Text Classification
Kyritsis, K., Liapis, C. M., Spatiotis, N., Perikos, I., & Paraskevas, M. (2024) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa62523.2024.10786704 · Cited by 7
“In [4], the authors propose a bench-marking framework that includes unified data sets, standardized evaluations, and state-of-the-art baselines.”
Towards Reflexive AI: A Comprehensive Exploration of Enhancing Social Science Research Through NLP
Shan, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-84460-7_49 · Cited by 7
Large Language Models in Crisis Informatics for Zero and Few-Shot Classification
Sánchez, C., Abeliuk, A., & Poblete, B. (2025) · ACM Transactions on the Web · doi:10.1145/3736160 · Cited by 7
“Yin et al. [43] proposes a textual entailment framework to perform zero-shot text classification, that is, without accessing any task-specific labeled data.”
Wencker, T., Borst, J., & Niekler, A. (2025) · Evaluation · doi:10.1177/13563890251330911 · Cited by 8
“Approaches include reusing models trained for other classification tasks (Muller et al., 2022; Schick and Schutze, 2021; Yin et al., 2019), predicting labels using generative foundation models like GPT-3 (Brown et al., 2020) or describing the task to instruction-following models such as Llama…”
Zero-Shot Turkish Text Classification
Birim, A., Erden, M., & Arslan, L. M. (2021) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu53274.2021.9477864 · Cited by 7
Bai, Y., Colas, A., & Wang, D. Z. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591907 · Cited by 7
“Following [16, 42], we leverage NLI models for the zero-shot stance detection task.”
MethodEntailment as Robust Self-Learner
Ge, J., Luo, H., Kim, Y., & Glass, J. R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.772 · Cited by 9
Zha, J., Li, Z., Wei, Y., & Zhang, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.383 · Cited by 7
KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification
Wu, L., Jiang, J., Zhao, H., Wang, H., Lian, D., Zhang, M., & Chen, E. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/262 · Cited by 7
Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization
Shen, M., Ma, J., Wang, S., Vyas, Y., Dixit, K., Ballesteros, M., & Benajiba, Y. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.142 · Cited by 7
Borst, J., Wehrheim, L., Niekler, A., & Burghardt, M. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f3725 · Cited by 7
Benchmarking Multilabel Topic Classification in the Kyrgyz Language
Alekseev, A., Nikolenko, S., & Kabaeva, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-54534-4_2 · Cited by 7
Benchmarking Zero-Shot Text Classification for Dutch
Langhe, L. D., Maladry, A., Vanroy, B., Bruyne, L. D., Singh, P., Lefever, E., & Clercq, O. D. (2024) · Computational Linguistics in the Netherlands Journal · Cited by 7
Qu, J., Hashimoto, K., Liu, W., Xiong, C., & Zhou, Y. (2021) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2021.nlp4convai-1.2 · Cited by 7
“DNNC (Zhang et al., 2020) Yin et al. (2019) explored to leverage short semantic labels.”
BackgroundEmbarrassingly Simple Performance Prediction for Abductive Natural Language Inference
Kadiķis, E., Srivastav, V., & Klinger, R. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.441 · Cited by 7
Political Event Coding as Text-to-Text Sequence Generation
Dai, Y., Radford, B. J., & Halterman, A. (2022) · Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE) · doi:10.18653/v1/2022.case-1.16 · Cited by 7
“These zero-shot methods rely primarily on textual entailment formulations of the event data coding task (Yin et al., 2019).”
MethodAkash, P. S., Huang, J., & Chang, K. C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.668 · Cited by 7
Normalization Matters in Zero-Shot LearningPreprint
Skorokhodov, I., & Elhoseiny, M. (2020) · arXiv (Cornell University) · Cited by 7
NatCat: Weakly Supervised Text Classification with Naturally Annotated ResourcesPreprint
Chu, Z., Stratos, K., & Gimpel, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14335 · Cited by 7
“, 2017, Yin et al., 2019]. We use NatCat as a pretraining resource in this experimental setup. We follow the same seen and unseen label splits as Yin et al. [2019], using their v0 and v1 splits, and same experimental setup. We train our models both starting from the original BERT-base-uncased model and the NatCat-pret…”
BackgroundMethodResultInfluentialTweets2Stance: Users stance detection exploiting Zero-Shot Learning Algorithms on TweetsPreprint
Gambini, M., Fagni, T., Senette, C., & Tesconi, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.10710 · Cited by 7
“Particularly, we exploited the zero-shot learning technique first proposed by [17] to use language models pre-trained on the Natural Language Inference (NLI) task [23] as generic classifiers able to quickly and accurately operate on non-labelled data.”
MethodEmpowering Sentence Encoders with Prompting and Label Retrieval for Zero-shot Text ClassificationPreprint
Hong, J., Park, J., Kim, D.-Y., Choi, S., Son, B., & Kang, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.10391 · Cited by 7
“Yin et al. (2019) solves the classification task by comparing the entailment logit scores where they concatenate the text input and label prompt as text pairs.”
BackgroundMethodInfluentialAdvancing Topical Text Classification: A Novel Distance-Based Method with Contextual Embeddings
Kosar, A., Pauw, G. D., Textgain, A., & Daelemans, W. M. P. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_064 · Cited by 6
“Yin et al. (2019) outlined three primary techniques for unsupervised text classification: 1) evaluating the frequency of class labels in a text, 2) measuring the distance between class labels and text in a shared vector space, and 3) leveraging nat-ural language inference with pre-trained…”
MethodInfluentialJeong, H., Luo, T., Kang, M., Garvey, W. F., Blankenau, G., Suk, J.-W., Tarzaben, M., & Hwang, S. (2024) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2024.111917 · Cited by 6
“To further refine our selection of articles, we employed a transformer-based bart-large model trained on the MultiNLI (MNLI) dataset to conduct zero-shot classification ( Yin et al., 2019).”
Large Language Models for Logical Fallacy Detection
Teo, N., Huang, D., Cambria, E., & Wang, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8197-6_29 · Cited by 6
Busta, L., & Oyler, A. R. (2025) · Quantitative Plant Biology · doi:10.1017/qpb.2025.10021 · Cited by 6
“For example, large language models have been utilized to perform tasks such as text classification, text summarization and question answering (Dalal et al., 2024; Guo et al., 2023; Riordan et al., 2024; Shiu & Lehti-Shiu, 2024; Yin et al., 2019).”
PReSS: An Automated Black-Box Framework for Evaluating Political Stance Stability in LLMs
Kabir, S., Dong, Y., & Esterling, K. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/35d8pipu4ofv · Cited by 6
“This zero-shot NLI approach (Yin et al., 2019) is widely adopted for stance classification in LLM political bias evaluations (Feng et al., 2023) and requires no task-specific training data.”
Wang, S., Zhu, Y., & Li, P. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104566 · Cited by 6
Physics-informed multimodal large model for machine fault diagnosis with dynamic vision
Lu, Z., Li, X., Liu, C., & Yu, S. (2026) · Structural Health Monitoring · doi:10.1177/14759217261445117 · Cited by 6
A Novel Approach for Cross-Selling Insurance Products Using Positive Unlabelled Learning
Sidorowicz, T., Peres, P. L. D., & Li, Y. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892762 · Cited by 6
“[13] built on this proposing a textual entailment framework achieving state-of-the-art results without annotated labels using Natural Language Inference (NLI).”
BackgroundMethodComparing User Perspectives in a Virtual Reality Cultural Heritage Environment
Bulla, L., Giorgis, S. D., Gangemi, A., Lucifora, C., & Mongiovì, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-34560-9_1 · Cited by 6
ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification
Guo, K., Yu, H., Liao, C., Li, J., & Zhang, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.200 · Cited by 6
“After fine-tuning with the labeled data for each dataset, BERT-MNLI greatly increases the macro F 1 -score, but still compares unfavorably with ZeroAE, probably because it cannot well generalize to unseen classes (Ma et al., 2021).”
BackgroundMethodInfluentialExtraction and Classification of Statute Facets using Few-shot Learning
Pawar, S. S., Ali, B., Palshikar, G. K., Singh, R., & Singh, D. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595134 · Cited by 6
“[20] proposed an interesting idea where text classification can be performed without any supervision or training using only a pre-trained model on NLI task.”
BackgroundRethinking the Event Coding Pipeline with Prompt Entailment
Lefebvre, C., & Stoehr, N. (2023) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2023.fever-1.1 · Cited by 6
A novel prompt-tuning method: Incorporating scenario-specific concepts into a verbalizer
Ma, Y., Luo, S., Shang, Y.-M., Li, Z., & Liu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123204 · Cited by 6
“Among the former, Yin et al. (Yin et al., 2019) manually design word lists related to class names as label word sets.”
BackgroundReading between the lines: The intersection of research attention and sustainable development goals
Funk, C., Tönjes, E., Teuber, R., & Breuer, L. (2024) · Sustainable Development · doi:10.1002/sd.2906 · Cited by 6
Hobbs, W., & Green, J. (2025) · Political Analysis · doi:10.1017/pan.2024.23 · Cited by 6
Zero-Shot Prediction of Conversational Derailment With Large Language Models
Nonaka, K., & Yoshida, M. (2025) · IEEE Access · doi:10.1109/access.2025.3554548 · Cited by 6
“As transformer models such as BERT and GPT demonstrate considerable advancement in most natural language processing tasks, they are capable of performing zero-shot classification based on the knowledge acquired during pre-training [25], [26], [27].”
MorphNLI: A Stepwise Approach to Natural Language Inference Using Text Morphing
Negru, V. A., Vacareanu, R., Lemnaru, C., Surdeanu, M., & Potolea, R. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.385 · Cited by 6
“…et al., 2005; Bow-man et al., 2015), is an important building block of many applications such as question answering, summarization, and dialogue systems, where understanding the logical connection between different pieces of information is essential (Yin et al., 2019; Sainz et al., 2021, 2022).”
DECIFE: Detecting Collusive Users Involved in Blackmarket Following Services on Twitter
Dutta, H. S., Aggarwal, K., & Chakraborty, T. (2021) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3465336.3475108 · Cited by 6
Understanding Class Representations : An Intrinsic Evaluation of Zero-Shot Text Classification
Hoppe, F., Dessì, D., & Sack, H. (2021) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.34657/7999 · Cited by 6
Generalised Zero-shot Learning for Entailment-based Text Classification with External Knowledge
Wang, Y., Wang, W., Chen, Q., Huang, K., Nguyen, A., & De, S. (2022) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp55677.2022.00018 · Cited by 6
DeepPress: guided press release topic-aware text generation using ensemble transformers
Rahali, A., & Akhloufi, M. A. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08393-4 · Cited by 6
“BART is trained on the datasets [65] for zero-shot classification and can easily be modified to handle different classification candidates.”
MethodInfluentialEthnicity on the table: A content analysis of online reviews for Syracuse's cuisines
Koytak, H. Z. (2024) · Sociology Compass · doi:10.1111/soc4.13195 · Cited by 6
Viegas, F. A. R., Canuto, S., da Cunha, W. L. M., França, C., de Andrade, C. M. V., Fonseca, G., Machado, A. C., da Rocha, L. C. D., & Gonçalves, M. A. (2024) · Journal on Interactive Systems · doi:10.5753/jis.2024.4117 · Cited by 6
“An alternative approach involves employing zero-shot learning with BERT pre-trained models for entailment tasks [Yin et al ., 2019], where the goal is to ascertain whether a premise sentence logically entails a hypothesis sentence corresponding to a particular label, such as positive or negative…”
MethodTowards Distribution-shift Robust Text Classification of Emotional Content
Bulla, L., Gangemi, A., & Mongiovì, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.524 · Cited by 6
“Yin et al. (Yin et al., 2019) provide datasets, a standard for evaluation and state-of-the-art baselines for zero-shot classification.”
InfluentialZhang, R. H., Fan, A. X., & Zhang, R. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.142 · Cited by 7
Towards Personalized Intelligence at ScalePreprint
Kang, Y., Mahendra, A., Clarke, C., Tang, L., & Mars, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.06668 · Cited by 6
Enhancing Metaphor Detection through Soft Labels and Target Word PredictionPreprint
Jia, K., & Li, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.18253 · Cited by 6
“Currently, prompt learning has found widespread application in diverse domains such as text classification (Yin et al., 2019), information extraction (Cui et al., 2021), question answering systems (Khashabi et al., 2020), and many others.”
BackgroundCoburn, B., He, J., Rollo, M. E., Dhaliwal, S. S., Kerr, D. A., & Zhu, F. M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.07048 · Cited by 6
“…name normalization (e.g., converting “Thai Basil Chicken” to “thai basil chicken” through lowercasing and punctuation removal); the classifier leverages natural-language classification [48] to effectively classify our cuisine types into predefined cuisine labels (such as American, Chinese, etc.).”
GLiGuard: Schema-Conditioned Classification for LLM SafeguardPreprint
Zaratiana, U., Newhauser, M., Hurn-Maloney, G., & Lewis, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.07982 · Cited by 6
“…but encoded directly into the model input as structured token sequences with natural-language descriptions, so different combinations of the supported tasks can be evaluated by composing their task and label blocks in the schema (Yin et al., 2019; Kumar et al., 2024b; Stepanov et al., 2025).”
Chinese-Jev: Bringing System One Model to Chinese-Language TasksPreprint
Wang, Z., Zhang, Z., Wang, X., Wang, P., Ye, Z., Zhao, H., Wu, Z., & Yan, S. (2026) · arXiv (Cornell University) · Cited by 6
Democracia aumentada: Inteligência Artificial como ferramenta de combate à desinformação
Alcoforado, A., Ferraz, T. P., Bustos, E., Oliveira, A. S., Gerber, R., Santoro, G. L. D. M., Fama, I. C., Veloso, B., Siqueira, F. L., & Costa, A. H. R. (2024) · Estudos Avançados · doi:10.1590/s0103-4014.202438111.021 · Cited by 5
Smishing Detection From a Messaging Platform View
Schwarz, S. F., Fonseca, P. D. T., & Rocha, A. D. R. (2025) · IEEE Access · doi:10.1109/access.2025.3597903 · Cited by 5
“For that, we leverage a zero-shot learning approach Yin et al. [27].”
Identifying large language model hallucinations in health communication
Galitsky, B. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30046-2.00012-0 · Cited by 5
Applying Large Language Models to investigate how people with disabilities interact at airports
McCullough, S. T., Grant, A., Mistur, E., & Park, J. Y. (2025) · Journal of Air Transport Management · doi:10.1016/j.jairtraman.2025.102880 · Cited by 5
A cross-country analysis on satisfaction and perception of non-alcoholic wine
Rui, M., Blanc, S., Brun, F., Sparacino, A., Mastromonaco, G., & Massaglia, S. (2025) · Food Quality and Preference · doi:10.1016/j.foodqual.2025.105774 · Cited by 5
“Zero-shot classification leverages pre-trained transformer models fine-tuned on natural language inference (NLI) tasks, enabling them to classify text into pre-defined topics without requiring task-specific training data (Yin, Hay, & Roth, 2019).”
Influential
Metadata
- Authors
- Yin, W., Hay, J., & Roth, D.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3912-3921
- OpenAlex
- W2970200208
- Cited by
- 537
- References
- 30
- Citations by year
- 2026: 66 · 2025: 86 · 2024: 75 · 2023: 141 · 2022: 94 · 2021: 64 · 2020: 11
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency (HR0011-15-C-0113)
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- zero-shot text classification, benchmarking, evaluation metrics, logical consequence, cross-domain classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

