Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Wenpeng Yin, Jamaal Hay, Dan Roth. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is evidenced by LLMs impressive zero-shot and few-shot learning ability [18,132], as well as the fact that in-context learning does not require weight update, which mechanically prevents unfavorable fine-tuning-related behavior like catastrophic forgetting.”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have been shown to outperform previous zero-shot methods [132].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“• Word2Vec 5 (Mikolov et al., 2013): Both the representations of the text and the labels are the addition of word embeddings element-wisely.”
BackgroundGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 11×
“For label-partially-unseen setup in which we intentionally provide annotated data, we first pretrain BERT on the MNLI/FEVER/RTE, then fine-tune on the provided training data.”
BackgroundMethodInfluentialA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 2×
“, 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al.”
BackgroundMethodCharacter-level Convolutional Networks for Text Classification
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 3,205 · cited 1×
“We use the large-scale Yahoo dataset released by Zhang et al. (2015).”
MethodComputing semantic relatedness using Wikipedia-based explicit semantic analysis
Gabrilovich, E., & Markovitch, S. (2007) · Cited by 1,981
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
(2018) · International Conference on Learning Representations · Cited by 1,621
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 1×
“In this work, we make use of the widely-recognized state of the art entailment technique – BERT (Devlin et al., 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al., 2005; Wang et al., 2019) and FEVER 3 (Thorne et al., 2018), respectively.”
Method
Research works printed but not linked: 2
- [8]8. Chenliang Li, Jian Xing, Aixin Sun, and Zongyang Ma. 2016a. Effective document labeling with very few seed words: A topic model approach. In CIKM, pages 85-94.
- [11]11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed repre- sentations of words and phrases and their composi- tionality. In NIPS, pages 3111-3119.
Cited by
Items 401–500 of 867
CYBY24 and Step-Wise Model for Thread-Based Fine-Grained Cyberbullying Detection
Alfurayj, H. S., Farid, D. M., Luna-Jiménez, C., & Lutfi, S. L. (2026) · IEEE Access · doi:10.1109/access.2026.3652469 · Cited by 5
“Zero-shot learning in text classification is a new method that operates without relying on labeled data and without any training, setting it apart from conventional models that heavily depend on labeled datasets [25].”
Zero-Shot Multi-Label Classification of Bangla Documents: Large Decoders Vs. Classic Encoders
Sarkar, S., Hasan, M. N., & Karmaker, S. (2025) · Workshop on Bangla Language Processing (BLP) · doi:10.18653/v1/2025.banglalp-1.7 · Cited by 5
Reassessing automated essay scoring with large language models: Evidence from API and GUI interfaces
Kim, J., & Holden, D. (2026) · Assessing Writing · doi:10.1016/j.asw.2026.101080 · Cited by 5
ProZe: Explainable and Prompt-Guided Zero-Shot Text Classification
Harrando, I., Reboud, A., Schleider, T., Ehrhart, T., & Raphaël Troncy, A. (2022) · IEEE Internet Computing · doi:10.1109/mic.2022.3187080 · Cited by 5
“Net to perform Zero-Shot classification; • Entail: this model was originally proposed in [9].”
MethodFrom Tweets to Stance: An Unsupervised Framework for User Stance Detection on Twitter
Gambini, M., Senette, C., Fagni, T., & Tesconi, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45275-8_7 · Cited by 5
Dense Retrieval as Indirect Supervision for Large-space Decision Making
Xu, N., Fei, W., Dong, M., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1002 · Cited by 5
Jin, Z., Cao, P., He, Z.-T., Chen, Y., Liu, K., & Zhao, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.974 · Cited by 5
“(2) BEM : Yin et al. (2019) propose to treat zero-shot text classification as a textual entailment task and classify whether a class description is entailed by the text.”
BackgroundInteractive Machine Teaching by Labeling Rules and Instances
Karamanolakis, G., Hsu, D., & Gravano, L. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00707 · Cited by 5
“…on each iteration, namely Teacher-Student co-training (Section 3.1), querying for instance feedback (Section 3.2), candidate rule extraction (Section 3.3), and querying for rule feedback (Section 3.4), and then we summarize the main ideas of our interactive machine teaching algorithm (Section 3.5).”
InfluentialCausal Reasoning of Occupational Incident Texts Using Large Language Models
Nakamura, M., Hayamizu, S., Hattori, M., Fuseya, T., Iwamatsu, H., & Terada, K. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.501 · Cited by 5
Visual Footprint of Separation Through Membrane Distillation on YouTube
Aytaç, E., & Khayet, M. (2025) · Data · doi:10.3390/data10020024 · Cited by 5
Working with Few Samples: Methods that Help Analyze Social Attitude and Personal Emotion
Liang, M., Zhou, J., Sun, Y., & He, L. (2021) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd49262.2021.9437799 · Cited by 5
BERT Based Topic-Specific Crawler
Tawil, Y., & Alqaraleh, S. (2021) · Innovations in Intelligent Systems and Applications Conference (ASYU) · doi:10.1109/asyu52992.2021.9599076 · Cited by 5
How Universal is Genre in Universal Dependencies?
Müller-Eberstein, M., van der Goot, R., & Plank, B. (2021) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.2112.04971 · Cited by 5
“Prior research has found that classifying sentences based solely on their cosine similarity to genre label strings in MLM embedding space can be remarkably effective (Veeranna et al., 2016; Yin et al., 2019; Davison, 2020).”
MethodKarapetian, K., Jeon, S. M., Kwon, J., & Suh, Y. (2022) · Journal of Medical Internet Research · doi:10.2196/41100 · Cited by 5
Identifying Topic and Cause for Sarcasm: An Unsupervised Knowledge-enhanced Prompt Method
Yuan, M., Li, Q., Mao, X. F., & Zeng, D. D. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587343 · Cited by 5
“• Entailment [15]: Entailment approach treats zero-shot text classifcation as a textual entailment problem.”
BackgroundMethodImproved Word Sense Disambiguation via Prompt-based Contextual Word Representation
He, Q., Zhang, J., & Huang, X. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191929 · Cited by 5
“[9] designed prefix templates for quiz, translation, and probe tasks, and the main research objectives of answer engineering are how to design the answer space and construct a mapping from answers to final output labels [10, 11].”
BackgroundGeneralization of finetuned transformer language models to new clinical contexts
Xie, K., Terman, S. W., Gallagher, R. S., Hill, C. E., Davis, K. A., Litt, B., Roth, D., & Ellis, C. A. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad070 · Cited by 5
Self-citationContextual stance classification using prompt engineering
da Fonseca, F. P. C., Paraboni, I., & Digiampietri, L. A. (2023) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2023.233708 · Cited by 5
“…architecture, addressed the issue of contextual stance classi(cid:2)cation in a standard fashion, that is, none of them addressed the task using zero-or few-shot prompt Examples of this kind are only beginning to emerge in the (cid:2)eld, and include, example, [Yin et al. 2019, Zhang et al. 2023].”
ChemDataWriter: a transformer-based toolkit for auto-generating books that summarise research
Huang, S., & Cole, J. M. (2023) · Digital Discovery · doi:10.1039/d3dd00159h · Cited by 5
Wu, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.982 · Cited by 5
Qu, J., Arguello, J., & Wang, Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658547 · Cited by 5
Using Twitter Dataset for Social Listening in Singapore
Wang, Q., Sailor, H. B., Lee, K. A., Ma, K., Goh, K. H., & Boh, W. F. (2024) · IEEE Access · doi:10.1109/access.2024.3427760 · Cited by 5
“Utilizing Hugging Face’s built-in zero-shot classification pipeline [19], [20], we acquire confidence scores.”
MethodAfreen, N., Balloccu, G., Boratto, L., Fenu, G., Malloci, F. M., Marras, M., & Martis, A. G. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728323 · Cited by 5
“In COCO , occupational groups were extracted from course metadata using ESCO 4 and linked via a zero-shot classifier [34].”
A Real-Time Semantic Model for Relevance and Novelty Detection from Group Messages
Fisher, D., Choudhary, R., Alsayed, O., Doboli, S., & Minai, A. A. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892268 · Cited by 5
“3) Zero-Shot Classifier: ZSCs ([38], [39]) can be used to predict whether a text and a chosen label are related or not without any labelled data.”
BackgroundRelevance Feedback Search Based on Automatic Annotation and Classification of Texts
Leal, R., Kesäniemi, J., Koho, M., & Hyvönen, E. (2021) · Aaltodoc (Aalto University) · doi:10.4230/oasics.ldk.2021.18 · Cited by 5
Narratives and Needs: Analyzing Experiences of Cyclone Amphan Using Twitter DiscoursePreprint
Crayton, A. S., Fonseca, J., Mehra, K., Ng, M., Ross, J., Sandoval-Castañeda, M., & von Gnechten, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05560 · Cited by 5
“Researchers have proposed an approach of using a pre-trained NLI (Natural Language Inference) sequence-pair classifier as a zero-shot text classifier [Yin et al., 2019].”
MethodAbbonato, D. (2024) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2411.14230 · Cited by 5
“I address this challenge using zero-shot classification, a machine learning paradigm in which models classify texts into categories not encountered during training (Yin et al., 2019).”
Dhima, A., Choudhary, S., Myrick, K. J., & Torous, J. B. (2025) · Digital Health · doi:10.1177/20552076251327044 · Cited by 4
Zhang, X., Zhou, Q., Liu, H., & Yu, H. Z. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.52 · Cited by 4
“SP RoBerta+template (Lamanov et al., 2022) utilizes templates to fuse utterances and intent labels, transforming intent recognition into a text entailment task (Yin et al., 2019), then it applies contrastive learning on multiple pairs of seen data and intent labels to enhance discriminability.”
Leveraging Text-to-Text Transformers as Classifier Chain for Few-Shot Multi-Label Classification
Nguyen, Q. A., Tomeh, N., Lebbah, M., Charnois, T., & Azzag, H. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1368 · Cited by 4
“We compare our models with the following approaches, which we adapt and train ourselves (appendix B): Head (Sun et al., 2020), PET (Schick and Schütze, 2021a,b), SetFit (Tunstall et al., 2022), BERT-LWAN (Mullenbach et al., 2018; Chalkidis et al., 2020), and NSP (Yin et al., 2019; Ma et al., 2021).”
Pattnaik, A., Vutla, S., Dev, H., Nandan, J., & George, C. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-industry.137 · Cited by 4
“LLMs perform well in zero-shot settings Yin et al. (2019), but degrade with large label spaces Zhang et al. (2023a) are sensitive to prompt formatting and label order Zhao et al. (2021); Lu et al. (2022), especially under high cardinality.”
When Personality Meets Platform: The Moderating Role of Human-Likeness in AI User Evaluations
Lee, S., & Choi, Y. K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2604629 · Cited by 4
“The Facebook Bart Large MNLI model was selected due to its established performance in entailment-based zero shot classification and its generalization capacity in semantically complex tasks (Yin et al., 2019).”
Lei, B., Wang, J., & Shen, C. (2026) · Scientific Reports · doi:10.1038/s41598-026-38214-2 · Cited by 4
Cosentino, C., Folino, F., Marozzo, F., & Pontieri, L. (2026) · Array · doi:10.1016/j.array.2026.100705 · Cited by 4
Contrastive Distillation of Emotion Knowledge from LLMS for Zero-Shot Emotion Recognition
Niu, M., & Provost, E. M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463749 · Cited by 4
Temporal Guardrails for LLM Conversations: A Runtime Verification Framework
Cohen, I., Havelund, K., Omer, M., & Peled, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-32357-6_7 · Cited by 4
“Under this view, classification reduces to deciding whether the text entails the label description [41].”
Domain-control prompt-driven zero-shot relational triplet extraction
Xu, L., Gao, C., & Tian, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127270 · Cited by 4
Data augmentation using instruction-tuned models improves emotion analysis in tweets
Yandrapati, P. B., & Eswari, R. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01325-7 · Cited by 4
“Nearest Neighbor Search is the simplest zero-shot text classification approach, which leverages the concept of similarity between the dense vectorial representations of the text to be classified and the labels (Yin et al. 2019).”
On Reference (In-)Determinacy in Natural Language Inference
Chen, S., Malaviya, C., Fabrikant, A., Taitelbaum, H., Schuster, T., Buthpitiya, S., & Roth, D. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.450 · Cited by 4
Self-citationAhmad, K. M., Liu, Q., Khan, A. A., Pang, R., Dai, T., & Gan, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107711 · Cited by 4
Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Models
Elshabrawy, A., Nguyen, T., Kang, Y., Feng, L., Jain, A., Shaikh, F. A., Mansurov, J., Imam, M. F. M., Ortiz-Barajas, J.-G., Chevi, R., et al. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.835 · Cited by 4
Task Calibration: Calibrating Large Language Models on Inference Tasks
Li, Y., Luo, Y., Xie, X., & Zhang, Y. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.362 · Cited by 4
You, J., Lee, J., Lee, S., & Kwon, H.-Y. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3777990 · Cited by 4
ASCM: An Answer Space Clustered Prompting Method without Answer Engineering
Wang, Z., Yang, Y., Xi, Z., Ma, B., Wang, L., Dong, R., & Anwar, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.193 · Cited by 4
“Yin et al. (2019) manually select words for each label in topic classification, emotion classification, and situation classification task.”
BackgroundZero-Shot Text Classification with Semantically Extended Textual Entailment
Liu, T., Hu, Y., Chen, P., Sun, Y., & Yin, B. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191094 · Cited by 4
“• MNLI-BERT [5]: On the basis of the above model, this method further adopts the mainstream entailment dataset - MNLI to pre-train BERT to improve domain shift and knowledge transfer.”
BackgroundMethodInfluentialConformal Predictor for Improving Zero-Shot Text Classification Efficiency
Choubey, P. K., Bai, Y., Wu, C.-S., Liu, W., & Rajani, N. F. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.196 · Cited by 4
Semantic matching for text classification with complex class descriptions
Silva, B. D., Huang, K.-W., Lee, G., Hovsepian, K., Xu, Y., & Shen, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.475 · Cited by 4
“This approach is closely related to the one proposed in Yin et al. 2019 (and, later, Wang et al. 2021; Halder et al. 2020), except for two key differences.”
BackgroundMethodResultInfluentialEfficient Zero-Shot Cross-lingual Inference via Retrieval
Winata, G. I., Xie, L., Radhakrishnan, K., Gao, Y., & Preoţiuc-Pietro, D. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-short.11 · Cited by 4
“Further, fine-tuning encoder models for sentence representations, for example using the natural language inference task, shows an ability to generalize for both monolingual (Yin et al., 2019) and multilingual classification tasks (Winata et al., 2021).”
Wang, G., Huang, Z., Li, D., & Liao, J. (2024) · Environmental science and engineering · doi:10.1007/978-3-031-53456-0_11 · Cited by 4
LabCLIP: Label-Enhanced Clip for Improving Zero-Shot Text Classification
Zhang, Y., Wang, P., Chen, Q., Zhou, J., Wang, Y., Li, M., & Qin, L. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446865 · Cited by 4
“To achieve zero-shot text classification, Yin et al. [3] pro-pose a textual entailment approach, which uses a descriptive sentence to replace the label to capture the label-related information.”
BackgroundMethodLow-resource multi-granularity academic function recognition based on multiple prompt knowledge
Liu, J., Xiong, Z., Jiang, Y., Ma, Y., Lu, W., Huang, Y., & Cheng, Q. (2024) · The Electronic Library · doi:10.1108/el-01-2024-0022 · Cited by 4
“However, tuning the extra classifier requires adequate training examples to achieve decent performance, it is still challenging to apply fine-tuning in low-resource settings, including few-shot and zero-shot learning scenarios [3, 70].”
BackgroundSingle Example Can Improve Zero-Shot Data Generation
Burnyshev, P., Malykh, V., Bout, A., Artemova, E., & Piontkovskaya, I. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.20 · Cited by 4
Zero-Shot Clinical Questionnaire Filling From Human-Machine Interactions
Toudeshki, F. G., Jolivet, P., Durand-Salmon, A., & Liednikova, A. (2021) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/2021.mrqa-1.5 · Cited by 4
“On the other hand, (Yin et al., 2019) shows how ZeroShot-TC models may be successfully used as NLI models, being partially inspired by the way (Obamuyide and Vla-chos, 2018) transformed relation classification task into NLI problem.”
MethodNuntachit, N., & Sugunnasil, P. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4030030 · Cited by 4
“[25] used the pre-trained Multi-genre NLI (MNLI) sequence-pair classifier as an out-of-the-box zero-shot text classifier, achieving an F1 score of 37.”
BackgroundMethodUsing Pseudo-Labelled Data for Zero-Shot Text Classification
Cong-cong, W., Nulty, P., & Lillis, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08473-7_4 · Cited by 4
“These are then expanded using BERT masked Topic Situation UnifyEmotion TwiEmotion Semantic matching based runs Label similarity [20] 34.62 40.75 26.21 56.03 Entail-single [28] 43.80 37.20 24.70 49.60 Entail-ensemble [28] 45.”
BackgroundMethodInfluentialJia, Q., Cao, Y., & Gehringer, E. F. (2022) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2022.bea-1.8 · Cited by 4
“The core idea of the method is to treat the problem of detecting quality feed-back as a natural language inference (NLI) task and utilize the pre-trained BART-based NLI model (Yin et al., 2019) to assess feedback quality.”
MethodInfluentialAutomatic Detection of Inconsistencies in Open-Domain Chatbots
Prats, J. M., Estecha-Garitagoitia, M., Rodríguez-Cantelar, M., & D’Haro, L. F. (2022) · IberSPEECH · doi:10.21437/iberspeech.2022-24 · Cited by 4
“Our algorithm is based on the zero-shot classification proposed in [10] which considers classification as the result of a natural language inference (NLI) process.”
MethodAd-Hoc Monitoring of COVID-19 Global Research Trends for Well-Informed Policy Making
Sarkar, S., Bijoy, B. S., Saba, S. J., Feng, D., Mahajan, Y., Amin, M. R., Islam, S. R., & Karmaker, S. K. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3576901 · Cited by 4
“For that reason, a group of researchers [69] employed Zero-Shot Learning for face detection by identifying the face mask.”
BackgroundMethodInfluentialMylonas, N., Karlos, S., & Tsoumakas, G. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102505 · Cited by 4
Identifying Checkworthy CURE Claims on Twitter
Gollapalli, S. D., Du, M., & Ng, S.-K. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583870 · Cited by 4
“In the zero-shot text classifcation approach proposed by Yin, et al., the problem is converted into an entailment problem (“this text is about <class-label>?") and solved using Natural Language Inference (NLI) [40].”
MethodImproving Zero-Shot Text Classification with Graph-based Knowledge Representations
Hoppe, F. (2022) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.34657/10667 · Cited by 4
Martinc, M., Montariol, S., Pivovarova, L., & Zosa, E. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/43pvk6nzi934 · Cited by 4
“8 The second evaluation approach is motivated by Yin et al. (2019), who used a pretrained natural language inference (NLI) sequence-pair classi-fier as a zero-shot text classifier.”
BackgroundDeng, P., Yuan, J., Zhao, Y., & Qin, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.02276 · Cited by 4
“Yin et al.,(2019) reformulate text classification tasks as a textual entailment formulation.”
BackgroundPrompt2Gaussia: Uncertain Prompt-learning for Script Event PredictionPreprint
Cui, S., Cong, X., Sheng, J., Wang, X., Liu, T., & Shi, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.02103 · Cited by 4
“Early research exploits the PVP construction using the manually-designed prompt [13, 57, 61] and label tokens [9, 55, 68].”
MethodA Search for Prompts: Generating Structured Answers from ContractsPreprint
Roegiest, A., Chitta, R., Donnelly, J., Lash, M., Vtyurina, A., & Longtin, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.10141 · Cited by 4
“…2022, we had spent time investigating the utility of MNLI [31] pre-trained encoder models 10 for structured question answering (described in Section 5.1) but found that the technique was only suitable when clauses and options could fit in relatively small context windows of such models [28,33].”
BackgroundMethodInfluentialAnswering Clean Tech Questions with Large Language ModelsPreprint
Stagnol, L., Cherief, A., Farah, Z., Guenedal, T. L., Sakout, S., & Sekine, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4663447 · Cited by 4
He, Z., Murugesan, B. R., Brandt, P. T., & Hu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.06781 · Cited by 4
“Zero-shot and NLI-based methods use label descriptions, hypotheses, or ontology information to transfer semantic knowledge to new classification schemas (Obamuyide and Vlachos, 2018; Yin et al., 2019; Huang et al., 2018; Geng et al., 2021; Hu et al., 2024).”
When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent WorkflowsPreprint
Ling, Y., Yu, S., Chen, Z., & Fang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.13092 · Cited by 4
“Entailment-based classifiers and GLiClass support labels supplied at inference time (Yin et al., 2019; Stepanov et al., 2025).”
Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning
Elshabrawy, A., Huang, Y., Gurevych, I., & Aji, A. F. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.465 · Cited by 3
Exploring Zero-Shot SLM ensembles as an alternative to LLMs for sentiment analysis
Cielen, D., Bock, K. W. D., & Florès, L. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103666 · Cited by 3
EcoLens GRIE: AI-Driven Detection of Greenwashing Claims Using Transformer-Based NLP
Mogili, U. (2026) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2026.78204 · Cited by 3
Lee, J. Y., Tian, L., Brillantes, A., Mihăiţă, A., & Rizoiu, M.-A. (2026) · Social Network Analysis and Mining · doi:10.1007/s13278-026-01633-0 · Cited by 3
“Two paradigms dominate the literature on misinformation response and stance classification (Augenstein et al. 2016; Mohammad et al. 2016; Hardalov et al. 2021): zero-shot inference (Yin et al. 2019; Schick and Schütze 2021) and supervised fine-tuning (Devlin et al. 2019; Liu et al. 2019).”
A Multi-instance Multi-label Weakly Supervised Approach for Dealing with Emerging MeSH Descriptors
Mylonas, N., Karlos, S., & Tsoumakas, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77211-6_47 · Cited by 3
Validating Ontology-based Annotations of Biomedical Resources using Zero-shot Learning
Koutsomitropoulos, D. A. (2021) · doi:10.1145/3486713.3486730 · Cited by 3
“The idea of using general-language models finetuned on natural language inference tasks to achieve zero-shot classification is proposed in [28] and is further discussed in Section 4.”
BackgroundInfluentialZero-Shot Data Maps. Efficient Dataset Cartography Without Model Training
Basile, A., Franco-Salvador, M., & Rosso, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.554 · Cited by 3
“Among the different architectures that have been shown capable of turning LLMs into zero-shot classifiers (Radford et al., 2019; Yin et al., 2019; Schick and Schütze, 2021), we focus on the bi-encoder (or Dual Encoder) class of models (Reimers and Gurevych, 2019; Mueller et al., 2022) due to their…”
BackgroundA Web-based Text Mining System for Analyzing Customer Feedback of Returned Products
Lin, M. S., & Wen, R. (2024) · International Conference on Computers in Management and Business (ICCMB) · doi:10.1145/3647782.3647784 · Cited by 3
“On the other hand, semantic similarity-based approaches [3, 13] involve calculating the semantic similarity of two texts, and more advanced approaches involve generalizing the label-target text setting to a hypothesis-premise setting, using a pre-trained MNLI model [9, 14].”
MethodExtremely Weakly-supervised Text Classification with Wordsets Mining and Sync-Denoising
Xiao, L. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.397 · Cited by 3
“There are many sources of weak supervision signals, including: 1) external knowledge bases(Gabrilovich et al., 2007; Yin et al., 2019), 2) seed words(Meng et al., 2020b; Zhang et al., 2021; Mekala and Shang, 2020; Wang et al., 2021; Zhao et al., 2023a), 3) heuristic rules(Badene et al., 2019; Shu…”
BackgroundTharayil, S. M., Alotaibi, N., Idris, M. A., & Aldhalaan, B. H. (2024) · Information systems engineering and management · doi:10.1007/978-3-031-65014-7_7 · Cited by 3
Estimating textual treatment effect via causal disentangled representation learning
Yang, Z., & Shen, B. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-024-06887-z · Cited by 3
“We employ the autocoder method from [35, 55] to infer the sentiment score for each review.”
Bisiani, S., Gulyás, Á., & Heravi, B. R. (2025) · Computational Humanities Research · doi:10.1017/chr.2025.10012 · Cited by 3
Automating the analysis of public saliency and attitudes toward biodiversity from digital media
Giebink, N., Gupta, A., Veríssimo, D., Chang, C. H., Chang, T., Brennan, A., Dickson, B. G., Bowmer, A., & BAILLIE, J. E. M. (2026) · Conservation Biology · doi:10.1111/cobi.70217 · Cited by 3
“These examples illustrate the versatility of our dataset in facilitating diverse analytical approaches to deepen our understanding of the discourse dynamics related to the target taxa.”
Reference-guided Style-Consistent Content Transfer
Chen, W.-F., Alshomary, M., Stahl, M., Al‐Khatib, K., Stein, B., & Wachsmuth, H. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3avi92gqgdxn · Cited by 3
“For style adherence, we follow Yin et al. (2019) to use a vanilla bart-large-mnli model as the zero-shot topic classifier to predict the probability that the generated text has the desired topic.”
MethodInstance-Based Zero-Shot learning for semi-Automatic MeSH indexing
Karlos, S., Mylonas, N., & Tsoumakas, G. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.08.009 · Cited by 3
Finding Appropriate User Feedback Analysis Techniques for Multiple Data Domains
Devine, P. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion55297.2022.9793732 · Cited by 3
Emotion Detection in Law Enforcement Interviews
Tshimula, J. M., Gray, S., Chikhaoui, B., & Wang, S. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00090 · Cited by 3
An efficient method for Natural Language Querying on Structured Data
Bhathena, H., Joshi, A., & Singh, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.31 · Cited by 3
“Based on the method put forth in (Yin et al., 2019) we formulate our task into one of textual entailment, where spans and the candidates are converted into (premise, hypothesis) pairs using a predefined template, with high entailment score signifying semantic similarity.”
MethodZhen, Z., Wang, L., Gu, D., Wu, H. K., Janfada, B., & Bidgoli, B. M. (2023) · International Journal of Information Technologies and Systems Approach · doi:10.4018/ijitsa.328681 · Cited by 3
PMMC: Prompt-based Multi-Modal Rumor Detection Model with Modality Conversion
Liu, Y., Ren, Y., & Sui, J. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650555 · Cited by 3
“This approach has shown promise in enhancing model performance in specialized domains [12]–[14].”
BackgroundVerma, A., & Nayak, J. K. (2025) · Journal of Promotion Management · doi:10.1080/10496491.2024.2443459 · Cited by 3
“Existing studies illustrate that the ZSC procedure is efficient when used in conjunction with pre-trained language models like BART or Roberta (Yin et al., 2019).”
Janjani, P., Palan, M., Shirude, S., Shegokar, N., & Kazi, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81542-3_14 · Cited by 3
Wehrheim, L., Borst, J., Liebl, B., Burghardt, M., & Spoerer, M. (2025) · Historical Methods A Journal of Quantitative and Interdisciplinary History · doi:10.1080/01615440.2025.2506427 · Cited by 3
“One successful approach for zero-shot-based text classification involves so-called entailment models, which are trained to recognize logical contradictions or implications between two sentences (Yin, Hay, and Roth 2019).”
Nguyen, Q. A., Tomeh, N., Lebbah, M., Charnois, T., Azzag, H., & Muñoz, S. C. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/48ic72qw7xdr · Cited by 3
Unsupervised Ranking and Aggregation of Label Descriptions for Zero-Shot Classifiers
Basile, A., Franco-Salvador, M., & Rosso, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08473-7_11 · Cited by 3
Using Bidirectional Prompt Learning in NLP Few Shot Tasks
Li, D., & Ye, S. (2023) · Frontiers in Computing and Intelligent Systems · doi:10.54097/fcis.v3i1.6362 · Cited by 3
FastClass: A Time-Efficient Approach to Weakly-Supervised Text Classification
Xia, T., Wang, Y., Tian, Y., & Chang, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.313 · Cited by 3
Named Entity Recognition in a Very Homogenous Domain
Agarwal, O., & Nenkova, A. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.138 · Cited by 3
“We performed domain classification experiments on our dataset via unsupervised clustering as well as zero-shot classification 5 (Yin et al., 2019), using both the known domains from the metadata and dummy domains as candidates.”
MethodKEViN: A Knowledge Enhanced Validity and Novelty Classifier for Arguments
Saadat-Yazdi, A., Li, X., Chausson, S., Belle, V., Ross, B., Pan, J. Z., & Kokciyan, N. (2022) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 3
Near-Zero-Shot Suggestion Mining with a Little Help from WordNet
Alekseev, A., Tutubalina, E. V., Kwon, S., & Nikolenko, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16500-9_3 · Cited by 3
“We extend the methodology recently proposed by [35] to this new task.”
MethodResultInfluentialVScript: Controllable Script Generation with Visual Presentation
Ji, Z., Xu, Y., Cheng, I.-T., Cahyawijaya, S., Frieske, R., Ishii, E., Zeng, M., Madotto, A., & Fung, P. (2022) · doi:10.18653/v1/2022.aacl-demo.1 · Cited by 3
Metadata
- Authors
- Yin, W., Hay, J., & Roth, D.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3912-3921
- OpenAlex
- W2970200208
- Cited by
- 537
- References
- 30
- Citations by year
- 2026: 66 · 2025: 86 · 2024: 75 · 2023: 141 · 2022: 94 · 2021: 64 · 2020: 11
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency (HR0011-15-C-0113)
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- zero-shot text classification, benchmarking, evaluation metrics, logical consequence, cross-domain classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

