Zero-Shot Learning — The Good, the Bad and the Ugly
Open accessAbstract
Due to the importance of zero-shot learning, the number of proposed approaches has increased steadily recently. We argue that it is time to take a step back and to analyze the status quo of the area. The purpose of this paper is three-fold. First, given the fact that there is no agreed upon zero-shot learning benchmark, we first define a new benchmark by unifying both the evaluation protocols and data splits. This is an important contribution as published results are often not comparable and sometimes even flawed due to, e.g. pre-training on zero-shot test classes. Second, we compare and analyze a significant number of the state-of-the-art methods in depth, both in the classic zero-shot setting but also in the more realistic generalized zero-shot setting. Finally, we discuss limitations of the current status of the area which can be taken as a basis for advancing it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Human-written text tends to have higher perplexity compared to AI-generated text.
“Similarly, human-written text tends to have higher perplexity compared to AI-generated text [85].”
From Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity (2024) · cites this paper as [85]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Feature Generating Networks for Zero-Shot Learning
Xian, Y., Lorenz, T., Schiele, B., & Akata, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00581 · Cited by 1,080 · see citing works
Self-citationA Survey of Zero-Shot Learning: Settings, Methods, and Applications
Wang, W., Zheng, V. W., Yu, H., & Miao, C. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3293318 · Cited by 528 · see citing works
Leveraging Uncertainty in Machine Learning Accelerates Biological Discovery and Design
Hie, B., Bryson, B. D., & Berger, B. (2020) · Cell Systems · doi:10.1016/j.cels.2020.09.007 · Cited by 154 · see citing works
Word meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131 · see citing works
A survey of multimodal deep generative models
Suzuki, M., & Matsuo, Y. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2035253 · Cited by 109 · see citing works
“[76] propose CADA-VAE for generalized zero-shot learning [70, 114], which is used to embed different modalities, namely image features and class attributes, into a shared representation.”
Joint feature synthesis and embedding: Adversarial cross-modal retrieval revisited
Xu, X., Lin, K., Yang, Y., Hanjalic, A., & Shen, H. T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045530 · Cited by 88 · see citing works
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Llorens Salvador, M. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-024-09576-x · Cited by 70 · see citing works
Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation
Al-Halah, Z., Ramakrishnan, S. K., & Grauman, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01652 · Cited by 58 · see citing works
Foundations of Multimodal Co-learning
Zadeh, A., Liang, P. P., & Morency, L.-P. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.001 · Cited by 40 · see citing works
Sfaira accelerates data and model reuse in single cell genomics
Fischer, D. S., Dony, L., König, M., Moeed, A., Zappia, L., Heumos, L., Tritschler, S., Holmberg, O., Aliee, H., & Theis, F. J. (2021) · Genome Biology · doi:10.1186/s13059-021-02452-6 · Cited by 33 · see citing works
Research progress of zero-shot learning beyond computer vision
Cao, W., Zhou, C., Wu, Y., Ming, Z., Xu, Z., & Zhang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60239-0_36 · Cited by 21 · see citing works
Prefix Conditioning Unifies Language and Label Supervision
Saito, K., Sohn, K., Zhang, X., Li, C.-L., Lee, C.-Y., Saenko, K., & Pfister, T. (2022) · arXiv · doi:10.48550/arxiv.2206.01125 · Cited by 17 · see citing works
Towards Recognizing Unseen Categories in Unseen Domains
Mancini, M., Akata, Z., Ricci, E., & Caputo, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58592-1_28 · Cited by 9 · see citing works
Self-citationPudasaini, S., Miralles-Pechuán, L., Lillis, D., & Salvador, M. L. (2024) · arXiv · doi:10.48550/arxiv.2407.13105 · Cited by 8 · see citing works
“However, when AIGC detection is solved from other approaches, such as zero-shot detection methods, a dataset may not be required for training [87].”
Method
- Authors
- Xian, Y., Schiele, B., & Akata, Z.
- Affiliations
- Max Planck Institute for Informatics (DE)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 3077-3086
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2596142952
- Cited by
- 891
- References
- 1
- Citations by year
- 2026: 44 · 2025: 95 · 2024: 95 · 2023: 101 · 2022: 98 · 2021: 123 · 2020: 113 · 2019: 114 · 2018: 91 · 2017: 16 · 2012: 1
- FWCI
- 41.66
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- zero-shot learning, benchmark definition, evaluation protocol, data splits, generalized zero-shot learning, method comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
