Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly
Open accessAbstract
Due to the importance of zero-shot learning, i.e., classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily. We argue that it is time to take a step back and to analyze the status quo of the area. The purpose of this paper is three-fold. First, given the fact that there is no agreed upon zero-shot learning benchmark, we first define a new benchmark by unifying both the evaluation protocols and data splits of publicly available datasets used for this task. This is an important contribution as published results are often not comparable and sometimes even flawed due to, e.g., pre-training on zero-shot test classes. Moreover, we propose a new zero-shot learning dataset, the Animals with Attributes 2 (AWA2) dataset which we make publicly available both in terms of image features and the images themselves. Second, we compare and analyze a significant number of the state-of-the-art methods in depth, both in the classic zero-shot setting but also in the more realistic generalized zero-shot setting. Finally, we discuss in detail the limitations of the current status of the area which can be taken as a basis for advancing it.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Transformer based cross-modal alignment has been studied for various tasks, e.g., speaker localization, speech translation, text-to-speech alignment, text-to-video retrieval, and visual grounding of natural language.
“Transformer based cross-modal alignment has been studied for various tasks, e.g., speaker localization in multi-speaker videos [250], speech translation [180], text-to-speech alignment [251], text-to-video retrieval [252], [253], [254], and visual grounding of natural language [255], [256], [257], [258], [259].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [251]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 3× · see citing works
“embeddings including attributes, word2vec [27], glove [56] and wordnet hierarchy [57].”
BackgroundChapelle, O., Schölkopf, B., & Zien, A. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262033589.001.0001 · Cited by 4,560 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
Optimizing search engines using clickthrough data
Joachims, T. (2002) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/775047.775067 · Cited by 3,921 · see citing works
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,856 · see citing works
(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534 · see citing works
DeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,309 · see citing works
Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4]
Bartels, R. H., & Stewart, G. W. (1972) · Communications of the ACM · doi:10.1145/361573.361582 · Cited by 1,957 · see citing works
Describing objects by their attributes
Farhadi, A., Endres, I., Hoiem, D., & Forsyth, D. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206772 · Cited by 1,917 · see citing works
Large Margin Methods for Structured and Interdependent Output Variables
Tsochantaridis, I., Joachims, T., Hofmann, T., & Altün, Y. (2005) · MPG.PuRe (Max Planck Society) · Cited by 1,879 · see citing works
Towards Open Set Deep Networks
Bendale, A., & Boult, T. E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.173 · Cited by 1,799 · see citing works
Attribute-Based Classification for Zero-Shot Visual Object Categorization
Lampert, C. H., Nickisch, H., & Stefan Harmeling, B. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.140 · Cited by 1,639 · see citing works
Self-citationScheirer, W. J., de Rezende Rocha, A., Sapkota, A., & Boult, T. E. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.256 · Cited by 1,573 · see citing works
An Embarrassingly Simple Approach to Zero-Shot Learning
Romera‐Paredes, B., & Torr, P. H. S. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-50077-5_2 · Cited by 1,113 · see citing works
Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]
Chapelle, O., Schölkopf, B., & Zien, E. A. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2015974 · Cited by 1,102 · see citing works
Evaluation of output embeddings for fine-grained image classification
Akata, Z., Reed, S., Walter, D., Lee, H., & Schiele, B. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298911 · Cited by 1,011 · see citing works
Self-citationZero-Shot Learning Through Cross-Modal Transfer
Socher, R., Ganjoo, M., Sridhar, H., Bastani, O., Manning, C. D., & Ng, A. Y. (2013) · arXiv · doi:10.48550/arxiv.1301.3666 · Cited by 997 · see citing works
Semantic Autoencoder for Zero-Shot Learning
Kodirov, E., Xiang, T., & Gong, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.473 · Cited by 960 · see citing works
Zero-Shot Learning by Convex Combination of Semantic EmbeddingsPreprint
Norouzi, M., Mikolov, T., Samy Bengio, A., Singer, Y., Shlens, J., Frome, A., Corrado, G. S., & Dean, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.5650 · Cited by 890 · see citing works
SUN attribute database: Discovering, annotating, and recognizing scene attributes
Patterson, G., & Hays, J. H. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247998 · Cited by 878 · see citing works
Zero-Shot Learning with Semantic Output Codes
Palatucci, M., Pomerleau, D., Hinton, G. E., et al. (2018) · Figshare · doi:10.1184/r1/6476456.v1 · Cited by 816 · see citing works
Learning Deep Representations of Fine-Grained Visual Descriptions
Reed, S., Akata, Z., Lee, H., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.13 · Cited by 802 · see citing works
Self-citationLabel-Embedding for Image Classification
Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2487986 · Cited by 785 · see citing works
Self-citationLearning a Deep Embedding Model for Zero-Shot Learning
Zhang, L., Xiang, T., & Gong, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.321 · Cited by 764 · see citing works
Latent Embeddings for Zero-Shot Classification
Xian, Y., Akata, Z., Sharma, G., Nguyen, Q., Hein, M., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.15 · Cited by 723 · see citing works
Self-citationSynthesized Classifiers for Zero-Shot Learning
Changpinyo, S., Chao, W., Gong, B., & Sha, F. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.575 · Cited by 720 · see citing works
Wsabie: Scaling up to large vocabulary image annotation
Weston, J., Samy Bengio, A., & Usunier, N. (2011) · Cited by 659 · see citing works
Label-Embedding for Attribute-Based Classification
Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.111 · Cited by 639 · see citing works
Self-citationZero-Shot Learning via Semantic Similarity Embedding
Zhang, Z., & Saligrama, V. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.474 · Cited by 629 · see citing works
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
Chao, W.-L., Changpinyo, S., Gong, B., & Sha, F. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_4 · Cited by 616 · see citing works
Evaluation of Output Embeddings for Fine-grained Image Classification
Akata, Z., Reed, S., Walter, D., Lee, H., & Schiele, B. (2015) · Max Planck Digital Library · Cited by 567 · see citing works
Self-citationTransductive Multi-View Zero-Shot Learning
Fu, Y., Hospedales, T. M., Xiang, T., & Gong, S. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2408354 · Cited by 514 · see citing works
Unsupervised Domain Adaptation for Zero-Shot Learning
Kodirov, E., Xiang, T., Fu, Z., & Gong, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.282 · Cited by 415 · see citing works
Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions
Ba, J. L., Swersky, K., Fidler, S., & Salakhutdinov, R. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.483 · Cited by 401 · see citing works
Multi-class Open Set Recognition Using Probability of Inclusion
Jain, L. P., Scheirer, W. J., & Boult, T. E. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_26 · Cited by 362 · see citing works
Zero-data learning of new tasks
Larochelle, H., Erhan, D., & Bengio, Y. (2008) · Cited by 345 · see citing works
Evaluating knowledge transfer and zero-shot learning in a large-scale setting
Rohrbach, M., Stark, M. M., & Schiele, B. (2011) · doi:10.1109/cvpr.2011.5995627 · Cited by 309 · see citing works
Self-citationMetric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost
Mensink, T., Verbeek, J., Perronnin, F., & Csurka, G. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_35 · Cited by 281 · see citing works
What helps where – and why? Semantic relatedness for knowledge transfer
Rohrbach, M., Stark, M. M., Szarvas, G., Gurevych, I., & Schiele, B. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540121 · Cited by 273 · see citing works
Self-citationWrite a Classifier: Zero-Shot Learning Using Purely Textual Descriptions
Elhoseiny, M., Saleh, B., & Elgammal, A. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.321 · Cited by 263 · see citing works
Zero Shot Recognition with Unreliable AttributesPreprint
Jayaraman, D., & Grauman, K. L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.4327 · Cited by 240 · see citing works
Zero-shot object recognition by semantic manifold distance
Fu, Z., Xiang, T., Kodirov, E., & Gong, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298879 · Cited by 217 · see citing works
A Simple Exponential Family Framework for Zero-Shot Learning
Verma, V. K., & Rai, P. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-71246-8_48 · Cited by 216 · see citing works
Transfer Learning in a Transductive Setting
Rohrbach, M., Ebert, S., & Schiele, B. (2013) · MPG.PuRe (Max Planck Society) · Cited by 212 · see citing works
Self-citationCOSTA: Co-Occurrence Statistics for Zero-Shot Classification
Mensink, T., Gavves, E., & Snoek, C. G. M. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.313 · Cited by 194 · see citing works
Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classiffication
Bucher, M., Herbin, S., & Jurie, F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_44 · Cited by 189 · see citing works
Less is More: Zero-Shot Learning from Online Textual Documents with Noise Suppression
Qiao, R., Liu, L., Shen, C., et al. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.247 · Cited by 187 · see citing works
Predicting Visual Exemplars of Unseen Classes for Zero-Shot Learning
Changpinyo, S., Chao, W., & Sha, F. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.376 · Cited by 174 · see citing works
From Zero-Shot Learning to Conventional Supervised Classification: Unseen Visual Data Synthesis
Long, Y., Liu, L., Shao, L., Shen, F., Ding, G., & Han, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.653 · Cited by 159 · see citing works
Ranking with ordered weighted pairwise classification
Usunier, N., Buffoni, D., & Gallinari, P. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553509 · Cited by 147 · see citing works
Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example
Yu, X., & Aloimonos, Y. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_10 · Cited by 145 · see citing works
Matrix Tri-Factorization with Manifold Regularizations for Zero-Shot Learning
Xu, X., Shen, F., Yang, Y., et al. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.217 · Cited by 136 · see citing works
Low-Rank Embedded Ensemble Semantic Dictionary for Zero-Shot Learning
Ding, Z., Shao, M., & Fu, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.636 · Cited by 135 · see citing works
Zero-Shot Classification with Discriminative Semantic Representation Learning
Ye, M., & Guo, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.542 · Cited by 129 · see citing works
Semi-supervised Vocabulary-Informed Learning
Fu, Y., & Sigal, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.576 · Cited by 120 · see citing works
Predicting Deep Zero-Shot Convolutional Neural Networks using Textual DescriptionsPreprint
Ba, J. L., Swersky, K., Fidler, S., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00511 · Cited by 117 · see citing works
Multi-cue Zero-Shot Learning with Strong Supervision
Akata, Z., Malinowski, M., Fritz, M., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.14 · Cited by 109 · see citing works
Self-citationGaze Embeddings for Zero-Shot Image Classification
Karessli, N., Akata, Z., Schiele, B., & Bulling, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.679 · Cited by 105 · see citing works
Self-citationOnline incremental attribute-based zero-shot learning
Kankuekul, P., Kawewong, A., Tangruamsub, S., & Hasegawa, O. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248112 · Cited by 101 · see citing works
Al-Halah, Z., Tapaswi, M., & Stiefelhagen, R. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.643 · Cited by 101 · see citing works
Zero-Shot Learning via Visual Abstraction
Antol, S., Zitnick, C. L., & Parikh, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_27 · Cited by 95 · see citing works
Semi-Supervised Zero-Shot Classification with Label Representation Learning
Li, X., Guo, Y., & Schuurmans, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.479 · Cited by 92 · see citing works
Zero-Shot Recognition via Structured Prediction
Zhang, Z., & Saligrama, V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_33 · Cited by 89 · see citing works
Online Collaborative Learning for Open-Vocabulary Visual Classifiers
Zhang, H., Shang, X., Yang, W., et al. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.307 · Cited by 48 · see citing works
Fujiwara, Y., & Irie, G. (2014) · Cited by 43 · see citing works
Gaussian Visual-Linguistic Embedding for Zero-Shot Recognition
Mukherjee, T., & Hospedales, T. M. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1089 · Cited by 34 · see citing works
Zero-Shot Learning with Generative Latent Prototype ModelPreprint
Li, Y., & Wang, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09474 · Cited by 17 · see citing works
Max-margin zero-shot learning for multi-class classification
Li, X., & Guo, Y. (2015) · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- [28][28] Y. Fu, T. M. Hospedales, T. Xiang, and S. Gong, 'Transductive multiview zero-shot learning,' TPAMI , 2015. 2, 3 -
- [37][37] Z. Zhang and V. Saligrama, 'Zero-shot learning via joint semantic similarity embedding,' in CVPR , 2016. 2 -
- [41][41] V. K. Verm and P. Rai, 'A simple exponential family framework for zeroshot learning,' in ECML , 2017, pp. 792-808. 2, 5, 7, 8, 9, 10, 11, 12, 13 -
- [59][59] W. J. Scheirer, A. Rocha, A. Sapkota, and T. E. Boult, 'Towards open set recognition,' TPAMI , vol. 36, 2013. 3 -
- [68][68] R. H. Bartels and G. Stewart, 'Solution of the matrix equation ax+ xb= c [f4],' Commun. ACM , vol. 15, no. 9, pp. 820-826, 1972. 4 -
- [75][75] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, 'Going deeper with convolutions,' CVPR , 2015. 8 -
- [76][76] S. Garcia and F. Herrera, 'An extension on'statistical comparisons of classifiers over multiple data sets'for all pairwise comparisons,' JLMR , vol. 9, pp. 2677-2694, 2008. 9 <!-- image --> Yongqin Xian received his B.Sc. degree in software engineering from Beijing Institute of Technology, China, in 2013 and his M.Sc. degree with honors in computer science from Saarland University, Germany in 2016. He is currently a PhD student in the Computer Vision and Multimodal Computing group at the Max-Planck Institute for Informatics in Saarbr¨ ucken, Germany. His research interests include machine learning and computer vision. 0 <!-- image --> Christoph Lampert received the PhD degree in mathematics from the University of Bonn in 2003. In 2010 he joined the Institute of Science and Technology Austria (IST Austria) first as an Assistant Professor and since 2015 as a Professor. His research on computer vision and machine learning has won several international and national awards, including the best paper prize at CVPR 2008. In 2012 he was awarded an ERC Starting Grant by the European Research Council. He is an Editor of the International Jour- nal of Computer Vision (IJCV), Action Editor of the Journal for Machine Learning Research (JMLR), and Associate Editor in Chief of the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). <!-- image --> Bernt Schiele received masters degree in computer science from the University of Karlsruhe and INP Grenoble in 1994 and the PhD degree from INP Grenoble in computer vision in 1997. He was a postdoctoral associate and visiting assistant professor with MIT between 1997 and 2000. From 1999 until 2004, he was an assistant professor with ETH Zurich and, from 2004 to 2010, he was a full professor of computer science with TU Darmstadt. In 2010, he was appointed a scientific member of the Max Planck Society and director at the Max Planck Institute for Informatics. Since 2010, he has also been a professor at Saarland University. His main interests are computer vision, perceptual computing, statistical learning methods, wearable computers, and integration of multimodal sensor data. He is particularly interested in developing methods which work under real world conditions. <!-- image --> Zeynep Akata holds a MSc degree from RWTH Aachen in 2010 and a PhD degree from INRIA of Universit´ e de Grenoble in 2014. She was a postdoctoral researcher in Max Planck Institute for Informatics between 2014-2017 and a visiting researcher at UC Berkeley in 2016-2017. In 2014, she received Lise-Meitner Award for Excellent Women in Computer Science from Max Planck Society. She is currently an assistant professor at the University of Amsterdam, Scientific Manager of the Delta Lab and a Senior Researcher at Max Planck Institute for Informatics. Her research interests include machine learning combined with vision and language for the task of explainable artificial intelligence (XAI).
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders
Schönfeld, E., Ebrahimi, S., Sinha, S., Darrell, T., & Akata, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00844 · Cited by 747 · see citing works
Self-citationDeep learning tools for the measurement of animal behavior in neuroscience
Mathis, M. W., & Mathis, A. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.10.008 · Cited by 450 · see citing works
Tiu, E., Talius, E., Patel, P., Langlotz, C. P., Ng, A. Y., & Rajpurkar, P. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00936-9 · Cited by 442 · see citing works
Large Language Models for Cyber Security: A Systematic Literature Review
Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3769676 · Cited by 189 · see citing works
Review on self-supervised image recognition using deep neural networks
Ohri, K., & Kumar, M. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107090 · Cited by 179 · see citing works
Attribute Attention for Semantic Disambiguation in Zero-Shot Learning
Liu, Y., Guo, J., Cai, D., & He, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00680 · Cited by 175 · see citing works
A Survey of IoT Privacy Security: Architecture, Technology, Challenges, and Trends
Sun, P., Shen, S., Wan, Y., Wu, Z., Fang, Z., & Gao, X.-Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3372518 · Cited by 172 · see citing works
Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective
Li, K., Min, M. R., & Fu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00368 · Cited by 144 · see citing works
Machine learning for perturbational single-cell omics
Ji, Y., Lotfollahi, M., Wolf, F. A., & Theis, F. J. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.016 · Cited by 137 · see citing works
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
Triguero, I., Molina, D., Poyatos, J., Del Ser, J., & Herrera, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102135 · Cited by 90 · see citing works
Joint feature synthesis and embedding: Adversarial cross-modal retrieval revisited
Xu, X., Lin, K., Yang, Y., Hanjalic, A., & Shen, H. T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045530 · Cited by 88 · see citing works
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71 · see citing works
Zero-Shot Clinical Trial Patient Matching with LLMs
Wornow, M., Lozano, A., Dash, D., Jindal, J., Mahaffey, K. W., & Shah, N. H. (2025) · NEJM AI · doi:10.1056/aics2400360 · Cited by 59 · see citing works
Parida, K. K., Matiyali, N., Guha, T., & Sharma, G. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093438 · Cited by 49 · see citing works
Linearly Mapping from Image to Text Space
Merullo, J., Castricato, L., Eickhoff, C., & Pavlick, E. (2022) · arXiv · doi:10.48550/arxiv.2209.15162 · Cited by 34 · see citing works
A Deep Multi-Modal Explanation Model for Zero-Shot Learning
Liu, Y., & Tuytelaars, T. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975980 · Cited by 31 · see citing works
Don't Make Your LLM an Evaluation Benchmark Cheater
Zhou, K., Zhu, Y., Chen, Z., Chen, W., Zhao, W. X., Chen, X., Lin, Y., Wen, J.-R., & Han, J. (2023) · arXiv · doi:10.48550/arxiv.2311.01964 · Cited by 25 · see citing works
Research progress of zero-shot learning beyond computer vision
Cao, W., Zhou, C., Wu, Y., Ming, Z., Xu, Z., & Zhang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60239-0_36 · Cited by 21 · see citing works
The University of North Carolina at Charlotte, Zhou, L., Rudin, C., Duke University, Gombolay, M., Georgia Institute of Technology, Spohrer, J., International Society of Service Innovation Professionals, Zhou, M., Juji, et al. (2023) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00085 · Cited by 20 · see citing works
GAN-MVAE: a discriminative latent feature generation framework for generalized zero-shot learning
Ma, P., Lu, H., Yang, B., & Ran, W. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.02.002 · Cited by 18 · see citing works
Towards Recognizing Unseen Categories in Unseen Domains
Mancini, M., Akata, Z., Ricci, E., & Caputo, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58592-1_28 · Cited by 9 · see citing works
Self-citationLanguage-Models-as-a-Service: Overview of a New Paradigm and its Challenges
La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Nazar, R., Cohn, A., Shadbolt, N., & Wooldridge, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15865 · Cited by 9 · see citing works
Losses over Labels: Weakly Supervised Learning via Direct Loss Construction
Sam, D., & Kolter, J. Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26159 · Cited by 6 · see citing works
Semi-supervised multimodal representation learning through a global workspace
Devillers, B., Maytié, L., & VanRullen, R. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3416701 · Cited by 5 · see citing works
Robust agents learn causal world models
Richens, J., & Everitt, T. (2024) · arXiv · doi:10.48550/arxiv.2402.10877 · Cited by 5 · see citing works
“…shifts ), as in domain adaptation and domain generalisation (Farahani et al., 2021; Wilson & Cook, 2020), or changes to the objective ( task shifts) as in zero shot learning (Xian et al., 2018), in-context learning (Brown et al., 2020) and multi-task reinforcement learning (Reed et al., 2022).”
Background
- Authors
- Xian, Y., Lampert, C. H., Schiele, B., & Akata, Z.
- Affiliations
- Max Planck Institute for Informatics (DE)
- Institute of Science and Technology Austria (AT)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-09-01
- Volume
- 41
- Issue
- 9
- Pages
- 2251-2265
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2963499153
- Cited by
- 1,678
- References
- 75
- Citations by year
- 2026: 147 · 2025: 221 · 2024: 231 · 2023: 229 · 2022: 219 · 2021: 266 · 2020: 205 · 2019: 117 · 2018: 39 · 2017: 3 · 2012: 1
- FWCI
- 88.24
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- zero-shot learning, benchmark definition, generalized zero-shot learning, image classification, limited labeled training data, method comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
