The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions
Abstract
One of the main challenges in Zero-Shot Learning of visual categories is gathering semantic attributes to accompany images. Recent work has shown that learning from textual descriptions, such as Wikipedia articles, avoids the problem of having to explicitly define these attributes. We present a new model that can classify unseen categories from their textual description. Specifically, we use text features to predict the output weights of both the convolutional and the fully connected layers in a deep convolutional neural network (CNN). We take advantage of the architecture of CNNs and learn features at different layers, rather than just learning an embedding space for both modalities, as is common with existing approaches. The proposed model also allows us to automatically generate a list of pseudo-attributes for each visual category consisting of words from Wikipedia articles. We train our models end-to-end using the Caltech-UCSD bird and flower datasets and evaluate both ROC and Precision-Recall curves. Our empirical results show that the proposed model significantly outperforms previous methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Self-citationImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“The recent success of the deep learning approaches to object recognition is supported by the collection of large datasets with millions of images and thousands of labels [3, 32].”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“In [9], label features are produced by an unsupervised skip-gram model [18] trained on Wikipedia articles, while the image features are produced by a CNN trained on Imagenet [14].”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Self-citationThe relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 14×
“It is common to boost the performance of a vision system by using the features from the fully connected layer of a CNN [4].”
BackgroundInfluential
Cited by
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Learning to Prompt for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01653-1 · Cited by 3,304
“For text features extraction, early work has mainly utilized pre-trained word vectors (Socher et al., 2013; Frome et al., 2013) or the hand-crafted TF-IDF features (Elhoseiny et al., 2013; Lei Ba et al., 2015).”
MethodConditional Prompt Learning for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01631 · Cited by 1,796
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly
Xian, Y., Lampert, C. H., Schiele, B., & Akata, Z. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2857768 · Cited by 1,678
Semantic Autoencoder for Zero-Shot Learning
Kodirov, E., Xiang, T., & Gong, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.473 · Cited by 921
Arandjelovic, R., & Zisserman, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.73 · Cited by 782
Learning Deep Representations of Fine-Grained Visual Descriptions
Reed, S., Akata, Z., Lee, H., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.13 · Cited by 760
Learning Deep Structure-Preserving Image-Text Embeddings
Wang, L., Li, Y., & Lazebnik, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.541 · Cited by 760
Learning a Deep Embedding Model for Zero-Shot Learning
Zhang, L., Xiang, T., & Gong, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.321 · Cited by 740
Synthesized Classifiers for Zero-Shot Learning
Changpinyo, S., Chao, W.-L., Gong, B., & Sha, F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.575 · Cited by 703
Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs
Wang, X., Ye, Y., & Gupta, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00717 · Cited by 648
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
Chao, W.-L., Changpinyo, S., Gong, B., & Sha, F. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_4 · Cited by 616
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Wang, L., Li, Y., Huang, J., & Lazebnik, S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797921 · Cited by 535
A Survey of Zero-Shot Learning: Settings, Methods, and Applications
Wang, W., Zheng, V. W., Yu, H., & Miao, C. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3293318 · Cited by 528
Multimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511
Where to Look: Focus Regions for Visual Question Answering
Shih, K. J., Singh, S., & Hoiem, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.499 · Cited by 493
A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts
Zhu, Y., Elhoseiny, M., Liu, B., Peng, X., & Elgammal, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00111 · Cited by 399
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2019) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-030-13969-8_18 · Cited by 371
Image Question Answering Using Convolutional Neural Network with Dynamic Parameter Prediction
Noh, H., Seo, P. H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.11 · Cited by 345
Kim, T., & Kim, H. Y. (2019) · PLOS ONE · doi:10.1371/journal.pone.0212320 · Cited by 342
Zero-Shot Learning via Joint Latent Similarity Embedding
Zhang, Z., & Saligrama, V. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.649 · Cited by 336
Leveraging the Invariant Side of Generative Zero-Shot Learning
Li, J., Jing, M., Lu, K., Ding, Z., Zhu, L., & Huang, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00758 · Cited by 328
Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks
Chen, L., Zhang, H., Xiao, J., Liu, W., & Chang, S.-F. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00115 · Cited by 309
Learning to Learn: Model Regression Networks for Easy Small Sample Learning
Wang, Y.-X., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_37 · Cited by 282
Multi-Level Semantic Feature Augmentation for One-Shot Learning
Chen, Z., Fu, Y., Zhang, Y., Jiang, Y.-G., Xue, X., & Sigal, L. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2910052 · Cited by 254
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Achlioptas, P., Abdelreheem, A., Xia, F., Elhoseiny, M., & Guibas, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_25 · Cited by 253
Learning Aligned Cross-Modal Representations from Weakly Aligned Data
Castrejon, L., Aytar, Y., Vondrick, C., Pirsiavash, H., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.321 · Cited by 188
“In a similar fashion, zero-shot learning [25, 31, 9, 2, 40] addresses the problem of learning new classifiers without training examples in a given domain, e.”
Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective
Li, K., Min, M. R., & Fu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00368 · Cited by 144
Munkhdalai, T., Sordoni, A., Wang, T., & Trischler, A. (2019) · arXiv · doi:10.48550/arxiv.1907.09720 · Cited by 31
“HyperNetworks generate context dependent weights as dynamic scaling terms for the weights of an RNN [10] and are closely related to conditional normalization techniques [21, 5, 31].”
MethodA Deep Multi-Modal Explanation Model for Zero-Shot Learning
Liu, Y., & Tuytelaars, T. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975980 · Cited by 31
(2017) · neural information processing systems
Metadata
- Authors
- Ba, J. L., Swersky, K., Fidler, S., & Salakhutdinov, R.
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 4247-4255
- OpenAlex
- W2962714319
- Cited by
- 324
- References
- 35
- FWCI
- 16.67
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.