Multi-attention Network for One Shot Learning
Abstract
One-shot learning is a challenging problem where the aim is to recognize a class identified by a single training image. Given the practical importance of one-shot learning, it seems surprising that the rich information present in the class tag itself has largely been ignored. Most existing approaches restrict the use of the class tag to finding similar classes and transferring classifiers or metrics learned thereon. We demonstrate here, in contrast, that the class tag can inform one-shot learning as a guide to visual attention on the training image for creating the image representation. This is motivated by the fact that human beings can better interpret a training image if the class tag of the image is understood. Specifically, we design a neural network architecture which takes the semantic embedding of the class tag to generate attention maps and uses those attention maps to create the image features for one-shot learning. Note that unlike other applications, our task requires that the learned attention generator can be generalized to novel classes. We show that this can be realized by representing class tags with distributed word embeddings and learning the attention map generator from an auxiliary training set. Also, we design a multiple-attention scheme to extract richer information from the exemplar image and this leads to substantial performance improvement. Through comprehensive experiments, we show that the proposed approach leads to superior performance over the baseline methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Quick adaptation studies are under intensive research.
“Such quick adaptation studies are also under intensive research [ 60 , 61 ].”
From Artificial intelligence in oil and gas upstream: Trends, challenges, and scenarios for the future (2021) · cites this paper as [61]
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Self-citationGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980 · see citing works
Siamese Neural Networks for One-shot Image Recognition
Koch, G. E., Zemel, R. S., & Salakhutdinov, R. (2015) · Cited by 3,712 · see citing works
One-shot learning of object categories
Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029 · see citing works
Stacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.10 · Cited by 2,036 · see citing works
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849 · see citing works
Attention to Scale: Scale-Aware Semantic Image Segmentation
Chen, L.-C., Yang, Y., Wang, J., Xu, W., & Yuille, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.396 · Cited by 1,462 · see citing works
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298754 · Cited by 1,305 · see citing works
An Embarrassingly Simple Approach to Zero-Shot Learning
Romera‐Paredes, B., & Torr, P. H. S. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-50077-5_2 · Cited by 1,113 · see citing works
Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., & Zhang, Z. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298685 · Cited by 807 · see citing works
DRAW: A Recurrent Neural Network For Image Generation
Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015) · Cited by 785 · see citing works
Mind's eye: A recurrent visual representation for image caption generation
Chen, X., & Zitnick, C. L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298856 · Cited by 496 · see citing works
Bayesian Face Revisited: A Joint Formulation
Chen, D., Cao, X., Wang, L., Wen, F., & Sun, J. Z. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33712-3_41 · Cited by 437 · see citing works
One-shot Learning with Memory-Augmented Neural NetworksPreprint
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06065 · Cited by 435 · see citing works
Cao, C., Liu, X., Yan, S., Yu, Y., Wang, J., Wang, Z., Huang, Y., Wang, L., Huang, C., Xu, W., et al. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.338 · Cited by 409 · see citing works
Recurrent Attentional Networks for Saliency Detection
Kuen, J., Wang, Z., & Wang, G. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.399 · Cited by 235 · see citing works
Visual Tracking Using Attention-Modulated Disintegration and Integration
Choi, J., Chang, H. J., Jeong, J., Demiris, Y., & Choi, J. Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.468 · Cited by 191 · see citing works
Recurrent Attention Models for Depth-Based Person Identification
Haque, A., Alahi, A., & Fei-Fei, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.138 · Cited by 172 · see citing works
Temporal Pyramid Pooling-Based Convolutional Neural Network for Action Recognition
Wang, P., Cao, Y., Shen, C., Liu, L., & Shen, H. T. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2576761 · Cited by 151 · see citing works
Self-citationOne-Shot Learning with a Hierarchical Nonparametric Bayesian Model
Salakhutdinov, R., Tenenbaum, J. B., & Torralba, A. B. (2011) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 100 · see citing works
Compositional Model Based Fisher Vector Coding for Image Classification
Liu, L., Wang, P., Shen, C., Wang, L., van den Hengel, A., Wang, C., & Shen, H. T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2651061 · Cited by 62 · see citing works
Self-citationZero-Shot Object Recognition Using Semantic Label Vectors
Naha, S., & Wang, Y. (2015) · Conference on Computer and Robot Vision (CRV) · doi:10.1109/crv.2015.21 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence in oil and gas upstream: Trends, challenges, and scenarios for the future
Koroteev, D., & Tekic, Z. (2021) · Energy and AI · doi:10.1016/j.egyai.2020.100041 · Cited by 285 · see citing works
- Authors
- Wang, P., Liu, L., Shen, C., Huang, Z., Van Den Hengel, A., & Shen, H. T.
- Affiliations
- The University of Adelaide (AU)
- The University of Queensland (AU)
- University of Electronic Science and Technology of China (CN)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 6212-6220
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2741727820
- Cited by
- 91
- References
- 26
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 4 · 2023: 3 · 2022: 15 · 2021: 17 · 2020: 15 · 2019: 22 · 2018: 11 · 2017: 2
- FWCI
- 6.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- UTS ePRESS (University of Technology Sydney) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- one-shot learning, multi-attention network, visual attention, image representation learning, novel class generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
