Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning
Open accessAbstract
Few-shot learning aims to learn latent patterns from few training examples and has shown promises in practice. However, directly calculating the distances between the query image and support image in existing methods may cause ambiguity because dominant objects can locate anywhere on images. To address this issue, this paper proposes a Semantic Alignment Metric Learning (SAML) method for few-shot learning that aligns the semantically relevant dominant objects through a "collect-and-select'' strategy. Specifically, we first calculate a relation matrix (RM) to "`collect" the distances of each local region pairs of the 3D tensor extracted from a query image and the mean tensor of the support images. Then, the attention technique is adapted to "select" the semantically relevant pairs and put more weights on them. Afterwards, a multi-layer perceptron (MLP) is utilized to map the reweighted RMs to their corresponding similarity scores. Theoretical analysis demonstrates the generalization ability of SAML and gives a theoretical guarantee. Empirical results demonstrate that semantic alignment is achieved. Extensive experiments on benchmark datasets validate the strengths of the proposed approach and demonstrate that SAML significantly outperforms the current state-of-the-art methods. The source code is available at https://github.com/haofusheng/SAML.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The alignment task can be divided into attentional alignment and semantic alignment.
“Generally, the alignment task can be divided into two subtasks: the attentional alignment task [170, 171] and the semantic alignment task [172, 173].”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [173]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512 · see citing works
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244 · see citing works
Prototypical Networks for Few-shot LearningPreprint
Snell, J., Swersky, K., & Zemel, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05175 · Cited by 5,863 · see citing works
Matching Networks for One Shot LearningPreprint
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04080 · Cited by 4,874 · see citing works
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,875 · see citing works
Siamese Neural Networks for One-shot Image Recognition
Koch, G. E., Zemel, R. S., & Salakhutdinov, R. (2015) · Cited by 3,712 · see citing works
One-shot learning of object categories
Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029 · see citing works
Optimization as a Model for Few-Shot Learning
Ravi, S., & Larochelle, H. (2017) · International Conference on Learning Representations · Cited by 2,766 · see citing works
Zagoruyko, S., & Komodakis, N. (2016) · arXiv · doi:10.48550/arxiv.1612.03928 · Cited by 2,027 · see citing works
Salient Object Detection: A Benchmark
Borji, A., Cheng, M.-M., Jiang, H., & Li, J. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487833 · Cited by 1,550 · see citing works
On First-Order Meta-Learning Algorithms
Nichol, A., Achiam, J., & Schulman, J. (2018) · arXiv · doi:10.48550/arxiv.1803.02999 · Cited by 1,438 · see citing works
Additive Margin Softmax for Face Verification
Wang, F., Cheng, J., Liu, W., & Liu, H. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2822810 · Cited by 1,326 · see citing works
Self-citationDynamic Few-Shot Visual Learning Without Forgetting
Gidaris, S., & Komodakis, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00459 · Cited by 1,128 · see citing works
Foundations of Machine Learning
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018) · doi:10.7551/mitpress/10290.001.0001 · Cited by 1,122 · see citing works
Foundations of Machine Learning
(2012) · The MIT Press eBooks · doi:10.7551/mitpress/8291.003.0006 · Cited by 1,102 · see citing works
Meta-SGD: Learning to Learn Quickly for Few-Shot LearningPreprint
Li, Z., Zhou, F., Chen, F., & Li, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09835 · Cited by 1,026 · see citing works
A Simple Neural Attentive Meta-LearnerPreprint
Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03141 · Cited by 961 · see citing works
Few-Shot Learning with Graph Neural NetworksPreprint
García, V., & Bruna, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04043 · Cited by 863 · see citing works
Low-Shot Visual Recognition by Shrinking and Hallucinating Features
Hariharan, B., & Girshick, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.328 · Cited by 824 · see citing works
Low-Shot Learning from Imaginary Data
Wang, Y.-X., Girshick, R., Hebert, M., & Hariharan, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00760 · Cited by 770 · see citing works
Spectrally-normalized margin bounds for neural networksPreprint
Bartlett, P. L., Foster, D. J., & Telgarsky, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.08498 · Cited by 609 · see citing works
Few-Shot Image Recognition by Predicting Parameters from Activations
Qiao, S., Liu, C., Shen, W., & Yuille, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00755 · Cited by 565 · see citing works
Memory Matching Networks for One-Shot Image Recognition
Cai, Q., Pan, Y., Yao, T., Yan, C. C., & Mei, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00429 · Cited by 318 · see citing works
MetaGAN: an adversarial approach to few-shot learning
Zhang, R., Che, T., Zoubin Ghahramani, A., Bengio, Y., & Song, Y. (2018) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 311 · see citing works
Improved Bilinear Pooling with CNNs
Lin, T., & Maji, S. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.117 · Cited by 220 · see citing works
Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation
Hu, V. T., Yang, P., Zhang, C., Yu, G., Mu, Y., & Snoek, C. G. M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018441 · Cited by 138 · see citing works
Incremental Few-Shot Learning with Attention Attractor NetworksPreprint
Ren, M., Liao, R., Fetaya, E., & Zemel, R. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.07218 · Cited by 124 · see citing works
Rapid Adaptation with Conditionally Shifted Neurons.
Munkhdalai, T., Yuan, X., Mehri, S., et al. (2018) · International Conference on Machine Learning · Cited by 121 · see citing works
Rapid Adaptation with Conditionally Shifted NeuronsPreprint
Munkhdalai, T., Yuan, X., Mehri, S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09926 · Cited by 106 · see citing works
Generative Adversarial Residual Pairwise Networks for One Shot LearningPreprint
Mehrotra, A., & Dukkipati, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08033 · Cited by 96 · see citing works
Discriminative k-shot learning using probabilistic modelsPreprint
Bauer, M., Rojas-Carulla, M., Świątkowski, J., Schölkopf, B., & Turner, R. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00326 · Cited by 48 · see citing works
A Dual Attention Network with Semantic Embedding for Few-Shot Learning
Yan, S., Zhang, S., & He, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019079 · Cited by 20 · see citing works
Deep Comparison: Relation Columns for Few-Shot Learning.Preprint
Zhang, X., Sung, F., Qiang, Y., et al. (2018) · arXiv (Cornell University) · Cited by 16 · see citing works
Learning Rapid-Temporal AdaptationsPreprint
Munkhdalai, T., Yuan, X., Mehri, S., Wang, T., & Trischler, A. (2017) · arXiv (Cornell University) · Cited by 10 · see citing works
Anchor-based Nearest Class Mean Loss for Convolutional Neural NetworksPreprint
Hao, F., Cheng, J. S., Wang, L., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.08087 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities
Song, Y., Wang, T., Cai, P., Mondal, S. K., & Sahoo, J. P. (2023) · ACM Computing Surveys · doi:10.1145/3582688 · Cited by 721 · see citing works
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
- Authors
- Hao, F., He, F., Cheng, J., Wang, L., Cao, J., & Tao, D.
- Affiliations
- Chinese University of Hong Kong (HK)
- The University of Sydney (AU)
- Xi'an Institute of Optics and Precision Mechanics (CN)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 8459-8468
- OpenAlex
- W2991559096
- Cited by
- 167
- References
- 40
- Citations by year
- 2026: 8 · 2025: 23 · 2024: 30 · 2023: 27 · 2022: 34 · 2021: 38 · 2020: 7
- FWCI
- 7.39
- Type
- conference paper
- Language
- en
- Open access
- green
- ir.opt.ac.cn
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Institutional Repository of Xi'an Institute of Optics and Fine Mechanics, Chinese Academy of Sciences (Xian Institute of Optics and Precision Mechanics) (submittedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- few-shot learning, metric learning, semantic alignment, relation matrix, attention mechanism, similarity scores
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
