The DOI matches its DataCite record; 53 of the 53 references it cites were found in a registry or an index.
Training Vision Transformers for Image Retrieval
Open accessAbstract
Transformers have shown outstanding results for natural language understanding and, more recently, for image classification. We here extend this work and propose a transformer-based approach for image retrieval: we adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective, which combines a contrastive loss with a differential entropy regularizer. Our results show consistent and significant improvements of transformers over convolution-based approaches. In particular, our method outperforms the state of the art on several public benchmarks for category-level retrieval, namely Stanford Online Product, In-Shop and CUB-200. Furthermore, our experiments on ROxford and RParis also show that, in comparable settings, transformers are competitive for particular object retrieval, especially in the regime of short vector representations and low-resolution images.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We have also seen emerged two properties that can be leveraged in future applications: the quality of the features in k -NN classification has a potential for image retrieval where ViT are already showing promising results [22]. The presence of information about the scene layout in the features can also benefit weakly supervised image segmentation.”
From Emerging Properties in Self-Supervised Vision Transformers (2021) · cites this paper as [22]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Cited by
Sorted by citations
Items 101–126 of 126
Breaking the Frame: Visual Place Recognition by Overlap Prediction
Wei, T., Lindenberger, P., Matas, J., & Baráth, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00232 · Cited by 2
Thi, H. N., Nguyen, H. Q., Dao, T. T. Q., & Le, N. T. T. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04205-5 · Cited by 2
Solving Math Word Problems Following Logically Consistent Template
Huang, Z., Zhang, X., Bai, J., Rong, W., Ouyang, Y., & Xiong, Z. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191776 · Cited by 2
Mabnet: Master Assistant Buddy Network With Hybrid Learning for Image Retrieval
Agarwal, R., Das, G., Aggarwal, S., Horsch, A., & Prasad, D. K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094987 · Cited by 1
TK-KNN: A Balanced Distance-Based Pseudo Labeling Approach for Semi-Supervised Intent Classification
Botzer, N., Vázquez, D., Weninger, T., & Laradji, I. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.429 · Cited by 2
An improved similarity matching model for the content-based image retrieval model
Asokaraj, M., Kumar, J. P., & Ashwin, N. (2025) · International Journal on Smart Sensing and Intelligent Systems · doi:10.2478/ijssis-2025-0031 · Cited by 1
TransWeaver: Weave Image Pairs for Class Agnostic Common Object Detection
Guo, X., Li, X., Wang, Y., & Jiang, S. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3275870 · Cited by 1
Images Classification Integrating Transformer with Convolutional Neural Network
Peng, Y. (2023) · Advances in Engineering Technology Research · doi:10.56028/aetr.6.1.621.2023 · Cited by 1
Deep permutation type design for Chinese characters
Chen, D., Terzidis, K., & Fabrocini, F. (2024) · The Design Journal · doi:10.1080/14606925.2024.2377455 · Cited by 1
Commodity Image Retrieval Method Based on CNN and Score Fusion
Liu, Z., Wu, X., Qian, J., & Zhu, Z. (2021) · International Conference on Culture-oriented Science & Technology (ICCST) · doi:10.1109/iccst53801.2021.00101
Havt: Hierarchical Attention Vision Transformer for Fine-Grained Visual ClassificationPreprint
Hu, X., Zhu, S. N., & Peng, T. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4132949
Industrial Part Image Retrieval Combining Convolution and Attention Features
Shao, Q., Qi, Y., Yang, L., & Wu, B. (2022) · International Conference on Computer Engineering and Artificial Intelligence (ICCEAI) · doi:10.1109/icceai55464.2022.00056
Whale Image Retrieval with Fused Mask Information
Shao, Q., Li, Y., & Wu, B. (2022) · International Conference on Artificial Intelligence and Advanced Manufacture (AIAM) · doi:10.1109/aiam57466.2022.00029
LM-Metric: Learned Pair Weighting and Contextual Memory for Deep Metric LearningPreprint
Yan, S., Xu, L., Shu, X., Lu, Z., & Shen, J. (2023) · doi:10.36227/techrxiv.22361128
Data Cleansing to Enhance Quality of Multimedia Data Using Convultional Neural Networks
Muthusi, K. A., Cherruyoit, W. K., & Rimiru, R. M. (2023) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect57570.2023.10117891
Linking Unknown Characters Via Oracle Bone Inscriptions RetrievalPreprint
Chen, X., Li, B., Liu, Y., Han, Y., Jiang, R., & Deng, L. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4576881
Lm-Metric: Learned Pair Weighting and Contextual Memory For Deep Metric LearningPreprint
Yan, S., Xu, L., Shu, X., Lu, Z., & Shen, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4600278
Image Retrieval Based on Tree Structures from Hierarchical Data
Shihong, L., Wang, Z., Gao, L., & Gong, K. (2024) · International Journal of Computer Science and Engineering · doi:10.14445/23488387/ijcse-v11i3p101
Delver Hamed : Content Based Image Retrieval Using BEIT Transformer and DeticPreprint
Jelveh, M. (2024) · SSRN Electronic Journal · doi:10.36227/techrxiv.171502898.86547280/v1
Siamese Networks and Adversarial Attacks: An Overview
Shaw, L., Hebli, P., & Ekin, T. (2024) · IEEE Conference on Prognostics and Health Management · doi:10.1109/icphm61352.2024.10626656 · Cited by 1
A Transformer-Based Architecture for Similar Image Retrieval with Combining Image and Text Features
Sheykhvand, Z., Farzalizadeh, M., & Meghdadı, M. (2024) · doi:10.1109/icee63041.2024.10667888
On the Vulnerability of Retrieval in High Intrinsic Dimensionality Neighborhood
Furon, T. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3553067
Refining Mushroom Image Recognition Using Deep Learning and Image Similarity Distance
Rangkuti, A. H., Kerta, J. M., Juarto, B., Kurniawan, A., Tandianto, A., & Tjandrasubrata, M. G. (2024) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis63449.2024.10956378
Vce: Visual Concept Embedding for Open-Set Fine-Grained Image RetrievalPreprint
Ye, S., Hou, W., chen, S., & Wang, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5231685
Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization
Thi, H. N., Huu, Q. N., Thuy, Q. D. T., Le, N. T. T., Van, T. N., & Huu, H. N. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03271-6
Metadata
- Authors
- El-Nouby, A., Neverova, N., Laptev, I., & Jégou, H.
- Affiliations
- École Normale Supérieure - PSL (FR)
- Meta (United States) (US)
- ORCID
- Neverova, Natalia
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2102.05644
- OpenAlex
- W3128099838
- Cited by
- 127
- References
- 52
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 25 · 2023: 39 · 2022: 32 · 2021: 22
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- vision transformer, image retrieval, image descriptors, metric learning, contrastive loss, object retrieval, low-resolution images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
