Training Vision Transformers for Image Retrieval
Open accessAbstract
Transformers have shown outstanding results for natural language understanding and, more recently, for image classification. We here extend this work and propose a transformer-based approach for image retrieval: we adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective, which combines a contrastive loss with a differential entropy regularizer. Our results show consistent and significant improvements of transformers over convolution-based approaches. In particular, our method outperforms the state of the art on several public benchmarks for category-level retrieval, namely Stanford Online Product, In-Shop and CUB-200. Furthermore, our experiments on ROxford and RParis also show that, in comparable settings, transformers are competitive for particular object retrieval, especially in the regime of short vector representations and low-resolution images.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two properties emerged that can be leveraged in future applications: the quality of features in k-NN classification has potential for image retrieval, and the presence of information about scene layout in the features can benefit weakly supervised image segmentation.
“We have also seen emerged two properties that can be leveraged in future applications: the quality of the features in k -NN classification has a potential for image retrieval where ViT are already showing promising results [22]. The presence of information about the scene layout in the features can also benefit weakly supervised image segmentation.”
From Emerging Properties in Self-Supervised Vision Transformers (2021) · cites this paper as [22]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · see citing works
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336 · see citing works
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., & Saul, L. K. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1577078 · Cited by 4,651 · see citing works
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,875 · see citing works
Object retrieval with large vocabularies and fast spatial matching
Philbin, J. F., Chum, O., Isard, M., Šivic, J., & Zisserman, A. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383172 · Cited by 2,931 · see citing works
Neighbourhood Components Analysis
Goldberger, J., Hinton, G. E., Roweis, S. T., & Salakhutdinov, R. (2004) · Cited by 1,855 · see citing works
DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations
Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.124 · Cited by 1,837 · see citing works
Deep Metric Learning via Lifted Structured Feature Embedding
Song, H. O., Yu, X., Jegelka, S., & Savarese, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.434 · Cited by 1,687 · see citing works
(2016) · neural information processing systems · Cited by 1,572 · see citing works
Fine-Tuning CNN Image Retrieval with No Human Annotation
Radenović, F., Tolias, G., & Chum, O. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2846566 · Cited by 1,515 · see citing works
Lost in quantization: Improving particular object retrieval in large scale image databases
Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587635 · Cited by 1,478 · see citing works
Neural Codes for Image Retrieval
Babenko, A., Slesarev, A., Chigorin, A., & Lempitsky, V. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_38 · Cited by 1,225 · see citing works
(2020) · International Conference on Machine Learning · Cited by 1,012 · see citing works
Stand-Alone Self-Attention in Vision Models
Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., & Shlens, J. (2019) · arXiv · doi:10.48550/arxiv.1906.05909 · Cited by 977 · see citing works
Wang, T., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10242 · Cited by 921 · see citing works
Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning
Wang, X., Han, X., Huang, W., Dong, D., & Scott, M. R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00516 · Cited by 893 · see citing works
Sampling Matters in Deep Embedding Learning
Wu, C.-Y., Manmatha, R., Smola, A. J., & Krähenbühl, P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.309 · Cited by 880 · see citing works
Deep Image Retrieval: Learning Global Representations for Image Search
Gordo, A., Almazán, J., Revaud, J., & Larlus, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_15 · Cited by 787 · see citing works
No Fuss Distance Metric Learning Using Proxies
Movshovitz-Attias, Y., Toshev, A., Leung, T. K., Ioffe, S., & Singh, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.47 · Cited by 658 · see citing works
Deep Metric Learning with Hierarchical Triplet Loss
Ge, W., Huang, W., Dong, D., & Scott, M. R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_17 · Cited by 467 · see citing works
(2021) · International Conference on Learning Representations · Cited by 449 · see citing works
A Metric Learning Reality Check
Musgrave, K., Belongie, S., & Lim, S.-N. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58595-2_41 · Cited by 447 · see citing works
SoftTriple Loss: Deep Metric Learning Without Triplet Sampling
Qian, Q., Shang, L., Sun, B., Hu, J., Tacoma, T., Li, H., & Jin, R. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00655 · Cited by 420 · see citing works
Learning With Average Precision: Training Image Retrieval With a Listwise Loss
Revaud, J., Almazan, J., Rezende, R., & Souza, C. D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00521 · Cited by 408 · see citing works
Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking
Radenović, F., İşcen, A., Tolias, G., Avrithis, Y., & Chum, O. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00598 · Cited by 406 · see citing works
Çakir, F., He, K., Xia, X., Kulis, B., & Sclaroff, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00196 · Cited by 260 · see citing works
Attention-Based Ensemble for Deep Metric Learning
Kim, W., Goyal, B., Chawla, K., Lee, J.-M., & Kwon, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_45 · Cited by 240 · see citing works
Aggregating Deep Convolutional Features for Image Retrieval
Babenko, A., & Lempitsky, V. (2015) · arXiv · doi:10.48550/arxiv.1510.07493 · Cited by 224 · see citing works
Classification is a Strong Baseline for Deep Metric LearningPreprint
Zhai, A., & Wu, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.12649 · Cited by 196 · see citing works
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182 · see citing works
Self-citation(2020) · doi:10.1007/978-3-030-58586-0_27 · Cited by 156 · see citing works
Learning Spread-Out Local Feature Descriptors
Zhang, X., Yu, F. X., Kumar, S., & Chang, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.492 · Cited by 137 · see citing works
A Unifying Mutual Information View of Metric Learning: Cross-Entropy vs. Pairwise Losses
Boudiaf, M., Rony, J., Ziko, I. M., Granger, É., Pedersoli, M., Piantanida, P., & Ayed, I. B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_33 · Cited by 134 · see citing works
Improving Generalization via Scalable Neighborhood Component Analysis
Wu, Z., Efros, A. A., & Yu, S. X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_42 · Cited by 132 · see citing works
UniformFace: Learning Deep Equidistributed Representation for Face Recognition
Duan, Y., Lu, J., & Zhou, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00353 · Cited by 129 · see citing works
RegularFace: Deep Face Recognition via Exclusive Regularization
Zhao, K., Xu, J., & Cheng, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00123 · Cited by 124 · see citing works
Particular Object Retrieval With Integral Max-Pooling of CNN Activations
Tolias, G., Sicre, R., & Jégou, H. (2016) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 122 · see citing works
Self-citationFrom single image query to detailed 3D reconstruction
Schönberger, J. L., Radenović, F., Chum, O., & Frahm, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299148 · Cited by 118 · see citing works
MIC: Mining Interclass Characteristics for Improved Metric Learning
Brattoli, B., Roth, K., & Ommer, B. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00809 · Cited by 83 · see citing works
Hybrid-Attention Based Decoupled Metric Learning for Zero-Shot Image Retrieval
Chen, B., & Deng, W. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00286 · Cited by 70 · see citing works
Metric Learning With HORDE: High-Order Regularizer for Deep Embeddings
Jacob, P., Picard, D., Histace, A., & Klein, E. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00664 · Cited by 68 · see citing works
Spreading vectors for similarity search
Sablayrolles, A., Douze, M., Schmid, C., & Jégou, H. (2018) · arXiv · doi:10.48550/arxiv.1806.03198 · Cited by 66 · see citing works
Investigating the Vision Transformer Model for Image Retrieval Tasks
Gkelios, S., Boutalis, Y. S., & Chatzichristofis, S. A. (2021) · International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) · doi:10.1109/dcoss52077.2021.00065 · Cited by 37 · see citing works
Bell, S., Liu, Y., Alsheikh, S., Tang, Y., Pizzi, E., Henning, M., Singh, K., Parkhi, O., & Borisyuk, F. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403311 · Cited by 27 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · see citing works
“We have also seen emerged two properties that can be leveraged in future applications: the quality of the features in k-NN classification has a potential for image retrieval where ViT are already showing promising results [22].”
BackgroundCSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
Dong, X., Bao, J., Chen, D., Zhang, W., Yu, N., Yuan, L., Chen, D., & Guo, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01181 · Cited by 1,458 · see citing works
Multiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407 · see citing works
Going deeper with Image Transformers
Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., & Jegou, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00010 · Cited by 1,259 · see citing works
CMT: Convolutional Neural Networks Meet Vision Transformers
Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., & Xu, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01186 · Cited by 1,005 · see citing works
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935 · see citing works
Self-citationSegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090 · see citing works
Task-Oriented Multi-User Semantic Communications
Xie, H., Qin, Z., Tao, X., & Letaief, K. B. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3191326 · Cited by 463 · see citing works
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y.-G., & Lim, S.-N. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01199 · Cited by 262 · see citing works
XCiT: Cross-Covariance Image TransformersPreprint
El-Nouby, A., Touvron, H., Caron, M., Bojanowski, P., Douze, M., Joulin, A., Laptev, I., Neverova, N., Synnaeve, G., Verbeek, J. J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09681 · Cited by 273 · see citing works
Self-citationA comprehensive survey on contrastive learning
Hu, H., Wang, X., Zhang, Y. T., Chen, Q., & Guan, Q. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128645 · Cited by 243 · see citing works
Deep Learning for Instance Retrieval: A Survey
Chen, W., Liu, Y., Wang, W., Bakker, E. M., Georgiou, T., Fieguth, P., Liu, L., & Lew, M. S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3218591 · Cited by 282 · see citing works
AnyLoc : Towards Universal Visual Place Recognition
Keetha, N., Mishra, A., Karhade, J., Murthy, J. K., Scherer, S., Krishna, K. M., & Garg, S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3343602 · Cited by 223 · see citing works
Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., & Goldstein, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00586 · Cited by 206 · see citing works
TransVPR: Transformer-Based Place Recognition with Multi-Level Attention Aggregation
Wang, R., Shen, Y., Zuo, W., Zhou, S., & Zheng, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01328 · Cited by 190 · see citing works
Hyperbolic Vision Transformers: Combining Improvements in Metric Learning
Ermolov, A., Mirvakhabova, L., Khrulkov, V., Sebe, N., & Oseledets, I. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00726 · Cited by 119 · see citing works
Feature Fusion Vision Transformer for Fine-Grained Visual Categorization
Wang, J., Yu, X., & Gao, Y. (2021) · Warwick Research Archive Portal (University of Warwick) · doi:10.48550/arxiv.2107.02341 · Cited by 106 · see citing works
Localizing Objects with Self-supervised Transformers and no LabelsPreprint
Siméoni, O., Puy, G., Vo, H. V., Roburin, S., Gidaris, S., Bursuc, A., P'erez, P., Marlet, R., & Ponce, J. (2021) · British Machine Vision Conference · doi:10.48550/arxiv.2109.14279 · Cited by 300 · see citing works
A Self-Supervised Descriptor for Image Copy Detection
Pizzi, E., Roy, S. D., Ravindra, S. N., Goyal, P., & Douze, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01413 · Cited by 101 · see citing works
All the attention you need: Global-local, spatial-channel attention for image retrieval
Song, C. H., Han, H. J., & Avrithis, Y. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00051 · Cited by 96 · see citing works
Deep Visual Geo-localization Benchmark
Berton, G., Mereu, R., Trivigno, G., Masone, C., Csurka, G., Sattler, T., & Barbara Caputo, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00532 · Cited by 96 · see citing works
Wang, Q., Wang, J., Deng, H., Wu, X. M., Wang, Y., & Hao, G. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109547 · Cited by 94 · see citing works
Vision Transformer Hashing for Image Retrieval
Dubey, S. R., Singh, S. K., & Chu, W.-T. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859900 · Cited by 92 · see citing works
LeViT: a Vision Transformer in ConvNet's Clothing for Faster InferencePreprint
Graham, B. V., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jeǵou, H., & Douze, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.01136 · Cited by 88 · see citing works
Self-citationContext-aware Scene Graph Generation with Seq2Seq Transformers
Lu, Y., Rai, H., Chang, J. S., Knyazev, B. A., Yu, G., Shekhar, S., Taylor, G. W., & Volkovs, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01563 · Cited by 85 · see citing works
Hybrid CNN-Transformer Features for Visual Place Recognition
Wang, Y., Qiu, Y., Cheng, P., & Zhang, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3212434 · Cited by 84 · see citing works
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers
Wei, S., Ye, T., Zhang, S., Tang, Y., & Liang, J. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.00208 · Cited by 152 · see citing works
Multilanguage Transformer for Improved Text to Remote Sensing Image Retrieval
Rahhal, M. M. A., Bazi, Y., Alsharif, N. A., Bashmal, L., Alajlan, N., & Melgani, F. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3215803 · Cited by 72 · see citing works
SG-Former: Self-guided Transformer with Evolving Token Reallocation
Ren, S., Yang, X., Liu, S.-H., & Wang, X. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.00552 · Cited by 80 · see citing works
ViTs as backbones: Leveraging vision transformers for feature extraction
Elharrouss, O., Himeur, Y., Mahmood, Y., Alrabaee, S., Ouamane, A., Bensaali, F., Bechqito, Y., & Chouchane, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102951 · Cited by 59 · see citing works
Tian, J., Xu, X., Shen, F., Yang, Y., & Shen, H. T. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20136 · Cited by 57 · see citing works
Boosting vision transformers for image retrieval
Song, C., Yoon, J., Choi, S., & Avrithis, Y. (2022) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00019 · Cited by 50 · see citing works
MSViT: Training Multiscale Vision Transformers for Image Retrieval
Li, X., Yu, J., Jiang, S., Lu, H., & Li, Z. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3304021 · Cited by 50 · see citing works
Learning Sequential Descriptors for Sequence-Based Visual Place Recognition
Mereu, R., Trivigno, G., Berton, G., Masone, C., & Barbara Caputo, B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3194310 · Cited by 47 · see citing works
Multiscale Vision TransformersPreprint
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11227 · Cited by 46 · see citing works
Zhou, H., Qin, Q., Hou, J., Dai, J., Huang, L., & Zhang, W. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122105 · Cited by 41 · see citing works
Person Re-Identification with a Locally Aware TransformerPreprint
Sharma, C., Kapil, S. R., & Chapman, D. (2021) · arXiv (Cornell University) · doi:10.13016/m2kqmp-wxwl · Cited by 40 · see citing works
RiceSeedNet: Rice seed variety identification using deep neural network
Rajalakshmi, R., Faizal, S., Sivasankaran, S., & Geetha, R. (2024) · Journal of Agriculture and Food Research · doi:10.1016/j.jafr.2024.101062 · Cited by 35 · see citing works
TransVLAD: Multi-Scale Attention-Based Global Descriptors for Visual Geo-Localization
Xu, Y., Shamsolmoali, P., Granger, É., Nicodème, C., Gardes, L., & Yang, J. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00286 · Cited by 34 · see citing works
Image Retrieval Using Convolutional Autoencoder, InfoGAN, and Vision Transformer Unsupervised Models
Sabry, E. S., Elagooz, S., El‐Samie, F. E. A., El‐Shafai, W., El-Bahnasawy, N. A. W., El‐Banby, G. M., Algarni, A. D., Soliman, N. F., & Ramadan, R. Α. (2023) · IEEE Access · doi:10.1109/access.2023.3241858 · Cited by 32 · see citing works
Ansari, S. A., Agrawal, A. P., Wajid, M. A., Wajid, M. S., & Zafar, A. (2024) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-024-00630-1 · Cited by 30 · see citing works
ETR: An Efficient Transformer for Re-ranking in Visual Place Recognition
Zhang, H., Chen, X., Jing, H., Zheng, Y., Wu, Y., & Jin, C. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00562 · Cited by 28 · see citing works
KMT-PLL: K-Means Cross-Attention Transformer for Partial Label Learning
Fan, J., Huang, L., Gong, C., You, Y., Gan, M., & Wang, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3347792 · Cited by 27 · see citing works
Graph Convolution Based Efficient Re-Ranking for Visual Retrieval
Zhang, Y., Qian, Q., Wang, H., Liu, C., Chen, W., & Wang, F. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3276167 · Cited by 28 · see citing works
HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization
Kim, S., Jeong, B., & Kwak, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01906 · Cited by 25 · see citing works
Transformer-Based Source-Free Domain AdaptationPreprint
Yang, G., Tang, H., Zhong, Z., Ding, M., Shao, L., Sebe, N., & Ricci, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14138 · Cited by 25 · see citing works
He, S., Yang, X., Jiang, C., Liang, G., Zhang, W., Pan, T., Wang, Q., Xu, F., Li, C., Liu, J., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02041 · Cited by 23 · see citing works
BodyMap: Learning Full-Body Dense Correspondence Map
Ianina, A., Sarafianos, N., Xu, Y., Rocco, I., & Tung, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01293 · Cited by 21 · see citing works
A Survey on Fashion Image Retrieval
Islam, S. M., Joardar, S., & Sekh, A. A. (2023) · ACM Computing Surveys · doi:10.1145/3636552 · Cited by 20 · see citing works
RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models
Long, Z., Killick, G., McCreadie, R., & Aragón-Camarasa, G. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610797 · Cited by 24 · see citing works
Robust and Decomposable Average Precision for Image RetrievalPreprint
Ramzi, E., Thome, N., Rambour, C., Audebert, N., & Bitot, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01445 · Cited by 19 · see citing works
Nguyen, S., Le, D. V., & Havinga, P. (2024) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2024.101912 · Cited by 18 · see citing works
KPPR: Exploiting Momentum Contrast for Point Cloud-Based Place Recognition
Wiesmann, L., Nunes, L., Behley, J., & Stachniss, C. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3228174 · Cited by 17 · see citing works
A Light-Weighted Hypergraph Neural Network for Multimodal Remote Sensing Image Retrieval
Yu, H., Deng, C., Zhao, L., Hao, L., Liu, X., Lu, W., & You, H. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3252670 · Cited by 17 · see citing works
Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image RecognitionPreprint
Wang, Y., Huang, R., Song, S., Huang, Z., & Huang, G. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.15075 · Cited by 101 · see citing works
Kong, Z., Ma, H., Yuan, G., Sun, M., Xie, Y., Dong, P., Meng, X., Shen, X., Tang, H., Qin, M., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26008 · Cited by 16 · see citing works
PlaceFormer: Transformer-Based Visual Place Recognition Using Multi-Scale Patch Selection and Fusion
Kannan, S. S., & Min, B. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3408075 · Cited by 16 · see citing works
Visual-Assisted Probe Movement Guidance for Obstetric Ultrasound Scanning Using Landmark Retrieval
Zhao, C., Droste, R., Drukker, L., Papageorghiou, A. T., & Noble, J. A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87237-3_64 · Cited by 15 · see citing works
SiT: Self-supervised vIsion Transformer
Atito, S., Awais, M., & Kittler, J. (2021) · arXiv · doi:10.48550/arxiv.2104.03602 · Cited by 75 · see citing works
AdaViT: Adaptive Vision Transformers for Efficient Image RecognitionPreprint
Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y., & Lim, S.-N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.15668 · Cited by 15 · see citing works
Robust facial expression recognition with Transformer Block Enhancement Module
Xie, Y., Tian, W., & Yu, Z. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106795 · Cited by 14 · see citing works
Transformer-based descriptors with fine-grained region supervisions for visual place recognition
Wang, Y., Qiu, Y., Cheng, P., & Zhang, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110993 · Cited by 12 · see citing works
Deep Factorized Metric Learning
Wang, C., Zheng, W., Li, J., Zhou, J., & Lu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00741 · Cited by 12 · see citing works
Coarse-to-Fine: Learning Compact Discriminative Representation for Single-Stage Image Retrieval
Zhu, Y., Gao, X., Ke, B., Qiao, R., & Sun, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01034 · Cited by 11 · see citing works
Visualizing Paired Image Similarity in Transformer Networks
Black, S., Stylianou, A., Pless, R. B., & Souvenir, R. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00160 · Cited by 10 · see citing works
Image Segmentation-based Unsupervised Multiple Objects Discovery
Kara, S., Ammar, H., Chabot, F., & Pham, Q.-C. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00329 · Cited by 10 · see citing works
Deep metric learning in projected-hypersphere space
Xu, Y., Chen, Z., & Hu, J. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111245 · Cited by 9 · see citing works
Wang, X., Jiang, B., Wang, X., Tang, J., & Luo, B. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3283132 · Cited by 9 · see citing works
Content based Image Retrieval: A Literature Review
Kulkarni, S., & Manu, T. M. (2022) · International Conference on Intelligent Computing and Control Systems (ICICCS) · doi:10.1109/iciccs53718.2022.9788375 · Cited by 9 · see citing works
Understanding Hyperbolic Metric Learning through Hard Negative Sampling
Yue, Y., Lin, F., Mou, G., & Zhang, Z. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00190 · Cited by 8 · see citing works
Nguyen, S. M., Le, D. V., & Havinga, P. (2023) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom56429.2023.10099376 · Cited by 8 · see citing works
Cross-Image-Attention for Conditional Embeddings in Deep Metric Learning
Kotovenko, D., Ma, P., Milbich, T., & Ommer, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01065 · Cited by 8 · see citing works
Writer Retrieval using Compact Convolutional Transformers and NetMVLAD
Peer, M., Kleber, F., & Sablatnig, R. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956155 · Cited by 8 · see citing works
A Light Touch Approach to Teaching Transformers Multi-view Geometry
Bhalgat, Y. S., Henriques, J. F., & Zisserman, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00480 · Cited by 8 · see citing works
Boosting cross-modal retrieval in remote sensing via a novel unified attention network
Choudhury, S., Saini, D., & Banerjee, B. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106718 · Cited by 7 · see citing works
Haihong, E., Ding, J., & Yuan, L. (2022) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml57342.2022.10009714 · Cited by 8 · see citing works
Locating Target Regions for Image Retrieval in an Unsupervised Manner
Zhang, B.-J., Liu, G., Li, Z., & Song, S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3363163 · Cited by 6 · see citing works
Learning Dynamic Batch-Graph Representation for Deep Representation Learning
Wang, X., Jiang, B., Wang, X., & Luo, B. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02175-8 · Cited by 6 · see citing works
A Noise-robust Locality Transformer for Fine-grained Food Image Retrieval
Song, J., Min, W., Liu, Y., Li, Z., Jiang, S., & Rui, Y. (2022) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr54900.2022.00068 · Cited by 5 · see citing works
Co-training 2L Submodels for Visual Recognition
Touvron, H., Cord, M., Oquab, M., Bojanowski, P., Verbeek, J. J., & Jeǵou, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01126 · Cited by 6 · see citing works
Unsupervised Deep-Embedding Global Feature Descriptor for Image Retrieval
He, Q. (2023) · Circuits Systems and Signal Processing · doi:10.1007/s00034-023-02545-6 · Cited by 5 · see citing works
Optimization of Rank Losses for Image Retrieval
Ramzi, E., Audebert, N., Rambour, C., Araújo, A. D. M., Bitot, X., & Thome, N. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3543846 · Cited by 5 · see citing works
LDGC-Net: learnable descriptor graph convolutional network for image retrieval
Wang, X., Wang, J., Kang, M., Feng, Z. K., Zhou, X., & Liu, B. (2022) · The Visual Computer · doi:10.1007/s00371-022-02753-2 · Cited by 5 · see citing works
Semantic VL-BERT: Visual Grounding via Attribute Learning
Wanigasekara, P., Qin, K., Barut, E., Yang, F., Ruan, W., & Su, C. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892420 · Cited by 5 · see citing works
Transformer-Based Distillation Hash Learning for Image Retrieval
Lv, Y., Wang, C., Yuan, W., Qian, X., Yang, W., & Zhao, W. (2022) · Electronics · doi:10.3390/electronics11182810 · Cited by 4 · see citing works
Revisiting Low-Resolution Images Retrieval with Attention Mechanism and Contrastive Learning
Vu, D. T., Yu, G.-H., & Kim, J. Y. (2021) · Applied Sciences · doi:10.3390/app11156783 · Cited by 4 · see citing works
Unsupervised Cross-Domain Deep-Fused Feature Descriptor for Efficient Image Retrieval
He, Q. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3519175 · Cited by 3 · see citing works
Nonoverlapping Feature Projection Convolutional Neural Network With Differentiable Loss Function
Abdelpakey, M. H., & Shehata, M. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3177394 · Cited by 4 · see citing works
Large-Scale Image Indexing and Retrieval Methods: A PRISMA-Based Review
Saouabe, A., Tkatek, S., Oualla, H., & Henriquez, C. S. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150732 · Cited by 3 · see citing works
Wang, F., Wang, R., Huang, Z., Dong, S., Wang, X., Zhou, Q., Zheng, S., & Liu, L. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17360-z · Cited by 3 · see citing works
LM-Metric: Learned pair weighting and contextual memory for deep metric learning
Yan, S., Xu, L. H., Shu, X., Lu, Z., & Shen, J. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110722 · Cited by 3 · see citing works
Large Kernel Attention Hashing for Efficient Image Retrieval
Zhao, X., Miao, Z., Wang, Y., Wang, J., & Li, Y. (2022) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp55476.2022.10039099 · Cited by 3 · see citing works
Davt: Data Augmentation Vision Transformer for Fine-Grained Visual CategorizationPreprint
Hu, X., Jiang, T., Zhu, S. N., Guo, J. Q., & Zhu, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4063510 · Cited by 3 · see citing works
Semantic Retrieval of Similar Radiological Images using Vision TransformersPreprint
Thakrar, A., Jayasuriya, M., Serapio, A., Wu, X. J., Davis, E., Schroeder, J., Vella, M., & Sohn, J. H. (2023) · medRxiv · doi:10.1101/2023.02.16.23286056 · Cited by 3 · see citing works
MCM-ViT:Mask-guided context-enhanced multi-scale transformer for fine-grained visual classification
Li, B., Wu, Y., Liu, Q., Chen, Y., He, K., & Yu, Y. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109888 · Cited by 2 · see citing works
Triplet Synthesis for Enhancing Composed Image Retrieval via Counterfactual Image Generation
Uesugi, K., Saito, N., Maeda, K., Ogawa, T., & Haseyama, M. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887762 · Cited by 2 · see citing works
Dynamic Knowledge Enhanced Neural Fashion Trend Forecasting with Quantile Loss
Rooholamini, F., Azmi, R., Khademhossein, M., & Zarvani, M. (2023) · International Conference on Computer and Knowledge Engineering · doi:10.1109/iccke60553.2023.10326316 · Cited by 2 · see citing works
ICAR: Image-Based Complementary Auto Reasoning
Wang, X., Liang, A., Liang, J., Lin, M. C., Lou, Y., & Yang, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28374 · Cited by 2 · see citing works
Image retrieval using underlying importance feature histogram
He, Q., & Liu, G. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09735-6 · Cited by 2 · see citing works
ClusVPR: Efficient Visual Place Recognition With Clustering-Based Weighted Transformer
Xu, Y., Shamsolmoali, P., Zareapoor, M., & Yang, J. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3510479 · Cited by 2 · see citing works
- Authors
- El-Nouby, A., Neverova, N., Laptev, I., & Jégou, H.
- Affiliations
- École Normale Supérieure - PSL (FR)
- Meta (United States) (US)
- ORCID
- Neverova, Natalia
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2102.05644
- OpenAlex
- W3128099838
- Cited by
- 127
- References
- 52
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 25 · 2023: 39 · 2022: 32 · 2021: 22
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- vision transformer, image retrieval, image descriptors, metric learning, contrastive loss, object retrieval, low-resolution images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
