The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Fine-Tuning CNN Image Retrieval with No Human Annotation
Open accessAbstract
Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide the selection of the training data. We show that both hard-positive and hard-negative examples, selected by exploiting the geometry and the camera positions available from the 3D models, enhance the performance of particular-object retrieval. CNN descriptor whitening discriminatively learned from the same training data outperforms commonly used PCA whitening. We propose a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and average pooling and show that it boosts retrieval performance. Applying the proposed method to the VGG network achieves state-of-the-art performance on the standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150
A comparison of affine region detectors
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Gool, L. V. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-3848-x · Cited by 2,939
Object retrieval with large vocabularies and fast spatial matching
Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383172 · Cited by 2,929
Self-citationNetVLAD: CNN Architecture for Weakly Supervised Place Recognition
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.572 · Cited by 2,584
MatConvNet: Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2807412 · Cited by 2,444
Deep Metric Learning Using Triplet Network
Hoffer, E., & Ailon, N. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24261-3_7 · Cited by 2,134
Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search
Jegou, H., Douze, M., & Schmid, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_24 · Cited by 1,901 · cited 1×
“We evaluate our approach on Oxford buildings [33], Paris [68] and Holidays(2) [69] datasets.”
MethodAggregating Local Image Descriptors into Compact Codes
Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.235 · Cited by 1,503
Lost in quantization: Improving particular object retrieval in large scale image databases
Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587635 · Cited by 1,478
Self-citationThree things everyone should know to improve object retrieval
Arandjelovic, R., & Zisserman, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248018 · Cited by 1,383
Learning Fine-Grained Image Similarity with Deep Ranking
Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., & Wu, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.180 · Cited by 1,269
Dollar, P., Tu, Z., Perona, P., & Belongie, S. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.91 · Cited by 1,237
Neural Codes for Image Retrieval
Babenko, A., Slesarev, A., Chigorin, A., & Lempitsky, V. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_38 · Cited by 1,192
Part-Based R-CNNs for Fine-Grained Category Detection
Zhang, N., Donahue, J., Girshick, R., & Darrell, T. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_54 · Cited by 1,150
Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., & Szeliski, R. (2011) · Communications of the ACM · doi:10.1145/2001269.2001293 · Cited by 1,072
Multi-scale Orderless Pooling of Deep Convolutional Activation Features
Gong, Y., Wang, L., Guo, R., & Lazebnik, S. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_26 · Cited by 1,030
Large-Scale Image Retrieval with Attentive Deep Local Features
Noh, H., Araujo, A., Sim, J., Weyand, T., & Han, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.374 · Cited by 847
Deep Image Retrieval: Learning Global Representations for Image Search
Gordo, A., Almazán, J., Revaud, J., & Larlus, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_15 · Cited by 787
Large-scale image retrieval with compressed Fisher vectors
Perronnin, F., Liu, Y., Sanchez, J., & Poirier, H. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540009 · Cited by 761
Particular object retrieval with integral max-pooling of CNN activations
Tolias, G., Sicre, R., & Jégou, H. (2015) · arXiv · doi:10.48550/arxiv.1511.05879 · Cited by 744
Self-citationArandjelovic, R., & Zisserman, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.207 · Cited by 713
Discriminative Deep Metric Learning for Face Verification in the Wild
Hu, J., Lu, J., & Tan, Y.-P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.242 · Cited by 682
DenseNet: Implementing Efficient ConvNet Descriptor Pyramids
Iandola, F., Moskewicz, M. W., Karayev, S., Girshick, R., Darrell, T. J., & Keutzer, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.1869 · Cited by 629
MatConvNet - Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4564 · Cited by 578
Building Rome on a Cloudless Day
Frahm, J.-M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., Lazebnik, S., et al. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_27 · Cited by 563
CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples
Radenović, F., Tolias, G., & Chum, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_1 · Cited by 551
Self-citationEnd-to-End Learning of Deep Visual Representations for Image Retrieval
Gordo, A., Almazán, J., Revaud, J., & Larlus, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1016-8 · Cited by 537
Location Recognition Using Prioritized Feature Matching
Li, Y., Snavely, N., & Huttenlocher, D. P. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15552-9_57 · Cited by 436
Cross-Dimensional Weighting for Aggregated Deep Convolutional Features
Kalantidis, Y., Mellina, C., & Osindero, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46604-0_48 · Cited by 405
From generic to specific deep representations for visual recognition
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., & Carlsson, S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301270 · Cited by 394
Negative Evidences and Co-occurences in Image Retrieval: The Benefit of PCA and Whitening
Jégou, H., & Chum, O. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_55 · Cited by 392
Self-citationGeneralizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and TreePreprint
Lee, C., Gallagher, P. W., & Zhuowen Tu, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.08985 · Cited by 386
Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors
Qin, D., Gammeter, S., Bossard, L., Quack, T., & van Gool, L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995373 · Cited by 305
Total recall II: Query expansion revisited
Chum, O., Mikulik, A., Perdoch, M., & Matas, J. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995601 · Cited by 287
Self-citation[Paper] Visual Instance Retrieval with Deep Convolutional Networks
Razavian, A. S., Sullivan, J., Carlsson, S., & Maki, A. (2016) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.4.251 · Cited by 276
Aggregating Deep Convolutional Features for Image Retrieval
Babenko, A., & Lempitsky, V. (2015) · arXiv · doi:10.48550/arxiv.1510.07493 · Cited by 224
Papandreou, G., Kokkinos, I., & Savalle, P.-A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298636 · Cited by 200
Good Practice in CNN Feature TransferPreprint
Zheng, L., Zhao, Y., Wang, S., Wang, J., & Tian, Q. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.00133 · Cited by 170
Bags of Local Convolutional Features for Scalable Instance Search
Mohedano, E., McGuinness, K., O'Connor, N. E., Salvador, A., Marqués, F., & Giró-i-Nieto, X. (2016) · International Conference on Multimedia Retrieval · doi:10.1145/2911996.2912061 · Cited by 161
Large-Scale Discovery of Spatially Related Images
Chum, O., & Matas, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.166 · Cited by 122
Self-citationFrom single image query to detailed 3D reconstruction
Schönberger, J. L., Radenović, F., Chum, O., & Frahm, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299148 · Cited by 118
Self-citationVisual query expansion with or without geometry: Refining local descriptors by feature aggregation
Tolias, G., & Jeǵou, H. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.04.007 · Cited by 104
Self-citationImproving Descriptors for Fast Tree Matching by Optimal Linear Projection
Mikolajczyk, K., & Matas, J. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408871 · Cited by 94
Geometric Latent Dirichlet Allocation on a Matching Graph for Large-scale Image Datasets
Philbin, J. F., Šivic, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-010-0363-5 · Cited by 69
Fracking Deep Convolutional Image DescriptorsPreprint
Simo‐Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., & Moreno-Noguer, F. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6537 · Cited by 67
Learning Vocabularies over a Fine Quantization
Mikulík, A., Perďoch, M., Chum, O., & Matas, J. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0600-1 · Cited by 67
Self-citationSpatially-Constrained Similarity Measurefor Large-Scale Object Retrieval
Shen, X., Lin, Z., Brandt, J., & Wu, Y. N. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.237 · Cited by 64
Approximate Gaussian Mixtures for Large Scale Vocabularies
Avrithis, Y., & Kalantidis, Y. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33712-3_2 · Cited by 45
Deep Convolutional Features for Image Based Retrieval and Scene CategorizationPreprint
Mousavian, A., & Košecká, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06033 · Cited by 38
Orientation Covariant Aggregation of Local Descriptors with Embeddings
Tolias, G., Furon, T., & Jeǵou, H. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_25 · Cited by 31
Self-citationRadenović, F., Jeǵou, H., & Chum, O. (2015) · International Conference on Multimedia Retrieval · doi:10.1145/2671188.2749366 · Cited by 26
Self-citationImage Retrieval for Online Browsing in Large Image Collections
Mikulík, A., Chum, O., & Matas, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41062-8_2 · Cited by 20
Self-citationNested Invariance Pooling and RBM Hashing for Image Instance Retrieval
Morère, O., Lin, J., Veillard, A., Duan, L., Chandrasekhar, V., & Poggio, T. (2017) · International Conference on Multimedia Retrieval · doi:10.1145/3078971.3078987 · Cited by 19
Cohen, N., Sharir, O., & Shashua, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.517 · Cited by 19
Research works printed but not linked: 3
- mikulík, 2014, Efficient image detail mining, Proc Asian Conf Comput Vis, 118
- radenovi?, 2016, From dusk till dawn: Modeling in the dark, Proc IEEE Conf Comput Vis Pattern Recognit, 5488
- 10.1109/ICCV.2013.432 · doi:10.1109/iccv.2013.432
Cited by
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638
“The features are obtained as the concatenation of the output [CLS] token and of the GeM pooled [54] output patch tokens.”
BackgroundDeep Learning for Person Re-Identification: A Survey and Outlook
Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., & Hoi, S. C. H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3054775 · Cited by 2,259
D2-Net: A Trainable CNN for Joint Description and Detection of Local Features
Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., & Sattler, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00828 · Cited by 1,179
DINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078
“A first family of self-supervised methods focuses on pretext tasks built from the image, i.e., extracting a signal from the image to be predicted from the rest of the image.”
BackgroundResMLP: Feedforward networks for image classification with data-efficient training
Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.03404 · Cited by 807
Metadata
- Authors
- Radenovic, F., Tolias, G., & Chum, O.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-07-01
- Volume
- 41
- Issue
- 7
- Pages
- 1655-1668
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2767621619
- Cited by
- 1,472
- References
- 67
- FWCI
- 53.46
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- MSMT (LL1303 ERC-CZ)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.