CoChat
Sign Up
PreprintCited by 744Advanced Image and Video Retrieval TechniquesImage Retrieval and Classification TechniquesAdvanced Neural Network Applications

Particular object retrieval with integral max-pooling of CNN activations

Open access
Tolias, Giorgos, Sicre, Ronan, Jégou, Hervé·2015·arXiv
DOI 10.48550/arxiv.1511.05879arXiv 1511.05879
Open this paper in CoChat — free accountOpen PDF

Abstract

Recently, image representation built upon Convolutional Neural Network (CNN) has been shown to provide effective descriptors for image search, outperforming pre-CNN features as short-vector representations. Yet such models are not compatible with geometry-aware re-ranking methods and still outperformed, on some particular object retrieval benchmarks, by traditional image search systems relying on precise descriptor matching, geometric re-ranking, or query expansion. This work revisits both retrieval stages, namely initial search and re-ranking, by employing the same primitive information derived from the CNN. We build compact feature vectors that encode several image regions without the need to feed multiple inputs to the network. Furthermore, we extend integral images to handle max-pooling on convolutional layer activations, allowing us to efficiently localize matching objects. The resulting bounding box is finally used for image re-ranking. As a result, this paper significantly improves existing CNN-based recognition pipeline: We report for the first time results competing with traditional methods on the challenging Oxford5k and Paris6k datasets.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1511.05879

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • DINO ViT features trained on GLDv2 are remarkably good, outperforming previously published methods based on off-the-shelf descriptors.

    “DINO ViT features trained on GLDv2 are remarkably good, outperforming previously published methods based on off-the-shelf descriptors [68, 57].”

    From Emerging Properties in Self-Supervised Vision Transformers (2021) · cites this paper as [68]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 44

  1. Agrawal, Pulkit, Girshick, Ross, and Malik, Jitendra. Analyzing the performance of multilayer neural networks for object recognition. In ECCV , 2014.
  2. An, Senjian, Peursum, Patrick, Liu, Wanquan, and Venkatesh, Svetha. Efficient algorithms for subwindow search in object detection and localization. In CVPR , 2009.
  3. Arandjelovic, Relja and Zisserman, Andrew. Three things everyone should know to improve object retrieval. In CVPR , Jun. 2012.
  4. Arandjelovic, Relja and Zisserman, Andrew. All about VLAD. In CVPR , Jun. 2013.
  5. Arandjelovic, Relja, Gronat, Petr, Torii, Akihiko, Pajdla, Tomas, and Sivic, Josef. Netvlad: Cnn architecture for weakly supervised place recognition. In arXiv , 2015.
  6. Avrithis, Yannis and Kalantidis, Yannis. Approximate gaussian mixtures for large scale vocabularies. In ECCV , 2012.
  7. Avrithis, Yannis and Tolias, Giorgos. Hough pyramid matching: Speeded-up geometry re-ranking for large scale image retrieval. IJCV , 2014.
  8. Azizpour, Hossein, Razavian, Ali Sharif, Sullivan, Josephine, Maki, Atsuto, and Carlsson, Stefan. From generic to specific deep representations for visual recognition. In arXiv , 2014.
  9. Babenko, Artem and Lempitsky, Victor. Aggregating deep convolutional features for image retrieval. In ICCV , 2015.
  10. Babenko, Artem, Slesarev, Anton, Chigorin, Alexandr, and Lempitsky, Victor. Neural codes for image retrieval. In ECCV , Sep. 2014.
  11. Bentley, Joe. Programming Pearls, 2/E . Addison-Wesley Professional, 1999.
  12. Chen, Qiang, Song, Zheng, Feris, Rogerio, Datta, Amitava, Cao, Liangliang, Huang, Zhongyang, and Yan, Shuicheng. Efficient maximum appearance search for large-scale object detection. In CVPR , 2013.
  13. Chum, Ondrej, Mikulik, A., Perdoch, M., and Matas, J. Total recall II: Query expansion revisited. In CVPR , Jun. 2011.
  14. Danfeng, Qin, Gammeter, S., Bossard, L., Quack, T., and Gool, L. Van. Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In CVPR , 2011.
  15. Doll´ ar, Piotr, Tu, Zhuowen, Perona, Pietro, and Belongie, Serge. Integral channel features. In BMVC , 2009.
  16. Donahue, Jeff, Jia, Yangqing, Vinyals, Oriol, Hoffman, Judy, Zhang, Ning, Tzeng, Eric, and Darrell, Trevor. Decaf: A deep convolutional activation feature for generic visual recognition. In arXiv , 2013.
  17. Girshick, Ross. Fast r-cnn. In arXiv , 2015.
  18. Girshick, Ross, Donahue, Jeff, Darrell, Trevor, and Malik, Jitendra. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014.
  19. Gong, Yunchao, Wang, Liwei, Guo, Ruiqi, and Lazebnik, Svetlana. Multi-scale orderless pooling of deep convolutional activation features. In ECCV , 2014.
  20. Iandola, Forrest, Moskewicz, Matt, Karayev, Sergey, Girshick, Ross, Darrell, Trevor, and Keutzer, Kurt. Densenet: Implementing efficient convnet descriptor pyramids. In arxiv , 2014.
  21. J´ egou, Herv´ e and Chum, Ondrej. Negative evidences and co-occurences in image retrieval: The benefit of PCA and whitening. In ECCV , Oct. 2012.
  22. J´ egou, Herv´ e and Zisserman, Andrew. Triangulation embedding and democratic aggregation for image search. In CVPR , 2014.
  23. J´ egou, Herv´ e, Douze, Matthijs, and Schmid, Cordelia. Improving bag-of-features for large scale image search. IJCV , 87(3), Feb. 2010.
  24. J´ egou, Herv´ e, Perronnin, Florent, Douze, Matthijs, S´ anchez, Jorge, P´ erez, Patrick, and Schmid, Cordelia. Aggregating local descriptors into compact codes. Trans. PAMI , Sep. 2012.
  25. Kalantidis, Yannis, Mellina, Clayton, and Osindero, Simon. Cross-dimensional weighting for aggregated deep convolutional features. In arXiv preprint arXiv:1512.04065 , 2015. · arXiv
  26. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In NIPS , 2012.
  27. Lampert, Christoph H. Detecting objects in large image collections and videos by efficient subimage retrieval. In ICCV , 2009.
  28. Lampert, Christoph H, Blaschko, Matthew B, and Hofmann, Thomas. Efficient subwindow search: A branch and bound framework for object localization. Trans. PAMI , 31(12):2129-2142, 2009.
  29. Lin, Zhe and Brandt, Jonathan. A local bag-of-features model for large-scale object retrieval. In ECCV , 2010.
  30. Mikulik, Andrej, Perdoch, Michal, Chum, Ondˇ rej, and Matas, Jiˇ r´ ı. Learning vocabularies over a fine quantization. IJCV , 103(1), 2013.
  31. Papandreou, George, Kokkinos, Iasonas, and Savalle, Pierre-Andr´ e. Untangling local and global deformations in deep convolutional networks for image classification and sliding window detection. In arXiv , 2014.
  32. Philbin, James, Chum, Ondrej, Isard, Michael, Sivic, Josef, and Zisserman, Andrew. Object retrieval with large vocabularies and fast spatial matching. In CVPR , Jun. 2007.
  33. Philbin, James, Chum, Ondrej, Isard, Michael, Sivic, Josef, and Zisserman, Andrew. Lost in quantization: Improving particular object retrieval in large scale image databases. In CVPR , Jun. 2008.
  34. Razavian, Ali Sharif, Sullivan, Josephine, Maki, Atsuto, and Carlsson, Stefan. A baseline for visual instance retrieval with deep convolutional networks. In arXiv , 2014b.
  35. Ren, Shaoqing, He, Kaiming, Girshick, Ross, and Sun, Jian. Faster r-cnn: Towards real-time object detection with region proposal networks. In arXiv , 2015.
  36. Shen, Xiaohui, Lin, Zhe, Brandt, Jonathan, and Wu, Ying. Spatially-constrained similarity measure for large-scale object retrieval. Trans. PAMI , 36(6):1229-1241, 2014.
  37. Simonyan, Karen and Zisserman, Andrew. Very deep convolutional networks for large-scale image recognition. In arXiv , 2014.
  38. Tao, Ran, Gavves, Efstratios, Snoek, Cees GM, and Smeulders, Arnold WM. Locality in generic instance search from one example. In CVPR , 2014.
  39. Tolias, Giorgos, Avrithis, Yannis, and J´ egou, Herv´ e. Image search with selective match kernels: aggregation across single and multiple images. IJCV , 2015.
  40. Uijlings, Jasper, Van de Sande, Koen, Gevers, Theo, and Smeulders, Arnold. Selective search for object recognition. IJCV , 104(2):154-171, 2013.
  41. Van de Sande, Koen EA, Snoek, Cees GM, and Smeulders, Arnold WM. Fisher and VLAD with flair. In CVPR , 2014.
  42. Vedaldi, Andrea and Lenc, Karel. Matconvnet-convolutional neural networks for matlab. In arXiv , 2014.
  43. Viola, Paul and Jones, Michael. Robust real-time object detection. IJCV , 4:34-47, 2001.
  44. Zhong, Zhiyuan, Zhu, Jianke, and Hoi, Steven CH. Fast object retrieval using direct spatial matching. IEEE Trans. on Multimedia , 17(8):1391-1397, 2015.

Cited by

Sorted by citations

Authors
Tolias, G., Sicre, R., & Jégou, H.
Affiliations
  • LINKMEDIA: Creating and exploiting explicit links between multimedia fragments (FR)
  • Institut de Recherche en Informatique et Systèmes Aléatoires (FR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2015
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1511.05879
arXiv
1511.05879
OpenAlex
W2174726731
Cited by
744
References
4
Citations by year
2026: 1 · 2025: 13 · 2024: 37 · 2023: 58 · 2022: 74 · 2021: 118 · 2020: 97 · 2019: 128 · 2018: 111 · 2017: 88 · 2016: 18 · 2015: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
integral image, max pooling, image re-ranking, object localization, bounding box estimation, image search

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review