Lost in quantization: Improving particular object retrieval in large scale image databases
Abstract
The state of the art in visual object retrieval from large databases is achieved by systems that are inspired by text retrieval. A key component of these approaches is that local regions of images are characterized using high-dimensional descriptors which are then mapped to ldquovisual wordsrdquo selected from a discrete vocabulary.This paper explores techniques to map each visual region to a weighted set of words, allowing the inclusion of features which were lost in the quantization stage of previous systems. The set of visual words is obtained by selecting words based on proximity in descriptor space. We describe how this representation may be incorporated into a standard tf-idf architecture, and how spatial verification is modified in the case of this soft-assignment. We evaluate our method on the standard Oxford Buildings dataset, and introduce a new dataset for evaluation. Our results exceed the current state of the art retrieval performance on these datasets, particularly on queries with poor initial recall where techniques like query expansion suffer. Overall we show that soft-assignment is always beneficial for retrieval with large vocabularies, at a cost of increased storage requirements for the index.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hamming embedding is better than directly using more Voronoi cells because using more cells incurs significant quantization error.
“This is better than directly using more Voronoi cells (i.e., more clusters) because using more cells will incur significant quantization error [23].”
From VCDB: A Large-Scale Database for Partial Copy Detection in Videos (2014) · cites this paper as [23]
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Baeza-Yates, R. A., & Ribeiro-Neto, B. A. (1999) · Cited by 11,498 · see citing works
Vector Quantization and Signal Compression
Gersho, A., & Gray, R. M. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3626-0 · Cited by 7,055 · see citing works
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · see citing works
Pattern Recognition and Machine Learning
Neal, R. M. (2007) · Technometrics · doi:10.1198/tech.2007.s518 · Cited by 4,629 · see citing works
Robust wide-baseline stereo from maximally stable extremal regions
Matas, J., Chum, O., Urban, M., & Pajdla, T. (2004) · Image and Vision Computing · doi:10.1016/j.imavis.2004.02.006 · Cited by 3,683 · see citing works
Self-citationScalable Recognition with a Vocabulary Tree
Nistér, D., & Stewénius, H. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.264 · Cited by 3,599 · see citing works
Scale & Affine Invariant Interest Point Detectors
Mikolajczyk, K., & Schmid, C. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000027790.02288.f2 · Cited by 3,436 · see citing works
Object retrieval with large vocabularies and fast spatial matching
Philbin, J. F., Chum, O., Isard, M., Šivic, J., & Zisserman, A. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383172 · Cited by 2,931 · see citing works
Self-citationWeb search for a planet: the google cluster architecture
Barroso, L. A., Dean, J., & Hölzle, U. (2003) · IEEE Micro · doi:10.1109/mm.2003.1196112 · Cited by 1,029 · see citing works
Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval
Chum, O., Philbin, J. F., Šivic, J., Isard, M., & Zisserman, A. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408891 · Cited by 860 · see citing works
Self-citationBook Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
A contextual dissimilarity measure for accurate and efficient image search
Jeǵou, H., Harzallah, H., & Schmid, C. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.382970 · Cited by 242 · see citing works
Vector Quantizing Feature Space with a Regular Lattice
Tuytelaars, T., & Schmid, C. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408924 · Cited by 168 · see citing works
Point matching as a classification problem for fast and robust object pose estimation
Lepetit, V., Pilet, J., & Fua, P. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315170 · Cited by 117 · see citing works
Multi-tier architecture for Web search engines
Risvik, K. M., Aasheim, Y., & Lidal, M. (2004) · doi:10.1109/laweb.2003.1250291 · Cited by 48 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · see citing works
“We consider the revisited [43] Oxford and Paris image retrieval datasets [40].”
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · see citing works
Product Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · see citing works
“First, the number of possible distances is significantly higher than for competing Hamming embedding methods [20], [17], [19], as the Hamming space used in these techniques allows for a few distinct distances only.”
MethodHamming Embedding and Weak Geometric Consistency for Large Scale Image Search
Jegou, H., Douze, M., & Schmid, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_24 · Cited by 1,901 · see citing works
Aggregating Local Image Descriptors into Compact Codes
Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.235 · Cited by 1,503 · see citing works
Understanding bag-of-words model: a statistical framework
Zhang, Y., Jin, R., & Zhou, Z.-H. (2010) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-010-0001-0 · Cited by 1,384 · see citing works
The devil is in the details: an evaluation of recent feature encoding methods
Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.76 · Cited by 834 · see citing works
Self-citationCNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607 · see citing works
Factors of Transferability for a Generic ConvNet Representation
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., & Carlsson, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2500224 · Cited by 301 · see citing works
[Paper] Visual Instance Retrieval with Deep Convolutional Networks
Razavian, A. S., Sullivan, J., Carlsson, S., & Maki, A. (2016) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.4.251 · Cited by 276 · see citing works
Unsupervised Pre-Training of Image Features on Non-Curated Data
Caron, M., Bojanowski, P., Mairal, J., & Joulin, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00305 · Cited by 258 · see citing works
Training Vision Transformers for Image Retrieval
El-Nouby, A., Neverova, N., Laptev, I., & Jégou, H. (2021) · arXiv · doi:10.48550/arxiv.2102.05644 · Cited by 127 · see citing works
Grauman, K., & Leibe, B. (2011) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00332ed1v01y201103aim011 · Cited by 126 · see citing works
VCDB: A Large-Scale Database for Partial Copy Detection in Videos
Jiang, Y.-G., Jiang, Y., & Wang, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_24 · Cited by 82 · see citing works
Partial Copy Detection in Videos: A Benchmark and an Evaluation of Popular Methods
Jiang, Y.-G., & Wang, J. (2016) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2016.2530714 · Cited by 72 · see citing works
- Authors
- Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- Center for Machine Perception, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic
- Microsoft (United States) (US)
- INRIA, WILLOW Project-Team, Laboratoire d'Informatique de l'Ecole Normale Superieure, Paris, France
- ORCID
- Zisserman, Andrew
- Venue
- 2008 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2008-06
- Pages
- 1-8
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2148809531
- Cited by
- 1,478
- References
- 16
- Citations by year
- 2026: 11 · 2025: 21 · 2024: 39 · 2023: 57 · 2022: 65 · 2021: 70 · 2020: 80 · 2019: 99 · 2018: 108 · 2017: 118 · 2016: 125 · 2015: 140 · 2014: 110 · 2013: 128 · 2012: 99
- FWCI
- 55.18
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- large-scale image databases, visual words, soft assignment, TF-IDF, spatial verification, query expansion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
