The devil is in the details: an evaluation of recent feature encoding methods
Abstract
A large number of novel encodings for bag of visual words models have been proposed in the past two years to improve on the standard histogram of quantized local features. Examples include locality-constrained linear encoding [23], improved Fisher encoding [17], super vector encoding [27], and kernel codebook encoding [20]. While several authors have reported very good results on the challenging PASCAL VOC classification data by means of these new techniques, differences in the feature computation and learning algorithms, missing details in the description of the methods, and different tuning of the various components, make it impossible to compare directly these methods and hard to reproduce the results reported. This paper addresses these shortcomings by carrying out a rigorous evaluation of these new techniques by: (1) fixing the other elements of the pipeline (features, learning, tuning); (2) disclosing all the implementation details, and (3) identifying both those aspects of each method which are particularly important to achieve good performance, and those aspects which are less critical. This allows a consistent comparative analysis of these encoding methods. Several conclusions drawn from our analysis cannot be inferred from the original publications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deep convolutional feature maps are analogous to feature maps in traditional methods.
“These feature maps generated by deep convolutional layers are analogous to the feature maps in traditional methods [27], [28].”
From Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2015) · cites this paper as [27]
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472 · see citing works
McLachlan, G., & Peel, D. (2000) · Wiley Series in Probability and Statistics · doi:10.1002/0471721182 · Cited by 10,991 · see citing works
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · see citing works
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · see citing works
Visual categorization with bags of keypoints
Csurka, G. (2004) · Cited by 4,379 · see citing works
PCA-SIFT: a more distinctive representation for local image descriptors
Ke, Y., & Sukthankar, R. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315206 · Cited by 3,237 · see citing works
Locality-constrained Linear Coding for image classification
Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540018 · Cited by 3,100 · see citing works
Vedaldi, A., & Fulkerson, B. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1874249 · Cited by 2,675 · see citing works
Self-citationFAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION
(2009) · Proceedings of the Fourth International Conference on Computer Vision Theory and Applications · doi:10.5220/0001787803310340 · Cited by 2,640 · see citing works
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588 · see citing works
The PASCAL visual object classes challenge 2006 (VOC2006) results
Everingham, M., Zisserman, A., Williams, C. K. I., & Gool, L. V. (2006) · Cited by 2,079 · see citing works
Self-citationTraining linear SVMs in linear time
Joachims, T. (2006) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1150402.1150429 · Cited by 1,932 · see citing works
Zhang, J., Marszałek, M., Lazebnik, S., & Schmid, C. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-9794-4 · Cited by 1,929 · see citing works
The pyramid match kernel: discriminative classification with sets of image features
Grauman, K. L., & Darrell, T. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.239 · Cited by 1,500 · see citing works
Lost in quantization: Improving particular object retrieval in large scale image databases
Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587635 · Cited by 1,478 · see citing works
Self-citationLearning mid-level features for recognition
Boureau, Y.-L., Bach, F., LeCun, Y., & Ponce, J. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539963 · Cited by 1,063 · see citing works
Nonlinear Learning using Local Coordinate Coding
Yu, K., Zhang, T., & Gong, Y. (2009) · Cited by 681 · see citing works
Kernel Codebooks for Scene Categorization
van Gemert, J. C., Geusebroek, J., Veenman, C. J., & Smeulders, A. W. M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88690-7_52 · Cited by 591 · see citing works
Image Classification Using Super-Vector Coding of Local Image Descriptors
Zhou, X., Yu, K., Zhang, T., & Huang, T. S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_11 · Cited by 503 · see citing works
Fast Discriminative Visual Codebooks using Randomized Clustering Forests
Moosmann, F., Triggs, B., & Jurie, F. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0128 · Cited by 467 · see citing works
A Bayesian Approach to Unsupervised One-Shot Learning of Object Categories
Fei-Fei, L., Fergus, R., & Perona, P. (2003) · Cited by 423 · see citing works
A Bayesian approach to unsupervised one-shot learning of object categories
Fe-Fei, L., Fergus, & Perona (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238476 · Cited by 423 · see citing works
Supervised translation-invariant sparse coding
Yang, J., Yu, K., & Huang, T. S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539958 · Cited by 359 · see citing works
Efficient additive kernels via explicit feature maps
Vedaldi, A., & Zisserman, A. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539949 · Cited by 259 · see citing works
Self-citationSupervised Learning of Quantizer Codebooks by Information Loss Minimization
Lazebnik, S., & Raginsky, M. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.138 · Cited by 220 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
“with respect to a dictionary, and Fisher Vector [30] can be formulated as a probabilistic version [18] of VLAD. Both of them are powerful shallow representations for image retrieval and classification [4,48]. For vector quantization, encoding residual vectors [17] is shown to be more effective than encoding original ve…”
BackgroundMomentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · see citing works
Image Classification with the Fisher Vector: Theory and Practice
Sánchez, J., Perronnin, F., Mensink, T., & Verbeek, J. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0636-x · Cited by 1,498 · see citing works
Learnable PINs: cross-modal embeddings for person identity
Nagrani, A., Albanie, S., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_5 · Cited by 151 · see citing works
Self-citationMultiple kernel learning for emotion recognition in the wild
Sikka, K., Dykstra, K., Sathyanarayana, S., Littlewort, G., & Bartlett, M. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531741 · Cited by 119 · see citing works
Monocular SLAM Supported Object Recognition
Pillai, S., & Leonard, J. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.034 · Cited by 118 · see citing works
Yu, Z., Tan, E.-L., Ni, D., Qin, J., Chen, S., Li, S., Lei, B., & Wang, T. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2705031 · Cited by 109 · see citing works
Towards Unified Object Detection and Semantic Segmentation
Dong, J., Chen, Q., Yan, S., & Yuille, A. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_20 · Cited by 70 · see citing works
NNIME: The NTHU-NTUA Chinese interactive multimodal emotion corpus
Chou, H.-C., Lin, W.-C., Chang, L.-C., Li, C.-C., Ma, H.-P., & Lee, C.-C. (2017) · 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2017.8273615 · Cited by 57 · see citing works
Event-based media processing and analysis: A survey of the literature
Tzelepis, C., Ma, Z., Mezaris, V., Ionescu, B., Kompatsiaris, I., Boato, G., Sebe, N., & Yan, S. (2016) · Image and Vision Computing · doi:10.1016/j.imavis.2016.05.005 · Cited by 49 · see citing works
On the Generality of Codebook Approach for Sensor-Based Human Activity Recognition
Shirahama, K., & Grzegorzek, M. (2017) · Electronics · doi:10.3390/electronics6020044 · Cited by 29 · see citing works
- Authors
- Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- Venue
- Procedings of the British Machine Vision Conference 2011
- Publisher
- British Machine Vision Association
- Published
- 2011
- Pages
- 76.1-76.12
- OpenAlex
- W1976921161
- Cited by
- 834
- References
- 26
- Citations by year
- 2025: 2 · 2024: 6 · 2023: 8 · 2022: 6 · 2021: 30 · 2020: 20 · 2019: 40 · 2018: 51 · 2017: 73 · 2016: 133 · 2015: 161 · 2014: 161 · 2013: 100 · 2012: 41
- FWCI
- 72.54
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- British Machine Vision Conference (BMVC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- bag of visual words, feature encoding, image classification, Pascal VOC, reproducibility, method comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
