Compact Deep Invariant Descriptors for Video Retrieval
Abstract
With emerging demand for large-scale video analysis, the Motion Picture Experts Group (MPEG) initiated the Compact Descriptor for Video Analysis (CDVA) standardization in 2014. In this work, we develop novel deep-learning features and incorporate them into the well-established CDVA evaluation framework to study its effectiveness in video analysis. In particular, we propose a Nested Invariance Pooling (NIP) method to obtain compact and robust Convolutional Neural Network (CNNs) descriptors. The CNNs descriptors are generated by applying three different pooling operations to the feature maps of CNNs in a nested way towards rotation and scale invariant feature representation. In particular, the rational, advantages and performance on the combination of CNNs and handcrafted descriptors are provided to better investigate the complementary effects of deep learnt and handcrafted features. Extensive experimental results show that the proposed CNNs descriptors outperform both state-of-the-art CNNs descriptors and canonical handcrafted descriptors adopted in CDVA Experimental Model (CXM) with significant mAP gains of 11.3% and 4.7%, respectively. Moreover, the combination of NIP derived deep invariant descriptors and handcrafted descriptors not only fulfills the lowest bitrate budget of CDVA, but also significantly advances the performance of CDVA core techniques.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Neural Codes for Image Retrieval
Babenko, A., Slesarev, A., Chigorin, A., & Lempitsky, V. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_38 · Cited by 1,225 · see citing works
Particular object retrieval with integral max-pooling of CNN activations
Tolias, G., Sicre, R., & Jégou, H. (2015) · arXiv · doi:10.48550/arxiv.1511.05879 · Cited by 744 · see citing works
Aggregating Local Deep Features for Image Retrieval
Yandex, A. B., & Lempitsky, V. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.150 · Cited by 679 · see citing works
From generic to specific deep representations for visual recognition
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., & Carlsson, S. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301270 · Cited by 401 · see citing works
Overview of the MPEG-CDVS Standard
Duan, L., Chandrasekhar, V., Chen, J., Lin, J., Wang, Z., Huang, T., Girod, B., & Gao, W. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2500034 · Cited by 135 · see citing works
Self-citationA practical guide to CNNs and Fisher Vectors for image instance retrieval
Chandrasekhar, V., Lin, J., Morère, O., Goh, H., & Veillard, A. (2016) · Signal Processing · doi:10.1016/j.sigpro.2016.05.021 · Cited by 82 · see citing works
Self-citationCompact Descriptors for Visual Search
Duan, L., Lin, J., Chen, J., et al. (2014) · IEEE Multimedia · doi:10.1109/mmul.2013.66 · Cited by 73 · see citing works
Self-citationWeighted Component Hashing of Binary Aggregated Descriptors for Fast Visual Search
Duan, L., Lin, J., Wang, Z., et al. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2419973 · Cited by 48 · see citing works
Self-citationLearning invariant representations and applications to face verification
Liao, Q., Leibo, J. Z., & Poggio, T. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 34 · see citing works
A deep representation for invariance and music classification
Zhang, C., Evangelopoulos, G., Voinea, S., et al. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854954 · Cited by 24 · see citing works
DeepHash: Getting Regularization, Depth and Fine-Tuning RightPreprint
Lin, J., Morère, O., Chandrasekhar, V., Veillard, A., & Goh, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.04711 · Cited by 23 · see citing works
Self-citationRepresentation Learning in Sensory Cortex: A Theory
Anselmi, F., & Poggio, T. (2022) · IEEE Access · doi:10.1109/access.2022.3208603 · Cited by 20 · see citing works
Interframe Coding of Global Image Signatures for Mobile Augmented Reality
Chen, D. M., Makar, M. A., Araújo, A. D. M., et al. (2014) · DCC · doi:10.1109/dcc.2014.8 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Video Copy Detection Using Spatio-Temporal CNN Features
Zhou, Z., Chen, J., Yang, C.-N., & Sun, X. (2019) · IEEE Access · doi:10.1109/access.2019.2930173 · Cited by 18 · see citing works
“[22] propose a Nested Invariance Pooling (NIP) method to obtain compact and robust CNN descriptors.”
MethodSelf-citation
- Authors
- Lou, Y., Bai, Y., Lin, J., Wang, S., Chen, J., Chandrasekhar, V., Duan, L.-Y., Huang, T., Kot, A. C., & Gao, W.
- Affiliations
- Peking University (CN)
- Agency for Science, Technology and Research (SG)
- Nanyang Technological University (SG)
- ORCID
- Venue
- 2017 Data Compression Conference (DCC)
- Publisher
- IEEE
- Published
- 2017-04
- Pages
- 420-429
- ISSN
- 1068-0314 (ISSN-L)
- OpenAlex
- W2614042628
- Cited by
- 40
- References
- 16
- Citations by year
- 2024: 1 · 2023: 2 · 2022: 6 · 2021: 6 · 2020: 7 · 2019: 8 · 2018: 9 · 2017: 1
- FWCI
- 2.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- DCC (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- video retrieval, rotation and scale invariance, descriptor fusion, MPEG standardization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
