The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Self-Supervised Video Hashing With Hierarchical Binary Auto-Encoder
Open accessAbstract
Existing video hash functions are built on three isolated stages: frame pooling, relaxed learning, and binarization, which have not adequately explored the temporal order of video frames in a joint binary optimization model, resulting in severe information loss. In this paper, we propose a novel unsupervised video hashing framework dubbed self-supervised video hashing (SSVH), which is able to capture the temporal nature of videos in an end-to-end learning to hash fashion. We specifically address two central problems: 1) how to design an encoder-decoder architecture to generate binary codes for videos and 2) how to equip the binary codes with the ability of accurate video retrieval. We design a hierarchical binary auto-encoder to model the temporal dependencies in videos with multiple granularities, and embed the videos into binary codes with less computations than the stacked architecture. Then, we encourage the binary codes to simultaneously reconstruct the visual content and neighborhood structure of the videos. Experiments on two real-world data sets show that our SSVH method can significantly outperform the state-of-the-art methods and achieve the current best performance on the task of unsupervised video retrieval.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Image retrieval: Ideas, influences, and trends of the new age
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,040
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062
Stacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.10 · Cited by 1,998
Gong, Y., Lazebnik, S., Gordo, A., & Perronnin, F. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.193 · Cited by 1,834
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Fang, C., & Luo, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.503 · Cited by 1,803
YFCC100M: the new data in multimedia research
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,541
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
Supervised hashing with kernels
Liu, W., Wang, J., Ji, R., Jiang, Y.-G., & Chang, S.-F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247912 · Cited by 1,351
Theano: A CPU and GPU Math Compiler in Python
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., & Bengio, Y. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-003 · Cited by 1,052
Wang, J., Zhang, T., Song, J., Sebe, N., & Shen, H. T. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699960 · Cited by 971
Self-citationAdversarial Learning for Neural Dialogue Generation
Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., & Jurafsky, D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1230 · Cited by 873
Semi-Supervised Hashing for Large-Scale Search
Wang, J., Kumar, S., & Chang, S.-F. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.48 · Cited by 810
HashNet: Deep Learning to Hash by Continuation
Cao, Z., Long, M., Wang, J., & Yu, P. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.598 · Cited by 777
Feature Learning based Deep Supervised Hashing with Pairwise LabelsPreprint
Li, W.-J., Wang, S., & Kang, W.-C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03855 · Cited by 643
Video Captioning With Attention-Based LSTM and Semantic Consistency
Gao, L., Guo, Z., Zhang, H., Xu, X., & Shen, H. T. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2729019 · Cited by 637
Self-citationDeep hashing for compact binary codes learning
Liong, V. E., Lu, J., Wang, G., Moulin, P., & Zhou, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298862 · Cited by 545
Learning to Hash for Indexing Big Data—A Survey
Wang, J., Liu, W., Kumar, S., & Chang, S.-F. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2487976 · Cited by 519
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1151 · Cited by 482
Radisic, A. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17117444 · Cited by 462
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
Pan, P., Xu, Z., Yang, Y., Wu, F., & Zhuang, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.117 · Cited by 412
Jiang, Y.-G., Wu, Z., Wang, J., Xue, X., & Chang, S.-F. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2670560 · Cited by 381
Multiple feature hashing for real-time large scale near-duplicate video retrieval
Song, J., Yang, Y., Huang, Z., Shen, H. T., & Hong, R. (2011) · Proceedings of the 19th ACM international conference on Multimedia · doi:10.1145/2072298.2072354 · Cited by 334
Self-citationGirshick, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 276
Two-Stream 3-D convNet Fusion for Action Recognition in Videos With Arbitrary Size and Length
Wang, X., Gao, L., Wang, P., Sun, X., & Liu, X. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2749159 · Cited by 263 · cited 1×
“But it is beyond a set of frames and the temporal information in video is important for video content analysis [33].”
BackgroundSelf-citationFrom Deterministic to Generative: Multimodal Stochastic RNNs for Video Captioning
Song, J., Guo, Y., Gao, L., Li, X., Hanjalic, A., & Shen, H. T. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2851077 · Cited by 259
Self-citationThe New Data and New Challenges in Multimedia Research.Preprint
Thomée, B., Shamma, D. A., Friedland, G. H., Elizalde, B., Ni, K. S., Poland, D., Borth, D., & Li, L.-J. (2015) · arXiv (Cornell University) · Cited by 247
Sequence to Sequence -- Video to TextPreprint
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R. J., Darrell, T., & Saenko, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00487 · Cited by 225
A Sparse Embedding and Least Variance Encoding Approach to Hashing
Zhu, X., Zhang, L., & Huang, Z. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2332764 · Cited by 220
Quantization-based hashing: a general framework for scalable image and video retrieval
Song, J., Gao, L., Liu, L., Zhu, X., & Sebe, N. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.03.021 · Cited by 182
Self-citationHierarchical LSTM with Adjusted Temporal Attention for Video Captioning
Song, J., Gao, L., Guo, Z., Liu, W., Zhang, D., & Shen, H. T. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/381 · Cited by 165
Self-citationEvent Retrieval in Large Video Collections with Circulant Temporal Encoding
Revaud, J., Douze, M., Schmid, C., & Jegou, H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.318 · Cited by 133
Answer-Type Prediction for Visual Question Answering
Kafle, K., & Kanan, C. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.538 · Cited by 128
Liong, V. E., Lu, J., Tan, Y., & Zhou, J. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2645404 · Cited by 115
Play and Rewind: Optimizing Binary Representations of Videos by Self-Supervised Temporal Hashing
Zhang, H., Wang, M., Hong, R., & Chua, T. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964308 · Cited by 94
Self-citationHarnessing Object and Scene Semantics for Large-Scale Video Understanding
Wu, Z., Fu, Y., Jiang, Y., & Sigal, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.339 · Cited by 89
Large-Scale Video Hashing via Structure Learning
Ye, G., Liu, D., Wang, J., & Chang, S. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.282 · Cited by 80
Distributed Adaptive Binary Quantization for Fast Nearest Neighbor Search
Liu, X., Li, Z., Deng, C. D., & Tao, D. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2729896 · Cited by 76
YFCC100M: The New Data in Multimedia Research
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J. (2015) · arXiv · doi:10.48550/arxiv.1503.01817 · Cited by 69
Supervised Recurrent Hashing for Large Scale Video Retrieval
Gu, Y., Ma, C., & Yang, J. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967225 · Cited by 56
Deep Region Hashing for Generic Instance Search from Images
Song, J., He, T., Gao, L., Xu, X., & Shen, H. T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11277 · Cited by 51
Self-citationSubmodular video hashing: a unified framework towards video pooling and indexing
Cao, L., Li, Z., Mu, Y., & Chang, S.-F. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2393393 · Cited by 50
Deep Region Hashing for Efficient Large-scale Instance Search from ImagesPreprint
Song, J., He, T., Gao, L., Xu, X., & Shen, H. T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.07901 · Cited by 32
Self-citation
Cited by
Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A. S., Al-dabbagh, B. S. N., Fadhel, M. A., Manoufali, M., Zhang, J., Al-Timemy, A. H., et al. (2023) · Journal of Big Data · doi:10.1186/s40537-023-00727-2 · Cited by 907
STAT: Spatial-Temporal Attention Mechanism for Video Captioning
Yan, C., Tu, Y., Wang, X., Zhang, Y., Hao, X., Zhang, Y., & Dai, Q. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2924576 · Cited by 399
Multi-level Feature Learning for Contrastive Multi-view Clustering
Xu, J., Tang, H., Ren, Y., Peng, L., Zhu, X., & He, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01558 · Cited by 395
Enhancing transportation systems via deep learning: A survey
Wang, Y., Zhang, D., Liu, Y., Dai, B., & Lee, L. H. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.12.004 · Cited by 312
Metadata
- Authors
- Song, J., Zhang, H., Li, X., Gao, L., Wang, M., & Hong, R.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2018-07
- Volume
- 27
- Issue
- 7
- Pages
- 3210-3221
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2786585376
- Cited by
- 276
- References
- 52
- FWCI
- 20.81
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.