The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis
Abstract
Previous work on action recognition has focused on adapting hand-designed local features, such as SIFT or HOG, from static images to the video domain. In this paper, we propose using unsupervised feature learning as a way to learn features directly from video data. More specifically, we present an extension of the Independent Subspace Analysis algorithm to learn invariant spatio-temporal features from unlabeled video data. We discovered that, despite its simplicity, this method performs surprisingly well when combined with deep learning techniques such as stacking and convolution to learn hierarchical representations. By replacing hand-designed features with our learned features, we achieve classification results superior to all previous published results on the Hollywood2, UCF, KTH and YouTube action recognition datasets. On the challenging Hollywood2 and YouTube action datasets we obtain 53.3% and 75.8% respectively, which are approximately 5% better than the current best published results. Further benefits of this method, such as the ease of training and the efficiency of training and prediction, will also be discussed. You can download our code and learned spatio-temporal features here: http://ai.stanford.edu/~wzou/.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557
A performance evaluation of local descriptors
Mikolajczyk, K., & Schmid, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.188 · Cited by 6,723
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
A comparison of affine region detectors
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Gool, L. V. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-3848-x · Cited by 2,939
Cited by
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Spectral Networks and Locally Connected Networks on Graphs
Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6203 · Cited by 3,983
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Deep learning applications and challenges in big data analytics
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015) · Journal of Big Data · doi:10.1186/s40537-014-0007-7 · Cited by 2,594
Robust physical-world attacks on deep learning visual classification
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00175 · Cited by 2,173
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164
Unsupervised Learning of Visual Representations Using Videos
Wang, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.320 · Cited by 1,005
(2015) · Cited by 989
Deep hashing for compact binary codes learning
Liong, V. E., Lu, J., Wang, G., Moulin, P., & Zhou, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298862 · Cited by 545
Shin, H.-C., Orton, M. R., Collins, D. J., Doran, S. J., & Leach, M. O. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.277 · Cited by 488
ModDrop: Adaptive Multi-Modal Gesture Recognition
Neverova, N., Wolf, C., Taylor, G., & Nebout, F. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2461544 · Cited by 420
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
Buhrmester, V., Münch, D., & Arens, M. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3040048 · Cited by 403
Big Data: Deep Learning for financial sentiment analysis
Sohangir, S., Wang, D., Pomeranets, A., & Khoshgoftaar, T. M. (2018) · Journal of Big Data · doi:10.1186/s40537-017-0111-6 · Cited by 390
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Liu, M., Shan, S., Wang, R., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.226 · Cited by 392
Combining modality specific deep neural networks for emotion recognition in video
Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., Ferrari, R. C., et al. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531745 · Cited by 391
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298
Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images
Wu, G., Kim, M., Wang, Q., Gao, Y., Liao, S., & Shen, D. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_80 · Cited by 149
(2016) · MPG.PuRe (Max Planck Society) · Cited by 107
Multimodal emotional state recognition using sequence-dependent deep hierarchical features
Barros, P., Jirak, D., Weber, C., & Wermter, S. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.09.009 · Cited by 86
A Multi-scale Approach to Gesture Detection and Recognition
Neverova, N., Wolf, C., Paci, G., Sommavilla, G., Taylor, G. W., & Nebout, F. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.69 · Cited by 52
Multi-modal Unsupervised Feature Learning for RGB-D Scene Labeling
Wang, A., Lu, J., Wang, G., Cai, J., & Cham, T.-J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_30 · Cited by 50
Deep Learning for Video Classification: A Review
Rehman, A., & Belhaouari, S. B. (2021) · doi:10.36227/techrxiv.15172920 · Cited by 17
Metadata
- Authors
- Le, Q. V., Zou, W. Y., Yeung, S. Y., & Ng, A. Y.
- Venue
- CVPR 2011
- Publisher
- IEEE
- Published
- 2011-06
- Pages
- 3361-3368
- OpenAlex
- W1999192586
- Cited by
- 960
- References
- 12
- FWCI
- 65.29
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
