The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Bag of visual words and fusion methods for action recognition: Comprehensive study and good practice
Open accessAbstract
A Comprehensive study on the BoVW pipeline for action recognition task.An evaluation and a generic analysis on 13 encoding methods.Intra-normalization for supervector based encoding methods.An evaluation on three fusion methods.Several good practices of the BoVW pipeline for action recognition task. Video based action recognition is one of the important and challenging problems in computer vision research. Bag of visual words model (BoVW) with local features has been very popular for a long time and obtained the state-of-the-art performance on several realistic datasets, such as the HMDB51, UCF50, and UCF101. BoVW is a general pipeline to construct a global representation from local features, which is mainly composed of five steps; (i) feature extraction, (ii) feature pre-processing, (iii) codebook generation, (iv) feature encoding, and (v) pooling and normalization. Although many efforts have been made in each step independently in different scenarios, their effects on action recognition are still unknown. Meanwhile, video data exhibits different views of visual patterns , such as static appearance and motion dynamics. Multiple descriptors are usually extracted to represent these different views. Fusing these descriptors is crucial for boosting the final performance of an action recognition system. This paper aims to provide a comprehensive study of all steps in BoVW and different fusion methods, and uncover some good practices to produce a state-of-the-art action recognition system. Specifically, we explore two kinds of local features, ten kinds of encoding methods, eight kinds of pooling and normalization strategies, and three kinds of fusion methods. We conclude that every step is crucial for contributing to the final recognition rate and improper choice in one of the steps may counteract the performance improvement of other steps. Furthermore, based on our comprehensive study, we propose a simple yet effective representation, called hybrid supervector, by exploring the complementarity of different BoVW frameworks with improved dense trajectories. Using this representation, we obtain impressive results on the three challenging datasets; HMDB51 (61.9%), UCF50 (92.3%), and UCF101 (87.9%).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit
Tropp, J. A., & Gilbert, A. C. (2007) · IEEE Transactions on Information Theory · doi:10.1109/tit.2007.909108 · Cited by 9,653
On Spectral Clustering: Analysis and an algorithm
Ng, A. Y., Jordan, M. I., & Weiss, Y. (2001) · Cited by 8,159
(2006) · Springer eBooks · Cited by 8,123
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2017) · doi:10.1109/cvpr35066.2017 · Cited by 7,141
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Hierarchical Clustering Schemes
Johnson, S. C. (1967) · Psychometrika · doi:10.1007/bf02289588 · Cited by 4,897
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Locality-constrained Linear Coding for image classification
Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540018 · Cited by 3,100 · cited 5×
“Reconstruction based encoding methods [61,50,63,42] are designed from the perspective of decoding process, where the codes s are enforced to reconstruct the input descriptor x.”
InfluentialLinear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., & Huang, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206757 · Cited by 2,828
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588
Behavior Recognition via Sparse Spatio-Temporal Features
Dollar, P., Rabaud, V., Cottrell, G., & Belongie, S. (2006) · 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance · doi:10.1109/vspets.2005.1570899 · Cited by 2,460
Efficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
Bruckstein, A. M., Donoho, D. L., & Elad, M. (2009) · SIAM Review · doi:10.1137/060657704 · Cited by 2,183
Human activity analysis: A review
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,056
Zhang, J., Marszałek, M., Lazebnik, S., & Schmid, C. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-9794-4 · Cited by 1,923
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszalek, M., & Schmid, C. (2008) · Procedings of the British Machine Vision Conference 2008 · doi:10.5244/c.22.99 · Cited by 1,846
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Aggregating Local Image Descriptors into Compact Codes
Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.235 · Cited by 1,503
Machine Recognition of Human Activities: A Survey
Turaga, P., Chellappa, R., Subrahmanian, V. S., & Udrea, O. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2008.2005594 · Cited by 1,333
Evaluation of local spatio-temporal features for action recognition
Wang, H., Ullah, M. M., Klaser, A., Laptev, I., & Schmid, C. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.124 · Cited by 1,296
Exploiting Generative Models in Discriminative Classifiers
Jaakkola, T., & Haussler, D. H. (1998) · Cited by 1,271
A Theoretical Analysis of Feature Pooling in Visual Recognition
Boureau, Y.-L., Ponce, J., & LeCun, Y. (2010) · Cited by 1,196
Action recognition with trajectory-pooled deep-convolutional descriptors
Wang, L., Qiao, Y., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299059 · Cited by 1,194
Self-citationArticulated pose estimation with flexible mixtures-of-parts
Yang, Y., & Ramanan, D. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995741 · Cited by 990
An Efficient Dense and Scale-Invariant Spatio-Temporal Interest Point Detector
Willems, G., Tuytelaars, T., & Van Gool, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88688-4_48 · Cited by 869
The devil is in the details: an evaluation of recent feature encoding methods
Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.76 · Cited by 834
ImageNet classification with deep convolutional neural networks
Krizhevsky (2012) · Medical Entomology and Zoology · Cited by 799
On feature combination for multiclass object classification
Gehler, P., & Nowozin, S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459169 · Cited by 797
Multiple kernels for object detection
Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459183 · Cited by 757
Nonlinear Learning using Local Coordinate Coding
Yu, K., Zhang, T., & Gong, Y. (2009) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 755
van Gemert, J. C., Veenman, C. J., Smeulders, A. W. M., & Geusebroek, J.-M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.132 · Cited by 737
Arandjelovic, R., & Zisserman, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.207 · Cited by 713
Action bank: A high-level representation of activity in video
Sadanand, S., & Corso, J. J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247806 · Cited by 691
Efficient Additive Kernels via Explicit Feature Maps
Vedaldi, A., & Zisserman, A. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.153 · Cited by 688
Recognizing 50 human action categories of web videos
Reddy, K. K., & Shah, M. (2013) · Machine Vision and Applications · doi:10.1007/s00138-012-0450-4 · Cited by 655
P-CNN: Pose-Based CNN Features for Action Recognition
Cheron, G., Laptev, I., & Schmid, C. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.368 · Cited by 633
Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks
Sun, L., Jia, K., Yeung, D.-Y., & Shi, B. E. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.522 · Cited by 577
Analyzing and recognizing walking figures in XYT
Niyogi, & Adelson (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323868 · Cited by 572
Image Classification Using Super-Vector Coding of Local Image Descriptors
Zhou, X., Yu, K., Zhang, T., & Huang, T. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_11 · Cited by 499
In defense of soft-assignment coding
Liu, L., Wang, L., & Liu, X. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126534 · Cited by 443
Self-citationVideo google: A text retrieval approach to object matching in videos
Šivic, J., & Zisserman, A. (2003) · Cited by 436
Action Recognition with Stacked Fisher Vectors
Peng, X., Zou, C., Qiao, Y., & Peng, Q. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_38 · Cited by 338
Self-citationParameterized Modeling and Recognition of Activities
Yacoob, Y., & Black, M. J. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1998.0726 · Cited by 310
Recognition of human body motion using phase space constraints
Campbell, L. W., & Bobick, A. F. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466880 · Cited by 296
Feature Coding in Image Classification: A Comprehensive Study
Huang, Y., Wu, Z., Wang, L., & Tan, T. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.113 · Cited by 242
Self-citationStructure from motion of rigid and jointed objects
Webb, J. A., & Aggarwal, J. K. (1982) · Artificial Intelligence · doi:10.1016/0004-3702(82)90023-6 · Cited by 213
Multi-view Super Vector for Action Recognition
Cai, Z., Wang, L., Peng, X., & Qiao, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.83 · Cited by 205
Self-citationTrajectory-Based Modeling of Human Actions with Motion Reference Points
Jiang, Y.-G., Dai, Q., Xue, X., Liu, W., & Ngo, C.-W. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_31 · Cited by 203
Motion Interchange Patterns for Action Recognition in Unconstrained Videos
Kliper-Gross, O., Gurovich, Y., Hassner, T., & Wolf, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_19 · Cited by 184
Motionlets: Mid-level 3D Parts for Human Motion Recognition
Wang, L., Qiao, Y., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.345 · Cited by 180
Self-citationA spatio-temporal descriptor based on 3D-gradients
Kläser, A., Marszałek, M., & Schmid, C. (2008) · Cited by 180
Multiple Kernel Learning and the SMO Algorithm
Sun, Z., Theera-Ampornpunt, N., Varma, M., et al. (2010) · neural information processing systems · Cited by 154
Salient coding for image classification
Huang, Y., Huang, K., Yu, Y., & Tan, T. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995682 · Cited by 141
Improved Local Coordinate Coding using Local Tangents
Yu, K., & Zhang, T. (2010) · Cited by 136
Tamrakar, A., Ali, S., Yu, Q., Liu, J., Javed, O., Divakaran, A., Cheng, H., & Sawhney, H. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248114 · Cited by 129
A Comparative Study of Encoding, Pooling and Normalization Methods for Action Recognition
Wang, X., Wang, L., & Qiao, Y. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37431-9_44 · Cited by 123
Self-citationLatent Hierarchical Model of Temporal Structure for Complex Activity Classification
Wang, L., Qiao, Y., & Tang, X. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2295753 · Cited by 102
Self-citationClassifying web videos using a global video descriptor
Solmaz, B., Assari, S. M., & Shah, M. A. (2012) · Machine Vision and Applications · doi:10.1007/s00138-012-0449-x · Cited by 94
Combining the Right Features for Complex Event Recognition
Tang, K., Yao, B., Fei-Fei, L., & Koller, D. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.335 · Cited by 90
Mining Motion Atoms and Phrases for Complex Action Recognition
Wang, L., Qiao, Y., & Tang, X. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.333 · Cited by 77
Self-citationTowards Good Practices for Action Video Encoding
Wu, J., Zhang, Y., & Lin, W. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.330 · Cited by 67
Ordered Trajectories for Large Scale Human Action Recognition
Murthy, O. V. R., & Goecke, R. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.61 · Cited by 45
Peng, X., Qiao, Y., Peng, Q., & Qi, X. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.59 · Cited by 43
Self-citationEvaluating multimedia features and fusion for example-based event detection
Myers, G. K., Nallapati, R. M., van Hout, J., et al. (2013) · Machine Vision and Applications · doi:10.1007/s00138-013-0527-8 · Cited by 31
Karaman, S., Seidenari, L., Bagdanov, A. D., & Bimbo, A. D. (2013) · Cited by 19
A Joint Evaluation of Dictionary Learning and Feature Encoding for Action Recognition
Peng, X., Wang, L., Qiao, Y., & Peng, Q. (2014) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2014.450 · Cited by 15
Self-citationHybrid Super Vector with Improved Dense Trajectories for Action Recognition
Peng, X., Wang, L., Cai, Z., 乔宇, & Peng, Q. (2013) · Cited by 15
Self-citation
Cited by
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Self-citationConvolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.213 · Cited by 2,684
A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks
Yin, C., Zhu, Y., Fei, J., & He, X. (2017) · IEEE Access · doi:10.1109/access.2017.2762418 · Cited by 2,100
Metadata
- Authors
- Peng, X., Wang, L., Wang, X., & Qiao, Y.
- Venue
- Computer Vision and Image Understanding
- Publisher
- Elsevier BV
- Published
- 2016-09
- Volume
- 150
- Pages
- 109-125
- ISSN
- 1077-3142 (ISSN-L)
- OpenAlex
- W1871385855
- Cited by
- 677
- References
- 77
- FWCI
- 87.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.