The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
A Robust and Efficient Video Representation for Action Recognition
Open accessAbstract
This paper introduces a state-of-the-art video representation and applies it to efficient action recognition and detection. We first propose to improve the popular dense trajectory features by explicit camera motion estimation. More specifically, we extract feature point matches between frames using SURF descriptors and dense optical flow. The matches are used to estimate a homography with RANSAC. To improve the robustness of homography estimation, a human detector is employed to remove outlier matches from the human body as human motion is not constrained by the camera. Trajectories consistent with the homography are considered as due to camera motion, and thus removed. We also use the homography to cancel out camera motion from the optical flow. This results in significant improvement on motion-based HOF and MBH descriptors. We further explore the recent Fisher vector as an alternative feature encoding approach to the standard bag-of-words (BOW) histogram, and consider different ways to include spatial layout information in these encodings. We present a large and varied set of evaluations, considering (i) classification of short basic actions on six datasets, (ii) localization of such actions in feature-length movies, and (iii) large-scale recognition of complex events. We find that our improved trajectory features significantly outperform previous dense trajectories, and that Fisher vectors are superior to BOW encodings for video recognition tasks. In all three tasks, we show substantial improvements over the state-of-the-art results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955 · cited 1×
“We extract speeded-up robust features (SURF; Bay et al, 2006) and match them based on the nearest neighbor rule.”
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Shi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724 · cited 4×
“We select motion vectors for salient feature points using the good-features-totrack criterion (Shi and Tomasi, 1994), i.”
InfluentialUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Self-citationAction Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Self-citationRecognizing human actions: a local SVM approach
Schuldt, C., Laptev, I., & Caputo, B. (2004) · Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. · doi:10.1109/icpr.2004.1334462 · Cited by 3,010
Two-Frame Motion Estimation Based on Polynomial Expansion
Farnebäck, G. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-45103-x_50 · Cited by 2,971
Recognizing human actions: A local SVM approach
Schüldt, C., Laptev, I., & Barbara Caputo, B. (2004) · Cited by 2,906
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Behavior Recognition via Sparse Spatio-Temporal Features
Dollar, P., Rabaud, V., Cottrell, G., & Belongie, S. (2006) · 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance · doi:10.1109/vspets.2005.1570899 · Cited by 2,460
International Journal of Computer Vision
(2013) · International Journal of Computer Vision · doi:10.1007/11263.1573-1405 · Cited by 2,340
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszalek, M., & Schmid, C. (2008) · Procedings of the British Machine Vision Conference 2008 · doi:10.5244/c.22.99 · Cited by 1,846
Self-citationDense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Self-citationHuman Detection Using Oriented Histograms of Flow and Appearance
Dalal, N., Triggs, B., & Schmid, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744047_33 · Cited by 1,652
Self-citationA 3-dimensional sift descriptor and its application to action recognition
Scovanner, P., Ali, S., & Shah, M. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291311 · Cited by 1,596
Image Alignment and Stitching: A Tutorial
Szeliski, R. (2007) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000009 · Cited by 1,593
Aggregating Local Image Descriptors into Compact Codes
Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.235 · Cited by 1,503
Self-citationImage Classification with the Fisher Vector: Theory and Practice
Sánchez, J., Perronnin, F., Mensink, T., & Verbeek, J. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0636-x · Cited by 1,498
Self-citationGorelick, L., Blank, M., Shechtman, E., Irani, M., & Basri, R. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70711 · Cited by 1,430
Three things everyone should know to improve object retrieval
Arandjelovic, R., & Zisserman, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248018 · Cited by 1,383
Evaluation of local spatio-temporal features for action recognition
Wang, H., Ullah, M. M., Klaser, A., Laptev, I., & Schmid, C. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.124 · Cited by 1,296
Self-citationMarszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
Self-citationRecognizing realistic actions from videos “in the wild”
Liu, J., Luo, J., & Shah, M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206744 · Cited by 1,075
Le, Q. V., Zou, W. Y., Yeung, S. Y., & Ng, A. Y. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995496 · Cited by 960
An Efficient Dense and Scale-Invariant Spatio-Temporal Interest Point Detector
Willems, G., Tuytelaars, T., & Van Gool, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88688-4_48 · Cited by 869
The devil is in the details: an evaluation of recent feature encoding methods
Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.76 · Cited by 834
van Gemert, J. C., Veenman, C. J., Smeulders, A. W. M., & Geusebroek, J.-M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.132 · Cited by 737
Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification
Niebles, J. C., Chen, C.-W., & Fei-Fei, L. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15552-9_29 · Cited by 700
Recognizing 50 human action categories of web videos
Reddy, K. K., & Shah, M. (2013) · Machine Vision and Applications · doi:10.1007/s00138-012-0450-4 · Cited by 655
Progressive search space reduction for human pose estimation
Ferrari, V., Marin-Jimenez, M., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587468 · Cited by 612
Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition
Gupta, A., Kembhavi, A., & Davis, L. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.83 · Cited by 502
Activity recognition using the velocity histories of tracked keypoints
Messing, R., Pal, C., & Kautz, H. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459154 · Cited by 455
Laptev, I., & Perez, P. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409105 · Cited by 416
Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking
Gauglitz, S., Höllerer, T., & Turk, M. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0431-5 · Cited by 397
Better Exploiting Motion for Better Action Recognition
Jain, M., Jegou, H., & Bouthemy, P. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.330 · Cited by 385
Action and Event Recognition with Fisher Vectors on a Compact Feature Set
Oneata, D., Verbeek, J., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.228 · Cited by 376
Self-citationHierarchical spatio-temporal context modeling for action recognition
Sun, J., Wu, X., Yan, S., Cheong, L.-F., Chua, T.-S., & Li, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206721 · Cited by 358
Learning latent temporal structure for complex event detection
Tang, K., Fei-Fei, L., & Koller, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247808 · Cited by 355
Local Trinary Patterns for human action recognition
Yeffet, L., & Wolf, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459201 · Cited by 339
Object, Scene and Actions: Combining Multiple Features for Human Action Recognition
Ikizler-Cinbis, N., & Sclaroff, S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15549-9_36 · Cited by 285
Automatic annotation of human actions in video
Duchenne, O., Laptev, I., Sivic, J., Bach, F., & Ponce, J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459279 · Cited by 275
Weakly Supervised Learning of Interactions between Humans and Objects
Prest, A., Schmid, C., & Ferrari, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.158 · Cited by 225
Self-citationTrajectons: Action recognition through the motion analysis of tracked features
Matikainen, P., Hebert, M., & Sukthankar, R. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457659 · Cited by 224
Trajectory-Based Modeling of Human Actions with Motion Reference Points
Jiang, Y.-G., Dai, Q., Xue, X., Liu, W., & Ngo, C.-W. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_31 · Cited by 203
Multimodal feature fusion for robust event detection in web videos
Natarajan, P., Wu, S., Vitaladevuni, S., Zhuang, X., Tsakalidis, S., Park, U., Prasad, R., & Natarajan, P. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247814 · Cited by 178
Wu, S., Oreifej, O., & Shah, M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126397 · Cited by 165
Modeling spatial layout with fisher vectors for image categorization
Krapac, J., Verbeek, J., & Jurie, F. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126406 · Cited by 163
Self-citationDynamic Eye Movement Datasets and Learnt Saliency Models for Visual Action Recognition
Mathe, S., & Sminchisescu, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_60 · Cited by 154
Actom sequence models for efficient action detection
Gaidon, A., Harchaoui, Z., & Schmid, C. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995646 · Cited by 145
Self-citationHuman Focused Action Localization in Video
Kläser, A., Marszałek, M., Schmid, C., & Zisserman, A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35749-7_17 · Cited by 137
Self-citationA Comparative Study of Encoding, Pooling and Normalization Methods for Action Recognition
Wang, X., Wang, L., & Qiao, Y. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37431-9_44 · Cited by 134
High Five: Recognising human interactions in TV shows
Patron-Perez, A., Marszałek, M., Zisserman, A., & Reid, I. D. (2010) · British Machine Vision Conference (BMVC) · doi:10.5244/c.24.50 · Cited by 134
Exploring Weak Stabilization for Motion Feature Extraction
Park, D., Zitnick, C. L., Ramanan, D., & Dollár, P. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.371 · Cited by 128
Activity representation with motion hierarchies
Gaidon, A., Harchaoui, Z., & Schmid, C. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0677-1 · Cited by 126
Self-citationSpace-Variant Descriptor Sampling for Action Recognition Based on Saliency and Eye Movements
Vig, E., Dörr, M., & Cox, D. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33786-4_7 · Cited by 124
Explicit Modeling of Human-Object Interactions in Realistic Videos
Prest, A., Ferrari, V., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.175 · Cited by 123
Self-citationModeling the spatial layout of images beyond spatial pyramids
Sánchez, J., Perronnin, F., & de Campos, T. E. (2012) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.07.019 · Cited by 120
Sampling Strategies for Real-Time Action Recognition
Shi, F., Petriu, E. M., & Laganière, R. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.335 · Cited by 110
Representing Pairwise Spatial and Temporal Relations for Action Recognition
Matikainen, P., Hebert, M., & Sukthankar, R. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_37 · Cited by 109
Action Recognition with Actons
Zhu, J., Wang, B., Yang, X., Zhang, W., & Tu, Z. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.442 · Cited by 98
Recognizing Complex Events Using Large Margin Joint Low-Level Event Model
Izadinia, H., & Shah, M. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_31 · Cited by 94
Combining the Right Features for Complex Event Recognition
Tang, K., Yao, B., Fei-Fei, L., & Koller, D. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.335 · Cited by 90
Large-scale web video event classification by use of Fisher Vectors
Sun, C., & Nevatia, R. (2013) · Proceedings · doi:10.1109/wacv.2013.6474994 · Cited by 80
Feature Tracking and Motion Compensation for Action Recognition
Uemura, H., Ishikawa, S., & Mikolajczyk, K. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.30 · Cited by 79
Mining Motion Atoms and Phrases for Complex Action Recognition
Wang, L., Qiao, Y., & Tang, X. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.333 · Cited by 77
Complex Events Detection Using Data-Driven Concepts
Yang, Y., & Shah, M. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33712-3_52 · Cited by 58
Compositional Models for Video Event Detection: A Multiple Kernel Learning Latent Variable Approach
Vahdat, A., Cannons, K. J., Mori, G., Oh, S., & Kim, I. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.463 · Cited by 49
Scene Aligned Pooling for Complex Video Recognition
Cao, L., Mu, Y., Natsev, A., Chang, S., Hua, G., & Smith, J. R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_49 · Cited by 48
Spatially Local Coding for Object Recognition
McCann, S., & Lowe, D. G. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37331-2_16 · Cited by 47
Ordered Trajectories for Large Scale Human Action Recognition
Murthy, O. V. R., & Goecke, R. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.61 · Cited by 47
Dynamic Pooling for Complex Event Recognition
Li, W., Yu, Q., Divakaran, A., & Vasconcelos, N. M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.339 · Cited by 47
Space-Time Robust Representation for Action Recognition
Ballas, N., Yang, Y., Lan, Z., Delezoide, B., Preteux, F. J., & Hauptmann, A. G. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.336 · Cited by 40
Learning discriminative space-time actions from weakly labelled videos
Sapienza, M., Fabio Cuzzolin, A., & Torr, P. H. S. (2012) · British Machine Vision Conference (BMVC) · doi:10.5244/c.26.123 · Cited by 37
Handling Uncertain Tags in Visual Recognition
Vahdat, A., & Mori, G. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.462 · Cited by 20
Karaman, S., Seidenari, L., Bagdanov, A. D., & Bimbo, A. D. (2013) · Cited by 19
Hybrid Super Vector with Improved Dense Trajectories for Action Recognition
Peng, X., Wang, L., Cai, Z., 乔宇, & Peng, Q. (2013) · Cited by 15
Space-Time Robust Video Representation for Action Recognition
Ballas, N., Yang, Y., Lan, Z., Delezoide, B., Preteux, F. J., & Hauptmann, A. G. (2013) · Cited by 3
Cited by
Human Action Recognition and Prediction: A Survey
Kong, Y., & Fu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01594-9 · Cited by 780
Molchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., & Kautz, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.456 · Cited by 695
A Comprehensive Survey of Vision-Based Human Action Recognition Methods
Zhang, H.-B., Zhang, Y.-X., Zhong, B., Lei, Q., Yang, L., Du, J.-X., & Chen, D.-S. (2019) · Sensors · doi:10.3390/s19051005 · Cited by 503
“[63] fused the results of human detection into iDT features, thereby reducing the interference of background trajectories.”
BackgroundOnline Real-Time Multiple Spatiotemporal Action Localisation and Prediction
Singh, G., Saha, S., Sapienza, M., Torr, P., & Cuzzolin, F. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.393 · Cited by 279
Learning to Track for Spatio-Temporal Action Localization
Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.362 · Cited by 278
Self-citation
Metadata
- Authors
- Wang, H., Oneata, D., Verbeek, J., & Schmid, C.
- Venue
- International Journal of Computer Vision
- Published
- 2015-07-17
- Volume
- 119
- Issue
- 3
- Pages
- 219-238
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W914561379
- Cited by
- 317
- References
- 81
- FWCI
- 42.17
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.