The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Towards Understanding Action Recognition
Abstract
Although action recognition in videos is widely studied, current methods often fail on real-world datasets. Many recent approaches improve accuracy and robustness to cope with challenging video sequences, but it is often unclear what affects the results most. This paper attempts to provide insights based on a systematic performance evaluation using thoroughly-annotated data of human actions. We annotate human Joints for the HMDB dataset (J-HMDB). This annotation can be used to derive ground truth optical flow and segmentation. We evaluate current methods using this dataset and systematically replace the output of various algorithms with ground truth. This enables us to discover what is important - for example, should we work on improving flow algorithms, estimating human bounding boxes, or enabling pose estimation? In summary, we find that high-level pose features greatly outperform low/mid level features, in particular, pose over time is critical, but current pose estimation algorithms are not yet reliable enough to provide this information. We also find that the accuracy of a top-performing action recognition framework can be greatly increased by refining the underlying low/mid level features, this suggests it is important to improve optical flow and human detection algorithms. Our analysis and J-HMDB dataset should facilitate a deeper understanding of action recognition algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Self-citationHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1×
“While marker-based motion capture systems provide accurate 3D ground-truth pose data [12, 15, 19, 25], they are impractical for recording realistic video data.”
BackgroundLearning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Self-citationTwo-Frame Motion Estimation Based on Polynomial Expansion
Farnebäck, G. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-45103-x_50 · Cited by 2,971
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Self-citationHuman Detection Using Oriented Histograms of Flow and Appearance
Dalal, N., Triggs, B., & Schmid, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744047_33 · Cited by 1,652
Self-citationMarszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
Self-citationSigal, L., Balan, A. O., & Black, M. J. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0273-6 · Cited by 1,164
Self-citationA survey of vision-based methods for action representation, segmentation and recognition
Weinland, D., Ronfard, R., & Boyer, E. (2011) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.10.002 · Cited by 997
Poselets: Body part detectors trained using 3D human pose annotations
Bourdev, L., & Malik, J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459303 · Cited by 961
"GrabCut": interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM SIGGRAPH 2004 Papers · doi:10.1145/1186562.1015720 · Cited by 935
Articulated Human Detection with Flexible Mixtures of Parts
Yang, Y., & Ramanan, D. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.261 · Cited by 852
Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation
Johnson, S., & Everingham, M. (2010) · Procedings of the British Machine Vision Conference 2010 · doi:10.5244/c.24.12 · Cited by 819
Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification
Niebles, J. C., Chen, C.-W., & Fei-Fei, L. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15552-9_29 · Cited by 700
Recognizing 50 human action categories of web videos
Reddy, K. K., & Shah, M. (2013) · Machine Vision and Applications · doi:10.1007/s00138-012-0450-4 · Cited by 655
Sun, D., Roth, S., & Black, M. J. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0644-x · Cited by 614
Self-citationProgressive search space reduction for human pose estimation
Ferrari, V., Marin-Jimenez, M., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587468 · Cited by 612
A database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521
Diagnosing Error in Object Detectors
Hoiem, D., Chodpathumwan, Y., & Dai, Q. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33712-3_25 · Cited by 520
Berkeley MHAD: A comprehensive Multimodal Human Action Database
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013) · 2013 IEEE Workshop on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2013.6474999 · Cited by 484
Detecting People Using Mutually Consistent Poselet Activations
Bourdev, L., Maji, S., Brox, T., & Malik, J. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15567-3_13 · Cited by 367
Parameterized Modeling and Recognition of Activities
Yacoob, Y., & Black, M. J. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1998.0726 · Cited by 310
Self-citationRecognition of human body motion using phase space constraints
Campbell, L. W., & Bobick, A. F. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466880 · Cited by 296
Human Pose Estimation Using Body Parts Dependent Joint Regressors
Dantone, M., Gall, J., Leistner, C., & Van Gool, L. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.391 · Cited by 287
Self-citationGuide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database
Torre, F. D. L., Hodgins, J. K., Bargteil, A. W., Martin, X., Macey, J., Collado, A., & Beltran, P. (2018) · KiltHub Repository · doi:10.1184/r1/6555020 · Cited by 228
Better appearance models for pictorial structures
Eichner, M., & Ferrari, V. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.3 · Cited by 219
Tenorth, M., Bandouch, J., & Beetz, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457583 · Cited by 214
Parsing human motion with stretchable models
Sapp, B., Weiss, D. A., & Taskar, B. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995607 · Cited by 143
Coupled Action Recognition and Pose Estimation from Multiple Views
Yao, A., Gall, J., & Gool, L. V. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0532-9 · Cited by 114
Self-citationFrom Pictorial Structures to deformable structures
Zuffi, S., Freifeld, O., & Black, M. J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248098 · Cited by 64
Self-citationThe role of features, algorithms and data in visual recognition
Parikh, D., & Zitnick, C. L. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539920 · Cited by 51
Action recognition in cluttered dynamic scenes using Pose-Specific Part Models
Singh, V. K., & Nevatia, R. (2011) · International Conference on Computer Vision · doi:10.1109/iccv.2011.6126232 · Cited by 37
Modeling Motion of Body Parts for Action Recognition
Tran, K., Kakadiaris, I. A., & Shah, S. K. (2011) · British Machine Vision Conference (BMVC) · doi:10.5244/c.25.64 · Cited by 24
Cited by
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification
Xie, S., Sun, C., Huang, J., Tu, Z., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_19 · Cited by 1,415
AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions
Gu, C., Sun, C., Ross, D. A., Vondrick, C., Pantofaru, C., Li, Y., Vijayanarasimhan, S., Toderici, G., Ricco, S., Sukthankar, R., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00633 · Cited by 986
Self-citationLi, D., Opazo, C. R., Yu, X., & Li, H. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093512 · Cited by 641
P-CNN: Pose-Based CNN Features for Action Recognition
Cheron, G., Laptev, I., & Schmid, C. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.368 · Cited by 633
Self-citation
Metadata
- Authors
- Jhuang, H., Gall, J., Zuffi, S., Schmid, C., & Black, M. J.
- Venue
- 2013 IEEE International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2013-12
- Pages
- 3192-3199
- OpenAlex
- W2034014085
- Cited by
- 879
- References
- 36
- FWCI
- 20.27
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.