The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Recognizing Human Actions as the Evolution of Pose Estimation Maps
Open accessAbstract
Most video-based action recognition approaches choose to extract features from the whole video to recognize actions. The cluttered background and non-action motions limit the performances of these methods, since they lack the explicit modeling of human body movements. With recent advances of human pose estimation, this work presents a novel method to recognize human action as the evolution of pose estimation maps. Instead of relying on the inaccurate human poses estimated from videos, we observe that pose estimation maps, the byproduct of pose estimation, preserve richer cues of human body to benefit action recognition. Specifically, the evolution of pose estimation maps can be decomposed as an evolution of heatmaps, e.g., probabilistic maps, and an evolution of estimated 2D human poses, which denote the changes of body shape and body pose, respectively. Considering the sparse property of heatmap, we develop spatial rank pooling to aggregate the evolution of heatmaps as a body shape evolution image. As body shape evolution image does not differentiate body parts, we design body guided sampling to aggregate the evolution of poses as a body pose evolution image. The complementary properties between both types of images are explored by deep convolutional neural networks to predict action label. Experiments on NTU RGB+D, UTD-MHAD and PennAction datasets verify the effectiveness of our method, which outperforms most state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Microsoft Kinect Sensor and Its Effect
Zhang, Z. (2012) · IEEE Multimedia · doi:10.1109/mmul.2012.24 · Cited by 2,487
Action recognition by dense trajectories
Wang, H., Klaser, A., Schmid, C., & Liu, C.-L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995407 · Cited by 2,154
Real-time human pose recognition in parts from single depth images
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., & Moore, R. (2013) · Communications of the ACM · doi:10.1145/2398356.2398381 · Cited by 2,044
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.82 · Cited by 1,517
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
Oreifej, O., & Liu, Z. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.98 · Cited by 956
Articulated Human Detection with Flexible Mixtures of Parts
Yang, Y., & Ramanan, D. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.261 · Cited by 852
Enhanced skeleton visualization for view invariant human action recognition
Liu, M., Liu, H., & Chen, C. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.02.030 · Cited by 842
Self-citationHuman Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805
A New Representation of Skeleton Sequences for 3D Action Recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaid, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.486 · Cited by 793
Chen, C., Jafari, R., & Kehtarnavaz, N. (2015) · 2015 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2015.7350781 · Cited by 765
Global Context-Aware Attention LSTM Networks for 3D Action Recognition
Liu, J., Wang, G., Hu, P., Duan, L.-Y., & Kot, A. C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.391 · Cited by 682
Spatiotemporal Multiplier Networks for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Wildes, R. P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.787 · Cited by 679
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.331 · Cited by 571
Modeling video evolution for action recognition
Fernando, B., Gavves, E., Jose Oramas, M., Ghodrati, A., & Tuytelaars, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299176 · Cited by 569
Gkioxari, G., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298676 · Cited by 528
Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations
Hussein, M. E., Torki, M., Gowayyed, M. A., & El-Saban, M. (2013) · Cited by 507
Human Pose Estimation via Convolutional Part Heatmap Regression
Bulat, A., & Tzimiropoulos, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_44 · Cited by 501
A new representation of skeleton sequences for 3D action recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaïd, F. (2017) · UWA Profiles and Research Repository (UWA) · Cited by 480
Super Normal Vector for Activity Recognition Using Depth Sequences
Yang, X., & Tian, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.108 · Cited by 388
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2640292 · Cited by 383
Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks
Wang, P., Li, Z., Hou, Y., & Li, W. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967191 · Cited by 383
An Approach to Pose-Based Action Recognition
Wang, C., Wang, Y., & Yuille, A. L. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.123 · Cited by 373
From Actemes to Action: A Strongly-Supervised Representation for Detailed Action Understanding
Zhang, W., Zhu, M., & Derpanis, K. G. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.280 · Cited by 361
Jointly learning heterogeneous features for RGB-D activity recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299172 · Cited by 358
Skeleton Optical Spectra-Based Action Recognition Using Convolutional Neural Networks
Hou, Y., Li, Z., Wang, P., & Li, W. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2628339 · Cited by 339
Joint Distance Maps Based Action Recognition With Convolutional Neural Networks
Li, C., Hou, Y., Wang, P., & Li, W. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2678539 · Cited by 283
Deep Learning on Lie Groups for Skeleton-Based Action Recognition
Huang, Z., Wan, C., Probst, T., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.137 · Cited by 276
Going Deeper into First-Person Activity Recognition
Ma, M., Fan, H., & Kitani, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.209 · Cited by 274
Skeletal Quads: Human Action Recognition Using Joint Quadruples
Evangelidis, G., Singh, G., & Horaud, R. (2014) · 2014 22nd International Conference on Pattern Recognition · doi:10.1109/icpr.2014.772 · Cited by 272
Spatiotemporal Pyramid Network for Video Action Recognition
Wang, Y., Long, M., Wang, J., & Yu, P. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.226 · Cited by 272
Beyond Gaussian Pyramid: Multi-skip Feature Stacking for action recognition
Lan, Z., Lin, M., Li, X., Hauptmann, A. G., & Raj, B. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298616 · Cited by 264
Multi-stream CNN: Learning representations based on human-related regions for action recognition
Tu, Z., Xie, W., Qin, Q., Poppe, R., Veltkamp, R. C., Li, B., & Yuan, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.01.020 · Cited by 247
Self-citationA Key Volume Mining Deep Framework for Action Recognition
Zhu, W., Hu, J., Sun, G., Cao, X., & Qiao, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.219 · Cited by 244
Pose Machines: Articulated Pose Estimation via Inference Machines
Ramakrishna, V., Munoz, D., Hebert, M., Andrew Bagnell, J., & Sheikh, Y. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_3 · Cited by 241
RPAN: An End-to-End Recurrent Pose-Attention Network for Action Recognition in Videos
Du, W., Wang, Y., & Qiao, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.402 · Cited by 199
First Person Action Recognition Using Deep Learned Descriptors
Singh, S., Arora, C., & Jawahar, C. V. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.287 · Cited by 192
Unsupervised Learning of Long-Term Motion Dynamics for Videos
Luo, Z., Peng, B., Huang, D.-A., Alahi, A., & Fei-Fei, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.751 · Cited by 189
Joint action recognition and pose estimation from video
Nie, B. X., Xiong, C., & Zhu, S.-C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298734 · Cited by 188
Discriminative Orderlet Mining for Real-Time Recognition of Human-Object Interaction
Yu, G., Liu, Z., & Yuan, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16814-2_4 · Cited by 156
Self-citationSpatio-Temporal Naive-Bayes Nearest-Neighbor (ST-NBNN) for Skeleton-Based Action Recognition
Weng, J., Weng, C., & Yuan, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.55 · Cited by 152
Self-citationAction Recognition Using 3D Histograms of Texture and A Multi-Class Boosting Classifier
Zhang, B., Yang, Y., Chen, C., Yang, L., Han, J., & Shao, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2718189 · Cited by 150
Learning Action Recognition Model from Depth and Skeleton Videos
Rahmani, H., & Bennamoun, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.621 · Cited by 125
Pose for Action - Action for Pose
Iqbal, U., Garbade, M., & Gall, J. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.61 · Cited by 115
Garcia-Hernando, G., & Kim, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.51 · Cited by 85
Depth Context: a new descriptor for human activity recognition by using sole depth sequences
Liu, M., & Liu, H. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.11.005 · Cited by 52
Self-citationSpace-time tree ensemble for action recognition
Ma, S., Sigal, L., & Sclaroff, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299137 · Cited by 43
Learning directional co-occurrence for human action classification
Liu, H., Liu, M., & Sun, Q. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853794 · Cited by 18
Self-citation
Cited by
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition
Liu, Z., Zhang, H., Chen, Z., Wang, Z., & Ouyang, W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00022 · Cited by 1,225
Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00119 · Cited by 647
Skeleton-Based Action Recognition With Multi-Stream Adaptive Graph Convolutional Networks
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3028207 · Cited by 553
MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation
Li, W., Liu, H., Tang, H., Wang, P., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01280 · Cited by 458
Metadata
- Authors
- Liu, M., & Yuan, J.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 1159-1168
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2799211965
- Cited by
- 336
- References
- 55
- FWCI
- 12.85
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.