The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
Learning Clip Representations for Skeleton-Based 3D Action Recognition
Abstract
This paper presents a new representation of skeleton sequences for 3D action recognition. Existing methods based on hand-crafted features or recurrent neural networks cannot adequately capture the complex spatial structures and the long-term temporal dynamics of the skeleton sequences, which are very important to recognize the actions. In this paper, we propose to transform each channel of the 3D coordinates of a skeleton sequence into a clip. Each frame of the generated clip represents the temporal information of the entire skeleton sequence and one particular spatial relationship between the skeleton joints. The entire clip incorporates multiple frames with different spatial relationships, which provide useful spatial structural information of the human skeleton. We also propose a multitask convolutional neural network (MTCNN) to learn the generated clips for action recognition. The proposed MTCNN processes all the frames of the generated clips in parallel to explore the spatial and temporal information of the skeleton sequences. The proposed method has been extensively tested on six challenging benchmark datasets. Experimental results consistently demonstrate the superiority of the proposed clip representation and the feature learning method for 3D action recognition compared to the existing techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Smoothing and Differentiation of Data by Simplified Least Squares Procedures.
Savitzky, A., & Golay, M. J. E. (1964) · Analytical Chemistry · doi:10.1021/ac60214a047 · Cited by 21,904
(2010) · International Conference on Machine Learning · Cited by 13,995
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
A Mathematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. S. (2017) · CRC Press eBooks · doi:10.1201/9781315136370 · Cited by 6,694
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Multi-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Sainath, T. N., Vinyals, O., Senior, A., & Sak, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178838 · Cited by 1,678
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
Luo, W., Li, Y., Urtasun, R., & Zemel, R. (2017) · arXiv · doi:10.48550/arxiv.1701.04128 · Cited by 1,575
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.82 · Cited by 1,517
Mining actionlet ensemble for action recognition with depth cameras
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247813 · Cited by 1,512
View invariant human action recognition using histograms of 3D joints
Xia, L., Chen, C.-C., & Aggarwal, J. K. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239233 · Cited by 1,452
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
Caruana, R. (1998) · Cited by 1,110
Temporal Segment Networks for Action Recognition in Videos
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2868668 · Cited by 918
Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., & Xie, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10451 · Cited by 915
MatchNet: Unifying feature and metric learning for patch-based matching
Han, X., Leung, T., Jia, Y., Sukthankar, R., & Berg, A. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298948 · Cited by 825
A New Representation of Skeleton Sequences for 3D Action Recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaid, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.486 · Cited by 793
Self-citationCross-View Action Modeling, Learning, and Recognition
Wang, J., Nie, X., Xia, Y., Wu, Y., & Zhu, S.-C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.339 · Cited by 659
(2014) · International Conference on Learning Representations · Cited by 634
Caruana, R. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_5 · Cited by 577
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks
Liu, J., Wang, G., Duan, L.-Y., Abdiyeva, K., & Kot, A. C. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2785279 · Cited by 537
Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks
Liu, J., Wang, G., Duan, L.-Y., Abdiyeva, K., & Kot, A. C. (2018) · Lancaster EPrints (Lancaster University) · Cited by 531
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Liu, J., Shahroudy, A., Xu, D., Kot, A. C., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2771306 · Cited by 530
Two-person interaction detection using body-pose features and multiple instance learning
Yun, K., Honorio, J., Chattopadhyay, D., Berg, T. L., & Samaras, D. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239234 · Cited by 523
Differential Recurrent Neural Networks for Action Recognition
Veeriah, V., Zhuang, N., & Qi, G.-J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.460 · Cited by 516
Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations
Hussein, M. E., Torki, M., Gowayyed, M. A., & El-Saban, M. (2013) · Cited by 507
Berkeley MHAD: A comprehensive Multimodal Human Action Database
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013) · 2013 IEEE Workshop on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2013.6474999 · Cited by 484
A new representation of skeleton sequences for 3D action recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaïd, F. (2017) · UWA Profiles and Research Repository (UWA) · Cited by 480
Self-citationEigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor
Yang, X., & Tian, Y. L. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239232 · Cited by 439
Super Normal Vector for Activity Recognition Using Depth Sequences
Yang, X., & Tian, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.108 · Cited by 388
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2640292 · Cited by 383
Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks
Wang, P., Li, Z., Hou, Y., & Li, W. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967191 · Cited by 383
An Approach to Pose-Based Action Recognition
Wang, C., Wang, Y., & Yuille, A. L. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.123 · Cited by 373
Jointly learning heterogeneous features for RGB-D activity recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299172 · Cited by 358
Recognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost
Lv, F., & Nevatia, R. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_28 · Cited by 351
3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold
Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., & Del Bimbo, A. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2350774 · Cited by 336
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.04.007 · Cited by 318
Space-time representation of people based on 3D skeletal data: A review
Han, F., Reily, B., Hoff, W., & Zhang, H. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.01.011 · Cited by 316
Robotic manipulators and the product of exponentials formula
Brockett, R. W. (2005) · Lecture Notes in Control and Information Sciences · doi:10.1007/bfb0031048 · Cited by 314
Action recognition from a distributed representation of pose and appearance
Maji, S., Bourdev, L., & Malik, J. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995631 · Cited by 302
Skeletal Quads: Human Action Recognition Using Joint Quadruples
Evangelidis, G., Singh, G., & Horaud, R. (2014) · 2014 22nd International Conference on Pattern Recognition · doi:10.1109/icpr.2014.772 · Cited by 272
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2_3 · Cited by 265
How to Construct Deep Recurrent Neural Networks
Pascanu, R., Gülçehre, Ç., Cho, K., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6026 · Cited by 228
Learning a Deep Model for Human Action Recognition from Novel Viewpoints
Rahmani, H., Mian, A., & Shah, M. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2691768 · Cited by 220
A Survey on Human Motion Analysis from Depth Data
Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., & Gall, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-44964-2_8 · Cited by 213
Wu, D., & Shao, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.98 · Cited by 211
Accurate 3D action recognition using learning on the Grassmann manifold
Slama, R., Wannous, H., Daoudi, M., & Srivastava, A. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.08.011 · Cited by 203
Bio-inspired Dynamic 3D Discriminative Skeletal Features for Human Action Recognition
Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., & Vidal, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.153 · Cited by 179
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.07.014 · Cited by 178
Does Human Action Recognition Benefit from Pose Estimation?
Yao, A., Gall, J., Fanelli, G., & Gool, L. V. (2011) · British Machine Vision Conference (BMVC) · doi:10.5244/c.25.67 · Cited by 172
SkeletonNet: Mining Deep Part Features for 3-D Action Recognition
Ke, Q., An, S., Bennamoun, M., Sohel, F., & Boussaid, F. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2690339 · Cited by 172
Self-citationDiscriminative virtual views for cross-view action recognition
Li, R., & Zickler, T. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248011 · Cited by 165
Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition
Du, Y., Fu, Y., & Wang, L. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2552404 · Cited by 151
Interactive body part contrast mining for human interaction recognition
Ji, Y., Ye, G., & Cheng, H. (2014) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2014.6890714 · Cited by 150
Recognize complex events from static images by fusing deep channels
Xiong, Y., Zhu, K., Lin, D., & Tang, X. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298768 · Cited by 139
Cooperative Training of Deep Aggregation Networks for RGB-D Action Recognition
Wang, P., Li, W., Wan, J., Ogunbona, P., & Liu, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12228 · Cited by 111
Category-Blind Human Action Recognition: A Practical Recognition System
Li, W., Wen, L., Chuah, M. C., & Lyu, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.505 · Cited by 74
Action recognition on motion capture data using a dynemes and forward differences representation
Kapsouras, I., & Nikolaidis, N. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2014.04.007 · Cited by 72
Joint Human Detection and Head Pose Estimation via Multistream Networks for RGB-D Videos
Zhang, G., Liu, J., Li, H., Chen, Y. Q., & Davis, L. S. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2731952 · Cited by 54
Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaïd, F. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2778559 · Cited by 49
Self-citationHuman Interaction Prediction Using Deep Temporal Features
Ke, Q., Bennamoun, M., An, S., Boussaïd, F., & Sohel, F. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48881-3_28 · Cited by 47
Self-citationLeveraging structural context models and ranking score fusion for human interaction prediction
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaïd, F. (2017) · Murdoch University Research Portal · Cited by 45
Self-citationIs Rotation a Nuisance in Shape Recognition?
Ke, Q., & Li, Y. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.528 · Cited by 33
Self-citationAction recognition from motion capture data using Meta-Cognitive RBF Network classifier
Vantigodi, S., & Babu, R. V. (2014) · International Conference on Intelligent Sensors, Sensor Networks and Information Processing · doi:10.1109/issnip.2014.6827664 · Cited by 30
Real-time human action recognition from motion capture data
Vantigodi, S., & Babu, R. V. (2013) · National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG) · doi:10.1109/ncvpripg.2013.6776204 · Cited by 29
Encoding Feature Maps of CNNs for Action Recognition
Peng, X., & Schmid, C. (2015) · HAL · Cited by 15
Physical blob detector and Multi-Channel Color Shape Descriptor for human detection
Zhang, G., Liu, J., Liu, Y., Zhao, J., Tian, L., & Chen, Y. Q. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.01.013 · Cited by 10
Cited by
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition
Liu, Z., Zhang, H., Chen, Z., Wang, Z., & Ouyang, W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00022 · Cited by 1,225
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., & Hu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01311 · Cited by 1,039
Skeleton-Based Action Recognition With Shift Graph Convolutional Network
Cheng, K., Zhang, Y., He, X., Chen, W., Cheng, J., & Lu, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00026 · Cited by 981
Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00119 · Cited by 647
Metadata
- Authors
- Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaid, F.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2018-06
- Volume
- 27
- Issue
- 6
- Pages
- 2842-2855
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2793547936
- Cited by
- 270
- References
- 78
- FWCI
- 15.83
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.