The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Skeleton Image Representation for 3D Action Recognition Based on Tree Structure and Reference Joints
Open accessAbstract
In the last years, the computer vision research community has studied on how to model temporal dynamics in videos to employ 3D human action recognition. To that end, two main baseline approaches have been researched: (i) Recurrent Neural Networks (RNNs) with Long-Short Term Memory (LSTM); and (ii) skeleton image representations used as input to a Convolutional Neural Network (CNN). Although RNN approaches present excellent results, such methods lack the ability to efficiently learn the spatial relations between the skeleton joints. On the other hand, the representations used to feed CNN approaches present the advantage of having the natural ability of learning structural information from 2D arrays (i.e., they learn spatial relations from the skeleton joints). To further improve such representations, we introduce the Tree Structure Reference Joints Image (TSRJI), a novel skeleton image representation to be used as input to CNNs. The proposed representation has the advantage of combining the use of reference joints and a tree structure skeleton. While the former incorporates different spatial relationships between the joints, the latter preserves important spatial relations by traversing a skeleton tree with a depth-first order algorithm. Experimental results demonstrate the effectiveness of the proposed representation for 3D action recognition on two datasets achieving state-of-the-art results on the recent NTU RGB+D 120 dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Pérez, M., Wang, G., Duan, L., & Kot, A. C. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,917
Mining actionlet ensemble for action recognition with depth cameras
Wang, J., Liu, Z., Wu, Y. N., & Yuan, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247813 · Cited by 1,556
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,369
Enhanced skeleton visualization for view invariant human action recognition
Liu, M., Liu, H., & Chen, C. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.02.030 · Cited by 871
A New Representation of Skeleton Sequences for 3D Action Recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaid, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.486 · Cited by 793
Zhang, P., Lan, C., Xing, J., Zeng, W., Xue, J., & Zheng, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.233 · Cited by 691
Global Context-Aware Attention LSTM Networks for 3D Action Recognition
Liu, J., Wang, G., Hu, P., Duan, L., & Kot, A. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.391 · Cited by 682
Li, C., Zhong, Q., Xie, D., & Pu, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/109 · Cited by 627
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks
Liu, J., Wang, G., Duan, L., Abdiyeva, K., & Kot, A. C. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2785279 · Cited by 612
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Liu, J., Shahroudy, A., Xu, D., Kot, A. C., & Wang, G. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2771306 · Cited by 548
Differential Recurrent Neural Networks for Action Recognition
Veeriah, V., Zhuang, N., & Qi, G.-J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.460 · Cited by 531
Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks
Liu, J., Wang, G., Duan, L.-Y., Abdiyeva, K., & Kot, A. C. (2018) · Lancaster EPrints (Lancaster University) · Cited by 531
An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton DataPreprint
Song, S., Lan, C., Xing, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06067 · Cited by 481
A new representation of skeleton sequences for 3D action recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F., & Boussaïd, F. (2017) · UWA Profiles and Research Repository (UWA) · Cited by 480
Zanfir, M., Leordeanu, M., & Sminchisescu, C. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.342 · Cited by 462
EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor
Yang, X., & Tian, Y. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239232 · Cited by 450
Skeleton based action recognition with convolutional neural network
Du, Y., Fu, Y., & Wang, L. (2015) · 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR) · doi:10.1109/acpr.2015.7486569 · Cited by 445
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition
Hu, J., Zheng, W., Lai, J., & Zhang, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2640292 · Cited by 392
Skeleton-based action recognition with convolutional neural networks
Li, C., Zhong, Q., Xie, D., & Pu, S. (2017) · IEEE International Conference on Multimedia & Expo Workshops (ICMEW) · doi:10.1109/icmew.2017.8026285 · Cited by 392
Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks
Wang, P., Li, Z., Hou, Y., & Li, W. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967191 · Cited by 391
Recognizing Human Actions as the Evolution of Pose Estimation Maps
Liu, M., & Yuan, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00127 · Cited by 359
3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold
Devanne, M., Wannous, H., Berretti, S., et al. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2350774 · Cited by 346
PoTion: Pose MoTion Representation for Action Recognition
Choutas, V., Weinzaepfel, P., Revaud, J., & Schmid, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00734 · Cited by 342
Space-time representation of people based on 3D skeletal data: A review
Han, F., Reily, B., Hoff, W., & Zhang, H. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.01.011 · Cited by 316
Learning Clip Representations for Skeleton-Based 3D Action Recognition
Ke, Q., Bennamoun, M., An, S., Sohel, F. A., & Boussaïd, F. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2812099 · Cited by 271
Action recognition based on joint trajectory maps with convolutional neural networks
Wang, P., Li, W., Li, C., & Hou, Y. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.05.029 · Cited by 230
Action Recognition With Spatio–Temporal Visual Attention on Skeleton Image Sequences
Yang, Z., Li, Y., Yang, J., & Luo, J. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2864148 · Cited by 176
Gowayyed, M. A., Torki, M., Hussein, M. E., & El-Saban, M. (2013) · Cited by 172
Skeleton-Based Online Action Prediction Using Scale Selection Network
Liu, J., Shahroudy, A., Wang, G., Duan, L., & Kot, A. C. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2898954 · Cited by 170
Early Action Prediction by Soft Regression
Hu, J., Zheng, W., Ma, L., Wang, G., Lai, J., & Zhang, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2863279 · Cited by 124
Mining Mid-Level Features for Action Recognition Based on Effective Skeleton Representation
Wang, P., Li, W., Ogunbona, P., et al. (2014) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2014.7008115 · Cited by 40
3D action recognition using data visualization and convolutional neural networks
Liu, M., Chen, C., & Liu, H. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019438 · Cited by 13
Research works printed but not linked: 1
- simonyan, 2015, Very deep convolutional networks for large-scale image recognition, ICLRE
Cited by
Sorted by citations
3D Human Action Representation Learning via Cross-View Consistency Pursuit
Li, L., Wang, M., Ni, B., Wang, H., Yang, J., & Zhang, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00471 · Cited by 236
“As shown in Table 6, TSRJI [2] supervisedly utilizes attention LSTM, ASCAL [39] adopts LSTM for skeleton modeling, and our method defeats the other unsupervised method and some of the supervised methods.”
Method
Metadata
- Authors
- Caetano, C., Bremond, F., & Schwartz, W. R.
- Affiliations
- Universidade Federal de Minas Gerais (BR)
- Venue
- 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 16-23
- OpenAlex
- W2972794281
- Cited by
- 149
- References
- 35
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 21 · 2023: 27 · 2022: 32 · 2021: 40 · 2020: 12
- FWCI
- 6.85
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (438629/2018-3, 204952/2017-4, 2016-5, 311053/2016‐5, 2018‐3)
- Fundação de Amparo à Pesquisa do Estado de Minas Gerais (APQ-00567-14, PPM-00540-17)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Keywords
- 3D human action recognition, depth-first traversal, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
