The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Global Context-Aware Attention LSTM Networks for 3D Action Recognition
Abstract
Long Short-Term Memory (LSTM) networks have shown superior performance in 3D human action recognition due to their power in modeling the dynamics and dependencies in sequential data. Since not all joints are informative for action analysis and the irrelevant joints often bring a lot of noise, we need to pay more attention to the informative ones. However, original LSTM does not have strong attention capability. Hence we propose a new class of LSTM network, Global Context-Aware Attention LSTM (GCA-LSTM), for 3D action recognition, which is able to selectively focus on the informative joints in the action sequence with the assistance of global contextual information. In order to achieve a reliable attention representation for the action sequence, we further propose a recurrent attention mechanism for our GCA-LSTM network, in which the attention performance is improved iteratively. Experiments show that our end-to-end network can reliably focus on the most informative joints in each frame of the skeleton sequence. Moreover, our network yields state-of-the-art performance on three challenging datasets for 3D action recognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
(2014) · Cited by 33,419
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Self-citationBeyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062
Attention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
LSTM neural networks for language modeling
Sundermeyer, M., Schlüter, R., & Ney, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-65 · Cited by 1,963
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.82 · Cited by 1,517
Mining actionlet ensemble for action recognition with depth cameras
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247813 · Cited by 1,512
View invariant human action recognition using histograms of 3D joints
Xia, L., Chen, C.-C., & Aggarwal, J. K. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239233 · Cited by 1,452
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
Self-citationSukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015) · arXiv · doi:10.48550/arxiv.1503.08895 · Cited by 1,345
Torch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,186
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133
Describing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., & Socher, R. (2015) · arXiv · doi:10.48550/arxiv.1506.07285 · Cited by 1,020
Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., & Xie, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10451 · Cited by 915
Dynamic Memory Networks for Visual and Textual Question Answering
Xiong, C., Merity, S., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.01417 · Cited by 603
End-to-End Learning of Action Detection from Frame Glimpses in Videos
Yeung, S., Russakovsky, O., Mori, G., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.293 · Cited by 585
Human activity recognition from 3D data: A review
Aggarwal, J. K., & Xia, L. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.04.011 · Cited by 576
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2_2 · Cited by 545
Two-person interaction detection using body-pose features and multiple instance learning
Yun, K., Honorio, J., Chattopadhyay, D., Berg, T. L., & Samaras, D. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239234 · Cited by 523
A Hierarchical Deep Temporal Model for Group Activity Recognition
Ibrahim, M. S., Muralidharan, S., Deng, Z., Vahdat, A., & Mori, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.217 · Cited by 523
Differential Recurrent Neural Networks for Action Recognition
Veeriah, V., Zhuang, N., & Qi, G.-J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.460 · Cited by 516
Learning Actionlet Ensemble for 3D Human Action Recognition
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.198 · Cited by 502
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Wu, Z., Wang, X., Jiang, Y.-G., Ye, H., & Xue, X. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806222 · Cited by 479
Zanfir, M., Leordeanu, M., & Sminchisescu, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.342 · Cited by 438
3D skeleton-based human action classification: A survey
Lo Presti, L., & La Cascia, M. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.11.019 · Cited by 397
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2640292 · Cited by 383
Learning Activity Progression in LSTMs for Activity Detection and Early Detection
Ma, S., Sigal, L., & Sclaroff, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.214 · Cited by 377
Jointly learning heterogeneous features for RGB-D activity recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299172 · Cited by 358
Effective 3D action recognition using EigenJoints
Yang, X., & Tian, Y. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.03.001 · Cited by 340
3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold
Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., & Del Bimbo, A. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2350774 · Cited by 336
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.04.007 · Cited by 318
Space-time representation of people based on 3D skeletal data: A review
Han, F., Reily, B., Hoff, W., & Zhang, H. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.01.011 · Cited by 316
RGB-D-based action recognition datasets: A survey
Zhang, J., Li, W., Ogunbona, P. O., Wang, P., & Tang, C. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.05.019 · Cited by 283
Skeletal Quads: Human Action Recognition Using Joint Quadruples
Evangelidis, G., Singh, G., & Horaud, R. (2014) · 2014 22nd International Conference on Pattern Recognition · doi:10.1109/icpr.2014.772 · Cited by 272
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos
Shahroudy, A., Ng, T.-T., Gong, Y., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2691321 · Cited by 264
Self-citationA Survey on Human Motion Analysis from Depth Data
Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., & Gall, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-44964-2_8 · Cited by 213
Group Sparsity and Geometry Constrained Dictionary Learning for Action Recognition from Depth Maps
Luo, J., Wang, W., & Qi, H. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.227 · Cited by 204
Accurate 3D action recognition using learning on the Grassmann manifold
Slama, R., Wannous, H., Daoudi, M., & Srivastava, A. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.08.011 · Cited by 203
Online Human Action Detection Using Joint Classification-Regression Recurrent Neural Networks
Li, Y., Lan, C., Xing, J., Zeng, W., Yuan, C., & Liu, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_13 · Cited by 202
Self-citationBio-inspired Dynamic 3D Discriminative Skeletal Features for Human Action Recognition
Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., & Vidal, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.153 · Cited by 179
Wang, P., Li, W., Gao, Z., Zhang, Y., Tang, C., & Ogunbona, P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.52 · Cited by 155
Interactive body part contrast mining for human interaction recognition
Ji, Y., Ye, G., & Cheng, H. (2014) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2014.6890714 · Cited by 150
Spatio-Temporal Naive-Bayes Nearest-Neighbor (ST-NBNN) for Skeleton-Based Action Recognition
Weng, J., Weng, C., & Yuan, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.55 · Cited by 138
Multimodal Multipart Learning for Action Recognition in Depth Videos
Shahroudy, A., Ng, T.-T., Yang, Q., & Wang, G. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505295 · Cited by 132
Self-citationGraph Based Skeleton Motion Representation and Similarity Measurement for Action Recognition
Wang, P., Yuan, C., Hu, W., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_23 · Cited by 119
Real time action recognition using histograms of depth gradients and random decision forests
Rahmani, H., Mahmood, A., Huynh, D. Q., & Mian, A. S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836044 · Cited by 116
Fusion of depth, skeleton, and inertial data for human action recognition
Chen, C., Jafari, R., & Kehtarnavaz, N. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472170 · Cited by 90
Hierarchical Attention Network for Action Recognition in VideosPreprint
Wang, Y., Wang, S., Tang, J., O’Hare, N., Chang, Y., & Li, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06416 · Cited by 82
Informative joints based human action recognition using skeleton contexts
Jiang, M., Kong, J., Bebis, G. N., & Huo, H. (2015) · Signal Processing Image Communication · doi:10.1016/j.image.2015.02.004 · Cited by 75
A novel hierarchical framework for human action recognition
Chen, H., Wang, G., Xue, J., & He, L. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.01.020 · Cited by 75
Self-citationCategory-Blind Human Action Recognition: A Practical Recognition System
Li, W., Wen, L., Chuah, M. C., & Lyu, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.505 · Cited by 74
Learning Maximum Margin Temporal Warping for Action Recognition
Wang, J., & Wu, Y. N. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.334 · Cited by 73
Tao, L., & Vidal, R. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.48 · Cited by 54
Multi-modal feature fusion for action recognition in RGB-D sequences
Shahroudy, A., Wang, G., & Ng, T.-T. (2014) · International Symposium on Communications, Control and Signal Processing · doi:10.1109/isccsp.2014.6877819 · Cited by 53
Self-citationLillo, I., Niebles, J. C., & Soto, Á. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.218 · Cited by 50
Mining Mid-Level Features for Action Recognition Based on Effective Skeleton Representation
Wang, P., Li, W., Ogunbona, P., et al. (2014) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2014.7008115 · Cited by 40
Recognizing Actions in 3D Using Action-Snippets and Activated Simplices
Wang, C., Flynn, J., Wang, Y., & Yuille, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10456 · Cited by 34
Spatial, Structural and Temporal Feature Learning for Human Interaction Prediction.Preprint
Ke, Q., Bennamoun, M., An, S., Boussaïd, F., & Sohel, F. A. (2016) · arXiv (Cornell University) · Cited by 11
Cited by
Attention gated networks: learning to leverage salient regions in medical images
Schlemper, J., Oktay, O., Schaap, M., Heinrich, M., Kainz, B., Glocker, B., & Rueckert, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.01.012 · Cited by 2,014
A review on the long short-term memory model
Van Houdt, G., Mosquera, C., & Nápoles, G. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09838-1 · Cited by 1,991
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Self-citationSkeleton-Based Action Recognition With Shift Graph Convolutional Network
Cheng, K., Zhang, Y., He, X., Chen, W., Cheng, J., & Lu, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00026 · Cited by 981
Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00119 · Cited by 647
Metadata
- Authors
- Liu, J., Wang, G., Hu, P., Duan, L.-Y., & Kot, A. C.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 3671-3680
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2736334449
- Cited by
- 682
- References
- 67
- FWCI
- 20.79
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.