The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks
Open accessAbstract
Human action recognition in 3D skeleton sequences has attracted a lot of research attention. Recently, long short-term memory (LSTM) networks have shown promising performance in this task due to their strengths in modeling the dependencies and dynamics in sequential data. As not all skeletal joints are informative for action recognition, and the irrelevant joints often bring noise which can degrade the performance, we need to pay more attention to the informative ones. However, the original LSTM network does not have explicit attention ability. In this paper, we propose a new class of LSTM network, global context-aware attention LSTM, for skeleton-based action recognition, which is capable of selectively focusing on the informative joints in each frame by using a global context memory cell. To further improve the attention capability, we also introduce a recurrent attention mechanism, with which the attention performance of our network can be enhanced progressively. Besides, a two-stream framework, which leverages coarse-grained attention and fine-grained attention, is also introduced. The proposed method achieves state-of-the-art performance on five challenging datasets for skeleton-based action recognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Self-citationBeyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062
LSTM neural networks for language modeling
Sundermeyer, M., Schlüter, R., & Ney, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-65 · Cited by 1,963
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.82 · Cited by 1,517
Mining actionlet ensemble for action recognition with depth cameras
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247813 · Cited by 1,512
View invariant human action recognition using histograms of 3D joints
Xia, L., Chen, C.-C., & Aggarwal, J. K. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239233 · Cited by 1,452
Enhanced Computer Vision With Microsoft Kinect Sensor: A Review
Han, J., Shao, L., Xu, D., & Shotton, J. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2265378 · Cited by 1,367
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
Self-citationStructural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133
Describing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035
Enhanced skeleton visualization for view invariant human action recognition
Liu, M., Liu, H., & Chen, C. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.02.030 · Cited by 842
Global Context-Aware Attention LSTM Networks for 3D Action Recognition
Liu, J., Wang, G., Hu, P., Duan, L.-Y., & Kot, A. C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.391 · Cited by 682
Self-citationEnd-to-End Learning of Action Detection from Frame Glimpses in Videos
Yeung, S., Russakovsky, O., Mori, G., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.293 · Cited by 585
Human activity recognition from 3D data: A review
Aggarwal, J. K., & Xia, L. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.04.011 · Cited by 576
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2_2 · Cited by 545
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Liu, J., Shahroudy, A., Xu, D., Kot, A. C., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2771306 · Cited by 530
Self-citationTwo-person interaction detection using body-pose features and multiple instance learning
Yun, K., Honorio, J., Chattopadhyay, D., Berg, T. L., & Samaras, D. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239234 · Cited by 523
A Hierarchical Deep Temporal Model for Group Activity Recognition
Ibrahim, M. S., Muralidharan, S., Deng, Z., Vahdat, A., & Mori, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.217 · Cited by 523
Differential Recurrent Neural Networks for Action Recognition
Veeriah, V., Zhuang, N., & Qi, G.-J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.460 · Cited by 516
Learning Actionlet Ensemble for 3D Human Action Recognition
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.198 · Cited by 502
Berkeley MHAD: A comprehensive Multimodal Human Action Database
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013) · 2013 IEEE Workshop on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2013.6474999 · Cited by 484
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Wu, Z., Wang, X., Jiang, Y.-G., Ye, H., & Xue, X. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806222 · Cited by 479
Zanfir, M., Leordeanu, M., & Sminchisescu, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.342 · Cited by 438
3D skeleton-based human action classification: A survey
Lo Presti, L., & La Cascia, M. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.11.019 · Cited by 397
Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks
Wang, P., Li, Z., Hou, Y., & Li, W. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967191 · Cited by 383
Learning Activity Progression in LSTMs for Activity Detection and Early Detection
Ma, S., Sigal, L., & Sclaroff, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.214 · Cited by 377
Jointly learning heterogeneous features for RGB-D activity recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299172 · Cited by 358
Effective 3D action recognition using EigenJoints
Yang, X., & Tian, Y. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.03.001 · Cited by 340
3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold
Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., & Del Bimbo, A. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2350774 · Cited by 336
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.04.007 · Cited by 318
Space-time representation of people based on 3D skeletal data: A review
Han, F., Reily, B., Hoff, W., & Zhang, H. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.01.011 · Cited by 316
RGB-D-based action recognition datasets: A survey
Zhang, J., Li, W., Ogunbona, P. O., Wang, P., & Tang, C. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.05.019 · Cited by 283
Skeletal Quads: Human Action Recognition Using Joint Quadruples
Evangelidis, G., Singh, G., & Horaud, R. (2014) · 2014 22nd International Conference on Pattern Recognition · doi:10.1109/icpr.2014.772 · Cited by 272
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos
Shahroudy, A., Ng, T.-T., Gong, Y., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2691321 · Cited by 264
Self-citationA Survey on Human Motion Analysis from Depth Data
Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., & Gall, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-44964-2_8 · Cited by 213
Group Sparsity and Geometry Constrained Dictionary Learning for Action Recognition from Depth Maps
Luo, J., Wang, W., & Qi, H. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.227 · Cited by 204
Accurate 3D action recognition using learning on the Grassmann manifold
Slama, R., Wannous, H., Daoudi, M., & Srivastava, A. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.08.011 · Cited by 203
Online Human Action Detection Using Joint Classification-Regression Recurrent Neural Networks
Li, Y., Lan, C., Xing, J., Zeng, W., Yuan, C., & Liu, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_13 · Cited by 202
Self-citationBio-inspired Dynamic 3D Discriminative Skeletal Features for Human Action Recognition
Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., & Vidal, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.153 · Cited by 179
SkeletonNet: Mining Deep Part Features for 3-D Action Recognition
Ke, Q., An, S., Bennamoun, M., Sohel, F., & Boussaid, F. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2690339 · Cited by 172
Interactive body part contrast mining for human interaction recognition
Ji, Y., Ye, G., & Cheng, H. (2014) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2014.6890714 · Cited by 150
Multimodal Multipart Learning for Action Recognition in Depth Videos
Shahroudy, A., Ng, T.-T., Yang, Q., & Wang, G. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505295 · Cited by 132
Self-citationGraph Based Skeleton Motion Representation and Similarity Measurement for Action Recognition
Wang, P., Yuan, C., Hu, W., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_23 · Cited by 119
Real time action recognition using histograms of depth gradients and random decision forests
Rahmani, H., Mahmood, A., Huynh, D. Q., & Mian, A. S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836044 · Cited by 116
Real-Time RGB-D Activity Prediction by Soft Regression
Hu, J., Zheng, W., Ma, L., Wang, G., & Lai, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_17 · Cited by 99
Self-citationFusion of depth, skeleton, and inertial data for human action recognition
Chen, C., Jafari, R., & Kehtarnavaz, N. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472170 · Cited by 90
Cross-View Action Recognition via Transferable Dictionary Learning
Zheng, J., Jiang, Z., & Chellappa, R. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2548242 · Cited by 89
Informative joints based human action recognition using skeleton contexts
Jiang, M., Kong, J., Bebis, G. N., & Huo, H. (2015) · Signal Processing Image Communication · doi:10.1016/j.image.2015.02.004 · Cited by 75
A novel hierarchical framework for human action recognition
Chen, H., Wang, G., Xue, J., & He, L. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.01.020 · Cited by 75
Self-citationCategory-Blind Human Action Recognition: A Practical Recognition System
Li, W., Wen, L., Chuah, M. C., & Lyu, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.505 · Cited by 74
Human Action Recognition in Unconstrained Videos by Explicit Motion Modeling
Jiang, Y., Dai, Q., Liu, W., Xue, X., & Ngo, C. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2456412 · Cited by 73
Learning Maximum Margin Temporal Warping for Action Recognition
Wang, J., & Wu, Y. N. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.334 · Cited by 73
Action recognition on motion capture data using a dynemes and forward differences representation
Kapsouras, I., & Nikolaidis, N. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2014.04.007 · Cited by 72
Simple to Complex Transfer Learning for Action Recognition
Liu, F., Xu, X., Qiu, S., Qing, C., & Tao, D. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2512107 · Cited by 61
Tao, L., & Vidal, R. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.48 · Cited by 54
Joint Human Detection and Head Pose Estimation via Multistream Networks for RGB-D Videos
Zhang, G., Liu, J., Li, H., Chen, Y. Q., & Davis, L. S. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2731952 · Cited by 54
Self-citationMulti-modal feature fusion for action recognition in RGB-D sequences
Shahroudy, A., Wang, G., & Ng, T.-T. (2014) · International Symposium on Communications, Control and Signal Processing · doi:10.1109/isccsp.2014.6877819 · Cited by 53
Self-citationLillo, I., Niebles, J. C., & Soto, Á. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.218 · Cited by 50
Mining Mid-Level Features for Action Recognition Based on Effective Skeleton Representation
Wang, P., Li, W., Ogunbona, P., et al. (2014) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2014.7008115 · Cited by 40
Action recognition from motion capture data using Meta-Cognitive RBF Network classifier
Vantigodi, S., & Babu, R. V. (2014) · International Conference on Intelligent Sensors, Sensor Networks and Information Processing · doi:10.1109/issnip.2014.6827664 · Cited by 30
Real-time human action recognition from motion capture data
Vantigodi, S., & Babu, R. V. (2013) · National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG) · doi:10.1109/ncvpripg.2013.6776204 · Cited by 29
Cited by
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Self-citationDisentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition
Liu, Z., Zhang, H., Chen, Z., Wang, Z., & Ouyang, W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00022 · Cited by 1,225
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., & Hu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01311 · Cited by 1,039
Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00119 · Cited by 647
Human Action Recognition From Various Data Modalities: A Review
Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., & Liu, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183112 · Cited by 601
Self-citation
Metadata
- Authors
- Liu, J., Wang, G., Duan, L.-Y., Abdiyeva, K., & Kot, A. C.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2018-04
- Volume
- 27
- Issue
- 4
- Pages
- 1586-1599
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2739179646
- Cited by
- 537
- References
- 67
- FWCI
- 16.73
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.