Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 257

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection

Song, Sijie, Lan, Cuiling, Xing, Junliang, Zeng, Wenjun, Liu, Jiaying·2018·IEEE Transactions on Image Processing
DOI 10.1109/tip.2018.2818328
Open this paper in CoChat — free account

Abstract

Human action analytics has attracted a lot of attention for decades in computer vision. It is important to extract discriminative spatio-temporal features to model the spatial and temporal evolutions of different actions. In this paper, we propose a spatial and temporal attention model to explore the spatial and temporal discriminative features for human action recognition and detection from skeleton data. We build our networks based on the recurrent neural networks with long short-term memory units. The learned model is capable of selectively focusing on discriminative joints of skeletons within each input frame and paying different levels of attention to the outputs of different frames. To ensure effective training of the network for action recognition, we propose a regularized cross-entropy loss to drive the learning process and develop a joint training strategy accordingly. Moreover, based on temporal attention, we develop a method to generate the action temporal proposals for action detection. We evaluate the proposed method on the SBU Kinect Interaction data set, the NTU RGB + D data set, and the PKU-MMD data set, respectively. Experiment results demonstrate the effectiveness of our proposed model on both action recognition and action detection.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tip.2018.2818328

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 7 references

Research works printed but not linked: 58

  1. 10.1109/ICCV.2017.393 · DOI · Paper page
  2. 10.1007/978-3-319-46493-0_45 · DOI · Paper page
  3. 10.1109/CVPR.2014.100 · DOI
  4. 10.1007/978-3-319-46487-9_47 · DOI · Paper page
  5. 10.5244/C.25.65 · DOI
  6. 10.1109/CVPR.2014.81 · DOI · Paper page
  7. 10.1109/CVPR.2016.119 · DOI · Paper page
  8. 10.1109/CVPR.2016.211 · DOI · Paper page
  9. 10.1109/CVPR.2014.101 · DOI
  10. 10.1109/CVPR.2015.7298676 · DOI · Paper page
  11. 10.1109/CVPR.2017.387 · DOI · Paper page
  12. 10.1109/CVPR.2014.82 · DOI · Paper page
  13. 10.1109/ICMEW.2014.6890714 · DOI
  14. 10.1109/ICPR.2014.772 · DOI · Paper page
  15. wang, 2014, Action recognition and detection by combining motion and appearance features, THUMOS Action Recognition challenge, 1, 2
  16. 10.1109/CVPR.2015.7299172 · DOI · Paper page
  17. 10.1109/ICCV.2013.228 · DOI · Paper page
  18. 10.1016/j.cviu.2010.10.002 · DOI · Paper page
  19. 10.5244/C.22.109 · DOI
  20. liu, 2017, PKU-MMD A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding
  21. song, 2017, An end-to-end spatio-temporal attention model for human action recognition from skeleton data, Proc AAAI Conf Artif Intell, 7
  22. 10.1016/j.patcog.2012.10.004 · DOI
  23. yu, 2014, Discriminative orderlet mining for real-time recognition of human-object interaction, Proc Asian Conf Comput Vis, 50
  24. karaman, 2014, Fast saliency based pooling of fisher encoded dense trajectories, Proc European Conf Comp Vis Workshop, 1, 5
  25. 10.1109/CVPR.2013.342 · DOI
  26. 10.1007/978-3-642-24797-2 · DOI · Paper page
  27. 10.1007/978-3-319-46487-9_50 · DOI · Paper page
  28. zaremba, 2014, Recurrent Neural Network Regularization
  29. 10.1109/CVPR.2016.115 · DOI · Paper page
  30. 10.1109/CVPRW.2012.6239234 · DOI · Paper page
  31. 10.1109/CVPR.2012.6248012 · DOI
  32. hochreiter, 2001, Gradient flow in recurrent nets: The difficulty of learning long-term dependencies, A Field Guide to Dynamical Recurrent Neural Networks
  33. 10.1109/CVPRW.2012.6239233 · DOI · Paper page
  34. 10.1109/CVPR.2012.6247813 · DOI · Paper page
  35. 10.1109/TCSVT.2018.2799968 · DOI · Paper page
  36. 10.1016/j.cviu.2016.04.005 · DOI
  37. zhu, 2016, Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks, Proc AAAI Conf Artif Intell, 8
  38. 10.1016/j.cviu.2006.07.014 · DOI
  39. du, 2015, Hierarchical recurrent neural network for skeleton based action recognition, Proc IEEE Conf Comput Vis Pattern Recognit, 1110
  40. 10.1007/978-3-319-46478-7_13 · DOI
  41. 10.1109/TIP.2016.2552404 · DOI
  42. hu, 2017, Temporal perceptive network for skeleton-based action recognition, Proc Brit Mach Vis Conf, 1
  43. carlsson, 2001, Action recognition by shape matching to key frames, Proc Workshop Models Versus Exemplars Comput Vis, 1, 1
  44. 10.1109/TIP.2014.2331759 · DOI
  45. 10.1109/TIP.2015.2456412 · DOI
  46. 10.3758/BF03212378 · DOI · Paper page
  47. 10.1109/TIP.2015.2490551 · DOI
  48. 10.1109/MMUL.2012.24 · DOI · Paper page
  49. 10.1016/j.cviu.2017.01.011 · DOI · Paper page
  50. yeung, 2017, Every moment counts: Dense detailed labeling of actions in complex videos, Int J Comput Vis, 126, 1
  51. 10.1109/CVPR.2011.5995316 · DOI · Paper page
  52. ba, 2015, Multiple object recognition with visual attention, Proc Int Conf Learn Represent, 1
  53. 10.1109/CVPR.2016.332 · DOI
  54. sharma, 2016, Action recognition using visual attention, Proc Workshop Int Conf Learn Represent, 1
  55. yu, 2010, An object-based visual attention model for robotic applications, IEEE Trans Syst Man Cybern B Cybern, 40, 1398, 10.1109/TSMCB.2009.2038895 · DOI
  56. 10.1109/TPAMI.2011.272 · DOI · Paper page
  57. bahdanau, 2015, Neural machine translation by jointly learning to align and translate, Proc Int Conf Learn Represent, 1
  58. bazzani, 2017, Recurrent mixture density network for spatiotemporal visual attention, Proc Int Conf Learn Represent, 1

Cited by

Metadata

Authors
Song, S., Lan, C., Xing, J., Zeng, W., & Liu, J.
Venue
IEEE Transactions on Image Processing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2018-07
Volume
27
Issue
7
Pages
3459-3471
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
DOI
10.1109/tip.2018.2818328
OpenAlex
W2792140610
Cited by
257
References
7
FWCI
13.21
Type
journal article
Language
en
Open access
closed
Licence
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Funders
  • National Natural Science Foundation of China (61772043, 61672519)
  • Microsoft Research Asia Fund (FY17-RES-THEME-013)
  • CCF-Tencent Open Research Fund

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.