CoChat
Sign Up
Cited by 17Human Pose and Action RecognitionHand Gesture Recognition SystemsContext-Aware Activity Recognition Systems

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Memory-Augmented Temporal Dynamic Learning for Action Recognition

Open access
Yuan, Yuan, Wang, Dong, Wang, Qi·2019·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v33i01.33019167
Open this paper in CoChat — free account

Abstract

Human actions captured in video sequences contain two crucial factors for action recognition, i.e., visual appearance and motion dynamics. To model these two aspects, Convolutional and Recurrent Neural Networks (CNNs and RNNs) are adopted in most existing successful methods for recognizing actions. However, CNN based methods are limited in modeling long-term motion dynamics. RNNs are able to learn temporal motion dynamics but lack effective ways to tackle unsteady dynamics in long-duration motion. In this work, we propose a memory-augmented temporal dynamic learning network, which learns to write the most evident information into an external memory module and ignore irrelevant ones. In particular, we present a differential memory controller to make a discrete decision on whether the external memory module should be updated with current feature. The discrete memory controller takes in the memory history, context embedding and current feature as inputs and controls information flow into the external memory module. Additionally, we train this discrete memory controller using straight-through estimator. We evaluate this end-to-end system on benchmark datasets (UCF101 and HMDB51) of human action recognition. The experimental results show consistent improvements on both datasets over prior works and our baselines.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aaai.v33i01.33019167

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Selective memory management improves video action recognition by focusing on key temporal elements.

    “This selective memory management improves video action recognition by focusing on key temporal elements.”

    From The AI Hippocampus: How Far are We From Human Memory? (2026) · cites this paper as Yuan et al., 2019

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 30

  1. [Bengio, L´ eonard, and Courville 2013] Bengio, Y.; L´ eonard, N.; and Courville, A. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 . · arXiv
  2. [Boiman and Irani 2007] Boiman, O., and Irani, M. 2007. Detecting irregularities in images and in video. International journal of computer vision 74(1):17-31.
  3. [Carreira and Zisserman 2017] Carreira, J., and Zisserman, A. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In IEEE Conference on Computer Vision and Pattern Recognition , 4724-4733.
  4. [Feichtenhofer, Pinz, and Zisserman 2016] Feichtenhofer, C.; Pinz, A.; and Zisserman, A. 2016. Convolutional two-stream network fusion for video action recognition. In IEEE Conference on Computer Vision and Pattern Recognition , 1933-1941.
  5. [Gao et al. 2018] Gao, J.; Ge, R.; Chen, K.; and Nevatia, R. 2018. Motion-appearance co-memory networks for video question answering. arXiv preprint arXiv:1803.10906 . · arXiv
  6. [Graves, Wayne, and Danihelka 2014] Graves, A.; Wayne, G.; and Danihelka, I. 2014. Neural turing machines. arXiv preprint arXiv:1410.5401 . · arXiv
  7. [Joulin and Mikolov 2015] Joulin, A., and Mikolov, T. 2015. Inferring algorithmic patterns with stack-augmented recurrent nets. In Advances in Neural Information Processing Systems , 190-198.
  8. [Karpathy et al. 2014] Karpathy, A.; Toderici, G.; Shetty, S.; Leung, T.; Sukthankar, R.; and Fei-Fei, L. 2014. Large-scale video classification with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition , 1725-1732.
  9. [Kuehne et al. 2011] Kuehne, H.; Jhuang, H.; Garrote, E.; Poggio, T.; and Serre, T. 2011. Hmdb: a large video database for human motion recognition. In IEEE International Conference on Computer Vision , 2556-2563.
  10. [Lin et al. 2008] Lin, W.; Sun, M.-T.; Poovandran, R.; and Zhang, Z. 2008. Human activity recognition for video surveillance. In International Symposium on Circuits and Systems , 2737-2740.
  11. [Long et al. 2018a] Long, X.; Gan, C.; de Melo, G.; Liu, X.; Li, Y.; Li, F.; and Wen, S. 2018a. Multimodal keyless attention fusion for video classification. In AAAI Conference on Artificial Intelligence .
  12. [Long et al. 2018b] Long, X.; Gan, C.; de Melo, G.; Wu, J.; Liu, X.; and Wen, S. 2018b. Attention clusters: Purely attention based local feature integration for video classification. In IEEE Conference on Computer Vision and Pattern Recognition , 7834-7843.
  13. [Long, Shelhamer, and Darrell 2015] Long, J.; Shelhamer, E.; and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition , 3431-3440.
  14. [Ma et al. 2017] Ma, C.-Y.; Chen, M.-H.; Kira, Z.; and AlRegib, G. 2017. Ts-lstm and temporal-inception: Exploiting spatiotemporal dynamics for activity recognition. arXiv preprint arXiv:1703.10667 . · arXiv
  15. [Paszke et al. 2017] Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; and Lerer, A. 2017. Automatic differentiation in pytorch. In Advances in Neural Information Processing Systems Workshop .
  16. [Sharma, Kiros, and Salakhutdinov 2015] Sharma, S.; Kiros, R.; and Salakhutdinov, R. 2015. Action recognition using visual attention. arXiv preprint arXiv:1511.04119 . · arXiv
  17. [Simonyan and Zisserman 2014] Simonyan, K., and Zisserman, A. 2014. Two-stream convolutional networks for ac- tion recognition in videos. In Advances in Neural Information Processing Systems , 568-576.
  18. [Soomro, Zamir, and Shah 2012] Soomro, K.; Zamir, A. R.; and Shah, M. 2012. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 . · arXiv
  19. [Sukhbaatar et al. 2015] Sukhbaatar, S.; Weston, J.; Fergus, R.; et al. 2015. End-to-end memory networks. In Advances in Neural Information Processing Systems , 2440-2448.
  20. [Sun et al. 2015] Sun, L.; Jia, K.; Yeung, D.-Y.; and Shi, B. E. 2015. Human action recognition using factorized spatio-temporal convolutional networks. In IEEE International Conference on Computer Vision , 4597-4605.
  21. [Sun et al. 2017] Sun, L.; Jia, K.; Chen, K.; Yeung, D.-Y.; Shi, B. E.; and Savarese, S. 2017. Lattice long short-term memory for human action recognition. In IEEE International Conference on Computer Vision , 2166-2175.
  22. [Tran et al. 2015] Tran, D.; Bourdev, L.; Fergus, R.; Torresani, L.; and Paluri, M. 2015. Learning spatiotemporal features with 3d convolutional networks. In IEEE International Conference on Computer Vision , 4489-4497.
  23. [Vu et al. 2018] Vu, T.-H.; Choi, W.; Schulter, S.; and Chandraker, M. 2018. Memory warps for learning long-term online video representations. arXiv preprint arXiv:1803.10861 . · arXiv
  24. [Wang et al. 2011] Wang, H.; Kl¨ aser, A.; Schmid, C.; and Liu, C.-L. 2011. Action recognition by dense trajectories. In IEEE Conference on Computer Vision and Pattern Recognition , 3169-3176.
  25. [Wang et al. 2016] Wang, L.; Xiong, Y.; Wang, Z.; Qiao, Y.; Lin, D.; Tang, X.; and Van Gool, L. 2016. Temporal segment networks: Towards good practices for deep action recognition. In European Conference on Computer Vision , 20-36.
  26. [Wang et al. 2018] Wang, Q.; Yuan, Z.; Du, Q.; and Li, X. 2018. Getnet: A general end-to-end 2-d cnn framework for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing .
  27. [Xingjian et al. 2015] Xingjian, S.; Chen, Z.; Wang, H.; Yeung, D.-Y.; Wong, W.-K.; and Woo, W.-c. 2015. Convolutional lstm network: A machine learning approach for precipitation nowcasting. In Advances in Neural Information Processing Systems , 802-810.
  28. [Yang and Chan 2018] Yang, T., and Chan, A. B. 2018. Learning dynamic memory networks for object tracking. arXiv preprint arXiv:1803.07268 . · arXiv
  29. [Yue-Hei Ng et al. 2015] Yue-Hei Ng, J.; Hausknecht, M.; Vijayanarasimhan, S.; Vinyals, O.; Monga, R.; and Toderici, G. 2015. Beyond short snippets: Deep networks for video classification. In IEEE Conference on Computer Vision and Pattern Recognition , 4694-4702.
  30. [Zha et al. 2015] Zha, S.; Luisier, F.; Andrews, W.; Srivastava, N.; and Salakhutdinov, R. 2015. Exploiting imagetrained cnn architectures for unconstrained video classification. arXiv preprint arXiv:1503.04144 . · arXiv

Cited by

Sorted by citations

Metadata

Authors
Yuan, Y., Wang, D., & Wang, Q.
Affiliations
Northwestern Polytechnical University (CN)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2019-07-17
Volume
33
Issue
01
Pages
9167-9175
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v33i01.33019167
OpenAlex
W2964084939
Cited by
17
References
1
Citations by year
2024: 3 · 2022: 2 · 2021: 7 · 2020: 4 · 2019: 1
FWCI
1.14
Type
conference paper
Language
en
Open access
Licence
www.aaai.org/ (unspecified)
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (U1864204, 61773316)
  • Chinese Academy of Sciences
  • Natural Science Foundation of Shaanxi Province (U1864204, 2018KJXX‐024)
  • Fundamental Research Funds for the Central Universities (3102017AX010)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
Keywords
human action recognition, memory-augmented networks, straight-through estimator

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review