Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
In this work, we aim to address the problem of human interaction recognition in videos by exploring the long-term inter-related dynamics among multiple persons. Recently, Long Short-Term Memory (LSTM) has become a popular choice to model individual dynamic for single-person action recognition due to its ability to capture the temporal motion information in a range. However, most existing LSTM-based methods focus only on capturing the dynamics of human interaction by simply combining all dynamics of individuals or modeling them as a whole. Such methods neglect the inter-related dynamics of how human interactions change over time. To this end, we propose a novel Hierarchical Long Short-Term Concurrent Memory (H-LSTCM) to model the long-term inter-related dynamics among a group of persons for recognizing human interactions. Specifically, we first feed each person's static features into a Single-Person LSTM to model the single-person dynamic. Subsequently, at one time step, the outputs of all Single-Person LSTM units are fed into a novel Concurrent LSTM (Co-LSTM) unit, which mainly consists of multiple sub-memory units, a new cell gate, and a new co-memory cell. In the Co-LSTM unit, each sub-memory unit stores individual motion information, while this Co-LSTM unit selectively integrates and stores inter-related motion information between multiple interacting persons from multiple sub-memory units via the cell gate and co-memory cell, respectively. Extensive experiments on several public datasets validate the effectiveness of the proposed H-LSTCM by comparing against baseline and state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 5×
“We also compare the recognition accuracy of H-LSTCM and the state-of-the-art methods, including some traditional methods (i.e., Choi et al. [19], Lan et al. [18], Choi et al. [3], Antic et al. [41], Liu et al. [16], Wang et al. [4], Lan et al. [20], Choi et al. [22], Kong et al. [1], Zhou et al. [42], and Hajimirsadeg…”
Influential
Cited by
Spatiotemporal Co-Attention Recurrent Neural Networks for Human-Skeleton Motion Prediction
Shu, X., Zhang, L., Qi, G.-J., Liu, W., & Tang, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050918 · Cited by 276
Self-citationHost–Parasite: Graph LSTM-in-LSTM for Group Activity Recognition
Shu, X., Zhang, L., Sun, Y., & Tang, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978942 · Cited by 258
Self-citationEfficient densely connected convolutional neural networks
Li, G., Zhang, M., Li, J., Lv, F., & Tong, G. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2020.107610 · Cited by 145
Associated Spatio-Temporal Capsule Network for Gait Recognition
Zhao, A., Dong, J., Li, J., Qi, L., & Zhou, H. (2021) · arXiv · doi:10.48550/arxiv.2101.02458 · Cited by 55
Multi-sensor-based Action Monitoring and Recognition via Hybrid Descriptors and Logistic Regression
Hafeez, S., Alotaibi, S. S., Alazeb, A., Mudawi, N. A., & Kim, W. (2023) · IEEE Access · doi:10.1109/access.2023.3275733 · Cited by 24
“Thus far, significant progress has been made in this area, and numerous effective HAR systems have been created for a variety of uses [24].”
Background
Metadata
- Authors
- Shu, X., Tang, J., Qi, G.-J., Liu, W., & Yang, J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-03-01
- Volume
- 43
- Issue
- 3
- Pages
- 1110-1118
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2974686944
- Cited by
- 213
- References
- 4
- FWCI
- 11.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition. https://app.cochat.ai/papers/10.1109/tpami.2019.2942030
- The paper
- Shu, X., Tang, J., Qi, G.-J., Liu, W., & Yang, J. (2021). Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2019.2942030
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.