Spatiotemporal Co-Attention Recurrent Neural Networks for Human-Skeleton Motion Prediction
The DOI matches its Crossref record; 67 of the 68 references it cites were found in a registry or an index.
Abstract
Human motion prediction aims to generate future motions based on the observed human motions. Witnessing the success of Recurrent Neural Networks (RNN) in modeling sequential data, recent works utilize RNNs to model human-skeleton motions on the observed motion sequence and predict future human motions. However, these methods disregard the existence of the spatial coherence among joints and the temporal evolution among skeletons, which reflects the crucial characteristics of human motions in spatiotemporal space. To this end, we propose a novel Skeleton-Joint Co-Attention Recurrent Neural Networks (SC-RNN) to capture the spatial coherence among joints, and the temporal evolution among skeletons simultaneously on a skeleton-joint co-attention feature map in spatiotemporal space. First, a skeleton-joint feature map is constructed as the representation of the observed motion sequence. Second, we design a new Skeleton-Joint Co-Attention (SCA) mechanism to dynamically learn a skeleton-joint co-attention feature map of this skeleton-joint feature map, which can refine the useful observed motion information to predict one future motion. Third, a variant of GRU embedded with SCA collaboratively models the human-skeleton motion and human-joint motion in spatiotemporal space by regarding the skeleton-joint co-attention feature map as the motion context. Experimental results of human motion prediction demonstrate that the proposed method outperforms the competing methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This encoder-decoder module pattern [5, 6] can prevent error accumulation caused by successive decoders from ASFormer [4].”
From Stacked encoder–decoder transformer with boundary smoothing for action segmentation (2022) · cites this paper as [6]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410 · cited 2×
“skeleton [21], [33], [34], [35], [36], [37].”
MethodHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.213 · Cited by 2,684
A survey on vision-based human action recognition
Poppe, R. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.11.014 · Cited by 2,242
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Generating Text with Recurrent Neural Networks
Sutskever, I., Martens, J., & Hinton, G. E. (2011) · International Conference on Machine Learning · Cited by 1,783
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.82 · Cited by 1,517
Research works printed but not linked: 1
- Li, 2019, Symbiotic graph neural networks for 3d skeleton-based human action recognition and motion prediction, arXiv:1910.02212 · arXiv
Cited by
Items 301–310 of 310
Fujii, S., & Tachibana, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-22539-9_4
Skeleton-Based Action Recognition Method Using Spatial-Temporal Graph Convolutional Networks
Kaya, M., Yıldız, G., & Özcan, C. (2026) · Data Engineering and Applications · doi:10.64972/dea.2026.v5i1.16911d:147-160
Dong, Q., Jiang, H., Bu, Q., & Liu, Q. (2026) · Mathematics · doi:10.3390/math14122033
Chen, X., Cao, W., & Zhong, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133341
Continuous Semantics and Intent Normalization Network for Human Pose Forecasting
Xiao, P., Xu, X., Liu, B., Chen, W., Xu, C., Zhang, H., & He, S. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3715057
Xing, Z., Liu, Z., Zhang, D.-F., Xie, K., Fang, J., Bi, X.-A., & Liu, T. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104225
Recent advances in the application of artificial intelligence and wearable devices in volleyball
Zhou, G., Shi, X., Wang, Y., Xu, K., Shi, W., Li, Q., & Park, J. (2026) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2026.1855108
Liu, S., Yin, X., Cao, W., & Ji, J. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134804
From Where to How: Continuous 4D Interaction Forecasting from Egocentric VideoPreprint
Chu, Q., Zhang, H., Liu, M., Shi, H., Jiang, D., & Nie, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.08636
Recurrent Neural Network for Human Fall Motion Prediction
Yunus, A. P., Pasha, B. R., & Rosita, Y. D. (2026) · Prosiding Seminar Nasional Universitas Ma Chung
Metadata
- Authors
- Shu, X., Zhang, L., Qi, G.-J., Liu, W., & Tang, J.
- Affiliations
- Nanjing University of Science and Technology (CN)
- Nanjing University of Aeronautics and Astronautics (CN)
- Huawei Cloud, Bellevue, WA, USA
- Tencent (China) (CN)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2022-06-01
- Volume
- 44
- Issue
- 6
- Pages
- 3300-3315
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2976669726
- Cited by
- 276
- References
- 68
- Citations by year
- 2026: 21 · 2025: 49 · 2024: 33 · 2023: 73 · 2022: 67
- FWCI
- 22.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Key Research and Development Program of China (2018AAA0102001)
- National Natural Science Foundation of China (61925204, 62072245, 61772268)
- Natural Science Foundation of Jiangsu Province (BK20190065)
- Qing Lan Project of Jiangsu Province
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- human motion prediction, co-attention, spatio-temporal modeling, GRU
- MeSH terms
- Algorithms, Attention, Humans, Motion, Skeleton, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.